Sanjeewa Malalgoda's Blog: May 2017

How to setup WSO2 API Manager with Oracle 11g using

In this post i will explain how you can setup WSO2 API Manager analytucs with oracle 11g using docker container. First you need to install docker on your machine. You can use same commands for any server to configure with oracle.

First run following command.

docker run -d -p 49160:22 -p 49161:1521 -e ORACLE_ALLOW_REMOTE=true wnameless/oracle-xe-11g
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5399cedca43c wnameless/oracle-xe-11g "/bin/sh -c '/usr/sbi" 2 minutes ago Up 2 minutes 8080/tcp, 0.0.0.0:49160->22/tcp, 0.0.0.0:49161->1521/tcp grave_hugle

Now Lets log into oracle and create

>>ssh root@localhost -p 49160
>>admin as password.
su oracle
sqlplus / as sysdba
SQL> create user testamdb identified by testamdb account unlock;
SQL> grant create session, dba to testamdb;
SQL> commit;
SQL> connect testamdb;

Now we have successfully connected to created database. Now lets add master-datasource config to api manager instance and analytics instance as follows. When analytics node run it will create required tables automatically. If you have schema you can create tables by yourself as well.

Data source

<datasource>
<name>WSO2AM_STATS_DB</name>

<description>The datasource used for setting statistics to API Manager</description>

<jndiConfig>

<name>jdbc/WSO2AM_STATS_DB</name>

</jndiConfig>

<definition type="RDBMS">

<configuration>

<url>jdbc:oracle:thin:@127.0.0.1:49161/xe</url>

<username>testamdb</username>

<password>testamdb</password> <driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>

<maxActive>50</maxActive>

<maxWait>60000</maxWait>

<testOnBorrow>true</testOnBorrow>

<validationQuery>SELECT 1</validationQuery>

<validationInterval>30000</validationInterval>

<defaultAutoCommit>false</defaultAutoCommit>

</configuration>

</definition>

</datasource>

Explanation about load balance endpoints and endpoint suspension

In this post i will provide explanation about load balance endpoints and endpoint suspension. If we have 2 endpoints in load balanced manner then it will behave as below.

If both endpoints are in working condition.

Requests with route to endpoint_1 and then next request will go to endpoint_2. Likewise it will repeat.

If endpoint_1 failed to serve requests.

If endpoint_1 failed to serve requests when load balanced endpoints used it will detect endpoint_1 failure and then route request to endpoint_2. You can see details in following log. It says it detect one endpoint failure and endpoint suspended for 30 seconds.

[2017-05-30 23:08:26,152]  WARN - ConnectCallback Connection refused or failed for : /172.17.0.1:8081
[2017-05-30 23:08:26,153]  WARN - EndpointContext Endpoint : admin--CalculatorAPI_APIproductionEndpoint_0_0 will be marked SUSPENDED as it failed
[2017-05-30 23:08:26,154]  WARN - EndpointContext Suspending endpoint : admin--CalculatorAPI_APIproductionEndpoint_0_0 - last suspend duration was : 30000ms and current suspend duration is : 30000ms - Next retry after : Tue May 30 23:08:56 IST 2017
[2017-05-30 23:08:26,154]  WARN - LoadbalanceEndpoint Endpoint [admin--CalculatorAPI_APIproductionEndpoint_0] Detect a Failure in a child endpoint : Endpoint [admin--CalculatorAPI_APIproductionEndpoint_0_0]

After you see above log in output it will not route requests to endpoint_1 for 30 seconds(30000ms). Then if you send request after 30 seconds it will again route to endpoint_1 and since its not available request will go to endpoint_2. Same cycle repeats until endpoint_1 available to serve requests.

If both endpoint_1 and endpoint_2 failed.

Then it will go to endpoint_1 and once failure detect it will go to endpoint_2. Then once it realized all endpoint belong to that load balanced endpoint it will not accept further requests and send error message. It will not go into loop and it will go through all endpoints onetime and stop processing request(by sending proper error). Please see below logs.

Detect first endpoint_1 failure
[2017-05-30 23:41:58,643] WARN - ConnectCallback Connection refused or failed for : /172.17.0.1:8081
[2017-05-30 23:41:58,646] WARN - EndpointContext Endpoint : admin--CalculatorAPI_APIproductionEndpoint_0_0 will be marked SUSPENDED as it failed
[2017-05-30 23:41:58,648] WARN - EndpointContext Suspending endpoint : admin--CalculatorAPI_APIproductionEndpoint_0_0 - last suspend duration was : 70000ms and current suspend duration is : 70000ms - Next retry after : Tue May 30 23:43:08 IST 2017
[2017-05-30 23:41:58,648] WARN - LoadbalanceEndpoint Endpoint [admin--CalculatorAPI_APIproductionEndpoint_0] Detect a Failure in a child endpoint : Endpoint [admin--CalculatorAPI_APIproductionEndpoint_0_0]

Detect endpoint_2 failure
[2017-05-30 23:41:58,651] WARN - ConnectCallback Connection refused or failed for : /172.17.0.1:8080
[2017-05-30 23:41:58,654] WARN - EndpointContext Endpoint : admin--CalculatorAPI_APIproductionEndpoint_0_1 will be marked SUSPENDED as it failed
[2017-05-30 23:41:58,656] WARN - EndpointContext Suspending endpoint : admin--CalculatorAPI_APIproductionEndpoint_0_1 - current suspend duration is : 30000ms - Next retry after : Tue May 30 23:42:28 IST 2017
[2017-05-30 23:41:58,657] WARN - LoadbalanceEndpoint Endpoint [admin--CalculatorAPI_APIproductionEndpoint_0] Detect a Failure in a child endpoint : Endpoint [admin--CalculatorAPI_APIproductionEndpoint_0_1]

Once it realized both load balanced endpoint failed it will print error saying no child endpoints to process requests.
[2017-05-30 23:41:58,657] WARN - LoadbalanceEndpoint Loadbalance endpoint : admin--CalculatorAPI_APIproductionEndpoint_0 - no ready child endpoints
[2017-05-30 23:41:58,667] INFO - LogMediator STATUS = Executing default 'fault' sequence, ERROR_CODE = 101503, ERROR_MESSAGE = Error connecting to the back end

When endpoint suspension happens, it will work as follows.
For the suspend duration after the first failure, this equation does not applies. Also when endpoint changed from active to suspend state, suspension duration will be exactly initial duration.

This equation only applies when endpoint already in suspended state and suspension duration expired.

next suspension time period = Min (Initial suspension duration * Progression Factor , Max suspend time).

How we can update swagger definition selectively and push that change to API gateway

In this post i will explain how we can update swagger definition selectively and push that change to API gateway with API Manager 1.10.

If we update swagger definition which is sub resource of the API resource it will work fine. And that operation is successful and you will see API definition updated in UI. But to reflect any change to API gateway(to update synapse API definition) we need to do complete API resource update as it will handle API gateway deployment process.

Obtain Access token to view APIs with both apim:api_create, apim:api_view scope. We need apim:api_create scope because we are going to update API and it need to done with token obtained for apim:api_create scope.

curl -k -d "grant_type=password&username=admin&password=admin&scope=apim:api_create apim:api_view" -H "Authorization: Basic cmpPeWZMTDRaMkhUVmVncnZUbERhRkFmeGZnYTpKUFJRVV9iNEM3WjY1RlBjUmhvQTJPWHh2dkFh" https://127.0.0.1:8243/token
{"access_token":"9f4553dc3c40abb81cf23c8a009821a2","refresh_token":"4db87bf23981e60b22c731aa42dcc6a8","scope":"apim:api_create apim:api_view","token_type":"Bearer","expires_in":3600}

Then get APIs available in the system.

curl -k -H "Authorization: Bearer 9f4553dc3c40abb81cf23c8a009821a2" https://127.0.0.1:9443/api/am/publisher/v0.9/apis

Now select correct ID associated with the API you need to access.

curl -k -H "Authorization: Bearer 9f4553dc3c40abb81cf23c8a009821a2" https://127.0.0.1:9443/api/am/publisher/v0.9/apis/8f8d859e-867e-45da-853b-23de80a44133

{"sequences":[],"tiers":["Unlimited"],"thumbnailUrl":null,"visibility":"PUBLIC","visibleRoles":[],"visibleTenants":[],"cacheTimeout":300,"endpointConfig":"{\"production_endpoints\":{\"url\":\"https://localhost:9443/am/sample/calculator/v1/api\",\"config\":null},\"sandbox_endpoints\":{\"url\":\"https://localhost:9443/am/sample/calculator/v1/api\",\"config\":null},\"implementation_status\":\"managed\",\"endpoint_type\":\"http\"}","subscriptionAvailability":null,"subscriptionAvailableTenants":[],"destinationStatsEnabled":"Disabled","apiDefinition":"{\"paths\":{\"\\/*\":{\"get\":{\"x-auth-type\":\"Application\",\"responses\":{\"200\":{\"description\":\"OK\"}},\"x-throttling-tier\":\"Unlimited\"}}},\"x-wso2-security\":{\"apim\":{\"x-wso2-scopes\":[]}},\"swagger\":\"2.0\",\"info\":{\"contact\":{\"name\":\"xx\",\"email\":\"xx@ee.com\"},\"description\":\"Verify a phone number\",\"title\":\"PhoneVerification\",\"version\":\"1.0.0\"}}","responseCaching":"Disabled","isDefaultVersion":false,"gatewayEnvironments":"Production and Sandbox","businessInformation":{"technicalOwner":null,"technicalOwnerEmail":null,"businessOwner":null,"businessOwnerEmail":null},"tags":["calculator"],"transport":["http","https"],"provider":"admin","version":"1.0","description":"Simple calculator API to perform addition, subtraction, multiplication and division.","status":"PUBLISHED","name":"CalculatorAPI","context":"/calc","id":"8f8d859e-867e-45da-853b-23de80a44133"}

Now update swagger content as follows with the token you obtained from previous step. I'm changing http verb get to put as follows.

curl -k -H "Authorization:Bearer 9f4553dc3c40abb81cf23c8a009821a2" -F apiDefinition="{\"paths\":{\"\\/*\":{\"put\":{\"x-auth-type\":\"Application\",\"responses\":{\"200\":{\"description\":\"OK\"}},\"x-throttling-tier\":\"Unlimited\"}}},\"x-wso2-security\":{\"apim\":{\"x-wso2-scopes\":[]}},\"swagger\":\"2.0\",\"info\":{\"contact\":{\"name\":\"xx\",\"email\":\"xx@ee.com\"},\"description\":\"Verify a phone number\",\"title\":\"PhoneVerification\",\"version\":\"1.0.0\"}}" -X PUT "https://127.0.0.1:9443/api/am/publisher/v0.9/apis/8f8d859e-867e-45da-853b-23de80a44133/swagger"

You will see below response.

{"paths":{"\/*":{"put":{"x-auth-type":"Application","responses":{"200":{"description":"OK"}},"x-throttling-tier":"Unlimited"}}},"x-wso2-security":{"apim":{"x-wso2-scopes":[]}},"swagger":"2.0","info":{"contact":{"name":"xx","email":"xx@ee.com"},"description":"Verify a phone number","title":"PhoneVerification","version":"1.0.0"}}

Now again get API and see your swagger changes are updated API properly. To do that you can make following API call.

curl -k -H "Authorization: Bearer 9f4553dc3c40abb81cf23c8a009821a2" https://127.0.0.1:9443/api/am/publisher/v0.9/apis/8f8d859e-867e-45da-853b-23de80a44133

You will get complete API definition as below.

{"sequences":[],"tiers":["Unlimited"],"thumbnailUrl":null,"visibility":"PUBLIC","visibleRoles":[],"visibleTenants":[],"cacheTimeout":300,"endpointConfig":"{\"production_endpoints\":{\"url\":\"https://localhost:9443/am/sample/calculator/v1/api\",\"config\":null},\"sandbox_endpoints\":{\"url\":\"https://localhost:9443/am/sample/calculator/v1/api\",\"config\":null},\"implementation_status\":\"managed\",\"endpoint_type\":\"http\"}","subscriptionAvailability":null,"subscriptionAvailableTenants":[],"destinationStatsEnabled":"Disabled","apiDefinition":"{\"paths\":{\"\\/*\":{\"put\":{\"x-auth-type\":\"Application\",\"responses\":{\"200\":{\"description\":\"OK\"}},\"x-throttling-tier\":\"Unlimited\"}}},\"x-wso2-security\":{\"apim\":{\"x-wso2-scopes\":[]}},\"swagger\":\"2.0\",\"info\":{\"contact\":{\"name\":\"xx\",\"email\":\"xx@ee.com\"},\"description\":\"Verify a phone number\",\"title\":\"PhoneVerification\",\"version\":\"1.0.0\"}}","responseCaching":"Disabled","isDefaultVersion":false,"gatewayEnvironments":"Production and Sandbox","businessInformation":{"technicalOwner":null,"technicalOwnerEmail":null,"businessOwner":null,"businessOwnerEmail":null},"tags":["calculator"],"transport":["http","https"],"provider":"admin","version":"1.0","description":"Simple calculator API to perform addition, subtraction, multiplication and division.","status":"PUBLISHED","name":"CalculatorAPI","context":"/calc","id":"8f8d859e-867e-45da-853b-23de80a44133"}

Up to this point we can see all UI elements and swagger file updated but it is not reflected to synapse configuration. Now you need to update API(not a swagger update but complete API update) using the exact same payload you obtained from previous step. So i will create data.json file and copy response of my previous step. Then invoke following operation.

curl -k -H "Authorization: Bearer 9f4553dc3c40abb81cf23c8a009821a2" -H "Content-Type: application/json" -X PUT -d @data.json https://127.0.0.1:9443/api/am/publisher/v0.9/apis/8f8d859e-867e-45da-853b-23de80a44133