WSO2 ESB - How to track messages between mediation flows

WSO2 ESB Services / APIs for Enterprise level applications, usually consume services deployed in other WSO2 products like DSS, BRS etc., or use services deployed in some legacy or third party application (can call, backend services) in order to complete the service request.

This means you have multiple endpoints configured and used, across different sequences encomposing the service.

Keeping track of these internal message flows between various services is important to debug service flows. This is more important and complex when debugging services which involves service timeouts and client or backend service errors.

Thankfully, as per the WS-Addressing specification, each message is assigned a unique message identifier or Message ID.
Synapse Engine takes care of the same and every Message is assigned an unique message ID.

Its available as MessageID and its a Synapse context message property that can be retrieved using the get-property() function in the Property mediator.

<property name="msgID" expression="get-property('MessageID')"/>

Its makes our world better... 😁
But wait...

For a sample flow like the following, the MessageID will vary across different integration points like ESB ↔ DSS, ESB ↔ TPL Application, etc.,

Sample service flow

This brings us to the core of the issue, how to track the message from the client Request (1) to the ESB and then subsequent calls to DSS services, backend services, etc., and finally the Response to the client.

This can be handled in a simple way as follows: 

1. Read the MessageID from the incoming client request through property mediator and assign to a custom property, say msgID.
      <property name="msgID" expression="get-property('MessageID')"/>

2. Log the custom property across different sequences and debug statements, as follows.

    <log level="custom">
        <property expression="get-property('msgID')" name="***MESSAGE_ID***"                       xmlns:ns="http://org.apache.synapse/xsd" xmlns:ns2="http://org.apache.synapse/xsd"/>
    </log>

This will enable tracking the entire message or service flow using a single MessageID

From a reusability and code quality standpoint, you can have these common log statements in a separate sequence and include this sequence in other In or Out sequences.
This will also be useful to maintain these sequences in Governance Registry.


Happy Reading..!!!

WSO2 ESB - Disable Endpoint Suspension

Endpoints

An endpoint in simple terms is a URL (destination) that can be used by any WSO2 service which needs to send a message to that particular destination / API. 

The endpoints can be configured for both external services and also internal, peer services running inside the same ESB instance or the host system. 

One of the important configurations that is often overlooked are the endpoints error handling. 

As any network oriented application, messages can get lost due to various tcp errors, connection timeouts, etc., Therefore for a successful and controlled behavior, endpoint error handling is very important. 

The default behavior of endpoints in WSO2 is, if messages in those endpoint are failed, the endpoint will be marked as "suspended" and thereby causing failure of the subsequent messages. 

This is more important if multiple internal services are consumed as part of an exposed Proxy service or API. To handle the different errors and timeouts from the internal services and thereby control the response and errors to the end client, its important to manage the errors of endpoints.


Few important configurations are listed out here and a working sample configuration.

Configurations:

"timeout" settings: 

duration - Connection timeout interval. If the service doesn´t respond by this time, the endpoint will be marked as "Timeout" state. In Timeout state, the endpoint can still send or receive messages but if the error continues, the endpoint will be marked as "suspended". 

responseAction - When a response is received to a timed out request, this parameter specifies whether to discard it or to invoke the fault handler. The default value is none. 

Sample Configuration

The following sample configuration can be used to completely disable the enpoint suspension behavior of the endpoints. 

Configure the Timeout, MarkForSuspension and suspendOnFailure settings as shown in the below configuration for the same.

<endpoint xmlns="http://ws.apache.org/ns/synapse" name="service_ep">
    <address statistics="disable" trace="disable" uri="http://localhost:9765/services/stores_Operation">
        <timeout>
            <duration>20000</duration>
            <responseAction>fault</responseAction>
        </timeout>
        <markForSuspension>
            <errorCodes>-1</errorCodes>
        </markForSuspension>
        <suspendOnFailure>
            <initialDuration>0</initialDuration>
            <maximumDuration>0</maximumDuration>
            <progressionFactor>1.0</progressionFactor>
        </suspendOnFailure>
    </address>
</endpoint>

WSO2 - Registry setup

All WSO2 products are shipped with a built-in registry, supported by the H2 Database packaged with the product. Though its sufficient for many applications, its not recommended for Enterprise integration applications and Production environments.

In our case the ESB registry is mounted with WSO2 Governance Registry (GREG). The below steps can be used to change the ATOM based registry mount (default) with JDBC based mount.

The following example was implemented on the systems which had deployed resources and services, servicing other applications.

GREG

1. In GREG, replace WSO2_CARBON_DB in master-datasources.xml with JDBC details.

<datasource>
   <name>WSO2_CARBON_DB</name>
   <description>The datasource used for registry and user manager</description>
   <jndiConfig>
        <name>jdbc/WSO2CarbonDB</name>
   </jndiConfig>
   <definition type="RDBMS">
       <configuration>
           <url>jdbc:oracle:thin:@172.17.0.2:1521:XE</url>
           <username>USER</username>
           <password>USER1234</password>
           <driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
        </configuration>
   </definition>
</datasource>

2. Since we are using a ORACLE Database, copy the driver ojdbc6.jar to the following path
<PRODUCT_HOME>/repository/components/lib/

3. Back up the .CAR files and configurations and packages

4. The following command will create the necessary tables on the target Datasource configured in the above step.
sh wso2server.sh -Dsetup

Validate the following

1. Tables and DB objects are created in the DB
2. Whether data is populated
3. Old resource content are also available
4. Existing CAR files got deployed successfully

Console:

User management

1. Existing user got updated in DB
2. Create a new user and the same should be populated in the DB

In ESB 4.9.0

1. Update the <PRODUCT_HOME>/repository/conf/datasource/master-datasources.xml
Add the following configuration

<datasource>
   <name>WSO2_CARBON_DB_GREG</name>
   <description>The datasource used for registry and user manager</description>
   <jndiConfig>
        <name>jdbc/WSO2CarbonDB_GREG</name>
   </jndiConfig>
   <definition type="RDBMS">
       <configuration>
           <url>jdbc:oracle:thin:@172.17.0.2:1521:XE</url>
           <username>USER</username>
           <password>USER1234</password>
           <driverClassName>oracle.jdbc.driver.OracleDriver</driverClassName>
           <maxActive>50</maxActive>
           <maxWait>60000</maxWait>
           <testOnBorrow>true</testOnBorrow>
           <validationQuery>SELECT 1</validationQuery>
           <validationInterval>30000</validationInterval>
        </configuration>
   </definition>
</datasource>

2. Update the registry configuration

<PRODUCT_ HOME>/repository/conf/registry.xml

<dbConfig name="remote_registry">
      <dataSource>jdbc/WSO2CarbonDB_GREG</dataSource>
</dbConfig>

-- Specify the remote Governance Registry instance --

<remoteInstance url="https://172.17.0.6:9443/registry">
    <id>instanceid</id>
    <dbConfig>remote_registry</dbConfig>
    <cacheId>USER@jdbc:oracle:thin:@172.17.0.2:1521:XE</cacheId>
    <readOnly>false</readOnly>
    <enableCache>false</enableCache>
    <registryRoot>/</registryRoot>
</remoteInstance>

Mount configuration
    <mount path="/_system/governance" overwrite="true">
        <instanceId>governanceRegistryInstance</instanceId>
        <targetPath>/_system/governance</targetPath>
    </mount>

2. Copy the JDBC driver - ojdbc6.jar to /repository/components/lib/

3. Restart ESB

Validate the following

1. Logs have the successful mounting of remote registry

References:

https://docs.wso2.com/display/ESB490/Remote+Instance+and+Mount+Configuration+Details
https://docs.wso2.com/display/ESB490/Governance+Partition+in+a+Remote+Registry