esb Posts

Logging and Circuit Breaker Mechanism with Hystrix on Red Hat Fuse

Red Hat have a middleware product called Red Hat Fuse, both Red hat Fuse it and Apache Camel as its OpenSource project are well known products for a message-oriented middleware and integration platform. Functionality-wise, there are no huge difference between those two (Fuse and Camel) besides of support, so sometimes it is a good approach to test the opensource version first before going with the supported one.

Right now, we are trying to create a simple Camel script which is doing a proxy request to a third party api. But we are doing some enhancement on the script, so it will not be just a reverse proxy but added with more features such as logging, statistics and a circuit breaker mechanism. We are adding both logging and statistics for seeing the healthiness of our api and third party api endpoints, while circuit braker is being use for handling scenarios where there a latency or slowness happen on our third party api.

The underlying concept is drawn on below diagram,

Okay, so lets start by creating a Fuse application with maven,

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.edw</groupId>
    <artifactId>HttpLoggingAndCircuitBreaker</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <fuse.version>7.2.0.fuse-720020-redhat-00001</fuse.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.jboss.redhat-fuse</groupId>
                <artifactId>fuse-springboot-bom</artifactId>
                <version>${fuse.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-http-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-servlet-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.camel</groupId>
            <artifactId>camel-hystrix-starter</artifactId>
        </dependency>

        <dependency>
            <groupId>com.netflix.archaius</groupId>
            <artifactId>archaius-core</artifactId>
            <version>0.7.1</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-hystrix-dashboard</artifactId>
            <version>1.4.7.RELEASE</version>
            <exclusions>
                <exclusion>
                    <groupId>com.netflix.archaius</groupId>
                    <artifactId>archaius-core</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-hystrix</artifactId>
            <version>1.4.7.RELEASE</version>
            <exclusions>
                <exclusion>
                    <groupId>com.netflix.archaius</groupId>
                    <artifactId>archaius-core</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.jboss.redhat-fuse</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                    <version>${fuse.version}</version>
                </plugin>
            </plugins>
        </pluginManagement>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>8</source>
                    <target>8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

And two simple rest api, written in Camel XML format, first is a simple hello world rest api, while the second one is a much more complex app which involved circuit breaking and logging,

<?xml version="1.0" encoding="UTF-8"?>
<rest path="/hello-world" xmlns="http://camel.apache.org/schema/spring">
    <get>
        <route>
            <setHeader headerName="Content-Type">
                <constant>application/json</constant>
            </setHeader>
            <setBody>
                <simple>{ "hello": "world" }</simple>
            </setBody>
        </route>
    </get>
</rest>
<?xml version="1.0" encoding="UTF-8"?>
<rest path="/circuit-breaker" xmlns="http://camel.apache.org/schema/spring">
    <post>
        <route>
            <setHeader headerName="timestamp">
                <simple>${bean:java.lang.System?method=currentTimeMillis}</simple>
            </setHeader>

            <hystrix>
                <!--
                    a message which exceed 1000ms will considered as timeout, and trigger fallback method.
                    It need a 5 timeouts in last ten seconds to trigger a 5000ms circuit break, before trying to re-connect again.
                -->
                <hystrixConfiguration executionTimeoutInMilliseconds="1000"
                                      circuitBreakerSleepWindowInMilliseconds="5000"
									  metricsRollingStatisticalWindowInMilliseconds="10000"
                                      circuitBreakerRequestVolumeThreshold="5" />
                <to uri="https://run.mocky.io/v3/bc6c24ff-d124-401f-bf84-31e0c6366af6?httpClient.soTimeout=1000&amp;bridgeEndpoint=true" />
                <onFallback>
                    <!--  reversal example when a message is considered timeout by hystrix -->
                    <doTry>
                        <to uri="https://run.mocky.io/v3/4eb219a6-a7b1-40b8-90e1-230052fc5823?httpClient.soTimeout=1000&amp;bridgeEndpoint=true" />

                        <doCatch>
                            <exception>java.lang.Exception</exception>
                            <bean ref="logHandler" method="logError('on catch')"/>
                        </doCatch>
                        <doFinally>
                            <setBody>
                                <simple>{ "hello": "fallback" }</simple>
                            </setBody>
                        </doFinally>
                    </doTry>
                </onFallback>
            </hystrix>

            <bean ref="logHandler" method="logTimestamp(${header.timestamp})"/>
        </route>
    </post>
</rest>

The first api is quite self-explanatory, use below curl to do testing

curl -kv http://localhost:8080/api/hello-world

But we are focusing to the second API, which is much more complicated because it consist of a circuit breaker functionality which will do a 5 seconds break if there are more than 5 errors on the last 10 seconds. Perhaps diagram below can help simplifying the basic concept,

And can test that url with this code,

curl -kv http://localhost:8080/api/circuit-breaker -X POST --header "Content-Type: application/json" --data '{"username":"xyz","password":"xyz"}'

Next is creating a surrounding support system for this app, such as a hystrix dashboard for displaying how much transaction is failing, response time average, circuit status and error percentage, and a hystrix stream for providing those data. In order to activate hystrix dashboard and stream, we need to register it on our project.

package com.edw;

import org.apache.camel.component.hystrix.metrics.servlet.HystrixEventStreamServlet;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;
import org.springframework.context.annotation.Bean;

@EnableCircuitBreaker
@EnableHystrixDashboard
@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Bean
    public HystrixEventStreamServlet hystrixServlet() {
        return new HystrixEventStreamServlet();
    }

    @Bean
    public ServletRegistrationBean servletRegistrationBean() {
        return new ServletRegistrationBean(new HystrixEventStreamServlet(), "/hystrix.stream");
    }
}

By default, we can see our hystrix dashboard on this url

http://localhost:8080/hystrix

Fill the data,

And see the result,

One other thing is that we can monitor the statistic and trace http history for our Camel by using our Actuator api,

curl -kv http://localhost:8080/metrics

curl -kv http://localhost:8080/trace

One thing that is worth mentioning is that we can see how much response time for each API from our Actuator trace API, which by default gives last 100 http request. For example below, it took 297ms for our API to give a response.

Lastly, we can see my code on below github url.

https://github.com/edwin/fuse-logging-circuit-breaker

Good luck, and have fun with Camel and Circuit Breaker.

Google+

[Mule] Error 400 Bad Request, Request Header Or Cookie Too Large

I had an error yersterday, when connecting Mule ESB to NGinx. Very weird, because somehow it never shown any errors when fired directly without a reverse proxy. Here is the complete stacktrace

<head><title>400 Request Header Or Cookie Too Large</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<center>Request Header Or Cookie Too Large</center>
<hr><center>nginx/1.4.4</center>
</body>
</html>
 (javax.xml.ws.soap.SOAPFaultException)
  org.apache.cxf.jaxws.JaxWsClientProxy:156 (http://java.sun.com/j2ee/sdk_1.3/techdocs/api/javax/xml/ws/soap/SOAPFaultException.html)
3. Response was of unexpected text/html ContentType.  Incoming portion of HTML stream: <html>
<head><title>400 Request Header Or Cookie Too Large</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<center>Request Header Or Cookie Too Large</center>
<hr><center>nginx/1.4.4</center>
</body>
</html>

It happens because Mule, by default, send a very large cookies called “X-MULE_SESSION”. After removing it, my ESB runs well again. This is how i remove it,

<http:connector name="NoSessionConnector">
	    <service-overrides sessionHandler="org.mule.session.NullSessionHandler"/>
</http:connector>
Google+

How to Handle Error on Mule, “Flow exception listener contains and exception strategy that doesn’t handle all request”

Today i had a very weird error on Mule ESB, that somehow never happened to my other colleague. This is the complete stacktrace,

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Failed to deploy artifact 'edw-soa', see below              +
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
org.mule.module.launcher.DeploymentInitException: FlowConstructInvalidException: Flow exception listener 
contains and exception strategy that doesn't handle all request, Perhaps there's an exception strategy 
with a when attribute set but it's not part of a catch exception strategy
	at org.mule.module.launcher.application.DefaultMuleApplication.init(DefaultMuleApplication.java:196) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.artifact.ArtifactWrapper$2.execute(ArtifactWrapper.java:62) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.artifact.ArtifactWrapper.executeWithinArtifactClassLoader(ArtifactWrapper.java:129) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.artifact.ArtifactWrapper.init(ArtifactWrapper.java:57) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.DefaultArtifactDeployer.deploy(DefaultArtifactDeployer.java:25) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.DefaultArchiveDeployer.guardedDeploy(DefaultArchiveDeployer.java:310) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.DefaultArchiveDeployer.deployArtifact(DefaultArchiveDeployer.java:330) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.DefaultArchiveDeployer.deployPackagedArtifact(DefaultArchiveDeployer.java:155) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.DefaultArchiveDeployer.deployPackagedArtifact(DefaultArchiveDeployer.java:256) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.DefaultArchiveDeployer.deployPackagedArtifact(DefaultArchiveDeployer.java:78) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.DeploymentDirectoryWatcher.deployPackedApps(DeploymentDirectoryWatcher.java:275) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.DeploymentDirectoryWatcher.start(DeploymentDirectoryWatcher.java:150) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.MuleDeploymentService.start(MuleDeploymentService.java:100) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.module.launcher.MuleContainer.start(MuleContainer.java:152) ~[mule-module-launcher-3.6.1.jar:3.6.1]
	at org.mule.tooling.server.application.ApplicationDeployer.main(ApplicationDeployer.java:15) ~[tooling-support-3.6.1.jar:?]

Basically, it happen because of MuleESB backward incompatibility. And after adding these line, the error seems to go away (forever hopefully :D )

<catch-exception-strategy doc:name="Catch Exception Strategy">
	<logger level="ERROR" message="[Exception thrown]" />
	<set-payload
		value="#[com.edw.soa.process.Utils.responseProcess(originalPayload,'Transaksi gagal','1')]" />
</catch-exception-strategy>

Thank you to Muhammad Reza Nurhakim and Adhi Wicaksono for solving this.

Google+