devops Posts

Create Async HTTP Process with Red Hat Fuse on Top of Openshift 3.11

Got a unique requirement yesterday where one process on Red Hat Fuse consuming a very long time, therefore creating lots of timeout error from frontend. It happen because some process on Fuse performing a very complicated task and we cannot modified existing api flow due to business requirements.

For this example, i want to simulate the same slowness on my Fuse + Spring Boot app,

package com.redhat.edw;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

A simple camel route,

package com.redhat.edw;

import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.model.rest.RestBindingMode;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

@Component
public class Routes extends RouteBuilder {
    @Override
    public void configure() throws Exception {
        restConfiguration("servlet")
                .bindingMode(RestBindingMode.json)
        ;

        rest()
                .get("hello")
                .route()
                .setBody(method("helloRouteHandler", "setHelloWithName"))
                .endRest()
        ;
    }
}

A placeholder bean,

package com.redhat.edw;

import lombok.*;

@Data
@ToString
@AllArgsConstructor
@NoArgsConstructor
@Builder
public class HelloResponse {
    private String content;
}

And a Handler class,

package com.redhat.edw;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

@Slf4j
@Component("helloRouteHandler")
public class HelloRouteHandler {

    @Value("${name}")
    private String name;

    public HelloResponse setHelloWithName() throws Exception {

        /*
         * simulate a very long process (10second)
         */
        Thread.sleep(10000);

        /*
         * this process will still getting called after 10 second regardless of sync or async
         */
        log.info("calling hello for "+name);

        return HelloResponse.builder().content(name).build();
    }
}

And two simple properties file,

# The Camel context name
camel.springboot.name=FuseHelloWorld

# enable all management endpoints
endpoints.enabled=true
management.security.enabled=false

camel.component.servlet.mapping.contextPath=/api/*

name=Edwin
# OpenShift ConfigMap name
spring.application.name=fuse-hello-world

spring.cloud.kubernetes.reload.enabled=true
spring.cloud.kubernetes.reload.strategy=restart_context
spring.cloud.kubernetes.reload.monitoring-config-maps=true
spring.cloud.kubernetes.reload.monitoring-secrets=true
spring.cloud.kubernetes.reload.mode=polling

Run the project and try calling /api/hello api and see how much time is needed to give response time.

As you can see, 10seconds is not an acceptable response for a regular web user and wee need to improve it. The workaround is quite simple, Spring have an @Async method which we will utilize on Handler class,

package com.redhat.edw;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Component;

@Slf4j
@Component("helloRouteHandler")
public class HelloRouteHandler {

    @Value("${name}")
    private String name;

    @Async
    public HelloResponse setHelloWithName() throws Exception {

        /*
         * simulate a very long process (10second)
         */
        Thread.sleep(10000);

        /*
         * this process will still getting called after 10 second regardless of sync or async
         */
        log.info("calling hello for "+name);

        return HelloResponse.builder().content(name).build();
    }
}

Which will give a faster response,

And we can deploy our code to Openshift by using this command,

mvn fabric8:deploy -Pfabric8

A successful deployment will looks like this,

And finally, the complete code can be downloaded from this url,

https://github.com/edwin/fuse-with-async-http
Google+

Do A Maven Owasp Library Scan from A Restricted Network

When we talk about DevSecOps, we are talking about a continous integration and delivery but embedded with a security scanning along the way. And one of the best tool for doing a security scanning for your application library is OWASP dependency-check, and thankfully we can embed it to our application and run it thru pipeline by using a Maven plugin.

There is a downside tho, Owasp Maven plugin need to update its vulnerability database regularly online from NVD database which is perhaps not convenient for most enterprise environment where online network access is very-very limited.

But there is one workaround, we can use our repository such as Nexus or JFrog to host our NVD vulnerability database. The concept is pretty much we can see on below diagram,

There are two repository needed to build for fulfilling Maven Owasp requirement. One for java library, and another one for javascript.

Once done, we can check our Maven Owasp scan by using this command,

mvn org.owasp:dependency-check-maven:check -DfailBuildOnCVSS=8 \
 -DcveUrlModified=http://nexus.example.com/repository/nvd/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz \
 -DcveUrlBase=http://nexus.example.com/repository/nvd/feeds/json/cve/1.1/nvdcve-1.1-%d.json.gz \
 -DretireJsUrl=http://nexus.example.com/repository/retireJsUrl/jsrepository.json -DretireJsAnalyzerEnabled=false \ 
 -DossindexAnalyzerEnabled=false

If build is success, we can see that both our newly-created repository folder is now have multiple files there,

And if failed, we can see this error happen

And if you want to ignore Owasp scan result, you can change failBuildOnCVSS parameter to 11.

Google+

Creating a Jenkins Slave Image with Maven 3.6, Java 11 and Skopeo

Openshift have a default maven Jenkins slave image, but too bad it is build on top of Java 8. And on this project which im currently working on, i need a custom Jenkins slave but with Java 11 and the ability to move images between Image Registry. Therefore i create a custom Dockerfile which contains Skopeo, Maven 3.6.3 and Java 11. Below is the detail Dockerfile which i created,

FROM openshift/jenkins-slave-base-centos7:v3.11

MAINTAINER Muhammad Edwin < edwin at redhat dot com >


ENV MAVEN_VERSION=3.6.3 \
    PATH=$PATH:/opt/maven/bin

# install skopeo
RUN yum install skopeo -y && yum clean all

# install java
RUN curl -L --output /tmp/jdk.tar.gz https://download.java.net/java/GA/jdk11/9/GPL/openjdk-11.0.2_linux-x64_bin.tar.gz && \
	tar zxf /tmp/jdk.tar.gz -C /usr/lib/jvm && \
	rm /tmp/jdk.tar.gz && \
	update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk-11.0.2/bin/java 20000 --family java-1.11-openjdk.x86_64 && \
	update-alternatives --set java /usr/lib/jvm/jdk-11.0.2/bin/java
	
# Install Maven
RUN curl -L --output /tmp/apache-maven-bin.zip  https://www-eu.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.zip && \
    unzip -q /tmp/apache-maven-bin.zip -d /opt && \
    ln -s /opt/apache-maven-${MAVEN_VERSION} /opt/maven && \
    rm /tmp/apache-maven-bin.zip && \
    mkdir -p $HOME/.m2

RUN chown -R 1001:0 $HOME && chmod -R g+rw $HOME

COPY run-jnlp-client /usr/local/bin/

USER 1001

Build by using this command,

docker build -t jenkins-slave-skopeo-jdk11-new -f skopeo-jdk11.dockerfile .

Pull the image to Openshift,

oc import-image docker.io/edwinkun/jenkins-slave-skopeo-jdk11-new --confirm

Register on Jenkins as a

And try on

node('maven') {
	stage('Clone') {
		sh "git config --global http.sslVerify false"
		sh "git clone https://github.com/edwin/hello-world.git"
	}
	stage('Build') {
		sh "mvn -v"
		sh "mvn clean package -f hello-world/pom.xml"
	}
}

This is the result,

Detail code can be seen on my github page, https://github.com/edwin/jenkins-slave-maven-jdk11-skopeo

Google+