Openshift | edwin's journey

Monitoring Kafka Topics with Dockerized Kafka Manager

By edwin / 19 January 2020 19 January 2020 / Openshift, Programming, Server Configuration / docker, kafka, middleware, openshift

Yesterday, Dimas (one of my colleague), are asking me on how to monitor Kafka which are running on top of Openshift using a tools which are accessible thru browser.

One of the tools im recommending is Kafka Manager, which we can download from below url,

https://github.com/yahoo/kafka-manager

Lets start from the beginning, from how to start Zookeeper, Kafka Server, until simulate a simple produce and consume and monitoring it using Kafka Manager.

First, download Kafka from Apache site, extract it, and open bin folder. We need Zookeeper to start before we start anything else. Fyi for this example im using Win10 as my primary Operating System, so all my command below can be different depends on what Operating System you are using.

cd D:\software\kafka_2.13-2.4.0\bin\windows
zookeeper-server-start.bat ..\..\config\zookeeper.properties

And run Kafka Server afterwards,

kafka-server-start.bat ..\..\config\server.properties

Create a topic,

kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic my-testing-topic

Try produce a simple echo message using Kafka Producer,

kafka-console-producer.bat --broker-list localhost:9092 --topic my-testing-topic

And listen to the sent message using Kafka Consumer,

kafka-console-consumer.bat --bootstrap-server localhost:9092 -topic  my-testing-topic --from-beginning

If you only want to get all the new message, ignoring the old one, just remove “–from-beginning” parameter. And use “–offset” parameter to get a specific offset.

Next is running my Kafka Manager using Docker command. Fyi, 192.168.1.20 is my laptop ip.

docker run --network host --add-host=moby:192.168.1.20 --add-host DESKTOP:192.168.1.20 -p 9000:9000 -e ZK_HOSTS="192.168.1.20:2181"  kafkamanager/kafka-manager

After Kafka-Manager is successfully started, we can browse our Kafka by opening thru browser,

Deploying Spring Boot with A Dynamic application.properties Location to Openshift

By edwin / 14 December 2019 27 December 2019 / Openshift, Programming / boot, Java, openshift, spring

I want to create a simple spring boot app, and deploy it to Openshift 4.2. It supposed to be a straigh forward task, but the problem is that it is required to externalize all configuration to a configmaps or secret so no need to recompile the whole app in case of configuration change.

There are several approach of externalizing configuraton to configmaps, one way is put it as a string literal, include on your pod and call on application via environment variables, or deploy the whole configuration file and mount it on your Openshift pod. The last approach is the one that we will be doing now today.

First lets start with deploying our properties to Openshift as configmaps,

oc create cm myapp-api-configmap --from-file=D:\source\my-app\src\main\resources\application.properties

We can check and validate the result,

oc get cm

oc describe cm myapp-api-configmap

After that, we can mount corresponding configmap to a specific folder on our Pod, on below example modification is done on DeploymentConfig.yaml and mounting application.properties to /deployments/config folder.

kind: DeploymentConfig
apiVersion: apps.openshift.io/v1
metadata:
  ........
    spec:
      volumes:
        - name: myapp-api-configmap-volume
          configMap:
            name: myapp-api-configmap
            defaultMode: 420
      containers:
        - name: myapp-api
          image: >-
            image-registry.openshift-image-registry.svc:5000/openshift/myapp@sha256:1127.....
          ports:
            - containerPort: 8778
              protocol: TCP
            - containerPort: 8080
              protocol: TCP
            - containerPort: 8443
              protocol: TCP
          resources: {}
          volumeMounts:
            - name: myapp-api-configmap-volume
              mountPath: /deployments/config
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: Always
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      securityContext: {}
      schedulerName: default-scheduler

A modification is also needed on my Dockerfile, pointing a new path for my properties file by using “spring.config.location” parameter,

FROM registry.access.redhat.com/openjdk/openjdk-11-rhel7

USER jboss
RUN mkdir -p /deployments/image && chown -R jboss:0 /deployments
EXPOSE 8080

COPY target/application-1.0.jar /deployments/application.jar
CMD ["java", "-jar", "/deployments/application.jar", "--spring.config.location=file:///deployments/config/application.properties"]

Build, deploy,and see that application is now taking configuration from external configuration file.

Run as a Root User on Openshift

By edwin / 21 November 2019 21 November 2019 / Openshift, Programming / configuration, openshift

Sometimes my docker images got permission issue when deployed to Openshift, due to Openshift gives a random userid as enforced by its default security policy. In order to “bypass” those constrain and run my image as root, i run below command,

oc adm policy add-scc-to-user anyuid -z default -n project-name

Deploying A Simple Hello World App using OpenLiberty S2I to Openshift

By edwin / 16 November 2019 22 November 2019 / Java, Openshift, Programming / Java, openliberty, openshift

For this example im using OpenLiberty version 19.0.0.6, and install corresponding image to my Openshift registry using below command,

oc import-image openliberty/open-liberty-s2i:19.0.0.6

Can check our list of images on our imagestream by using this command,

oc get is

Next is creating a simple hello-world webapps, with below pom

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>hello-world-servlet</groupId>
    <artifactId>com.edw</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>war</packaging>

    <dependencies>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
            <version>4.0.1</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

    <build>
        <sourceDirectory>src/main/java</sourceDirectory>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.4</version>
                <configuration>
                    <failOnMissingWebXml>false</failOnMissingWebXml>
                    <webXml>src\main\webapp\WEB-INF\web.xml</webXml>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

And web.xml,

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE web-app
        PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
        "http://java.sun.com/dtd/web-app_2_3.dtd">
<web-app>
    <display-name>My Web Application</display-name>

    <servlet>
        <servlet-name>helloServlet</servlet-name>
        <servlet-class>com.edw.MyServlet</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>helloServlet</servlet-name>
        <url-pattern>/hello.servlet</url-pattern>
    </servlet-mapping>

    <welcome-file-list>
        <welcome-file>/hello.jsp</welcome-file>
    </welcome-file-list>
    <session-config>
        <session-timeout>30</session-timeout>
    </session-config>
</web-app>

A simple JSP file,

<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<head>
    <title>Hello World</title>
</head>
<body>
Hello World
</body>
</html>

And a simple java file,

package com.edw;

import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;

public class MyServlet  extends HttpServlet {
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws IOException {
        response.getWriter().println("Hello");
    }
}

And a simple server.xml file,

<?xml version="1.0" encoding="UTF-8"?>
<server description="OpenLiberty Server">
    <httpEndpoint id="defaultHttpEndpoint" host="*" httpPort="9080" httpsPort="9443"/>
    <webApplication location="com.edw-1.0-SNAPSHOT.war"/>
</server>

After project is properly setup, we can do a simple mvn build,

mvn clean package

And push our application to Openshift, run below command on the root of your project location

oc new-build --name=my-openliberty-full --image-stream=open-liberty-s2i:19.0.0.6 --binary=true

oc start-build my-openliberty-full --from-dir=.

oc new-app my-openliberty-full --name=my-openliberty-full

We can access our newly created app directly thru browser,

Creating a Simple Openshift Pipeline for NodeJS 10 Apps with Jenkins Slave

By edwin / 3 October 2019 3 October 2019 / Openshift, Programming / openshift, pipeline

Jenkins pipeline build have a slave mechanism, where it will spawn a new pod based on a specific image and will build on top of it. Slave mechanism have several benefits compared to traditional build, and one of the benefit is it can build with a different environment compared to jenkins master’s environment.

So, lets start with a simple docker file. We’ll create an imagestream using it, and will be used as a slave image. Basically it will use jenkins-slave-base-rhel7 as base image, and will install nodejs 10 on top of it.

oc new-build -D $'
FROM registry.access.redhat.com/openshift3/jenkins-slave-base-rhel7:v3.11
\n
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash
\n
ENV NVM_DIR=/home/jenkins/.nvm \ NODE_VERSION=10.16.0
\n
RUN . "$NVM_DIR/nvm.sh" && nvm install ${NODE_VERSION} && nvm use v${NODE_VERSION} && nvm alias default v${NODE_VERSION}
\n
ENV PATH="/home/jenkins/.nvm/versions/node/v${NODE_VERSION}/bin/:${PATH}"
\n
RUN node --version && npm --version
\n
USER 1001' --name=new-jenkins-slave-node10-rhel7

And you can see the imagestream on Openshift,

Next step is, creating a Jenkins ephemeral on Openshift and creating new Jenkins slave with our newly created image.

oc new-app jenkins-persistent --param ENABLE_OAUTH=true --param MEMORY_LIMIT=2Gi --param VOLUME_CAPACITY=4Gi --param DISABLE_ADMINISTRATIVE_MONITORS=true -e OPENSHIFT_JENKINS_JVM_ARCH=i386

For creating new slave, we can login to Jenkins page, open manage Jenkins menu, and go to Configure System menu, press Add Pod Template button.

Once successfully add new pod, we can start build our pipeline. Select New Item menu, add select Pipeline after that.Add put below code on Pipeline script,

def gitRepo="https://github.com/ariemay/node-test-app.git"
def branch="master"

node('new-jenkins-slave-node10-rhel7') {
    stage('test npm') {
        sh("node --version")
        sh("npm --version")
        sh("oc whoami")
    }
    stage ('pull code') {
        git branch: branch, url: gitRepo
    }
    stage ('build') {
        sh("npm install")
        sh("npm run build")
    }
    stage('check and prepare') {
        sh("cd /tmp")
        sh("pwd")
        sh("ls -alrth")
    }
    stage ('deploy') {
        try {
            sh("oc delete bc hello-react")
        } catch (Exception e) {
            sh("echo \"fail deleting bc \"")
        }
        try {
            sh("oc delete is hello-react")
        } catch (Exception e) {
            sh("echo \"fail deleting is \"")
        }
        try {
            sh("oc delete svc hello-react")
        } catch (Exception e) {
            sh("echo \"fail deleting svc \"")
        }
        try {
            sh("oc delete route hello-react")
        } catch (Exception e) {
            sh("echo \"fail deleting route \"")
        }
        sh("oc new-build --binary=true --name=hello-react --image-stream=nginx-112-rhel7")
        sh("oc start-build hello-react --from-dir=build --follow --wait" )

        try {
            sh("oc new-app  hello-react --name=hello-react" )
        } catch (Exception e) {
            sh("echo \"fail creating new-app, dc exists \"")
        }

        sh("oc expose svc/hello-react --name=hello-react")
    }
}

Press Build Now in order to see the build result,

We can see the url for result pod, and click it to see the built webpage.

So simple right?