openshift Posts

Monitoring Kafka Topics with Dockerized Kafka Manager

19 Jan , 2020 No Comments

Yesterday, Dimas (one of my colleague), are asking me on how to monitor Kafka which are running on top of Openshift using a tools which are accessible thru browser.

One of the tools im recommending is Kafka Manager, which we can download from below url,

https://github.com/yahoo/kafka-manager

Lets start from the beginning, from how to start Zookeeper, Kafka Server, until simulate a simple produce and consume and monitoring it using Kafka Manager.

First, download Kafka from Apache site, extract it, and open bin folder. We need Zookeeper to start before we start anything else. Fyi for this example im using Win10 as my primary Operating System, so all my command below can be different depends on what Operating System you are using. 

cd D:\software\kafka_2.13-2.4.0\bin\windows
zookeeper-server-start.bat ..\..\config\zookeeper.properties

And run Kafka Server afterwards,

kafka-server-start.bat ..\..\config\server.properties

Create a topic,

kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic my-testing-topic

Try produce a simple echo message using Kafka Producer,

kafka-console-producer.bat --broker-list localhost:9092 --topic my-testing-topic

And listen to the sent message using Kafka Consumer,

kafka-console-consumer.bat --bootstrap-server localhost:9092 -topic  my-testing-topic --from-beginning

If you only want to get all the new message, ignoring the old one, just remove “–from-beginning” parameter. And use “–offset” parameter to get a specific offset.

Next is running my Kafka Manager using Docker command. Fyi, 192.168.1.20 is my laptop ip.

docker run --network host --add-host=moby:192.168.1.20 --add-host DESKTOP:192.168.1.20 -p 9000:9000 -e ZK_HOSTS="192.168.1.20:2181"  kafkamanager/kafka-manager

After Kafka-Manager is successfully started, we can browse our Kafka by opening thru browser,

Google+
Read More

Deploying Spring Boot with A Dynamic application.properties Location to Openshift

14 Dec , 2019 No Comments

I want to create a simple spring boot app, and deploy it to Openshift 4.2. It supposed to be a straigh forward task, but the problem is that it is required to externalize all configuration to a configmaps or secret so no need to recompile the whole app in case of configuration change.

There are several approach of externalizing configuraton to configmaps, one way is put it as a string literal, include on your pod and call on application via environment variables, or deploy the whole configuration file and mount it on your Openshift pod. The last approach is the one that we will be doing now today.

First lets start with deploying our properties to Openshift as configmaps,

oc create cm myapp-api-configmap --from-file=D:\source\my-app\src\main\resources\application.properties

We can check and validate the result,

oc get cm

oc describe cm myapp-api-configmap

After that, we can mount corresponding configmap to a specific folder on our Pod, on below example modification is done on DeploymentConfig.yaml and mounting application.properties to /deployments/config folder.

kind: DeploymentConfig
apiVersion: apps.openshift.io/v1
metadata:
  ........
    spec:
      volumes:
        - name: myapp-api-configmap-volume
          configMap:
            name: myapp-api-configmap
            defaultMode: 420
      containers:
        - name: myapp-api
          image: >-
            image-registry.openshift-image-registry.svc:5000/openshift/myapp@sha256:1127.....
          ports:
            - containerPort: 8778
              protocol: TCP
            - containerPort: 8080
              protocol: TCP
            - containerPort: 8443
              protocol: TCP
          resources: {}
          volumeMounts:
            - name: myapp-api-configmap-volume
              mountPath: /deployments/config
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: Always
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      securityContext: {}
      schedulerName: default-scheduler

A modification is also needed on my Dockerfile, pointing a new path for my properties file by using “spring.config.location” parameter,

FROM registry.access.redhat.com/openjdk/openjdk-11-rhel7

USER jboss
RUN mkdir -p /deployments/image && chown -R jboss:0 /deployments
EXPOSE 8080

COPY target/application-1.0.jar /deployments/application.jar
CMD ["java", "-jar", "/deployments/application.jar", "--spring.config.location=file:///deployments/config/application.properties"]

Build, deploy,and see that application is now taking configuration from external configuration file.

Google+
Read More

Run as a Root User on Openshift

21 Nov , 2019 No Comments

Sometimes my docker images got permission issue when deployed to Openshift, due to Openshift gives a random userid as enforced by its default security policy. In order to “bypass” those constrain and run my image as root, i run below command,

oc adm policy add-scc-to-user anyuid -z default -n project-name
Google+
Read More

Deploying A Simple Hello World App using OpenLiberty S2I to Openshift

16 Nov , 2019 No Comments

For this example im using OpenLiberty version 19.0.0.6, and install corresponding image to my Openshift registry using below command,

oc import-image openliberty/open-liberty-s2i:19.0.0.6

Can check our list of images on our imagestream by using this command,

oc get is

Next is creating a simple hello-world webapps, with below pom

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>hello-world-servlet</groupId>
    <artifactId>com.edw</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>war</packaging>

    <dependencies>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
            <version>4.0.1</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

    <build>
        <sourceDirectory>src/main/java</sourceDirectory>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.4</version>
                <configuration>
                    <failOnMissingWebXml>false</failOnMissingWebXml>
                    <webXml>src\main\webapp\WEB-INF\web.xml</webXml>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

And web.xml,

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE web-app
        PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
        "http://java.sun.com/dtd/web-app_2_3.dtd">
<web-app>
    <display-name>My Web Application</display-name>

    <servlet>
        <servlet-name>helloServlet</servlet-name>
        <servlet-class>com.edw.MyServlet</servlet-class>
    </servlet>

    <servlet-mapping>
        <servlet-name>helloServlet</servlet-name>
        <url-pattern>/hello.servlet</url-pattern>
    </servlet-mapping>

    <welcome-file-list>
        <welcome-file>/hello.jsp</welcome-file>
    </welcome-file-list>
    <session-config>
        <session-timeout>30</session-timeout>
    </session-config>
</web-app>

A simple JSP file,

<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<head>
    <title>Hello World</title>
</head>
<body>
Hello World
</body>
</html>

And a simple java file,

package com.edw;

import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;

public class MyServlet  extends HttpServlet {
    public void doGet(HttpServletRequest request, HttpServletResponse response)
            throws IOException {
        response.getWriter().println("Hello");
    }
}

And a simple server.xml file,

<?xml version="1.0" encoding="UTF-8"?>
<server description="OpenLiberty Server">
    <httpEndpoint id="defaultHttpEndpoint" host="*" httpPort="9080" httpsPort="9443"/>
    <webApplication location="com.edw-1.0-SNAPSHOT.war"/>
</server>

After project is properly setup, we can do a simple mvn build,

mvn clean package

And push our application to Openshift, run below command on the root of your project location

oc new-build --name=my-openliberty-full --image-stream=open-liberty-s2i:19.0.0.6 --binary=true

oc start-build my-openliberty-full --from-dir=.

oc new-app my-openliberty-full --name=my-openliberty-full

We can access our newly created app directly thru browser,

Google+
Read More

Fixing Error “null username” when Integrating RedHat Single Sign On to Active Directory

17 Oct , 2019 No Comments

Previously never had any issue when integrating RedHat SSO (Keycloak) to LDAP, but now got a very weird issue because now im trying to connecting RHSSO to Microsoft Active Directory instead of standard LDAP.

One biggest difference is that ActiveDirectory is using “sAMAccountName” field for user primarykey mapping, and somehow RHSSO is always get null value when trying to synchronize with existing user. Below is the complete stacktrace. 

11:59:45,031 ERROR [org.keycloak.storage.ldap.LDAPStorageProviderFactory] (default task-122) Failed during import user from LDAP: 
org.keycloak.models.ModelException: User returned from LDAP has null username! 
Check configuration of your LDAP mappings. Mapped username LDAP attribute: sAMAccountName, 
user DN: CN=XXX,OU=User,OU=HO,DC=llll,DC=co,DC=id, attributes from LDAP: 
{whenChanged=[20191016020643.0Z], whenCreated=[20170105023800.0Z], mail=[xxx@lll.co.id], givenName=[cccc], sn=[dddd], cn=[ccccc dddd], userAccountControl=[512], pwdLastSet=[132156652033202194]}
	at org.keycloak.storage.ldap.LDAPUtils.getUsername(LDAPUtils.java:113)
	at org.keycloak.storage.ldap.LDAPStorageProviderFactory$3.run(LDAPStorageProviderFactory.java:542)
	at org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:227)
	at org.keycloak.storage.ldap.LDAPStorageProviderFactory.importLdapUsers(LDAPStorageProviderFactory.java:535)
	at org.keycloak.storage.ldap.LDAPStorageProviderFactory.syncImpl(LDAPStorageProviderFactory.java:490)
	at org.keycloak.storage.ldap.LDAPStorageProviderFactory.sync(LDAPStorageProviderFactory.java:428)
	at org.keycloak.services.managers.UserStorageSyncManager$2$1.call(UserStorageSyncManager.java:107)
	at org.keycloak.services.managers.UserStorageSyncManager$2$1.call(UserStorageSyncManager.java:102)
	at org.keycloak.cluster.infinispan.InfinispanClusterProvider.executeIfNotExecuted(InfinispanClusterProvider.java:78)
	at org.keycloak.services.managers.UserStorageSyncManager$2.run(UserStorageSyncManager.java:102)
	at org.keycloak.models.utils.KeycloakModelUtils.runJobInTransaction(KeycloakModelUtils.java:227)
	at org.keycloak.services.managers.UserStorageSyncManager.syncAllUsers(UserStorageSyncManager.java:92)
	at org.keycloak.services.resources.admin.UserStorageProviderResource.syncUsers(UserStorageProviderResource.java:142)
	at sun.reflect.GeneratedMethodAccessor891.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)

It turns out that i have to mapping “sAMAccountName” field to username. Can find the complete screenshot below,

Cheers (^)

Google+
Read More