docker Posts

Building Containerized Images on Openshift 4 and Push the Result to Third Party Image Registry

Sometimes in our pipeline, we need to build a docker images based on a specific Dockerfile and push the result to an external Image Registry such as Quay, Docker Hub or even on-premise Nexus or JFrog.

On this example, im trying to simulate build a simple java application, containerized it, and push it to Quay. The rough concept can be seen below,

1. Jenkins pull latest java code from Github, do testing and Maven build
2. Containerizing Maven build result and push it to Quay
3. Openshift Pre-Prod and Prod will pull from Quay, if build result is considered stable enough

For this example, im using a simple Dockerfile,

FROM registry.access.redhat.com/ubi8/ubi-minimal:8.0

MAINTAINER Muhammad Edwin < edwin at redhat dot com >

LABEL BASE_IMAGE="registry.access.redhat.com/ubi8/ubi-minimal:8.0"
LABEL JAVA_VERSION="11"

RUN microdnf install --nodocs java-11-openjdk-headless && microdnf clean all

WORKDIR /work/
COPY target/*.jar /work/application.jar

EXPOSE 8080
CMD ["java", "-jar", "application.jar"]

And build it in a Jenkins pipeline, on this example im deploying to Quay

node('maven') {
    stage ('pull code') {
        sh "git clone https://github.com/edwin/hello-world-java-docker.git source"
    }
    stage ('mvn build') {
        dir("source") {
            sh "mvn clean package"
        }
    }
    stage ('build and push') {
        dir("source") {
            sh "oc new-build --strategy docker --name=hello-world-java-docker \
                        --binary --to-docker \
                        --to=quay.io/edwinkun/hello-world-java-docker || true"
            sh "oc start-build hello-world-java-docker --from-dir=. --follow --wait "
        }
    }
}

One thing you need to remember is that we need to register our Quay credentials in order to be able to push there. And we can achieve it by using this command,

oc create secret docker-registry --docker-server=quay.io \
	--docker-username=edwinkun --docker-password=******* \
	--docker-email=unused \
	quay-login

oc secrets link default quay-login

Run our Jenkins pipeline and we can see the result on Jenkins dashboard,

When successfully deployed, we can see the pipeline log result will be like this,

And lastly we can see that the containerized image is successfully deployed to Quay

Code for above example can be found on this Github link,

https://github.com/edwin/hello-world-java-docker
Google+

Migrating Image Stream from One Openshift Image Registry to Another Image Registry with Skopeo

I have a requirement where i need to move all images from Image Registry on Openshift 3, to Image Registry on Openshift 4. There are a lot of ways to do it, such as mounting the same disk to multiple Openshift instance or move in manually using docker pull, tag and then push.

After brainstorming for quite some time, i come up with a solution of using Skopeo as a tools to do image migration. It’s a very convenient tool for handling image copying from one image registry to another.

It is actually a very simple script, first we need to capture all images within every OCP3 project,

	
oc get project -o template --template='{{range.items}}{{.metadata.name}}{{"\n"}}{{end}}' | while read line
do
	oc get imagestreamtag -n $line -o template \ 
		--template='{{range.items}}{{.metadata.namespace}}{{"/"}}{{.metadata.name}}{{"\n"}}{{end}}' > images.txt
done

Use this command to capture your OCP username and token,

# capturing your username
oc whoami 

#capturing your token
oc whoami -t

And then we need to iterate the content of generated file with the username and token you get from previous command.

cat images.txt | while read line
do
	skopeo copy  --src-creds ocp3username:ocp3token --src-tls-verify=false \
		--dest-creds ocp4username:ocp4token  --dest-tls-verify=false \
		docker://docker-registry-from.ocp3/$line \
		docker://image-registry-target.apps.ocp4/$line
done

After all is done, what is left is do a simple validation to count how many images has been migrated.

 oc get imagestreamtag --no-headers | wc -l
Google+

How to Display How Many Images are Available on Our Openshift Image Registry

Openshift is a very convenient platform, not only it provides an enterprise kubernetes cluster, but also provide its own image registry bundled within it. So we can push images and deploy it to our namescpace within our cluster in a timely manner. But there are times when i need to count how many images are resides in my existing Openshift cluster. After googling quite some time, i found the solution and write it here.

First we need to check where is our Openshift image registry url,

C:\>oc project default
Already on project "default" on server "https://console.example.com:8443".

C:\>oc get route
NAME               HOST/PORT                                                PATH      SERVICES           PORT       TERMINATION   WILDCARD
docker-registry    docker-registry-default.apps.example.com              docker-registry    5000-tcp   reencrypt     None
registry-console   registry-console-default.apps.example.com             registry-console   <all>      passthrough   None

Next step is login to our oc cluster by using this command, and insert the right username and password.

oc login https://console.example.com:8443 

And see the oc login token

oc whoami -t

Use both username and token to do a simple curl to your docker registry url,

C:\>curl -X GET https://docker-registry-default.apps.example.com/v2/_catalog -k -u <my-username>:<my-token>

The result of that api contains list of images available on your Openshift’s Image Registry.

Google+

Creating a Jenkins Slave Image with Maven 3.6, Java 11 and Skopeo

Openshift have a default maven Jenkins slave image, but too bad it is build on top of Java 8. And on this project which im currently working on, i need a custom Jenkins slave but with Java 11 and the ability to move images between Image Registry. Therefore i create a custom Dockerfile which contains Skopeo, Maven 3.6.3 and Java 11. Below is the detail Dockerfile which i created,

FROM openshift/jenkins-slave-base-centos7:v3.11

MAINTAINER Muhammad Edwin < edwin at redhat dot com >


ENV MAVEN_VERSION=3.6.3 \
    PATH=$PATH:/opt/maven/bin

# install skopeo
RUN yum install skopeo -y && yum clean all

# install java
RUN curl -L --output /tmp/jdk.tar.gz https://download.java.net/java/GA/jdk11/9/GPL/openjdk-11.0.2_linux-x64_bin.tar.gz && \
	tar zxf /tmp/jdk.tar.gz -C /usr/lib/jvm && \
	rm /tmp/jdk.tar.gz && \
	update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk-11.0.2/bin/java 20000 --family java-1.11-openjdk.x86_64 && \
	update-alternatives --set java /usr/lib/jvm/jdk-11.0.2/bin/java
	
# Install Maven
RUN curl -L --output /tmp/apache-maven-bin.zip  https://www-eu.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.zip && \
    unzip -q /tmp/apache-maven-bin.zip -d /opt && \
    ln -s /opt/apache-maven-${MAVEN_VERSION} /opt/maven && \
    rm /tmp/apache-maven-bin.zip && \
    mkdir -p $HOME/.m2

RUN chown -R 1001:0 $HOME && chmod -R g+rw $HOME

COPY run-jnlp-client /usr/local/bin/

USER 1001

Build by using this command,

docker build -t jenkins-slave-skopeo-jdk11-new -f skopeo-jdk11.dockerfile .

Pull the image to Openshift,

oc import-image docker.io/edwinkun/jenkins-slave-skopeo-jdk11-new --confirm

Register on Jenkins as a

And try on

node('maven') {
	stage('Clone') {
		sh "git config --global http.sslVerify false"
		sh "git clone https://github.com/edwin/hello-world.git"
	}
	stage('Build') {
		sh "mvn -v"
		sh "mvn clean package -f hello-world/pom.xml"
	}
}

This is the result,

Detail code can be seen on my github page, https://github.com/edwin/jenkins-slave-maven-jdk11-skopeo

Google+

Monitoring Kafka Topics with Dockerized Kafka Manager

Yesterday, Dimas (one of my colleague), are asking me on how to monitor Kafka which are running on top of Openshift using a tools which are accessible thru browser.

One of the tools im recommending is Kafka Manager, which we can download from below url,

https://github.com/yahoo/kafka-manager

Lets start from the beginning, from how to start Zookeeper, Kafka Server, until simulate a simple produce and consume and monitoring it using Kafka Manager.

First, download Kafka from Apache site, extract it, and open bin folder. We need Zookeeper to start before we start anything else. Fyi for this example im using Win10 as my primary Operating System, so all my command below can be different depends on what Operating System you are using.

cd D:\software\kafka_2.13-2.4.0\bin\windows
zookeeper-server-start.bat ..\..\config\zookeeper.properties

And run Kafka Server afterwards,

kafka-server-start.bat ..\..\config\server.properties

Create a topic,

kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic my-testing-topic

Try produce a simple echo message using Kafka Producer,

kafka-console-producer.bat --broker-list localhost:9092 --topic my-testing-topic

And listen to the sent message using Kafka Consumer,

kafka-console-consumer.bat --bootstrap-server localhost:9092 -topic  my-testing-topic --from-beginning

If you only want to get all the new message, ignoring the old one, just remove “–from-beginning” parameter. And use “–offset” parameter to get a specific offset.

Next is running my Kafka Manager using Docker command. Fyi, 192.168.1.20 is my laptop ip.

docker run --network host --add-host=moby:192.168.1.20 --add-host DESKTOP:192.168.1.20 -p 9000:9000 -e ZK_HOSTS="192.168.1.20:2181"  kafkamanager/kafka-manager

After Kafka-Manager is successfully started, we can browse our Kafka by opening thru browser,

Google+