docker Posts

Migrating Image Stream from One Openshift Image Registry to Another Image Registry with Skopeo

I have a requirement where i need to move all images from Image Registry on Openshift 3, to Image Registry on Openshift 4. There are a lot of ways to do it, such as mounting the same disk to multiple Openshift instance or move in manually using docker pull, tag and then push.

After brainstorming for quite some time, i come up with a solution of using Skopeo as a tools to do image migration. It’s a very convenient tool for handling image copying from one image registry to another.

It is actually a very simple script, first we need to capture all images within every OCP3 project,

	
oc get project -o template --template='{{range.items}}{{.metadata.name}}{{"\n"}}{{end}}' | while read line
do
	oc get imagestreamtag -n $line -o template \ 
		--template='{{range.items}}{{.metadata.namespace}}{{"/"}}{{.metadata.name}}{{"\n"}}{{end}}' > images.txt
done

Use this command to capture your OCP username and token,

# capturing your username
oc whoami 

#capturing your token
oc whoami -t

And then we need to iterate the content of generated file with the username and token you get from previous command.

cat images.txt | while read line
do
	skopeo copy  --src-creds ocp3username:ocp3token --src-tls-verify=false \
		--dest-creds ocp4username:ocp4token  --dest-tls-verify=false \
		docker://docker-registry-from.ocp3/$line \
		docker://image-registry-target.apps.ocp4/$line
done

After all is done, what is left is do a simple validation to count how many images has been migrated.

 oc get imagestreamtag --no-headers | wc -l
Google+

How to Display How Many Images are Available on Our Openshift Image Registry

Openshift is a very convenient platform, not only it provides an enterprise kubernetes cluster, but also provide its own image registry bundled within it. So we can push images and deploy it to our namescpace within our cluster in a timely manner. But there are times when i need to count how many images are resides in my existing Openshift cluster. After googling quite some time, i found the solution and write it here.

First we need to check where is our Openshift image registry url,

C:\>oc project default
Already on project "default" on server "https://console.example.com:8443".

C:\>oc get route
NAME               HOST/PORT                                                PATH      SERVICES           PORT       TERMINATION   WILDCARD
docker-registry    docker-registry-default.apps.example.com              docker-registry    5000-tcp   reencrypt     None
registry-console   registry-console-default.apps.example.com             registry-console   <all>      passthrough   None

Next step is login to our oc cluster by using this command, and insert the right username and password.

oc login https://console.example.com:8443 

And see the oc login token

oc whoami -t

Use both username and token to do a simple curl to your docker registry url,

C:\>curl -X GET https://docker-registry-default.apps.example.com/v2/_catalog -k -u <my-username>:<my-token>

The result of that api contains list of images available on your Openshift’s Image Registry.

Google+

Creating a Jenkins Slave Image with Maven 3.6, Java 11 and Skopeo

Openshift have a default maven Jenkins slave image, but too bad it is build on top of Java 8. And on this project which im currently working on, i need a custom Jenkins slave but with Java 11 and the ability to move images between Image Registry. Therefore i create a custom Dockerfile which contains Skopeo, Maven 3.6.3 and Java 11. Below is the detail Dockerfile which i created,

FROM openshift/jenkins-slave-base-centos7:v3.11

MAINTAINER Muhammad Edwin < edwin at redhat dot com >


ENV MAVEN_VERSION=3.6.3 \
    PATH=$PATH:/opt/maven/bin

# install skopeo
RUN yum install skopeo -y && yum clean all

# install java
RUN curl -L --output /tmp/jdk.tar.gz https://download.java.net/java/GA/jdk11/9/GPL/openjdk-11.0.2_linux-x64_bin.tar.gz && \
	tar zxf /tmp/jdk.tar.gz -C /usr/lib/jvm && \
	rm /tmp/jdk.tar.gz && \
	update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk-11.0.2/bin/java 20000 --family java-1.11-openjdk.x86_64 && \
	update-alternatives --set java /usr/lib/jvm/jdk-11.0.2/bin/java
	
# Install Maven
RUN curl -L --output /tmp/apache-maven-bin.zip  https://www-eu.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.zip && \
    unzip -q /tmp/apache-maven-bin.zip -d /opt && \
    ln -s /opt/apache-maven-${MAVEN_VERSION} /opt/maven && \
    rm /tmp/apache-maven-bin.zip && \
    mkdir -p $HOME/.m2

RUN chown -R 1001:0 $HOME && chmod -R g+rw $HOME

COPY run-jnlp-client /usr/local/bin/

USER 1001

Build by using this command,

docker build -t jenkins-slave-skopeo-jdk11-new -f skopeo-jdk11.dockerfile .

Pull the image to Openshift,

oc import-image docker.io/edwinkun/jenkins-slave-skopeo-jdk11-new --confirm

Register on Jenkins as a

And try on

node('maven') {
	stage('Clone') {
		sh "git config --global http.sslVerify false"
		sh "git clone https://github.com/edwin/hello-world.git"
	}
	stage('Build') {
		sh "mvn -v"
		sh "mvn clean package -f hello-world/pom.xml"
	}
}

This is the result,

Detail code can be seen on my github page, https://github.com/edwin/jenkins-slave-maven-jdk11-skopeo

Google+

Monitoring Kafka Topics with Dockerized Kafka Manager

Yesterday, Dimas (one of my colleague), are asking me on how to monitor Kafka which are running on top of Openshift using a tools which are accessible thru browser.

One of the tools im recommending is Kafka Manager, which we can download from below url,

https://github.com/yahoo/kafka-manager

Lets start from the beginning, from how to start Zookeeper, Kafka Server, until simulate a simple produce and consume and monitoring it using Kafka Manager.

First, download Kafka from Apache site, extract it, and open bin folder. We need Zookeeper to start before we start anything else. Fyi for this example im using Win10 as my primary Operating System, so all my command below can be different depends on what Operating System you are using.

cd D:\software\kafka_2.13-2.4.0\bin\windows
zookeeper-server-start.bat ..\..\config\zookeeper.properties

And run Kafka Server afterwards,

kafka-server-start.bat ..\..\config\server.properties

Create a topic,

kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic my-testing-topic

Try produce a simple echo message using Kafka Producer,

kafka-console-producer.bat --broker-list localhost:9092 --topic my-testing-topic

And listen to the sent message using Kafka Consumer,

kafka-console-consumer.bat --bootstrap-server localhost:9092 -topic  my-testing-topic --from-beginning

If you only want to get all the new message, ignoring the old one, just remove “–from-beginning” parameter. And use “–offset” parameter to get a specific offset.

Next is running my Kafka Manager using Docker command. Fyi, 192.168.1.20 is my laptop ip.

docker run --network host --add-host=moby:192.168.1.20 --add-host DESKTOP:192.168.1.20 -p 9000:9000 -e ZK_HOSTS="192.168.1.20:2181"  kafkamanager/kafka-manager

After Kafka-Manager is successfully started, we can browse our Kafka by opening thru browser,

Google+

Dockerfile for Creating A UBI-based Docker Image and OpenJDK

I use UBI 8 as base images for basically almost every java deployment. It’s lightweight, freely redistributable, and easy to configure.

This is my simplified Dockerfile script, which consist of UBI8 and OpenJDK installation.

FROM registry.access.redhat.com/ubi8/ubi-minimal

MAINTAINER Muhammad Edwin < emuhamma at redhat dot com >

# Some version information
LABEL io.k8s.description="Platform for running plain Java applications (fat-jar and flat classpath)" \
      io.k8s.display-name="My Simple Java Applications" \
      io.openshift.tags="builder,java" \
      org.jboss.deployments-dir="/deployments" \
      com.redhat.deployments-dir="/deployments" \
      com.redhat.dev-mode="JAVA_DEBUG:false" \
      com.redhat.dev-mode.port="JAVA_DEBUG_PORT:5005"

# Install Java runtime
RUN microdnf install java-11-openjdk-headless --nodocs \
 && microdnf install shadow-utils && microdnf clean all \
 && mkdir /deployments

# Add user & group
RUN groupadd jboss && useradd jboss -g jboss

# Use /dev/urandom to speed up startups & Add jboss user to the root group
RUN echo securerandom.source=file:/dev/urandom >> /usr/lib/jvm/jre/lib/security/java.security \
 && usermod -g root -G jboss jboss

# set working directory at /deployments
WORKDIR /deployments

# copy corresponding jar file
COPY demo.jar demo.jar

# Necessary to permit running with a randomised UID
RUN mkdir -p /deployments/data \
 && chmod -R "g+rwX" /deployments \
 && chown -R jboss:jboss /deployments \
 && chmod 664 /etc/passwd

# gives uid
USER 185

# run it
CMD ["java","-jar","demo.jar"]
Google+