ocp Posts

How to Solve Openshift “Failed to pull image, unauthorized: authentication required”

Just recently got an unique error, this happens when my application is pulling an image within a different Openshift namespace. In this example, im creating my application in “xyz-project” and try to pull image from “abc-project”. Here’s the complete error detail,

Failed to pull image "image-registry.openshift-image-registry.svc:5000/abc-project/image01@sha256:xxxxxxxxxxxx": 
rpc error: code = Unknown desc = Error reading manifest sha256:xxxxxxxxxxxx in 
image-registry.openshift-image-registry.svc:5000/abc-project/image01: unauthorized: authentication required

Solution for this is quite easy, actually we need to give a specific access right in order for “xyz-project” to be able to pull image from “abc-project”.

oc policy add-role-to-user system:image-puller system:serviceaccount:xyz-project:default -n abc-project

Hope it helps.

Get ImageStream Name and SHA from All DeploymentConfig within a Namespace on Openshift 4

There are times where we want to display list of DC within one Namespace, and want to see what are the images involved within it. We can do that easily by using a simple OC command like below,

oc get dc -n  <namespace> --no-headers -o template \
     --template='{{range.items}}{{.metadata.namespace}}{{"/"}}{{.metadata.name}}{{" - "}}
     {{(index .spec.template.spec.containers 0).image}}{{"\n"}}{{end}}'

Delete All Pods Within a Specific Project or Namespace in Openshift 4

Sometimes we got some pods stuck in our namespace and unable to be deleted within a specific timeframe so we need to delete them forcefully, but sometimes it becomes problematic when we have like hundreds of them. Well deleting all of them is actually quite easy, all i have to do is run below command.

oc delete pod  -n youropenshiftprojectname --grace-period=0 --force --all

How to Handle CORS in Red Hat Process Automation Manager

One feature on Red Hat Process Automation Manager (RHPAM) is the ability to provide an API endpoint which can be accessed from multiple applications. But when we call them directly from javascript within a browser, sometimes it would shows a CORS error.

Workaround is quite easy, either we add a CORS header on RHPAM API response or change call method from a browser call into server-to-server call. For this article i would go with the first approach, and that is adding a CORS header on API that RHPAM provides.

Luckily our RHPAM deployment is on top of Openshift, and by adding below key-value parameters to RHPAM kie-server’s Deployment Config is good enough to solve CORS issue.

         - name: FILTERS
           value: "AC_ALLOW_ORIGIN,AC_ALLOW_METHODS,AC_ALLOW_HEADERS,AC_ALLOW_CREDENTIALS,AC_MAX_AGE"
         - name: AC_ALLOW_ORIGIN_FILTER_RESPONSE_HEADER_NAME
           value: "Access-Control-Allow-Origin"
         - name: AC_ALLOW_ORIGIN_FILTER_RESPONSE_HEADER_VALUE
           value: "*"
         - name: AC_ALLOW_METHODS_FILTER_RESPONSE_HEADER_NAME
           value: "Access-Control-Allow-Methods"
         - name: AC_ALLOW_METHODS_FILTER_RESPONSE_HEADER_VALUE
           value: "POST,GET,OPTIONS,PUT"
         - name: AC_ALLOW_HEADERS_FILTER_RESPONSE_HEADER_NAME
           value: "Access-Control-Allow-Headers"
         - name: AC_ALLOW_HEADERS_FILTER_RESPONSE_HEADER_VALUE
           value: "*"
         - name: AC_ALLOW_CREDENTIALS_FILTER_RESPONSE_HEADER_NAME
           value: "Access-Control-Allow-Credentials"
         - name: AC_ALLOW_CREDENTIALS_FILTER_RESPONSE_HEADER_VALUE
           value: "true"
         - name: AC_MAX_AGE_FILTER_RESPONSE_HEADER_NAME
           value: "Access-Control-Max-Age"
         - name: AC_MAX_AGE_FILTER_RESPONSE_HEADER_VALUE
           value: "86400"

A Simple Load Testing Pipeline on Openshift 4 and Jenkins

Theres one thing needed to be done before deploying your app to production environment, and that is ensuring that your app able to perform well under a high load of transaction. One way to achieve that is by doing a load testing and stress testing internally before deploying to production, but there are times when load testing are being done at the end of development phase with not many time left for developer to do performance tuning. Therefore the better approach is by “shifting left” both load and stress testing phase to an earlier phase, and that is since development phase.

The concept on this blog is doing a load testing on a temporary deployed application, with a maximum one percent acceptable fail. Why i need to deploy the application first before doing a load testing? Because im trying to simulate the exact same condition with production, where each application is a standalone pod, with a specific memory and cpu allocation.

Everything is automated, monitored and managed thru jenkins pipeline with a specific load testing scenario created separatedly in a regular JMeter desktop ui, saved and mapped to a specific application. The detail can be see on on below image where scenario 1 is a scenario created for application 1.

The hard part is creating a JMeter application that is able to consume different scenario, with a parameterized thread and testing endpoint. Thats why im leveraging jmeter-maven-plugin for this, because it’s so lightweight and have a dynamic configuration.

It consist only a one pom file with multiple parameterized fields,

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.edw</groupId>
    <artifactId>JMeterLoadTesting</artifactId>
    <version>1.0-SNAPSHOT</version>
    <name>JMeterLoadTesting</name>
    <description>A Load Testing tool</description>

    <properties>
        <java.version>11</java.version>
    </properties>

    <dependencies>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>com.lazerycode.jmeter</groupId>
                <artifactId>jmeter-maven-plugin</artifactId>
                <version>3.1.0</version>
                <executions>
                    <execution>
                        <id>configuration</id>
                        <goals>
                            <goal>configure</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>jmeter-tests</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>jmeter</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>jmeter-check-results</id>
                        <goals>
                            <goal>results</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <testFilesIncluded>
                        <jMeterTestFile>${testfile}</jMeterTestFile>
                    </testFilesIncluded>
                    <propertiesJMeter>
                        <threads>${threads}</threads>
                        <rampup>${rampup}</rampup>
                        <loops>${loops}</loops>
                        <url>${url}</url>
                        <port>${port}</port>
                    </propertiesJMeter>
                    <errorRateThresholdInPercent>1</errorRateThresholdInPercent>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

Next is we need to create a JMeter test scenario, a simple http GET to root url. And save it to test01.jmx, and put it on /test/jmeter folder so that jmeter-maven-plugin can detect this scenario.

We can test our JMeter script with below command, below example we are running test01.jmx which is doing a 25 hit testing within a 5 seconds timeframe.

mvn clean verify -Dthreads=5 -Dloops=5 -Drampup=5 \
 -Durl=localhost -Dport=8080 -Dtestfile=test01.jmx

The next complicated task is to create a simple Jenkins pipeline script to run this. It needs to have the ability to build and deploy an apps on a temporary pod, do load testing, and clean all resources once load testing is done.

node('maven2') {
    def appname = "app-loadtest-${env.BUILD_NUMBER}"
    try {
        stage ('pull code') {
            sh "git clone https://github.com/edwin/app-loadtest.git source"
        }
        stage ('deploy to ocp') {
            dir("source") {
                sh "oc new-build jenkins2/openjdk-11-rhel7 --name=${appname} --binary "
                sh "oc start-build ${appname} --from-dir=. --follow --wait"
                sh "oc new-app --docker-image=image-registry.openshift-image-registry.svc:5000/jenkins2/${appname}:latest --name=${appname} || true"
                sh "oc set resources dc ${appname} --limits=cpu=500m,memory=1024Mi --requests=cpu=200m,memory=256Mi"
            }
        }
        stage ('do load test') {
            sh "git clone https://github.com/edwin/jmeter-loadtesting.git load"
            dir("load") {
                // 5 threads x 5 loops in 5 seconds
                sh "mvn clean verify -Dthreads=5 -Dloops=5 -Drampup=5 -Durl=${appname} -Dport=8080 -Dtestfile=test01.jmx"
            }
        }
    } catch (error) {
       throw error
    } finally {
        stage('housekeeping') {
            sh "oc delete svc ${appname}"
            sh "oc delete bc ${appname}"
            sh "oc delete is ${appname}"
            sh "oc delete dc ${appname}"
        }
    }
}

If we run the pipeline, we can see that it will spawn an appication pod. We can check whether application runs perfectly or not, by running terminal directly inside it.

The result on Jenkins Dashboard will be like this,

As for the loadtest result, we can see those on our Jenkins logs

All codes are available on github,

https://github.com/edwin/app-loadtest

https://github.com/edwin/jmeter-loadtesting

So, have fun with Jenkins and JMeter 🙂