Deploying Python App using S2I to Openshift Container Platform

For this sample, we are trying to deploy a Flask application to Openshift without any CI/CD tools involved. Means we are deploying directly by using S2I functionality thru OCP Dashboard.

Lets start with a simple hello world Python application, it is basically one simple py file

from flask import Flask

app = Flask(__name__)

@app.route("/")
def index():
    return "<h1>Hello, World!</h1>"

app.run(host="0.0.0.0", port=8080)

and one requirements.txt file,

flask

And after that, we can push our complete Python code to a Git repository. For example, im using below Git repo,

https://github.com/edwin/hello-world-flask

We can deploy to Openshift from the Developer dashboard,

Put the Git repo URL there

It will trigger a build process

Would took a while

But it will deployed successfully after a while

We can access the deployed apps directly after we expose a Route to our app.

Build a Native Quarkus and Camel Application using Mandrel and Docker

Apache Camel is a Java framework for routing and integration, and when we talk about integration means we are talking about lightweight and fast response time. And this is where Apache Camel and Quarkus comes into the picture.

Utilizing Quarkus capability of native compilation, we can compile our Camel Framework application into a native application without the necessity of using JVM. Therefore making a lightweight Apache Camel into more lighweight and faster.

For this project, we will start with a simple Maven file

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.edw</groupId>
    <artifactId>quarkus-camel-native</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <compiler-plugin.version>3.13.0</compiler-plugin.version>
        <maven.compiler.release>21</maven.compiler.release>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
        <quarkus.platform.group-id>io.quarkus.platform</quarkus.platform.group-id>
        <quarkus.platform.version>3.16.3</quarkus.platform.version>
        <skipITs>true</skipITs>
        <surefire-plugin.version>3.5.0</surefire-plugin.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>${quarkus.platform.artifact-id}</artifactId>
                <version>${quarkus.platform.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>quarkus-camel-bom</artifactId>
                <version>${quarkus.platform.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-direct</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-jackson</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-rest</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-arc</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-junit5</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>${quarkus.platform.group-id}</groupId>
                <artifactId>quarkus-maven-plugin</artifactId>
                <version>${quarkus.platform.version}</version>
                <extensions>true</extensions>
                <executions>
                    <execution>
                        <goals>
                            <goal>build</goal>
                            <goal>generate-code</goal>
                            <goal>generate-code-tests</goal>
                            <goal>native-image-agent</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>${compiler-plugin.version}</version>
                <configuration>
                    <parameters>true</parameters>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${surefire-plugin.version}</version>
                <configuration>
                    <systemPropertyVariables>
                        <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                        <maven.home>${maven.home}</maven.home>
                    </systemPropertyVariables>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>${surefire-plugin.version}</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <systemPropertyVariables>
                        <native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path>
                        <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                        <maven.home>${maven.home}</maven.home>
                    </systemPropertyVariables>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <profiles>
        <profile>
            <id>native</id>
            <activation>
                <property>
                    <name>native</name>
                </property>
            </activation>
            <properties>
                <skipITs>false</skipITs>
                <quarkus.native.enabled>true</quarkus.native.enabled>
            </properties>
        </profile>
    </profiles>

</project>

a properties file,

# default
quarkus.http.port=8080
quarkus.log.level=INFO
quarkus.log.category."com.edw".level=DEBUG

quarkus.log.console.format=%d{yyyy-MM-dd HH:mm:ss,SSS} %h %N[%i] %-5p [%c{3.}] (%t) %s%e%n

# disable sending anonymous statistics
quarkus.analytics.disabled=true

and with a simple Java file,

package com.edw.route;

import jakarta.enterprise.context.ApplicationScoped;
import org.apache.camel.builder.RouteBuilder;

@ApplicationScoped
public class HelloWorldRoute extends RouteBuilder {

    @Override
    public void configure() throws Exception {
        rest("/api")
                .get("/hello-world")
                .produces("application/json")
                .to("direct:hello-world");

        from("direct:hello-world")
                .routeId("hello-world-api")
                .log("calling getHelloWorld")
                .setBody(constant("{\"hello\":\"world\"}"));
    }
}

and finally, a Dockerfile

## Stage 1 : build with maven builder image with native capabilities
FROM quay.io/quarkus/ubi-quarkus-mandrel-builder-image:jdk-21 AS build
COPY --chown=quarkus:quarkus --chmod=0755 mvnw /code/mvnw
COPY --chown=quarkus:quarkus .mvn /code/.mvn
COPY --chown=quarkus:quarkus pom.xml /code/
USER quarkus
WORKDIR /code
RUN ./mvnw -B org.apache.maven.plugins:maven-dependency-plugin:3.1.2:go-offline
COPY src /code/src
RUN ./mvnw package -Dnative

## Stage 2 : create the docker final image
FROM quay.io/quarkus/quarkus-micro-image:2.0
WORKDIR /work/
COPY --from=build /code/target/*-runner /work/application

# set up permissions for user `1001`
RUN chmod 775 /work /work/application \
  && chown -R 1001 /work \
  && chmod -R "g+rwX" /work \
  && chown -R 1001:root /work

EXPOSE 8080
USER 1001

CMD ["./application", "-Dquarkus.http.host=0.0.0.0"]

As we can see, it is a multi-stage docker build and we are using Mandrel to compile Quarkus into a native application. Next is to do a docker build, and see our Quarkus add compiled into native

$ podman build -t quarkus-camel-native -f multistage.dockerfile  .

[1/8] Initializing...                                                                                   (11.5s @ 0.12GB)
 Java version: 21.0.5+11-LTS, vendor version: Mandrel-23.1.5.0-Final
 Graal compiler: optimization level: 2, target machine: x86-64-v3
 C compiler: gcc (redhat, x86_64, 8.5.0)
 Garbage collector: Serial GC (max heap size: 80% of RAM)
 4 user-specific feature(s):
 - com.oracle.svm.thirdparty.gson.GsonFeature
 - io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
 - io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase
 - org.eclipse.angus.activation.nativeimage.AngusActivationFeature
 
 .......
 
Produced artifacts:
 /code/target/quarkus-camel-native-1.0-SNAPSHOT-native-image-source-jar/build-artifacts.json (build_info)
 /code/target/quarkus-camel-native-1.0-SNAPSHOT-native-image-source-jar/quarkus-camel-native-1.0-SNAPSHOT-runner (executable)
 /code/target/quarkus-camel-native-1.0-SNAPSHOT-native-image-source-jar/quarkus-camel-native-1.0-SNAPSHOT-runner-build-output-stats.json (build_info)
========================================================================================================================
Finished generating 'quarkus-camel-native-1.0-SNAPSHOT-runner' in 3m 24s.

We can gain some benefits from native compilation such as a lighter image and less utilization

$ podman stats -a
ID            NAME               CPU %       MEM USAGE / LIMIT  MEM %       NET IO      BLOCK IO      PIDS        CPU TIME    AVG CPU %
f8745a7b7ce2  gracious_goldberg  0.01%       37.86MB / 4.097GB  0.92%       0B / 0B     0B / 12.29kB  12          1.155723s   0.41%

Code for this activity can be found here,

https://github.com/edwin/quarkus-camel-native

How to Fix Jenkins “No such DSL method ‘node’ found among steps”

Had this error today,

Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: 417038a0-74ea-4878-a8b3-6a832a793150
java.lang.NoSuchMethodError: No such DSL method 'node' found among steps [archive, bat, build, catchError, checkout, container,
 containerLog, deleteDir, dir, echo, envVarsForTool, error, fileExists, getContext, git, input, isUnix, library, libraryResource, load,
 mail, milestone, node, parallel, podTemplate, powershell, properties, pwd, pwsh, readFile, readTrusted, resolveScm, retry, script, sh,
 sleep, stage, stash, step, timeout, tool, unarchive, unstable, unstash, validateDeclarativePipeline, waitForBuild, waitUntil, warnError,
 withContext, withCredentials, withEnv, wrap, writeFile, ws] or symbols [GitUsernamePassword, agent, all, allBranchesSame, allOf, always,
 any, anyOf, apiToken, apiTokenProperty, architecture, archiveArtifacts, artifactManager, assembla, authorInChangelog, batchFile,
 bitbucket, bitbucketServer, booleanParam, branch, browser, buildButton, buildDiscarder, buildDiscarders, buildRetention,
 buildSingleRevisionOnly, buildingTag, builtInNode, caseInsensitive, caseSensitive, certificate, cgit, changeRequest, changelog,
 changelogBase, changelogToBranch, changeset, checkoutOption, checkoutToSubdirectory, choice, choiceParam, cleanAfterCheckout,
 cleanBeforeCheckout, clock, cloneOption, command, computerRetentionCheckInterval, configMapVolume, consoleUrlProvider, containerEnvVar,
 containerLivenessProbe, containerTemplate, cps, credentials, cron, crumb, default, defaultDisplayUrlProvider, defaultFolderConfiguration, 
defaultView, demand, disableConcurrentBuilds, disableRestartFromStage, disableResume, discoverOtherRefs, discoverOtherRefsTrait, diskSpace, diskSpaceMonitor, downstream, dumb, durabilityHint, dynamicPVC, emptyDirVolume, emptyDirWorkspaceVolume, envVar, envVars, envVarsFilter, 
environment, equals, experimentalFlags, expression, file, fileParam, filePath, fingerprint, fingerprints, firstBuildChangelog, fisheye, 
frameOptions, freeStyle, freeStyleJob, fromScm, fromSource, genericEphemeralVolume, git, gitBranchDiscovery, gitHooks, gitLab, gitList, 
gitSCM, gitTagDiscovery, gitTool, gitUsernamePassword, gitWeb, gitblit, github, gitiles, gogs, headRegexFilter, headWildcardFilter, 
hostPathVolume, hostPathWorkspaceVolume, hyperlink, hyperlinkToModels, ignoreOnPush, inbound, installSource, isRestartedRun, jdk, jgit,
 jgitapache, jnlp, jobBuildDiscarder, jobName, kiln, kubeconfig, kubernetes, kubernetesAgent, label, lastDuration, lastFailure, 
lastGrantedAuthorities, lastStable, lastSuccess, legacy, legacySCM, lfs, list, local, localBranch, localBranchTrait, location, logRotator,
 loggedInUsersCanDoAnything, mailer, masterBuild, maven, maven3Mojos, mavenErrors, mavenGlobalConfig, mavenMojos, mavenWarnings, merge, 
modernSCM, multiBranchProjectDisplayNaming, multibranch, myView, namedBranchesDifferent, never, nfsVolume, nfsWorkspaceVolume, node, 
nodeProperties, none, nonresumable, not, onFailure, organizationFolder, override, overrideIndexTriggers, paneStatus, 
parallelsAlwaysFailFast, parameters, password, pattern, perBuildTag, permanent, persistentVolumeClaim, 
persistentVolumeClaimWorkspaceVolume, phabricator, pipeline, pipelineTriggers, plainText, plugin, podAnnotation, podEnvVar, podLabel, 
pollSCM, portMapping, preserveStashes, prism, projectNamingStrategy, proxy, pruneStaleBranch, pruneStaleTag, pruneTags, 
queueItemAuthenticator, quietPeriod, rateLimit, rateLimitBuilds, redmine, refSpecs, remoteName, resourceRoot, responseTime, 
retainOnlyVariables, rhodeCode, run, runParam, schedule, scmGit, scmRetryCount, scriptApproval, scriptApprovalLink, search, secretEnvVar, 
secretVolume, security, shell, simpleBuildDiscarder, skipDefaultCheckout, skipStagesAfterUnstable, slave, sourceRegexFilter, 
sourceWildcardFilter, sparseCheckout, sparseCheckoutPaths, sshUserPrivateKey, standard, status, string, stringParam, submodule, 
submoduleOption, suppressAutomaticTriggering, suppressFolderAutomaticTriggering, swapSpace, tag, teamFoundation, text, textParam, timezone, 
tmpSpace, toolLocation, triggeredBy, unsecured, untrusted, upstream, userIdentity, userSeed, usernameColonPassword, usernamePassword, 
viewgit, viewsTabBar, weather, zip] or globals [currentBuild, env, params, pipeline, scm]
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:219)
	at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:124)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:98)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1225)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1034)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:41)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
	at PluginClassLoader for script-security//org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:180)
	at PluginClassLoader for script-security//org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at PluginClassLoader for script-security//org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:163)

when building Jenkins Pipeline using Script from SCM. The weird thing is the same script is working perfectly well when being executed directly from Jenkins, without the SCM / Git part.

The rootcause is actually simple, we are storing the Jenkins pipeline in Git using UTF8 With BOM with a specific Windows encoding. Saving it as a UTF8 directy and push it to Git can solve this problem.

ps.
opening the log in notepad++, after choosing the option to show all character, will give this result.

As we can see from above image, there is a “hidden” character with the name of “ZWNBSP” that is invisible to naked eyes. However it makes Jenkins failed to parse the corresponding pipeline.

How to Solve “error, cannot create resource in API group” when Deploying Application with Jenkins and OpenShift

Had this error when im using Jenkins and integrate it to Openshift Container Platform

--> Creating resources with label build=hello-world-dot-net-core ...
    error: imagestreams.image.openshift.io is forbidden: User "system:serviceaccount:cicd:default" cannot create resource "imagestreams" in API group "image.openshift.io" in the namespace "cicd"
    error: buildconfigs.build.openshift.io is forbidden: User "system:serviceaccount:cicd:default" cannot create resource "buildconfigs" in API group "build.openshift.io" in the namespace "cicd"
--> Failed

How to solve this issue is actually quite simple, running this below command can solve it directly.

$ oc policy add-role-to-user admin system:serviceaccount:cicd:default -n cicd

Error While Running dotnet test

Had this error while running automated test on my dotnet core application thru Bitbucket Pipeline

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: < 1 ms - micro-service-test.dll (net8.0)
Testhost process for source(s) '/opt/atlassian/pipelines/agent/build/micro-service/bin/Debug/net8.0/micro-service.dll' exited with error: Error:
  An assembly specified in the application dependencies manifest (testhost.deps.json) was not found:
    package: 'Microsoft.TestPlatform.CommunicationUtilities', version: '17.11.1-release-24455-02'
    path: 'Microsoft.TestPlatform.CommunicationUtilities.dll'
. Please check the diagnostic logs for more information.

Apperently this happen because of my main project is detected as test folder. We can exclude it by adding this configuration in .csproj file

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <IsTestProject>false</IsTestProject>
  </PropertyGroup>
</Project>