edwin Posts

Migrating Image Stream from One Openshift Image Registry to Another Image Registry with Skopeo

I have a requirement where i need to move all images from Image Registry on Openshift 3, to Image Registry on Openshift 4. There are a lot of ways to do it, such as mounting the same disk to multiple Openshift instance or move in manually using docker pull, tag and then push.

After brainstorming for quite some time, i come up with a solution of using Skopeo as a tools to do image migration. It’s a very convenient tool for handling image copying from one image registry to another.

It is actually a very simple script, first we need to capture all images within every OCP3 project,

	
oc get project -o template --template='{{range.items}}{{.metadata.name}}{{"\n"}}{{end}}' | while read line
do
	oc get imagestreamtag -n $line -o template \ 
		--template='{{range.items}}{{.metadata.namespace}}{{"/"}}{{.metadata.name}}{{"\n"}}{{end}}' > images.txt
done

Use this command to capture your OCP username and token,

# capturing your username
oc whoami 

#capturing your token
oc whoami -t

And then we need to iterate the content of generated file with the username and token you get from previous command.

cat images.txt | while read line
do
	skopeo copy  --src-creds ocp3username:ocp3token --src-tls-verify=false \
		--dest-creds ocp4username:ocp4token  --dest-tls-verify=false \
		docker://docker-registry-from.ocp3/$line \
		docker://image-registry-target.apps.ocp4/$line
done

After all is done, what is left is do a simple validation to count how many images has been migrated.

 oc get imagestreamtag --no-headers | wc -l
Google+

Securing Connection Between Pods in Openshift with SSL

On this post, im trying to create a simple microservices application on top of Openshift 3.11 and each services will do a simple secure connection between it by using a self-sign SSL which are managed by Openshift.

The goal of why Openshift are managing SSL certificate thru Openshift Secret is to have a rolling or rotating certificate feature on each services but can be triggered by Openshift without have to replace SSL on each services manually.

First is generate a p12 certificate by using keytool

cert>keytool -genkey -alias edw 
	-keystore edw.p12 -storetype PKCS12 
	-keyalg RSA -storepass password 
	-validity 730 -keysize 4096
What is your first and last name?
  [Unknown]:  Edwin
What is the name of your organizational unit?
  [Unknown]:  Company 01
What is the name of your organization?
  [Unknown]:  IT
What is the name of your City or Locality?
  [Unknown]:  Jakarta
What is the name of your State or Province?
  [Unknown]:  Jakarta
What is the two-letter country code for this unit?
  [Unknown]:  ID
Is CN=Edwin, OU=Company 01, O=IT, L=Jakarta, ST=Jakarta, C=ID correct?
  [no]:  yes

Next is creating two java projects which are connected one and another,

https://github.com/edwin/ssl-pods-example
https://github.com/edwin/ssl-pods-example-2

There are several part of the code that need mentioning,

First is making sure https option is active on application.properties, include our p12 certificate and make certificate password as parameterized. This parameter later on will be injected as environment variables on Openshift.

server.ssl.key-store-type=PKCS12
server.ssl.key-store=cert/edw.p12
server.ssl.key-store-password=${SSLPASSWORD}
server.ssl.key-alias=edw

server.port=8443
server.ssl.enabled=true

And the next is because we are using a custom certificate, dont forget to include it on RestTemplate.

@Configuration
public class MyRestTemplate {

    @Value("${server.ssl.key-store}")
    private String sslKeyStore;

    @Value("${server.ssl.key-store-password}")
    private String sslPassword;

    @Bean
    public RestTemplate restTemplate() throws Exception {
        KeyStore clientStore = KeyStore.getInstance("PKCS12");
        clientStore.load(new FileInputStream(sslKeyStore), sslPassword.toCharArray());

        SSLContext sslContext = SSLContextBuilder
                .create()
                .loadTrustMaterial(clientStore, new TrustSelfSignedStrategy())
                .build();
        SSLConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE);
        HttpClient httpClient = HttpClients.custom()
                .setSSLSocketFactory(socketFactory)
                .build();
        HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory(httpClient);

        return new RestTemplate(factory);
    }
}

Deploy those two application to Openshift,

oc new-app registry.access.redhat.com/openjdk/openjdk-11-rhel7~https://github.com/edwin/ssl-pods-example

oc new-app registry.access.redhat.com/openjdk/openjdk-11-rhel7~https://github.com/edwin/ssl-pods-example-2

Deploy certificate as OCP Secret and mount it as a volume on our application,

oc create secret generic cert --from-file=cert\edw.p12

oc set volume dc ssl-pods-example --add -t secret -m /deployments/cert --name cert --secret-name cert
oc set volume dc ssl-pods-example-2 --add -t secret -m /deployments/cert --name cert --secret-name cert

And our certificate password as OCP Secret and inject it as environment variable to our application

oc create secret generic sslpassword --from-literal=SSLPASSWORD=password

oc set env dc ssl-pods-example --from=secret/sslpassword 
oc set env dc ssl-pods-example-2 --from=secret/sslpassword 

After all deployed on OCP, next is give a route for our application. Im using re-encrypt method for ensuring an end to end encryption within the app. In order to do so, we need to include our application CA certificate as our route’s destination certificate. We can do so by exporting our certificate from p12 file using this command,

keytool -exportcert -keystore edw.p12 -storetype PKCS12 -storepass password -alias edw -file edw.crt -rfc

And paste the certificate on our route,

The end result would be like below image,

And as you can see, we are using certificate from end to end for securing our connection.

Google+

Deploy a New Application and Building It Using Openshift S2I Feature and a Custom Base Image

Lots of ways to deploy apps to Openshift, one of it is by using oc new-app command. We are trying now to create a new app using corresponding command, but specifying a custom base image for it. For this example, im using a OpenJDK 11 and RHEL 7 base image.

The command is quite easy, run it on your code folder

D:\source> oc new-app registry.access.redhat.com/openjdk/openjdk-11-rhel7~. --name=spring-boot-2

D:\source> oc start-build spring-boot-2 --from-dir=.

It will create a BuildConfig with the name of spring-boot-2,

D:\source> oc get bc spring-boot-2
NAME            TYPE      FROM      LATEST
spring-boot-2   Source    Binary    3

We can see the detail of our BuildConfig by running this command,

D:\source> oc describe bc spring-boot-2

....
Strategy:       Source
From Image:     ImageStreamTag openjdk-11-rhel7:latest
Output to:      ImageStreamTag spring-boot-2:latest
Binary:         provided on build
....

And if we have some code change and want to redeploy, we can run this command

D:\source> oc start-build spring-boot-2 --from-dir=.

It will rebuild the whole image, and using new code which are uploaded from existing source directory.

Google+

Encrypt Values on Database using Spring Boot, JPA and Jasypt

There are times when you want to encrypt a specific sensitive data on database, like field Salary or Account Number, so that nobody can see the value directly. And there are many approach available to achieve this, and one of it is by using encryption from application side.

For encryption on application level we can use manual encryption or can use Jasypt, which is a very convenient library for handling encryption on database level. On this example, we are using Jasypt.

First as always, a simple pom.xml file,

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.redhat.edw</groupId>
    <artifactId>TestFieldEncryption2</artifactId>
    <version>1.0-SNAPSHOT</version>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.1.RELEASE</version>
        <relativePath/>
    </parent>

    <properties>
        <java.version>11</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
        </dependency>

        <dependency>
            <groupId>org.jasypt</groupId>
            <artifactId>jasypt</artifactId>
            <version>1.9.3</version>
        </dependency>
        <dependency>
            <groupId>org.jasypt</groupId>
            <artifactId>jasypt-hibernate5</artifactId>
            <version>1.9.3</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>

            <plugin>
                <artifactId>maven-deploy-plugin</artifactId>
                <version>2.8.1</version>
                <executions>
                    <execution>
                        <id>default-deploy</id>
                        <phase>deploy</phase>
                        <goals>
                            <goal>deploy</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

        </plugins>
    </build>
</project>

A main java class,

package com.redhat.edw;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

Several java class for creating api and database query,

package com.redhat.edw.controller;

import com.redhat.edw.model.UserAccount;
import com.redhat.edw.repository.UserAccountRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

import java.util.List;

@RestController
public class IndexController {
    @Autowired
    private UserAccountRepository userAccountRepository;

    @GetMapping("/")
    public List<UserAccount> showAll() {
        return userAccountRepository.findAll();
    }

    @PostMapping("/")
    public UserAccount save(@RequestBody UserAccount userAccount) {
        return userAccountRepository.save(userAccount);
    }
}
package com.redhat.edw.model;

import org.hibernate.annotations.Type;

import javax.persistence.*;
import java.io.Serializable;

@Entity
@Table(name = "t_user_account")
public class UserAccount implements Serializable {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Integer id;

    @Column(name = "user_id")
    private Integer userId;

    @Type(type="encryptedString")
    @Column(name = "account_no", length = 300)
    private String accountNo;

    public Integer getId() {
        return id;
    }

    public void setId(Integer id) {
        this.id = id;
    }

    public Integer getUserId() {
        return userId;
    }

    public void setUserId(Integer userId) {
        this.userId = userId;
    }

    public String getAccountNo() {
        return accountNo;
    }

    public void setAccountNo(String accountNo) {
        this.accountNo = accountNo;
    }
}
package com.redhat.edw.repository;

import com.redhat.edw.model.UserAccount;
import org.springframework.data.jpa.repository.JpaRepository;

public interface UserAccountRepository extends JpaRepository<UserAccount, Integer> {
}

Now start with the fun part, where we do all the encryption. First creating a Configuration class for handling encryption configuration

package com.redhat.edw.service;

import org.jasypt.encryption.pbe.PooledPBEStringEncryptor;
import org.jasypt.hibernate5.encryptor.HibernatePBEEncryptorRegistry;
import org.jasypt.iv.RandomIvGenerator;
import org.springframework.context.annotation.Configuration;

@Configuration
public class EncryptionConfig {

    public EncryptionConfig() {
        PooledPBEStringEncryptor encryptor = new PooledPBEStringEncryptor();
        encryptor.setPoolSize(4);
        encryptor.setPassword("MY-PASSPASSPASS123");
        encryptor.setAlgorithm("PBEWithMD5AndTripleDES");
        encryptor.setIvGenerator(new RandomIvGenerator());
        encryptor.setKeyObtentionIterations(1500);

        HibernatePBEEncryptorRegistry registry = HibernatePBEEncryptorRegistry.getInstance();
        registry.registerPBEStringEncryptor("myStringEncryptor", encryptor);
    }
}

And then a standalone typedef,

@TypeDefs({
        @TypeDef(
                name="encryptedString",
                typeClass= EncryptedStringType.class,
                parameters= {
                        @Parameter(name="encryptorRegisteredName", value="myStringEncryptor")
                }
        )
})
package com.redhat.edw.model;

import org.hibernate.annotations.Parameter;
import org.hibernate.annotations.TypeDef;
import org.hibernate.annotations.TypeDefs;
import org.jasypt.hibernate5.type.EncryptedStringType;

We can add new data by using a simple curl,

curl -L -X POST 'http://localhost:8080/' \
-H 'Content-Type: application/json' \
--data-raw '{
	"accountNo":"1234567890",
	"userId":"5"
}'

And querying using this curl command,

curl -L -X GET 'http://localhost:8080/'

A successful json response will looks like this,

[
    {
        "id": 4,
        "userId": 4,
        "accountNo": "1234567890"
    },
    {
        "id": 3,
        "userId": 1,
        "accountNo": "1234567890"
    },
    {
        "id": 5,
        "userId": 3,
        "accountNo": "1234567890"
    },
    {
        "id": 6,
        "userId": 5,
        "accountNo": "1234567890"
    }
]

Despite on database looks like this,

For full code, can access my Github page,

https://github.com/edwin/spring-boot-jpa-jasypt
Google+

How to Connect to Red Hat AMQ using Artemis and SSL

We can protect our AMQ end point using a specific SSL, preventing those who doesnt have the exact certificate to connecting to my AMQ server’s endpoint.

In order to do so, first we need to create a very simple certificate to be use by our AMQ server with a simple keytool command

keytool -genkey -alias artemists -keyalg RSA -sigalg SHA1withRSA -keystore artemis.ts -keysize 2048

It will generate a new file, artemis.ts

And generate a new keystore,

keytool -genkey -v -keystore broker02.ks -alias broker02 -sigalg SHA1withRSA  -keyalg RSA -keysize 2048 -validity 10000

Put string “password” if you need to input a password while generating those two items.

And reference those files on our AMQ broker.xml configuration,

<acceptor name="core">
tcp://0.0.0.0:61617?protocols=CORE;sslEnabled=true;
	keyStorePath=D:/tmp/broker02.ks;keyStorePassword=password;
	trustStorePath=D:/tmp/artemis.ts;trustStorePassword=password;
	enabledCipherSuites=SSL_RSA_WITH_RC4_128_SHA,SSL_DH_anon_WITH_3DES_EDE_CBC_SHA,RSA;enabledProtocols=TLSv1,TLSv1.1,TLSv1.2;
	sslProvider=JDK;sniHost=localhost;anycastPrefix=jms.queue.;multicastPrefix=jms.topic;tcpSendBufferSize=1048576;
	tcpReceiveBufferSize=1048576;useEpoll=true;
	amqpCredits=1000;amqpMinCredits=300
</acceptor>

Start our server,

artemis run

We can connect using our artemis client with below command,

artemis producer --url tcp://127.0.0.1:61617?sslEnabled=true --message-count 1

The first time connected, it will shows error on your client’s side,

Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
        at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
        at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
        at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)

And on your server’s side,

Caused by: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
        at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) [java.base:]
        at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) [java.base:]
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308) [java.base:]
        at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:285) [java.base:]
        at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:181) [java.base:]
        at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) [java.base:]
        at java.base/sun.security.ssl.SSLEngineImpl.decode(SSLEngineImpl.java:685) [java.base:]

Means you need to use your server’s cert on your client while making request, there are multiple ways of doing that. The simplest is by putting your server’s certificate on your local client’s trust store.

keytool -exportcert -alias artemists -keystore artemis.ts -file artemis.cer
keytool -exportcert -alias broker-server -keystore broker02.ks -file broker02.cer

keytool -import -alias artemists -file artemis.cer -cacerts
keytool -import -alias broker-server -file broker02.cer -cacerts

Use password “changeit” while importing your certificate to client’s cacerts.

When re-run artemis producer again, it should gives a successful message like this

\bin>artemis producer --url tcp://127.0.0.1:61617?sslEnabled=true --message-count 1
Producer ActiveMQQueue[TEST], thread=0 Started to calculate elapsed time ...

Producer ActiveMQQueue[TEST], thread=0 Produced: 1 messages
Producer ActiveMQQueue[TEST], thread=0 Elapsed time in second : 0 s
Producer ActiveMQQueue[TEST], thread=0 Elapsed time in milli second : 27 milli seconds
Google+