After creating self-signed wildcard certificates for modern browsers a while back, self-signed certificates came back to haunt me once again. This post describes a solution for creating self-signed keystores (pkcs12 or jks) in OpenShift and Kubernetes to use in Spring Boot applications to re-encrypt traffic. Actually I present 2 solutions, but the second one only works in OpenShift. The first one will leverage the Cert-Manager to create a keystore, the second one relies on OpenShifts service-serving certificate. The goal is to have OpenShift re-encrypt traffic with a route to a Spring Boot application. The keystore will be used as an application facing certificate for OpenShift to validate.
1. Using Cert-Manager
This approach requires an already working cert-manager instance in your cluster.
Create a new issuer for self-signed certificates:
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: private-cert-issuer
namespace: somewhere
spec:
selfSigned: {}
Next we need a certificate. It’s important to set isCa: true for OpenShift to accept this certificate as a destination CA. secretName references the secret which will be created by cert-manager containing the keystore. Another secret needs to exist for this to work which contains the password encrypting the keystore. This secret is referenced at the end of the yaml here. Name matches the secret name and key the data entry inside the secret.
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: app-https
namespace: somewhere
spec:
dnsNames:
- "chart-example.local"
commonName: my-selfsigned-ca
isCA: true
issuerRef:
group: cert-manager.io
kind: Issuer
name: private-cert-issuer
secretName: app-https
keystores:
pkcs12: # switch to jks to create a jks keystore
create: true
passwordSecretRef: # Password used to encrypt the keystore
key: password
name: keystore-pw
The secret containing the password may look like this:
apiVersion: v1
kind: Secret
metadata:
name: keystore-pw
type: Opaque
data:
password: somepass
After creating the cert-manager resources a new secret should be created by the cert-manager containing the keystore. This keystore can now be mounted inside the Spring Boot application. The password can be passed as an environment variable referencing the same secret used to create the keystore. In this example the keystore is mounted to the /certs directory.
# deployment.yaml
env:
- name: server_ssl_key_store_password
valueFrom:
secretKeyRef:
name: keystore-pw
key: password
...
volumeMounts:
- name: https-certs-volume
mountPath: /certs
...
volumes:
- name: https-certs-volume
secret:
secretName: app-https
defaultMode: 420
Spring Boot needs to pick up the certificate and use it for SSL. This can be achieved using the spring variables, e.g. contained inside the values.yaml when using Helm:
#values.yml
server:
ssl:
key-store: "/certs/keystore.jks"
key-alias: "certificate"
key-store-type: "PKCS12"
For OpenShift to validate the certificate when re-encrypting traffic, the destinationCACertificate property in the OpenShift route has to be set with the created certificate from the keystore secret. When using OpenShift 4.11 this can be automated by leveraging an annotation as described here. However using OpenShift there is another way to create the necessary certificates.
2. OpenShift Service Signer
This approach only works in OpenShift as it relies on OpenShift features to create the needed secret. A simple annotation can be used to annotate the service resource to instruct OpenShift to create a secret containing self-signed certificates.
#service.yaml
...
metadata:
annotations:
service.beta.openshift.io/serving-cert-secret-name: reencrypt-certs
...
This will create a secret named keystore-reencrypt which contains only a certificate and a key and may look like this:
kind: Secret
apiVersion: v1
metadata:
name: reencrypt-certs
namespace: somewhere
annotations:
service.alpha.openshift.io/expiry: '2024-11-06T12:10:22Z'
service.beta.openshift.io/expiry: '2024-11-06T12:10:22Z'
service.beta.openshift.io/originating-service-name: service-name
service.beta.openshift.io/originating-service-uid: 0f31e820-b49d-4619-8ef5-65f1c85293e3
data:
tls.crt: >-
LS0tLS1CRUdJT[...]
tls.key: >-
LS0tLS1CRUdJT[...]
type: kubernetes.io/tls
To use the self-signed certificate within the secret to create a keystore an init container within the deployment resource can be used. Any image containing openssl should be possible. The init container reads the created certificate and key from the volume mount, creates a pkcs12 keystore and writes it to an emptyDir. In this example a secret containing a password is referenced as an environment variable to be used in the init container.
#deployment.yaml
...
spec:
volumes:
- name: reencrypt-certs
secret:
secretName: reencrypt-certs
defaultMode: 420
- name: reencrypt-keystore
emptyDir: {}
...
initContainers:
- name: gen-reencrypt-keystore
image: image-with-openssl:tag
command:
- sh
args:
- -c
- |
echo "creating reencrypt keystore"
openssl pkcs12 -export -in /reencrypt-certs/tls.crt -inkey /reencrypt-certs/tls.key -name "https" -out /certs/reencrypt.p12 -passout pass:$server_ssl_key_store_password
volumeMounts:
- name: reencrypt-certs
mountPath: /reencrypt-certs
- name: reenrcypt-keystore
mountPath: /certs
env:
- name: server_ssl_key_store_password
valueFrom:
secretKeyRef:
name: reencrypt-keystore-pw
key: password
...
The actual application receives the same volume mount and the environment variable reference to decrypt the keystore so Spring Boot can start up.
#deployment.yaml
...
containers:
volumeMounts:
- name: reenrcypt-keystore
mountPath: /certs
...
env:
- name: server_ssl_key_store_password
valueFrom:
secretKeyRef:
name: reencrypt-keystore-pw
key: password
...
The location of the keystore and the alias can be added as environment variables as well. I would however advise to have these settings within the application config or in the values.yaml when using Helm to achieve a slimmer deployment resource.
#values.yaml
server:
ssl:
key-store: "/certs/reencrypt.p12"
key-alias: "https"
After these settings Spring Boot will locate the keystore /certs/reencrypt.p12 on startup, decrypt it using the password passed within the environment variables and run the application. The real neat part here is that no further settings have to be made to the OpenShift route, as the self-signed certificate created by OpenShift already contains the OpenShift signer CA which is automatically trusted as a destination CA. Even though this property is left empty inside the route resource, OpenShift will trust it’s own certificate.
Although the second option to create a keystore is more straightforward in my eyes, using Cert Manager does have it’s benefits if it’s already running in a cluster. If Helm is used to deploy application be aware that the created certificate by the Cert Manager is not recognized in the lifecycle of the deployment triggering the creation of the certificate in Cert Manager. An uninstall in Helm will not remove the keystore secret. This will become an issue if the password is created in a non deterministic way or is just different each time, as the password secret will be created anew and the certificate will not be overwritten, resulting in a mismatch.
If you happen to know a different approach to create self-signed keystores in OpenShift / Kubernetes to re-encrypt traffic, please let me know in the comments below or contacting me directly.
