How to set up Mutual TLS (mTLS) Authentication for Spinnaker Services

Spinnaker services communicate and exchange sensitive data with each other. When TLS (Transport Level Security) is enabled between the services it ensures that all of this data is encrypted. Communication between services happens only when they have valid certificates.

Mutual authentication or two-way authentication refers to two parties authenticating each other at the same time. Enabling Mutual TLS (mTLS) provides an additional layer of security for the Spinnaker services as only validated clients can interact with the services.

When a client connects to a server:

1. The server responds with its certificate signed by a valid CA (certificate authorities) and the client validates it.

2. The server sends requests for a certificate from the client and validates the same after receiving it.

How to create certificates for mutual tls using cert-manager?

To enable mutual TLS, you need to get a certificate (a type of file) from a Certificate Authority (CA). The cert-manager is a native Kubernetes certificate management controller. It can help with issuing certificates from a variety of CA (certificate authorities) sources, such as Let’s Encrypt, HashiCorp Vault, Venafi, a simple signing key pair, or self-signed. It will ensure certificates are valid and up to date, and attempt to renew certificates at a configured time before expiry. Here’s how you can create certificates using the cert-manager:

Pre-requisites:

Kubernetes, cert-manager

Steps:

1. Create a cluster issuer to issue self-signed certificates using the below YAML code with kubectl create -f as shown below:

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata: name: selfsigned-issuer
spec:selfSigned: {}
kubectl create -f clusterissuer.yml
kubectl get clusterissuer

2. Create a certificate authority (CA) certificate that can use the above self-signed issuer. Change the namespace below to the namespace where spinnaker is installed. Also, include any other Subject Alternate Names in the dnsNames field.

apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
 name: mtlsca
 namespace: spintest
spec:
 secretName: cacert
 isCA: true
 issuerRef:
   name: selfsigned-issuer
   kind: ClusterIssuer
 commonName: mtlsca
 dnsNames:
  – “*.spintest.svc”
  – localhost
kubectl create -f cacert.yml
kubectl -n spintest get certs
kubectl -n spintest get secret

3. Create a certificate authority issuer that can use the above ca certificate. Change the namespace below to the namespace where spinnaker is installed.

apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
 name: caissuer
 namespace: spintest
spec:
 ca:
   secretName: cacert
kubectl -n spintest create -f caissuer.yml
kubectl -n spintest get issuer

4. Create a certificate using the caissuer. Change the namespace below to the namespace where the spinnaker is installed. Also, change the dnsNames. This expects a pkcs12 passphrase in a secret called passphrasesecret.

kubectl -n spintest create secret generic passphrasesecret –from-literal=passphrase=mysecrepassphrase
This secret will be used later in configuring the spinnaker files.
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: mtlscerts-pkcs12
  namespace: spintest
spec:
  secretName: mtlscerts-pkcs12
  duration: 2160h # 90d
  renewBefore: 360h # 15d
  commonName: spintest.svc
  keystores:
   pkcs12:
     create: true
     passwordSecretRef: 
       name: passphrasesecret
       key: passphrase
  dnsNames:
  – “*.spintest.svc”
  – localhost
  usages:
  – digital signature
  – key encipherment
  – server auth
  – client auth
  issuerRef:
    name: caissuer
    kind: Issuer
kubectl create -f mtlscerts.yml
kubectl -n spintest get certs
kubectl  -n spintest get secret mtlscerts-pkcs12 -o yaml should show ca.crt, tls.crt,tls.key and keystore.p12
kubectl -n spintest get secret mtlscerts-pkcs12 -o jsonpath='{.data.ca\.crt}’ | base64 -d >ca.crt

5. From clouddriver pod get the cacerts file:

kubectl -n spintest cp clouddriverpod:/etc/ssl/certs/java/cacerts cacerts
keytool -import -file ca.crt -keystore cacerts   
kubectl -n spintest create secret generic cacerts –from-file=cacerts

6. Make the following changes in spinnaker by exec into halyard pod:

• In /home/spinnaker/.hal/default/service-settings , change svc.yaml ( example echo.yml, clouddriver.yml) to mount secret on to svc and overridebaseurl from http to https:

kubernetes:
  volumes:
  – id: cacerts
    mountPath: /etc/ssl/certs/java
    type: secret
    readOnly: true
  – id: mtlscerts-pkcs12
    mountPath: /pkcs12
    type: secret
    readOnly: true
overrideBaseUrl: 
https://spin-clouddriver.spintest.svc:7002

• Change the service name, namespace and port accordingly. In /home/spinnaker/.hal/default/profiles, change svc-local.yml ( example echo-local.yml, clouddriver-local.yml) to add https to server and okHttpClient:

server:
  port: 7002
  ssl:
    enabled: true
    keyStore: /pkcs12/keystore.p12
    keyStoreType: PKCS12
    keyStorePassword: changeit  # from the passphrase secret
    trustStore: /etc/ssl/certs/java/cacerts
    trustStoreType: JKS
    trustStorePassword: changeit # from the passphrase secret
    clientAuth: need
okHttpClient:
  keyStore: /pkcs12/keystore.p12
  keyStorePassword: changeit # from he passphrase secret
  trustStore: /etc/ssl/certs/java/cacerts
  propagateSpinnakerHeaders: true
  connectTimeoutMs: 60000
  readTimeoutMs: 60000

7. Hal deploy apply after you are done.

Conclusion:

After applying the above configuration changes to your Spinnaker deployment, the Mutual TLS (mTLS) Authentication for Spinnaker Services is enabled thereby making it secure to communicate securely over the network with other services.