Vault is a tool for securely accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, or certificates. Vault provides a unified interface to any secret, while providing tight access control and recording a detailed audit log.
Vault Installation
Vault Configuration with Kubernetes and Spinnaker
Verification of Vault Integration
The following steps include the deployment of Vault Server on Linux (Ubuntu) in a standalone mode and it is managed by a daemon (called, ‘vault’):
Download the latest version of vault binary zip file from vault release page and unzip it.
sudo wget https://releases.hashicorp.com/vault/1.1.3/vault_1.1.3_linux_amd64.zipsudo unzip vault_1.1.3_linux_amd64.zip -d .
Note: Need to download Available latest packages [(https://releases.hashicorp.com/vault)] here is a direct link for vault.
Copy vault binary into /usr/bin. This will allow us to execute vault binary system wide.
sudo cp vault /usr/local/bin/
Create a vault config directory under /etc, a vault data directory and logs directory.
sudo mkdir /etc/vaultsudo mkdir /vault-datasudo mkdir -p /logs/vault/
Create a config.json file and add the vault configuration.
sudo vi /etc/vault/config.json{"listener": [{"tcp": {"address" : "0.0.0.0:8200","tls_disable" : 1}}],"api_addr": "http://<Vault-Server-IP>:8200","storage": {"file": {"path" : "/vault-data"}},"max_lease_ttl": "100h","default_lease_ttl": "100h","ui":true}
To create a vault service file.
sudo vi /etc/systemd/system/vault.service[Unit]Description=vault serviceRequires=network-online.targetAfter=network-online.targetConditionFileNotEmpty=/etc/vault/config.json[Service]EnvironmentFile=-/etc/sysconfig/vaultEnvironment=GOMAXPROCS=2Restart=on-failureExecStart=/usr/local/bin/vault server -config=/etc/vault/config.jsonStandardOutput=/logs/vault/output.logStandardError=/logs/vault/error.logLimitMEMLOCK=infinityExecReload=/bin/kill -HUP $MAINPIDKillSignal=SIGTERM[Install]WantedBy=multi-user.target
To start the vault service.
sudo systemctl start vaultsudo systemctl status vault
Login as root and Export VAULT_ADDR environment variable, don’t forget to add this to ~/.bashrc file. Change the IP to you vault server public/private IP.
export VAULT_ADDR=http://<Vault-Server-IP>:8200echo "export VAULT_ADDR=http://<Vault-Server-IP>:8200" >> ~/.bashrc
Execute the below command to Iniate Vault Init file
vault operator init > /etc/vault/init.file
Note: This command should be executed as a root user.
To check the vault status execute the below command. Output looks like below
vault statusKey Value--- -----Seal Type shamirInitialized trueSealed trueTotal Shares 5Threshold 3Unseal Progress 0/3Unseal Nonce n/aVersion 1.1.3HA Enabled false
Now vault is initiated but sealed.
To unseal the vault, Check the init.file for the vault token's to unseal.
cat /etc/vault/init.file
Unseal vault using ‘unseal’ command. There are 5 unseal tokens. You need to execute the unseal command with a minimum of three unseal token to unseal vault.
vault operator unseal <Vault Non-Root Token>vault operator unseal <Vault Non-Root Token>vault operator unseal <Vault Non-Root Token>
We follow the ‘Kubernetes auth method’ for authenticating with Kubernetes service accounts and storing secrets. Configuration of Vault for the ‘Kubernetes auth method’ requires configuring both ‘Vault’ and ‘Kubernetes’. Prerequisites: 1. A running Kubernetes cluster 2. A running vault cluster
Create a service account called 'spin-vault-token' in a specific namespace (ex: vaultspinnaker), that Vault will use it login to Kuberenetes:
kubectl create namespace vaultspinnakerkubectl create serviceaccount spin-vault-token -n <NameSpace>
Create a clusterrolebinding for the 'spin-vault-token' serviceaccount access:
vi vault-token-binding.yamlkubectl create -f vault-token-binding.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1kind: ClusterRoleBindingmetadata:name: vault-spin-token-bindingnamespace: vaultspinnakerroleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:auth-delegatorsubjects:- kind: ServiceAccountname: vault-spin-tokennamespace: vaultspinnaker
Then check the secrets by running the below command:
kubectl get secrets
Now, get the JWT, Kubernetes API URL and certificate authority cert from the serviceaccount by running below commands.
VAULT_ACCOUNT_TOKEN is the final JWT token
SECRET_NAME=$(kubectl -n vaultspinnaker get serviceaccount vault-spin-token -o jsonpath='{.secrets[0].name}')VAULT_ACCOUNT_TOKEN=$(kubectl -n vaultspinnaker get secret ${SECRET_NAME} -o jsonpath='{.data.token}' | base64 -d)
kubectl cluster-info
export VAULT_SA_NAME=$(kubectl get sa vault-spin-token -n vaultspinnaker -o jsonpath="{.secrets[*]['name']}")export SA_CA_CRT=$(kubectl get secret -n vaultspinnaker $VAULT_SA_NAME -o jsonpath="{.data['ca\.crt']}" | base64 -d; echo)
Now, validate the exported environment variables with the below values
echo $SECRET_NAMEecho $VAULT_ACCOUNT_TOKENkubectl cluster-infoecho $VAULT_SA_NAMEecho $SA_CA_CRT
Enable the 'Kubernetes auth method' in the vault and authenticate by running below command (So that configured Vault can talk to Kubernetes API)
vault auth enable -path=spin-k8spath kubernetesvault write auth/spin-k8spath/config kubernetes_host="https://Kubernetes-Hostname/IP:6443" kubernetes_ca_cert="$SA_CA_CRT" token_reviewer_jwt=$VAULT_ACCOUNT_TOKEN
Now, configure role and policy. The Kubernetes backend authorizes an entity by granting it a role mapped to a serviceaccount. A role is configured with policies which control the entity’s access to paths and operations in Vault.
vault secrets enable -path=spin-hal-path kv
Create a new policy spin-policy using an example policy file, ‘policy.hcl’
vi policy.hclpath "spin-hal-path/*" {capabilities = ["create", "read", "delete"]}
Above command simply grants the privileges to create/read/delete the secrets in 'spin-hal-path/' path.
vault write sys/policy/spin-policy policy=@policy.hcl
Next, create a role for binding the policy to a service account.
vault write auth/spin-k8spath/role/spin-role bound_service_account_names=vault-spin-token bound_service_account_namespaces=vaultspinnaker policies=spin-policy ttl=48h
Now, authenticate with the role we just created. For that, get the service account token by running the below command
VAULT_ACCOUNT_TOKEN=$(kubectl get secret -n vaultspinnaker VAULT_SA_NAME -o jsonpath="{.data['token']}" | base64 -d; echo)echo $VAULT_ACCOUNT_TOKEN
Now, login to the vault with the above token generated from the below command
vault write auth/spin-k8spath/login role=spin-role jwt=${VAULT_ACCOUNT_TOKEN}
Copy the Token value from the above login and execute the below command to export it as a Environment Variable
export VAULT_TOKEN="LoginTokenValue"
Now, install spinnaker in the same namespaces as vault is created(vaultspinnaker).
The Vault HTTP API gives you full access to Vault via HTTP. Every aspect of Vault can be controlled via this API. The Vault CLI uses the HTTP API to access Vault.
Login to the ‘spin-halyard’ pod and perform the below curl HTTP API commands for verifying the secrets access from the pod
cat /run/secrets/kubernetes.io/serviceaccount/token*curl --request POST --data "{\"jwt\": \"`cat /run/secrets/kubernetes.io/serviceaccount/token`\", \"role\": \"spin-role\"}" http://Vault-Server-IP:8200/v1/auth/spin-k8spath/loginexport VAULT_TOKEN = $(curl --request POST --data "{\"jwt\": \"`cat /run/secrets/kubernetes.io/serviceaccount/token`\", \"role\": \"spin-role\"}" http://Vault-Server-IP:8200/v1/auth/spin-k8spath/login | jq -r .auth.client_token)echo $VAULT_TOKEN
The following curl request writes a secret, ‘halconfigPath’ with the value,’/root/.hal/config’
curl -H "X-Vault-Token: $VAULT_TOKEN" -X POST -d '{"halconfigPath":"/root/.hal/config"}' http://Vault-Server-IP:8200/v1/spin-hal-path/halconfigfile
The following curl request deletes the secret from the specified path
curl --header "X-Vault-Token: $VAULT_TOKEN" --request DELETE http://Vault-Server-IP:8200/v1/spin-hal-path/halconfigfile*
Above list of curl commands writes the halyard config file(halconfig) as a secret into vault. By converting it into encoded value and creates a JSON file(config.json) and creates a secret with the JSON file.
Encode the config file
base64 config > encoded-config.txt
Create a JSON file with the name "config.json" and enter the above encoded values into the same.
Store this 'config.json' in the vault by running the below command:
curl --header "X-Vault-Token: $VAULT_TOKEN" --request POST --data @config.json http://<Vault-Server-IP>:8200/v1/spin-hal-path/spinhalconfig
Execute the below command to read back the encoded values from Vault.
curl --header "X-Vault-Token: $VAULT_TOKEN" http://<Vault-Server-IP>:8200/v1/spin-hal-path/spinhalconfig| jq -r .data.spinhalyardconfig > encodedhalconfig.binbase64 -d encodedhalconfig.bin > config-new
By executing the above step, you should be able to verify the decoded config file with the original file.