Upgrade from 4.0.4.1 to 4.0.4.2
Upgrading instructions
Follow these instructions if you are upgrading from 4.0.4.1 to 4.0.4.2 The current installation (4.0.4.1) can be installed using Helm (Scenario A) or the GitOps installer (Scenario B). Please follow the steps as per your present scenario.
WARNING: Before starting the upgrade, take a backup of all the databases, particularly the Postgres DB. The backup procedures may vary depending on your use of external DBs and Spinnaker configuration.
Scenario A
Use these instructions if:
You have a 4.0.4.1 installed using the helm installer and
Already have a gitops-repo for Spinnaker Configuration
Have values.yaml that was used for helm installation
Execute these commands, replacing gitops-repo with your repo
Copy the existing values.yaml, that was used for previous installation into this folder. We will call it values-4041.yaml.
Update the values-4041.yaml as per the requirement.
Copy the updated values file as "values.yaml" (file name is important).
Create gittoken secret. This token will be used to authenticate to the gitops-repo
kubectl -n opsmx-isd create secret generic gittoken --from-literal gittoken=PUT_YOUR_GITTOKEN_HERE
Create the secrets mentioned above.
You only need to create these secrets if they are changed from the default
Scenario B
Use these instructions if:
a) You have a 4.0.4.1 installed using gitops installer.
b) Already have a gitops-repo for ISD (AP and Spinnaker) Configuration.
Execute these commands, replacing "gitops-repo" with your repo. Execute these commands, replacing "gitops-repo" with your repo
Check that a "values.yaml" file exists in this directory (root of the gitops-repo).
Common Steps
Upgrade sequence: (4.0.4.1 to 4.0.4.2)
Ensure that "default" account is configured to deploy to the ISD namespace (e.g. opsmx-isd).
If you have modified "sampleapp" or "opsmx-gitops" applications, please backup them up using "syncToGit" pipeline opsmx-gitops application.
Copy the bom from standard-isd-gitops.git to the gitops-repo
cp -r standard-isd-gitops/bom gitops-repo/
If there are any custom settings done for spinnaker please update those changes accordingly in gitops-repo/default/profiles.
cd upgrade
Update upgrade-inputcm.yaml:
url, username and gitemail MUST be updated. TIP: if you have install/inputcm.yaml from previous installation, simply copy-paste these lines here
If ISD Namespace is different from "opsmx-isd": Update namespace (default is opsmx-isd) to the namespace where ISD is installed
If ISD Namespace is different from "opsmx-isd": Edit serviceaccount.yaml and edit "namespace:" to update it to the ISD namespace (e.g.oes)
Update the values.yaml:
(Optional step) Refer Enabling Pipeline Insights and Stage Insights in ISD if you want to enable the new Insights pages (Pipeline Insights and Stage Insights) added to ISD.
DB Upgrade: Upgrade from ISD 4.0.4.1 to 4.0.4.2 doesn't include DB changes so, dbmigration flag in values.yaml should be set to false.
Push changes to git:
git add -A; git commit -m"Upgrade related changes";git push
kubectl -n opsmx-isd apply -f upgrade-inputcm.yaml
kubectl patch configmap/upgrade-inputcm --type merge -p '{"data":{"release":"isd"}}' -n opsmx-isd
# Default release name is "isd". Please update it accordingly and apply the commandkubectl -n opsmx-isd apply -f serviceaccount.yaml
# Edit namespace if changed from the default "opsmx-isd"kubectl -n opsmx-isd replace --force -f ISD-Generate-yamls-job.yaml
[ Wait for isd-generate-yamls-* pod to complete ]Once the pod is completed please check the pod logs to verify manifest files are updated in GIt or not.
kubectl -n opsmx-isd logs isd-generate-yamls-xxx -c git-clone
#Replacing the name of the pod name correctly, check if your gitops-repo is cloned correctly
Compare and merge branch: This job should have created a branch on the gitops-repo with the helmchart version number specified in upgrade-inputcm.yaml. Raise a PR and check what changes are being made. Once satisfied, merge the PR.
kubectl -n opsmx-isd replace --force -f ISD-Apply-yamls-job.yaml
Wait for isd-yaml-update-* pod to completeOnce pod will completed so please check the pod logs to verify manifest files are updated in Git or not.
kubectl -n opsmx-isd logs isd-apply-yamls-xxx -c git-clone
#Replacing the name of the pod name correctly, check if your gitops-repo is cloned correctlykubectl -n opsmx-isd logs isd-apply-yamls-xxx -c script
#Replacing the name of the pod name correctly, check the log of the script that pushes the yamls and applies them
isd-spinnaker-halyard-0 pod should restart automatically. If not, execute the following command:
kubectl -n opsmx-isd delete po isd-spinnaker-halyard-0
Restart all pods:
kubectl -n opsmx-isd scale deploy -l app=oes --replicas=0
Wait for a min or twokubectl -n opsmx-isd scale deploy -l app=oes --replicas=1
Wait for all pods to come to ready stateIf you enabled new Insights feature in step 8, please follow the post installation steps listed in Enabling Pipeline Insights and Stage Insights in ISD.
Go to ISD UI and check that version number has changed in the top right corner (under Help menu)
Wait for about 5 min for autoconfiguration to take place.
If required: a) Connect Spinnaker again b) Configure pipeline-promotion again. To do this, in the ISD UI:
Click setup
Click Spinnaker tab at the top. Check if "External Accounts" and "Pipeline-promotion" columns show "yes". If any of them is "no":
Click "edit" on the 3 dots on the far right. Check the values already filled in, make changes if required and click "update".
Restart the halyard pod by clicking "Sync Accounts to Spinnaker" in the Cloud Accounts tab or simply delete the halayard pod
If an upgrade fails
As we have a gitops installer, recovering from a failed install/upgrade is very easy. In summary, we simply delete all objects are re-apply. Please follow the steps below to recover.
As a first step. Please try the Troubleshooting section in the Installation document.
Reinstall ISD
Make changes to uppgrade-inputcm and/or values.yaml as required. Ensure that the changes are pushed to git
Wait for all the pods to come up
Last updated