Rollback Instructions

Follow the instructions below to roll back from v4.0.3 to v3.12.x


Take the 3.12.x instance external database backup and then revert the database dumps while rolling back (When reverted the endpoints will change).

  1. Assume we have installed the 4.0.3 environment and wanted to rollback to the 3.12.x environment. These are the steps to be followed when you have installed ISD using the gitops style installation.

  2. We will use the same git repo which we used for the installation. Then copy the service settings for the 3.12.x ISD version from the standard-isd-gitops repo. (We can also copy bom file from standard-isd-gitops repo for overwriting the service settings images). cp -r /default/servicesettings/ * /default/service settings/ * <your-repo>/default/service settings/

  3. Change the spinnaker version in the config file in your repo to the respective version (for example 1.26.6 for 3.12.5).

  4. Update the upgrade-inputcm.yaml file the following fields like chartVersion to 3.12.x and make sure the branch is master/main. Verify the branch in serviceaccount.yaml is the same as your repo branch master/main.

  5. To check the chartVersion for specific version of isd run the following command helm search repo opsmx --versions.

  6. In the values.yaml file in the global section change the common gate to required value (true/false) and the sapor gate will be in the global section in 4.0.3. Copy the sapor gate section and remove it from the global section and paste it below rabbitmq as a separate section.

  7. The db endpoints will change when the 3.12.x dumps are restored. Update the mysql db endpoints in the clouddriver-caching-local.yml, front50-local.yml and the orca-local.yml. These files are in cd default/profiles.

  8. Update the psql db endpoint in the values.yaml file.

  9. While upgrading from 3.12.x to 4.0.3 we have created a branch and merged that branch with the master/main. As now we are rolling back to 3.12.x we have to revert back the commit. Follow the link below to revert back the commit.

  10. Take backup of the databases which are being used by spinnaker and ISD as we are using the external database. (i.e 4.0.3 database)

  11. Now push all the changes to your repo to git master/main branch using git add . → git commit -m “message” → git push origin master/main.

  12. Delete the old mysqlcredentials secret and create mysql password for the database with the updated end point. Use the command for reference. kubectl -n create secret generic mysqlcredentials --from-literal host="<reverted database url>" --from-literal username="<username>" --from-literal password="<password>"

  13. Now cd into the upgrade folder and enter the following command: kubectl replace -f upgrade-inputcm.yaml -n <namespace> --force kubectl replace -f serviceaccount.yaml -n <namespace> --force

  14. After that execute the following command: kubectl replace -f ISD-Generate-yamls-job.yaml -n <namespace> --force

  15. This will create a branch in the git repo and merge the branch with master/main by creating a pull request.

    NOTE:- If compare & pull request is not raised automatically, manually raise the PR and merge the branch to the master/main.

  16. After merging execute the command: kubectl replace -f ISD-Apply-yamls-job.yaml -n <namespace> --force

  17. Wait for isd-yaml-update-*pod to complete and once all the pods are stabilised then hit the ingress, login to the ISD and spinnaker and check the version it should be 3.12.x.

Last updated