Delivery Verification

Integrate automated verification with your deployment to automatically verify new application versions based on errors, exceptions, and performance after deployment. You can promote a release to the next stage or restore your service with the auto rollback in seconds based on intelligence-based guidance.

Note: OpsMx provides a repository containing all the sample files that are being used in this document to integrate automated verification with your deployment. Refer to the below link to access the sample files.

https://github.com/OpsMx/Delivery-Verification-Examples/tree/main/Opsmx-Provider-Configgithub.com

Pre-requisites:

• OpsMx Profile

• Service Account

Note: Please note that the above pre-requisites get installed by default during the “ISD-Argo Full Installation” and “ISD-Argo Rollouts Installation”. For the modes of installation where Argo CD/Argo Rollout installation is not done using ISD installer then the pre-requisites need to be created manually.

• To create an "OpsMx Profile", refer here. Sample “OpsMx Profile” yaml file is available here.

• To create a "Service Account", refer here. Sample “Service Account” yaml file is available here.

Enable Verification:

Follow the instructions below to integrate automated verification with your deployment.

1. Connect your log and metric sources on the ISD “Integrations” page. To add an integration as your data source to allow Autopilot to perform your log analysis, refer here.

2. Create Log and Metric templates. We support the following two modes of template creation.

   • Gitops mode template creation. For more information on how to create Log and Metric templates in git, refer here.

   • Template creation in ISD UI. For more information on how to create Log and Metric templates in ISD UI refer here.

3. Create “OpsMx Provider Configmap” with opsmx metric provider information, including Log and Metric templates as shown below. Sample “OpsMx Provider Configmap” yaml file is available here. To create “OpsMx Provider Configmap” refer here.

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: opsmx-provider-config
   data:
     providerConfig: |
       application: rollout-app
       user: admin
       gateUrl: 'https://isd-argofull.isd-dev.opsmx.net/'
       lifetimeMinutes: 30
       intervalTime: 10
       lookBackType: sliding
       gitops: true
       passScore: 80
       serviceList:
       - serviceName: rollout-app-svc
         metricScopeVariables: "${app_key},${host_key}"
         baselineMetricScope: "rollout-app,{{env.STABLE_POD_HASH}}"
         canaryMetricScope: "rollout-app,{{env.LATEST_POD_HASH}}"
         metricTemplateName: "newrelic-verifier"
         metricTemplateVersion: "v1.0"
         logTemplateName: elasticsearch-verifier
         logScopeVariables: "resource.labels.pod_name"
         baselineLogScope: ".*{{env.STABLE_POD_HASH}}.*"
         canaryLogScope: ".*{{env.LATEST_POD_HASH}}.*"
         logTemplateVersion: "v1.0"

4. Create an Analysis template as shown below. Specify the “OpsMx Provider Configmap” under the job section in the Analysis template. Sample "Analysis Template" is available here. To create an Analysis Template, refer here.

   apiVersion: argoproj.io/v1alpha1
   kind: AnalysisTemplate
   metadata:
     name: opsmx-analysis
   spec:
     args:
       - name: canary-hash
       - name: baseline-hash
     metrics:
       - name: opsmx-analysis
         provider:
           job:
             spec:
               backoffLimit: 2
               template:
                 spec:
                   serviceAccountName: opsmx-provider-account
                   containers:
                   - name: opsmx-analysis
                     image: 'quay.io/opsmxpublic/argo-isd-metric-provider-job:v2' 
                     imagePullPolicy: Always
                     env:
                       - name: MY_POD_NAME
                         valueFrom:
                           fieldRef:
                             fieldPath: metadata.name
                       - name: STABLE_POD_HASH
                         value: "{{args.baseline-hash}}"
                       - name: LATEST_POD_HASH
                         value: "{{args.canary-hash}}"
                     volumeMounts:
                       - name: provider-config-volume
                         mountPath: /etc/config/provider
                       - name: metric-config-volume
                         mountPath: /etc/config/templates
                       - name: secret-config-volume
                         mountPath: /etc/config/secrets
                         readOnly: true
                   restartPolicy: Never
                   volumes:
                   - name: provider-config-volume
                     configMap:
                       name: opsmx-provider-config
                   - name: metric-config-volume
                     configMap:
                       name: metrixtemplates
                   - name: secret-config-volume
                     secret:
                       secretName: opsmx-profile
                       optional: false

5. Modify the rollout.yaml and include the analysis step as shown below. Sample rollout.yaml is available here.

   strategy:
   canary:
     steps:
       - setWeight: 25
       - pause: { duration: 60s }
       - analysis:
           templates:
             - templateName: isd-analysis-template
           args:
             - name: canary-hash
               valueFrom:
                 podTemplateHashValue: Latest
             - name: baseline-hash
               valueFrom:
                 podTemplateHashValue: Stable

6. Deploy the application. To deploy the application, refer here.

   Note: Please make sure all the above configuration files are stored in the folder where the Rollouts.yaml manifest file is stored.

7. If a newer version of application is deployed, Rollouts strategy will be invoked and analysis run will be triggered.

   Note: For the first version of application deployment, Rollout strategy is not invoked. The Rollout strategy is followed only when a newer version of the application is deployed.

8. View the Analysis report. To view the Analysis report from Rollouts Dashboard, refer here.

9. For Advanced Configuration refer to the below sections.

   • Interval Analysis

   • Scoring algorithm