Delivery Verification

Integrate automated verification with your deployment to automatically verify new application versions based on errors, exceptions, and performance after deployment. You can promote a release to the next stage or restore your service with the auto rollback in seconds based on intelligence-based guidance.

Note: OpsMx provides a repository containing all the sample files that are being used in this document to integrate automated verification with your deployment. Refer to the below link to access the sample files.

Delivery-Verification-Examples/Opsmx-Provider-Config at main · OpsMx/Delivery-Verification-ExamplesGitHub

Pre-requisites:

OpsMx Profile
Service Account

Note: Please note that the above pre-requisites get installed by default during the “ISD-Argo Full Installation” and “ISD-Argo Rollouts Installation”. For the modes of installation where Argo CD/Argo Rollout installation is not done using ISD installer then the pre-requisites need to be created manually.

To create an "OpsMx Profile", refer here. Sample “OpsMx Profile” yaml file is available here.
To create a "Service Account", refer here. Sample “Service Account” yaml file is available here.

Enable Verification:

Follow the instructions below to integrate automated verification with your deployment.

Connect your log and metric sources on the ISD “Integrations” page. To add an integration as your data source to allow Autopilot to perform your log analysis, refer here.
Create Log and Metric templates. We support the following two modes of template creation.
- Gitops mode template creation. For more information on how to create Log and Metric templates in git, refer here.
- Template creation in ISD UI. For more information on how to create Log and Metric templates in ISD UI refer here.

Create “OpsMx Provider Configmap” with opsmx metric provider information, including Log and Metric templates as shown below. Sample “OpsMx Provider Configmap” yaml file is available here. To create “OpsMx Provider Configmap” refer here.

apiVersion: v1
kind: ConfigMap
metadata:
  name: opsmx-provider-config
data:
  providerConfig: |
    application: rollout-app
    user: admin
    gateUrl: 'https://isd-argofull.isd-dev.opsmx.net/'
    lifetimeMinutes: 30
    intervalTime: 10
    lookBackType: sliding
    gitops: true
    passScore: 80
    serviceList:
    - serviceName: rollout-app-svc
      metricScopeVariables: "${app_key},${host_key}"
      baselineMetricScope: "rollout-app,{{env.STABLE_POD_HASH}}"
      canaryMetricScope: "rollout-app,{{env.LATEST_POD_HASH}}"
      metricTemplateName: "newrelic-verifier"
      metricTemplateVersion: "v1.0"
      logTemplateName: elasticsearch-verifier
      logScopeVariables: "resource.labels.pod_name"
      baselineLogScope: ".*{{env.STABLE_POD_HASH}}.*"
      canaryLogScope: ".*{{env.LATEST_POD_HASH}}.*"
      logTemplateVersion: "v1.0"

Create an Analysis template as shown below. Specify the “OpsMx Provider Configmap” under the job section in the Analysis template. Sample "Analysis Template" is available here. To create an Analysis Template, refer here.

apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: opsmx-analysis
spec:
  args:
    - name: canary-hash
    - name: baseline-hash
  metrics:
    - name: opsmx-analysis
      provider:
        job:
          spec:
            backoffLimit: 2
            template:
              spec:
                serviceAccountName: opsmx-provider-account
                containers:
                - name: opsmx-analysis
                  image: 'quay.io/opsmxpublic/argo-isd-metric-provider-job:v2' 
                  imagePullPolicy: Always
                  env:
                    - name: MY_POD_NAME
                      valueFrom:
                        fieldRef:
                          fieldPath: metadata.name
                    - name: STABLE_POD_HASH
                      value: "{{args.baseline-hash}}"
                    - name: LATEST_POD_HASH
                      value: "{{args.canary-hash}}"
                  volumeMounts:
                    - name: provider-config-volume
                      mountPath: /etc/config/provider
                    - name: metric-config-volume
                      mountPath: /etc/config/templates
                    - name: secret-config-volume
                      mountPath: /etc/config/secrets
                      readOnly: true
                restartPolicy: Never
                volumes:
                - name: provider-config-volume
                  configMap:
                    name: opsmx-provider-config
                - name: metric-config-volume
                  configMap:
                    name: metrixtemplates
                - name: secret-config-volume
                  secret:
                    secretName: opsmx-profile
                    optional: false

Modify the rollout.yaml and include the analysis step as shown below. Sample rollout.yaml is available here.

strategy:
canary:
  steps:
    - setWeight: 25
    - pause: { duration: 60s }
    - analysis:
        templates:
          - templateName: isd-analysis-template
        args:
          - name: canary-hash
            valueFrom:
              podTemplateHashValue: Latest
          - name: baseline-hash
            valueFrom:
              podTemplateHashValue: Stable

Deploy the application. To deploy the application, refer here.
Note: Please make sure all the above configuration files are stored in the folder where the Rollouts.yaml manifest file is stored.
If a newer version of application is deployed, Rollouts strategy will be invoked and analysis run will be triggered.
Note: For the first version of application deployment, Rollout strategy is not invoked. The Rollout strategy is followed only when a newer version of the application is deployed.
View the Analysis report. To view the Analysis report from Rollouts Dashboard, refer here.
For Advanced Configuration refer to the below sections.
- Interval Analysis
- Scoring algorithm

PreviousDelete Application NextOpsMx Provider Configmap

Last updated 1 year ago