OpsMx
OpsMx
OpsMx
  • Home
  • Overview
    • OpsMx Intelligent Software Delivery (ISD) Platform - Spinnaker
      • Overview
      • Orchestration Module - OpsMx Enterprise for Spinnaker (OES)
        • OES Features
          • Automated Workflows
          • Multi-cloud Deployments
          • Safe-Deployment strategies
          • Scalable & Extensible
        • OES Concepts
          • Application Management
          • Application Deployment
      • Data and Intelligence Module - Autopilot
        • Overview
        • Observability
        • Audit and Traceability
        • Continuous Compliance - Policy
          • Policy - Overview
  • Operator Manual
    • Installation and Configuration
      • Quick Installation
        • ISD v4.0
        • ISD v4.0.1
        • ISD v4.0.2
        • ISD v4.0.3
        • ISD v4.0.4
      • Standard Installation
        • ISD v4.0
          • GitOps Based Installation
        • ISD v4.0.1
          • GitOps Based Installation
        • ISD v4.0.2
          • GitOps Based Installation
        • ISD v4.0.3
          • GitOps Based Installation
        • ISD v4.0.4
          • GitOps Based Installation
        • ISD v4.0.4.1
          • GitOps Based Installation
          • Instructions to Update the Parameters in value.yaml
        • ISD v4.0.4.2
          • GitOps Based Installation
          • Enabling Pipeline Insights and Stage Insights in ISD
        • ISD v4.0.4.3
          • GitOps Based Installation
          • Enabling Pipeline Insights and Stage Insights in ISD
          • Instructions to Update the Parameters in value.yaml
        • ISD v2024.06.00
          • GitOps Based Installation
        • ISD v2024.12.00
          • GitOps Based Installation
        • ISD v2025.03.00
          • Enabling Pipeline Insights or Deployment Insights or Stage Insights in ISD
          • GitOps Based Installation
      • Environment setup for OpsMx ISD
      • ISD On-Prem POV Infrastructure requirements
      • ISD On-Prem Production Infrastructure requirements
      • Stormdriver Installation
    • Access Management
      • User Role
      • Feature Visibility
    • SlackOps Interactive Notification
  • User Guide
    • Dashboard
      • Application Dashboard
      • Application Service View
    • Manage Application
      • Create an Application
      • Configure an Application
      • Delete an Application
    • Manage Pipelines
      • Create a Pipeline
      • Add Stage
      • Add Trigger
      • Edit Pipeline
      • Run Pipeline
      • Rename Pipeline
      • Delete Pipeline
      • Disable Pipeline
      • Lock Pipeline
      • View and Restore Pipeline
      • Pipeline level RBAC
      • Add Intelligent Gates to the Pipeline
        • Approval Gate
        • Verification Gate
        • Test Verification Gate
        • Policy Gate
      • Execute pipelines (Run deployments)
        • Rolling Restart for EKS manifest
        • Deployment to Kubernetes namespace with Manifest file
        • Deployment to Kubernetes namespace with Git based Manifest
        • Manual Approval before deployment
        • Blue/Green deployment using Manifest
        • Helm based deployment to Kubernetes namespace
    • Approvals
    • Policy Evaluation
    • Verification History
    • Infrastructure
      • Cluster
    • Integrations
      • Available Integrations
        • Artifact
        • CI
        • Governance
        • Monitoring Tools
        • Notifications
        • Policy
        • SAST/DAST
        • Pub/Sub Subscription Name
        • Verification
      • Used Integrations
        • Artifact
        • SAST / DAST
        • Monitoring Tools
    • Pipeline Templates
    • Cloud Targets
      • Create a new Cloud Target account
      • Cloud Target account operations
    • Environment Mapping
    • Custom Stages
      • ServiceNow
      • Pipeline Promotion
      • JIRA
      • Update PR
      • Custom notification
      • ServiceDesk
      • Ansible
      • Terraform
      • Ansible AWX
    • OpsMx Agent
      • Agent Overview
      • Agent Creation
      • Agent Service Configuration
      • Agent Installation
      • Delete an Agent
      • Agent and Controller Version Compatibility
    • Verification
    • CD Integration
    • Informed Approval
      • Overview
      • Artifactory
      • Bitbucket
      • Bitbucket Server
      • GITHUB
      • Bamboo
      • Jenkins
      • JIRA
      • ServiceNow
      • Aquawave
      • HCL AppScan
      • JFrog Xray
      • Prisma Cloud
      • SonarQube
      • Verification
    • Manage Policy
      • Create Policy
      • Edit Policy
      • Delete Policy
    • Continuous Verification
      • Overview
      • Templates
        • Log Template
        • Metric Template
      • Log Analysis
        • Events and Clusters
        • Cardinality consideration of Error Events
        • Interpreting the cluster graph
        • Interpreting the score
        • Perceived Risk and Sensitivity
        • Canary Threshold Calibration
        • Info-Cluster Scoring
      • Metric Analysis
      • Test Verification
      • Scoring Algorithms
        • Canary Scoring
        • Autonomous Scoring
        • Definite Scoring
        • Comprehensive Scoring
      • Enable integration into CI/CD pipelines
        • Integrate with CI/CD
        • Integrate with Jenkins
      • Continuous Verification Report Details
    • Unified logs report to track issues
    • Observability Plugin
    • Audit
      • Pipeline Execution
      • Pipeline Audit
      • Policy Audit
      • User Audit
    • Insights
      • Delivery Insights
      • Usage Insights
      • Pipeline Insights
      • User Insights
      • Stage Insights
      • Deployment Insights
  • Release Notes
    • OpsMx Enterprise for Spinnaker
      • OES Release Notes for Spinnaker Version v1.33.3
      • OES Release Notes for Spinnaker Version v1.30.1
    • ISD 4.0.x Release Notes
    • OES 3.9.X Release Notes
    • OES 3.8.X Release Notes
    • OES 3.7.X Release Notes
    • OES 3.6.X Release Notes
    • OES 3.5.X Release Notes
    • OES 3.4.X Release Notes
    • OES 3.3.X Release Notes
    • OES 3.2.X Release Notes
    • OES 3.1.X Release Notes
    • OES 3.0.X Release Notes
  • Additional Resources
    • Configuration Changes for 2025.03.00
      • Configuration changes for Opentelemetry with ISD
      • Configuration changes for Kafka Addition
      • Configuration changes for Secure Redis
      • Configuration changes to Disable Auto Pilot
      • Configuration changes for Custom Environment Mapping from Pipeline Name
    • Configuration Changes to Enable / Disable Insights Pages
    • Multi Spinnaker
    • API Reference
    • OpsMx Argo Sandbox Guide
    • Code Labs
      • Red Hat Certified Spinnaker Operator
      • How to Integrate Vault with Spinnaker
      • How to Create Terraform Custom Job in Spinnaker
      • How to Create Terraform Webhook Stage in Spinnaker
      • How to Create Ansible Custom Job in Spinnaker
      • How to set up Mutual TLS (mTLS) Authentication for Spinnaker Services
      • SSH
      • X.509
    • FAQs
    • Glossary
    • Previous releases
      • ISD 3.12
        • OpsMx Intelligent Software Delivery Platform
          • Overview
          • ISD Architecture
            • ISD Deployment Architecture
          • ISD Installation
            • ISD Installation Configuration
            • Helm Chart based installation
            • ISD Installation on OpenShift
            • ISD Installation Guide
            • Routing Web URLs to ISD services
            • ISD On-Prem POV Infrastructure requirements
            • ISD - Commonly used Commands
            • ISD Service Catalogue
            • Environment setup for OpsMx ISD
          • Life Cycle Management
          • OpsMx ISD Setup
            • Applications
              • Add Applications
              • Services and Pipelines
                • Overview
                • Add services and pipelines
                • Multiple services
              • Group Permissions
              • Edit Applications
              • Delete Application
            • Integrations
              • Available Integrations
            • Spinnaker Setup
            • Cloud Providers
              • Add new cloud provider account
              • Cloud provider account operations
            • OpsMx Agent
              • Agent Overview
              • Agent Installation
              • Agent Service Configuration
              • Agent Service Credentials
              • Agent Service Type
              • Delete an Agent
        • Orchestration Module - OpsMx Enterprise for Spinnaker (OES)
          • OES Features
            • Automated Workflows
            • Multi-cloud Deployments
            • Safe-Deployment strategies
            • Scalable & Extensible
          • OES Concepts
            • Application Management
            • Application Deployment
          • Feature Configuration
            • Application Management
              • Create an Application
              • Configure an Application
              • Delete an Application
            • Pipeline Management
              • Create a Pipeline
              • Add a Stage
              • Add a Trigger
              • Run a Pipeline
              • Pipeline with Parameter
              • Disable a Pipeline
              • Delete a pipeline
              • Edit a Pipeline
              • View and Restore Pipeline
              • Lock a Pipeline
              • Rename a Pipeline
            • Add Trigger Support
            • Configure Artifact Support
              • Google Cloud Storage
              • GitHub
              • GitHub Artifacts Spinnaker
              • GitLab
              • HTTP
            • Configure the Image Bakery
              • Overview
              • Google Compute Engine
            • Secure Spinnaker
              • Secure Spinnaker Installation
              • Authentication
                • Overview
                • SSL
                • Methods
                  • OAuth 2.0
                  • SAML
                  • LDAP
              • Authorization
                • Overview
                • User Role Providers
                  • Google Group
                  • GitHub Teams
                  • LDAP
                  • SAML
                • Service Account
            • Add CI System
              • Overview and Compatible List of CI Systems by Spinnaker
              • Steps to Configure CI Systems
                • Google Cloud Build
                • Jenkins
                • Travis CI
                • Wercker
            • Enable Monitoring
              • Steps to Setup Supported Monitoring Tools
                • Promethues and Grafana
                • Promethues and Kubernetes
            • List of Custom Stages
              • ServiceNow
              • Pipeline Promotion
              • JIRA
              • Update PR
              • Custom notification
              • ServiceDesk
              • Ansible
              • Terraform
            • SlackOps Interactive Notification
            • How to Setup Replication for Minio Storage Service on Openshift
        • Data and Intelligence Module - Autopilot
          • Overview
          • Installing Autopilot
          • Observability
            • Overview
            • Application Dashboard
            • Application Service View
          • Audit and Traceability
            • Overview
            • Application Audit
              • Pipeline Execution
              • Pipeline
              • Policy Audit
          • Insights
            • Delivery Insights
            • Usage Insights
          • Informed Approval
            • Overview
            • Artifactory
            • Bitbucket
            • Bitbucket Server
            • GITHUB
            • Bamboo
            • Jenkins
            • JIRA
            • ServiceNow
            • Aqua Wave
            • HCL AppScan
            • JFrog Xray
            • Prisma Cloud
            • SonarQube
            • Autopilot
          • Continuous Security
            • Audit Trail
            • Access Management
          • Continuous Compliance - Policy
            • Policy Management - Overview
            • Static Policies
            • Runtime Policies
          • Continuous Verification
            • Overview
            • Analysis Setup
              • Templates
                • Log Template
                • Metric Template
            • Log Analysis
              • Events & Clusters
              • Cardinality consideration of Error Events
              • Interpreting the cluster graph
              • Interpreting the score
              • Perceived Risk and Sensitivity
              • Canary Threshold Calibration
              • Contextual Clustering
              • Info-Cluster Scoring
            • Metric Analysis
            • AppDynamics Verification
              • Using AppDynamics Provider for verification
            • Test Verification
              • Analyze Test runs
            • Scoring Algorithms
              • Canary Scoring
              • Autonomous Scoring
              • Definite Scoring
              • Comprehensive Scoring
            • Enable integration into CI/CD pipelines
              • Integrate with CI/CD
              • Integrate with Jenkins
            • Report Details
            • Supervised Learning
        • Quickstart Guide
          • Contents
          • Quick ISD Installation
            • ISD/Spinnaker (OES) 3.12 Quick Installation
            • Helm based Installation
          • Setup & Configure ISD product
            • ISD Installation Configuration
            • Routing Web URLs to ISD services
            • ISD On-Prem POV Infrastructure requirements
          • Manage Applications
            • Create an Application
            • Configure an Application
            • Delete an Application
          • Manage Pipelines
            • Create a Pipeline
            • Edit a Pipeline
            • Add Intelligent Gates to the Pipeline
              • Add Approval Gate
              • Add Verification Gate
              • Add Test Verification Gate
              • Add Policy Gate
            • Run deployments (Execute pipelines)
              • Deployment to Kubernetes namespace with Manifest file
              • Deployment to Kubernetes namespace with Git based Manifest
              • Manual Approval before deployment
              • Blue/Green deployment using Manifest
              • Helm based deployment to Kubernetes namespace
          • Configure Integrations
            • Add Integrations
            • Edit and Delete Integrations
          • Manage Deployment Accounts
            • Add Deployment Accounts
            • Edit Deployment Accounts
            • Delete Deployment Accounts
          • Manage Policy
            • Create Policy
            • Edit Policy
            • Delete Policy
          • Configure & Setup Spinnaker with ISD
          • Agent Configuration
            • Add a New Agent
            • Configure an Agent
            • Delete an Agent
        • Trial User Guide
          • User Guide
          • SaaS Trial Usage: Instruction Video
          • Copy of Trial User Guide
        • Codelabs
          • Red Hat Certified Spinnaker Operator
          • How to Integrate Vault with Spinnaker
          • How to Create Terraform Custom Job in Spinnaker
          • How to Create Terraform Webhook Stage in Spinnaker
          • How to Create Ansible Custom Job in Spinnaker
          • How to set up Mutual TLS (mTLS) Authentication for Spinnaker Services
          • SSH
          • X.509
        • API Reference
        • FAQs
        • Glossary
        • Release Notes
          • ISD 3.12.x Release Notes
          • ISD 3.11.x Release Notes
          • ISD 3.10.x Release Notes
          • OES 3.9.X Release Notes
          • OES 3.8.X Release Notes
          • OES 3.7.X Release Notes
          • OES 3.6.X Release Notes
          • OES 3.5.X Release Notes
          • OES 3.4.X Release Notes
          • OES 3.3.X Release Notes
          • OES 3.2.X Release Notes
          • OES 3.1.X Release Notes
          • OES 3.0.X Release Notes
        • Upgrade
          • Upgrade from 3.11.x to 3.12.x
          • Upgrade from 3.11.x to 3.11.2
          • Upgrade from 3.11.x to 3.11.1
          • Upgrade from 3.10.x to 3.11
          • Upgrade from 3.9.x to 3.10
          • Upgrade from 3.9.x to 3.9.5
          • Upgrading to OES 3.7
      • ISD 3.10
        • OpsMx Intelligent Software Delivery Platform
          • Overview
          • ISD Architecture
            • ISD Deployment Architecture
          • ISD Installation
            • ISD Installation Configuration
            • Helm Chart based installation - Detailed
            • ISD Installation on OpenShift
          • Life Cycle Management
          • OpsMx ISD Setup
            • Applications
              • Add Applications
              • Services and Pipelines
                • Overview
                • Add services and pipelines
                • Multiple services
              • Group Permissions
              • Edit Applications
              • Delete Application
            • Integrations
              • Available Integrations
            • Spinnaker Setup
            • Cloud Providers
              • Add new cloud provider account
              • Cloud provider account operations
            • OpsMx Agent
              • Agent Overview
              • Agent Installation
              • Agent Service Configuration
              • Agent Service Credentials
              • Agent Service Type
              • Deleting an Agent
        • Orchestration Module - OpsMx Enterprise for Spinnaker (OES)
          • Overview
            • Automated Workflows
            • Multi-cloud Deployments
            • Safe-Deployment strategies
            • Scalable & Extensible
          • Additional Feature Configuration
            • Configure Artifact Support
              • Overview
              • Google Cloud Storage
              • GitHub
              • GitHub Artifacts Spinnaker
              • GitLab
              • HTTP
            • Configure the Image Bakery
              • Overview
              • Google Compute Engine
            • Secure Spinnaker
              • Secure Spinnaker Installation
              • Authentication
                • Overview
                • SSL
                • Methods
                  • OAuth 2.0
                  • SAML
                  • LDAP
                  • X.509
              • Authorization
                • Overview
                • User Role Providers
                  • Google Group
                  • GitHub Teams
                  • LDAP
                  • SAML
                • Service Account
            • Setup Triggers
              • Google Cloud Pub/Sub
              • GitHub WebHook
            • Add CI System
              • Overview and Compatible List of CI Systems by Spinnaker
              • Steps to Configure CI Systems
                • Google Cloud Build
                • Jenkins
                • Travis CI
                • Wercker
            • Enable Monitoring
              • Overview
              • Steps to Setup Supported Monitoring Tools
                • Datadog
                • Promethues and Grafana
                • Promethues and Kubernetes
                • Stackdriver
            • Steps to Setup Canary Support
            • Additional Features Setup
              • Productionize Spinnaker
                • Overview
                • Configure Caching
                  • Configure Scaling
                    • Steps to Scale Orca
                    • Steps to Scale Clouddriver
                  • Configure Persistence
                    • Steps to Setup Set up Orca to use SQL
                  • Steps to Externalize Redis
                  • Steps to Configure Spinnaker’s Usage for Redis
              • Configure Notifications
                • Email
                • HipChat
                • Slack
                • SMS via Twilio
              • Configure User Data(Metadata)
            • Configure Script Stage
            • How to Setup Replication for Minio Storage Service on Openshift
        • Data and Intelligence Module - Autopilot
          • Overview
          • Installing Autopilot
          • Observability
            • Overview
            • Application Dashboard
            • Application Service View
          • Audit and Traceability
            • Overview
            • Application Audit
              • Pipeline Execution
              • Pipeline Changes
            • Policy Audit
          • Insights
            • Overview
            • Delivery Insights
            • Usage Insights
          • Informed Approval
            • Overview
            • JIRA
            • GIT
            • JENKINS
            • AUTOPILOT
            • SONARQUBE
          • Continuous Security
            • Overview
            • Security
            • Access Management
          • Continuous Compliance - Policy
            • Policy Management - Overview
            • Static Policies
            • Runtime Policies
          • Continuous Verification
            • Overview
            • Analysis Setup
              • Templates
                • Log Template
                • Metric Template
            • Log Analysis
              • Events & Clusters
              • Cardinality consideration of Error Events
              • Interpreting the cluster graph
              • Interpreting the score
              • Perceived Risk and Sensitivity
              • Canary Threshold Calibration
              • Contextual Clustering
              • Info-Cluster Scoring
            • Metric Analysis
            • Test Verification
              • Analyze Test runs
            • Scoring Algorithms
              • Canary Scoring
              • Definite Scoring
              • Comprehensive Scoring
              • Autonomous Scoring
            • Enable integration into CI/CD pipelines
              • Integrate with CI/CD
              • Integrate with Jenkins
              • Integrate with Spinnaker (ACA and Canary)
            • Report Details
            • Supervised Learning
      • Spinnaker
        • Spinnaker Concepts
        • Spinnaker Architecture
        • Spinnaker Installation
        • Spinnaker Application
        • Spinnaker Pipeline
        • Cloud & Storage Providers
        • Continuous Integration Providers
        • Trigger Support
        • Authentication and Authorization
        • Caching Agents
        • Configure Caching Agents
        • Configure Slack notifications
        • Integrate Jira with Spinnaker
        • Continuous Deployment to Kubernetes using GitHub triggered Spinnaker pipelines
        • Configure GitHub OAuth
        • Add CI System
          • Overview and Compatible List of CI Systems by Spinnaker
          • Steps to Configure CI Systems
            • Google Cloud Build
            • Jenkins
            • Travis CI
            • Wercker
    • Spinnaker Reference
      • Spinnaker Concepts
      • Spinnaker Architecture
      • Spinnaker Installation
        • Halyard Installation
        • Configure cloud provider
        • Choose your Environment
          • Local Git installation
        • Configure Storage Service
        • Deploy Spinnaker
      • Spinnaker Application
      • Spinnaker Pipeline
      • Cloud & Storage Providers
      • Continuous Integration Providers
      • Trigger Support
      • Authentication and Authorization
      • Caching Agents
      • Configure Caching Agents
      • Configure Slack notifications
      • Integrate Jira with Spinnaker
      • Continuous Deployment to Kubernetes using GitHub triggered Spinnaker pipelines
      • Configure GitHub OAuth
    • Troubleshooting
      • Troubleshooting ISD GitOps Installation Issues
    • Rollback Instructions
      • Revert the GitHub Commit
    • Database
      • Periodic Backup and Recovery of ISD DB
      • Periodic Cleanup of Historic Data in ISD Database
    • Upgrade
      • Upgrade to 2025.03.00
      • Upgrade to 2024.12.00
      • Upgrade to 2024.06.00
      • Upgrade 4.0.3.1 to 4.0.4.3
      • Upgrade from 4.0.4.1 to 4.0.4.2
      • Upgrade from 4.0.4 to 4.0.4.1
      • Upgrade from 4.0.3 to 4.0.4
      • Upgrade from 4.0.3 to 4.0.3.1
      • Upgrade from 4.0.2 to 4.0.3
      • Upgrade from 3.12.x to 4.0.3
        • Changes from 3.12 to 4.0.3 branch
      • Upgrade from 4.0.1 to 4.0.2
      • Upgrade from 3.12.x to 4.0.2
      • Upgrade from 3.12.x to 4.0
      • Upgrade from 3.11.x to 3.12.x
      • Upgrade from 3.11 to 3.11.x
      • Upgrade from 3.10.x to 3.11
      • Upgrade from 3.9.x to 3.10
      • Upgrade from 3.9.x to 3.9.5
      • Upgrading to OES 3.7
  • OpsMx Intelligent Software Delivery (ISD) Platform - Argo
    • Overview
    • Intelligent Software Delivery (ISD) for Argo
    • OpsMx Enterprise for Argo(OEA)
    • Getting Started
      • Platform Installation
      • Getting started with Automated Analysis
      • Automated Analysis User Guide
      • Trial Sandbox User Guide
    • Operator Manual
      • ISD-Argo Standard Installation
        • ISD-Argo Installation with Argo CD and Argo Rollouts
        • ISD-Argo Platform Standard Installation
        • Additional Argo Installation
      • Access Management
      • Configure Authentication Providers through ISD UI
      • Slack Interactive Notification
    • User Guide
      • Application Dashboard
      • Manage Application
        • Create Application
        • Edit Application
        • Sync Application
        • Deploy Application
        • Sync Application Status
        • History and Rollback
        • Refresh Application
        • Delete Application
      • Delivery Verification
        • OpsMx Provider Configmap
        • Templates
          • Create Log Template in ISD UI
          • Create Metric Template in ISD UI
          • Create Log Template in Git
          • Create Metric Template in Git
        • Analysis Template
        • Deployments
        • Analysis History
        • Log Analysis
        • Metric Analysis
        • Interval Analysis
        • Scoring Algorithms
          • Canary Scoring
          • Autonomous Scoring
          • Definite Scoring
          • Comprehensive Scoring
        • Argo Rollouts analysis with ISD and NewRelic
        • Argo Rollouts analysis with ISD and Prometheus
        • Argo Rollouts analysis with ISD and Stackdriver
        • Argo Rollouts analysis with ISD and Elasticsearch
        • Deployments
      • Integrations
        • Available Integrations
      • Notification
      • Argo CD Integration
        • Argo CD Integration with ISD
        • Mapping of host URL with agent-grpc service
      • Insights
      • Audit
        • Deployments Audit
        • System Audit
      • Opsmx Agent
        • Agent Overview
        • Agent Creation
        • Agent Configuration and Installation
        • Edit and Delete Agent
    • Release Notes
      • ISD 4.1 - Release Notes (ISD for Argo)
    • Additional Resources
      • Troubleshooting
        • Troubleshooting ISD Argo Installation Issues
        • Troubleshooting connectivity issues between Argo CD and ISD
      • Upgrade
        • Upgrade ISD-Argo with Argo CD and Argo Rollouts from v4.1.2 to v4.1.3
        • Upgrade ISD-Argo Platform from v4.1.2 to v4.1.3
        • Upgrade ISD-Argo with Argo CD and Argo Rollouts from v4.1.1 to v4.1.2
        • Upgrade ISD-Argo Platform from v4.1.1 to v4.1.2
        • Upgrade Additional Argo from v4.1.1 to v4.1.2
      • Configuration Changes
        • Configuration Changes for 2025.04.00
        • Configuration Changes for 2025.01.00
        • Configuration Changes for 2024.08.00
          • Configuration for System Audit
          • Configuration for Notification
        • Configuration Changes for 4.1.9
        • Configuration Changes for 4.1.8
        • Steps to Generate Token for Agent/Controller v5.x
        • SAML 2 Configuration
        • Controller and Agent Changes for 4.1.7
      • Configuring Application Synchronization (Refresh) Duration
      • Environment setup for ISD-Argo
      • Create API token in Argo CD
      • Create new account for Argo CD with API key and RBAC permissions
      • ISD agent-grpc external IP
      • Previous releases
        • Install Instructions: ISD-Argo v4.1.2
          • ISD-Argo Installation with Argo CD and Argo Rollouts
          • ISD-Argo Platform Standard Installation
          • Additional Argo Installation
        • ISD-Argo v4.1.1
          • Overview
          • Intelligent Software Delivery (ISD) for Argo
          • OpsMx Enterprise for Argo(OEA)
          • Getting Started
            • ISD-Argo Quick Installation
          • Operator Manual
            • Installation and Configuration
              • Installation via curl command
                • ISD-Argo Full Installation via curl command
                • Autopilot Installation via curl command
                • Worker OEA Installation via curl command
              • ISD-Argo Standard Installation
                • ISD-Argo Full Installation
                • ISD-Argo Rollouts Installation
                • Autopilot Installation
                • Worker OEA Installation
              • ISD-Argo On-Prem POV Infrastructure Requirements
              • ISD-Argo On-Prem Production Infrastructure Requirements
              • Environment setup for ISD-Argo
              • Mapping of host URL with agent-grpc service
            • Access Management
            • Slack Interactive Notification
          • User Guide
            • Application Dashboard
            • Manage Application
              • Create Application
              • Edit Application
              • Sync Application
              • Deploy Application
              • Sync Application Status
              • History and Rollback
              • Refresh Application
              • Delete Application
            • Delivery Verification
              • OpsMx Provider Configmap
              • Templates
                • Create Log Template in ISD UI
                • Create Metric Template in ISD UI
                • Create Log Template in Git
                • Create Metric Template in Git
              • Analysis Template
              • Analysis History
              • Log Analysis
              • Metric Analysis
              • Interval Analysis
              • Scoring Algorithms
                • Canary Scoring
                • Autonomous Scoring
                • Definite Scoring
                • Comprehensive Scoring
            • Integrations
              • Available Integrations
            • CD Integration
            • Insights
            • Deployments Audit
            • Opsmx Agent
              • Agent Overview
              • Agent Creation
              • Agent Configuration and Installation
              • Edit and Delete Agent
      • Best Practices for setting up Verification
      • Worker OEA Installation
  • OpsMx Delivery Shield Platform
    • Delivery Shield Overview
    • Getting Started
      • Delivery Shield Feature Overview
      • Installing Delivery Shield
      • Integration with Kubernetes Cluster
        • Integrating Kube Detector
        • Enable Deployment Firewall in Target Clusters
      • Integrating Registry in Delivery Shield
        • ACR
        • ECR
        • GCR
      • Integrating CI and CD tools in Delivery Shield
        • GitHub
        • GitLab
        • Bitbucket
        • Jenkins
        • ArgoCD
        • Spinnaker
        • Quay
        • Docker
        • Google Artifact Registry
        • JFrog Artifactory
        • Sysdig
      • Integrating Security Scanning tools in Delivery Shield
        • Grype
        • VirusTotal
        • Kubescape
        • Snyk
        • Trivy
        • Semgrep
        • Codacy
        • Sonarqube
        • ZAP
        • JFrog Xray
      • Integrating Cloud Service Providers in Delivery Shield
        • AWS
        • ScouteSuite
      • Integrating Other tools in Delivery Shield
        • ChatGPT
        • Slack
        • JIRA
        • MobSF
        • TFsec
        • Custom Policy Source
      • Support Matrix
    • User Guide
      • View Security Posture
        • Organization Security Posture
        • Application Security Posture
          • View Version History
          • Context Graph
        • View Open Security Issues
        • View Current Deployments
        • View Deployment History
      • Manage Policies
        • Global Policies
        • Application Policies
      • Vulnerability Management
      • License Scan
      • Artifact Security
        • Deployed Artifacts
        • Generated Artifacts
        • Plugin Artifacts
        • Mobile Artifacts
      • Global Risk Management
        • OSS Risk
        • Cloud Security
          • Compute
          • Network
          • Security
      • Adhoc Scan
      • Managing Audit
        • Exceptions
      • Deployment Firewall
      • Delivery Bill of Materials (DBOM)
      • Software Bill of Materials (SBOM)
      • Compare Environments using Smart Diff
      • Manage Teams and Access
      • Managing Clusters
      • Compliance Automation
        • NIST 800-53
        • FedRAMP
        • OpenSSF ScoreCard
        • OWASP Top 10 CI CD Security Risks
        • NSA CISA Top 10
        • MITRE-ATT&CK
        • CIS Benchmark Kubernetes
      • AI Powered Features
        • Rules Genie
        • Security Issues Genie
    • Release Notes
    • Additional Resources
      • Whitelisting API Calls for Self-Hosted Delivery Shield
Powered by GitBook
On this page
  • How to do Automated Canary Analysis for Kubernetes and Prometheus
  • Background:
  • Steps to Enable Kayenta for Kubernetes and Prometheus

Was this helpful?

  1. Additional Resources
  2. Previous releases
  3. ISD 3.10
  4. Orchestration Module - OpsMx Enterprise for Spinnaker (OES)
  5. Additional Feature Configuration
  6. Enable Monitoring
  7. Steps to Setup Supported Monitoring Tools

Promethues and Kubernetes

PreviousPromethues and GrafanaNextStackdriver

Last updated 3 years ago

Was this helpful?

This is an older version of the document. To view the most recent version of the document, click .

How to do Automated Canary Analysis for Kubernetes and Prometheus

Spinnaker automated canary analysis tool has been available starting with version 1.7. In this blog we will cover how to setup Kayenta for a Kubernetes provider with Prometheus being used for monitoring.

Background:

Canary deployments has been one of the best practices in the software delivery for some time now. The reason for doing Canary deployment is to reduce the risk of introducing new update with issues to 100 % of the customers like American Airline did and resulted in

Automated canary analysis is way to use automation to detecting issues with the new software using the metrics generated by the new application and comparing to the current production version. Netflix and Google has teamed up to release an open source version of this analysis tool called Kayenta and make it available as part of the Spinnaker project. Note that the Kayenta architecture is designed in a modular fashion allowing for use of 3rd party or custom judges created by the users. For more information about Kayenta architecture, the recent meetup from the architect of Kayenta.

Steps to Enable Kayenta for Kubernetes and Prometheus

Kayenta supports the following monitoring services - Stackdriver, Prometheus and DataDog. You can deploy Kayenta to Kubernetes running on AWS. In our case, we have Spinnaker instance deploying to Kubernetes running on AWS. Prometheus is the monitoring service used for service level metrics monitoring (Tomcat service).

Let’s begin

Step 1: Enable Canary.

Ensure that you are running 1.7 or later version of the Spinnaker software. You can use the following command to ve rify your spinnaker version.

    hal config version

If you are running prior version, make sure you upgrade the version to the latest version and try again. You can use the following command.

    hal config version edit --version 1.7.6

Then use the below command to enable Canary analysis option for all applications. You can turn off canary availability option for individual applications as desired.

    hal config canary enable

Step 2: Enable Storage for Canary.

Enable the default storage account with the following command.

    hal config canary edit --default-storage-account ACCOUNTName --bucket StorageBucketName --root-folder RootFolderlName

For example:

    hal config canary edit --default-storage-account aws-s3-kayenta --bucket kayentasetup --root-folder kayenta

Step 3: Enable Prometheus Monitoring Service Integration.

Next, we have to enable for Kayenta to able to read Prometheus metric store. Authentication for prometheus is not enabled in the given example. However, it can be setup easily if authentication at Prometheus is enabled. Use the following command to enable Prometheus metric store

    hal config canary prometheus enable
    hal config canary prometheus account add ACCOUNTName --base-url ENDPointOfPrometheus

For example:

    hal config canary prometheus account add prometheus-account --base-url http://a23f9cc3537ee12345678910-655574421.us-east-2.elb.amazonaws.com:9090 
    hal config canary edit --default-metric-account ACCOUNTName

For example:

    hal config canary edit --default-metric-account prometheus-account

Step 4: Create Canary Config

Add canary config in the form of metric template of your monitoring data store (Prometheus)

image 1

After that, click on ‘Add Configuration’, ‘Add configuration name and description’.

image 2

Add Name and Metric name. We need to add filter template in case of kubernete cluster because pod names are saved in the format - container_label_{your pod name} in prometheus server. So for the filter template, be sure to attach the “container_label_” string before your pod name which you will give in canary analysis stage in pipeline and spinnaker will use internally to query metric data from prometheus server. Label Bindings are another way to filter metrics for aggregation. Internally, if the query to Prometheus results in more than one metric (due to multiple tags/dimensions that match metric name) then they are averaged to form one series of data. Labels allow filtering to support only the required tags to be used in generating metric time series. This is also a simple way to utilize labels assigned to Kubernetes pods to select in metric time series query. In summary, before adding anything in this pop up template, Add template in filter templates button (image-1) and then come to this pop up, configure metric and attached filter template. Direction: Customize for change in higher or lower values. For example, measuring throughput, higher is acceptable, however, for latency lower is acceptable Filter templates allow customization of metrics to use in canary analysis. Some implicit binding variables available in filter templates are scope, project, location, resourceType. Filter templates allow literal bindings along with variable bindings. For example, in Prometheus metrics, if only cpu idle time is of interest then literal filter can be set to filter only idle time and not utilization. Another example is when the canary comparison is using a single pod and not the entire server group then the filter can be specified as shown in image-3.

Click on Add template under filter template section on canary configuration.

image 3

${scope} will fill with baseline and canary version dynamically i.e. pod name of your kubernetes cluster. Now you are done with the canary configuration.

image 4

image 5

image 6

Step 5: Run the pipeline / Get scores and Review the scores.

Over ‘Canary reports’ tab, under the ‘delivery tab’ on application page one can see the reports of metrics which we configured in the canary config.

image 7

image 6

Canary needs a storage account to store the canary data such as canary configurations and time series data retrieved from metric stores. If you need help configuring a storage account, check

Click on canary config page under delivery tab on application tab. (image-1)

Click on ‘Add Metric’, which will result in a pop-up as shown in (image-2).

here
(Kayenta)
all their airplanes grounded for 6 hours.
watch
documentation.
Screenshot
Screenshot