Continuous Verification

This is an older version of the document. To view the most recent version of the document, click here.

Software risk verification of new releases is performed at various stages in a software delivery cycle. These verifications happen during integration testing, staging, and production deployment through canary, blue-green, or progressive delivery (like rolling-update).

Verification is performed through analysis of real-time data observed while the new release is running in these environments. The types of data are Log, APM, Distributed Tracing, and Metrics. This data is used to form a baseline version/patterns and the risk assessment is done by comparing the new version data to the baseline pattern, and KPI metrics of the new version by comparing to the baseline version. These verifications are tedious, error-prone, and time-consuming. OpsMx ISD using machine-learning and statistical algorithms, automated identify risk in the new version by processing the logs and metrics during the test or deploy process.

OpsMx ISD integrates with the log monitoring tools like Splunk, Elastic, Sumo Logic, Graylog, and metrics monitoring tools like Datadog, Prometheus, Stackdriver to immediately identify the risks.

Log analysis is done using machine learning Natural Language Processing algorithms like Roberta parses 1000s of log messages into logical clusters and then classifies unexpected clusters that are unique or significantly varied in the new release compared to the old release. The classification could be critical, error or warning, or ignored, which determines the impact on the final risk score. These classifications can be modified with Supervised inputs that form the basis of further analysis. The classification engine can also use a pre-trained classification model/template for quicker convergence for a service. Diagnostics of the error found is made easier with the easy-to-understand contextual information. If there are multiple contexts for the same error message, it is highlighted to assist the developer in root-cause all possible issues.

The metric analysis uses a 3-layer Backpropagation Neural network to determine individual metric weights that impact the final score. The higher the relevance or importance (again, supervised input is accepted), the higher the impact on the final score if that metric fails. The metrics are analyzed for failure using statistical algorithms comparing the new release to the baseline.

Verification handles multi-service analysis as it is common for an application to release multiple services at the same time, and it could be challenging to identify the failing service.

Verification is also extended to support analysis during integration testing. In a typical integration testing, multiple test cases (typically 100s) are performed on the new release, and it is critical to understand the risk at an individual test cases level to diagnose the issues found.

Increase customer satisfaction by minimizing errors during build, deploy, test, and release process. Validate health of a new software release through AI/ML technology that uses data from metrics and logs.

This feature enables you to verify new software releases for suitability in production, including reliability and business impact, in an automated fashion.

It uses machine learning based intelligence to control app rollout via canary, blue/green or rolling update.