Canary Scoring
Last updated
Last updated
This is an older version of the document. To view the most recent version of the document, click here.
Canary scoring works with the principle of comparing the log files of two different releases. The log files of the v1 release also known as the baseline is compared with the log files of the v2 release also known as the new release/canary. In this scoring method, the two log files are compared to figure out the unexpected events (CRITICAL ERROR,ERROR, WARNING) and determine the shift from v1 to v2. The lesser the shift in v2 the closer the score is to 100. Canary scoring works best when the size of v1 and v2 logs are almost the same and when log files are large in size.
Canary scoring is done based on the following factors:
For every unexpected cluster present in v2 and not in v1, score of the canary is reduced
For all unexpected clusters present in both v1 and v2, the score of the canary is reduced only when the number of logs of that cluster are higher in v2.
Canary score also takes into account errors in v1 by introducing a normalization factor.
Score Reduction is higher at higher sensitivity levels.
Score Reduction is higher for clusters with higher criticality.
By default, canary scoring will give a score of 0 (fail) whenever there is a critical error.
To set the Canary Scoring algorithm follow the steps below:
Click Setup in the navigation menu as shown below:
2. Add/Edit an application. Click Add Application and Edit Application to know more.
3. After entering the application details you need to add a service.
Enter the Service Name in the text box and click Save. Refer to the image below:
5. Next, you get the option to select the type of service you want to add. The options are:
Deployment Verification
Visibility
Release Management
Select Deployment Verification. The following options appear:
Click the + icon to add a log template.
6. The create Template screen appears as shown below:
7. Click Log Tags. Select Canary Scoring from the drop-down as shown below:
