Argo Rollouts analysis with ISD and Elasticsearch
Last updated
Was this helpful?
Last updated
Was this helpful?
This document explains the end-to-end flow for the analysis run with the Elasticsearch data source.
Follow the steps below for the end-to-end workflow for the analysis run and to get an analysis report.
Note: You must add all the integrations through ISD UI only.
To add an Elasticsearch integration as your data source to allow ISD to perform your log analysis, follow the steps given below:
From the ISD dashboard, click "Setup" and click "Integrations" and then Click "+New Integration" button as shown in the image below:
The list of available integrators and their respective fields appears. Select the Elasticsearch integration and fill out the information about it that appears on the right pane of the screen.
Update the following information on the above screen:
Account Name: Name of the Elasticsearch account to operate on
Elastic End Point: End of the Elasticsearch URL for the account to operate on
User Name: Provider the user name for the ElasticSearch instance
Password: Provider the password for the ElasticSearch instance
Kibana End Point(Optional): Endpoint URL for the Kibana instance
Kibana User Name(Optional): Provide the username for the Kibana instance
Kibana Password(Optional): Provide the password for the Kibana instance
Permissions: To restrict permissions to this account, you can select the User Groups.
Once you have updated all the information, click "Save" and the newly created Elasticsearch integrator appears as shown below:
Create a log template after adding the Elasticsearch integrator.
We support the following two modes of template creation
Gitops mode template creation
Template creation in ISD UI
You can create a log template in a GitHub repository where your deployment manifest files are saved. The sample Log Template for Elasticsearch data source is available here.
OpsMx provides the two types of sample log Templates for Elasticsearch as follows:
Minimal yaml file (elasticsearch-log-generic-minimal.yaml)
Extended yaml file (elasticsearch-log-generic-extended.yaml)
Minimal yaml file:
The following sample Elasticsearch minimal yaml file contains the mandatory parameters to create a log template. You can use this sample yaml file instead of creating a new template.
Parameters details are as follows:
name: Give a name to the Log Template (This Log Template name, must be used in OpsMx Provider configmap file ).
monitoringProvider: Provide the name of “ELASTICSEARCH”.
accountName: Log provider account name (Must be provided the same account name, which you have given while adding the Elasticsearch integrator in ISD UI)
index: Index containing logs for processing
filterKey: Unique Key which identifies logs to be processed in the index
responseKeywords: Field name in the index containing logs to be processed.
Extended yaml file:
The following sample Elasticsearch extended yaml file contains all the available parameters even non-mandatory to create a log template. You can use this sample yaml file instead of creating a new template.
Parameters details are as follows:
name: Give a name to the Log Template (This Log Template name, must be used in OpsMx Provider configmap file ).
monitoringProvider: Provide the name of “ELASTICSEARCH”.
accountName: Log provider account name (Must be provided the same account name, which you have given while adding the Elasticsearch integrator in ISD UI)
index: Index containing logs for processing
filterKey: Unique Key which identifies logs to be processed in the index
responseKeywords: The element in Elasticsearch record referring to the actual log line. e.g. log, message, etc.
errorTopics: Error Topics can be defined to filter the attached errorString to categorize the logs carrying it into a severity level. The definition of an errorTopic contains a combination of errorString and severity to associate with it. errorTopics can have 4 severity levels: CRITICAL, ERROR, WARN and INFO
tags: Tags are used for future reference of an issue captured during analysis. An error string (as configured) when found in a log cluster, the cluster gets tagged and comments can be associated with the tag for future reference.
You can create a log Template on the Setup → Analysis Templates page in ISD UI. To create a log template in ISD UI, follow the steps below:
From the application dashboard, click "Setup" and click “Analysis Templates” and then click "+New Template" button. Refer to the image below.
After clicking “+New Template” button, two options appear for you to choose the type of template you want to create. Select the “Log Template” from the available options as shown in the below image.
The New Log Template window appears and it has three sections to update the necessary parameters as shown below:
Log Provider: Select the data source for analysis and provide relevant parameters
Log Topics: Strings that appear in logs with their characterization
Log Tags: Create custom tags based on business logic.
Select the data source for analysis and update the relevant parameters as per the below instructions.
Log Template Name: Provide a unique name to the Log Template in the text box.
Provider: Select the data source from the Provider drop-down. Based on the selection there will be new options added. In this section, we have selected Elastic Search as an example. Once selected, the new options appear as shown in the image below: