Create Log Template in Git

A log template is a type of template which is used for doing log analysis. Users can create a log template in GitHub. To create a log template follow the steps below:

Users can create a log template after creating an application. If you haven’t created an application already, click here. You can create a log template in a GitHub repository where your deployment manifest files are saved. OpsMx provides two types of sample Log Templates for each data source type that supports Log Analysis. You can use these sample yaml files instead of creating a new template.

Sample templates:

The sample templates for each data source are available here. For example, sample Log Templates for Elasticsearch data source are as follows:

• Minimal yaml file (elasticsearch-log-generic-minimal.yaml)

• Extended yaml file (elasticsearch-log-generic-extended.yaml)

Minimal yaml file:

The following sample Elasticsearch minimal yaml file contains the mandatory parameters to create a log template. You can use this sample minimal yaml file instead of creating a new template.

apiVersion: v1
kind: ConfigMap
metadata:
  name: elasticsearch-generic-minimal
data:
  elasticsearch-generic-minimal: |
    monitoringProvider: ELASTICSEARCH
    accountName: elastic-account-name
    index: kubernetes*
    filterKey: kubernetes.pod_name
    responseKeywords: log

Parameters details are as follows:

• name: Give a name to the Log Template (This Log Template name, must be used in OpsMx Provider configmap file ).

• monitoringProvider: Provide the name of “ELASTICSEARCH”.

• accountName: Log provider account name (Must be provided the same account name, which you have given while adding the Elasticsearch integrator in ISD UI)

• index: Index containing logs for processing

• filterKey: Unique Key which identifies logs to be processed in the index

• responseKeywords: Field name in the index containing logs to be processed.

Extended yaml file:

The following sample Elasticsearch extended yaml file contains all the available parameters even non-mandatory to create a log template. You can use this sample extended yaml file instead of creating a new template.

apiVersion: v1
kind: ConfigMap
metadata:
  name: elasticsearch-log-generic-ext
data:
  elasticsearch-log-generic-ext: |
    monitoringProvider: ELASTICSEARCH
    accountName: elastic-account-name
    index: kubernetes*
    filterKey: kubernetes.pod_name
    responseKeywords: log
    # errorTopics array if not defined, default set of error topics are applied.
    # errorTopics array if given and disableDefaultErrorTopics is set to false(default), adds the given list into default applied list and 
    # if errorString matches with existing errorTopic,default is overridden by the custom one.
    # errorTopics array if given and disableDefaultErrorTopics is set to true, only the given list will be applied.
    errorTopics: 
    - errorString: ArrayIndexOutOfBounds
      topic: ERROR
    - errorString: NullPointerException
      topic: ERROR
    tags:
    - errorString: FATAL
      tag: FatalErrors

Parameters details are as follows:

• name: Give a name to the Log Template (This Log Template name, must be used in OpsMx Provider configmap file ).

• monitoringProvider: Provide the name of “ELASTICSEARCH”.

• accountName: Log provider account name (Must be provided the same account name, which you have given while adding the Elasticsearch integrator in ISD UI)

• index: Index containing logs for processing

• filterKey: Unique Key which identifies logs to be processed in the index

• responseKeywords: The element in Elasticsearch record referring to the actual log line. e.g. log, message, etc.

• errorTopics: Error Topics can be defined to filter the attached errorString to categorize the logs carrying it into a severity level. The definition of an errorTopic contains a combination of errorString and severity to associate with it. errorTopics can have 4 severity levels: CRITICAL, ERROR, WARN and INFO.

• tags: Tags are used for future reference of an issue captured during analysis. An error string (as configured) when found in a log cluster, the cluster gets tagged and comments can be associated with the tag for future reference.

After creating the log template in git, you need to create the “OpsMx Provider Configmap”. To create the “OpsMx Provider Configmap” refer here.