Generating events from logs


Use alert policies to monitor and manage system health by generating an event to notify you about a critical or important log message. An alert policy consists of the following details:

  • Name, description, and precedence.
  • Policy selection criteria or the conditions that generate an event in BMC Helix Operations Management. Configure the policy selection criteria based on the fields available in the logs. The operators that you can use are Equals, Not Equal to, and Contains. Combine these conditions with the AND and OR logical operators. Optionally, group these conditions on a particular field, such as when status Equals 401 for a particular host. In this case, you group the condition on the host field. Next, define the time period for these conditions to be true. As an example, generate an event if status Equals 401 for 5 times (minimum) in the past 10 minutes.  

    Important

    While using the Contains operator in the policy selection criteria, ensure that you use the complete word present in the log string of a field. For example, if the value of the Country field is "United States of America", set the criteria as Country Contains United or Country Contains America. Do not set the criteria for partial words, such as Country Contains Unite or Country Contains Amer. 

  • Host name, which can be either a static value that you type or a field in the logs that you select. If you select a log field, ensure that you select the same log field in the Group by field. 
  • Additional Details are the values from the logs that are added to the fields of the generated event. These values can be either static values that you type or a field in the logs that you select. The additional details that you can add to the event are described as slots on this page: Log Alert event class. Fields of type Enum accept only preconfigured values. If you enter a value that is not preconfigured, the default value is added to the slot in the event. 
    To add custom fields to an event, see Managing events with REST APIs..  

The generated events are shown in BMC Helix Operations ManagementThe class of these events is LOGALERT_EV. To help you analyze the issue for which the event is generated, a cross-launch link from BMC Helix Operations Management to BMC Helix Log Analytics is provided in the Search Parameters field. For a service or host name, these events are correlated in BMC Helix AIOps. 

AIOpsEntities.png 

Use visalizations in BMC Helix Dashboards to track and monitor these events.

LogEventsinHelixDashboards.png

To create an alert policy

The following video (3:02) illustrates the steps to create an alert policy.


icon-play@2x.pnghttps://youtu.be/363A--xCKfs

  1. Click the Alerts menu.
  2. On the Alert Policies page, click Create.
  3. Enter a unique name such as Authentication Failure, and an optional description.
  4. In the Precedence field, set a precedence for the policy.
    Note that the lower the number, the higher the precedence. Policies are applied in the ascending order of precedence.
  5. In the Policy Selection Criteria field, configure the condition for which the event will be generated.
    For example, enter status Equals 401. When you click in the box, you are prompted to make a selection after which, you are progressively prompted to make another selection.
  6. In the Group by field, to group occurrences of a condition, enter the values by which you want to group.
    For example, to group all occurrences of status 401 on a particular host name, enter the host name. You can enter a maximum of three values, but one must be the host name.
    Alternatively, you can add a host name from a log field. Click in the field and select an appropriate option.
  7. In the Alert Condition field, decide how many times the condition must occur in a time period to generate the event and the status of the event and enter and select the values in the Minutes, Minimum count is, and Alert fields.
    For example, when status 401 is reported a minimum of 50 times within a 5-minute period, a critical event is generated.
  8. In the Alert Parameters section, in the Hostname field, enter or select a host name that is added to the event.
    To add a host name from a log field, click in the field and select the appropriate option. This value helps you correlate events in BMC Helix AIOps. 
  9. In the Message field, change the default message, if required.
    To use a log field value in the message, put double curly brackets around the field name such as {{ $.location }}.
  10. In Additional Details, configure additional event parameters like source identifier.
    These values are set for the generated event.
  11. Enable and save the policy.
    View all your policies on the Alert Policies page. To edit, enable, disable, or delete a policy, use the Actions menu.
    AlertPolicies.png

To understand the number of events generated

Let's consider the following examples to understand how many events are generated.

Configurations in an alert policy

Incoming logs

Number of events generated

Details

Policy selection criteria: status Equals 401

Group by: blank

Hostname: blank or static value

For last: 5 minutes; When minimum count is: 10

The condition is satisfied 22 times in the last 5 minutes.

1

The event is generated after the criteria is satisfied the first 10 times in the logs. When it is satisfied another 10 times, for the same event, the Repeated count field is updated as 1.

Policy selection criteria: status Equals 401

Group by: hostname

Hostname: $.hostname

For last: 5 minutes; When minimum count is: 10

The condition is satisfied 11 times for host 1 and 20 times for host 2 in the last 5 minutes.

2

  • One event is generated for host 1 because the criteria for it is satisfied 11 times.
  • One event is generated for host 2 after the criteria is satisfied the first 10 times in the logs. When it is satisfied another 10 times, for the same event, the Repeated count field is updated as 1.

Policy selection criteria: status Equals 401

Group by: blank

Hostname: host 1 (static value)

For last: 5 minutes; When minimum count is: 10

The condition is satisfied 11 times for host 1 in the last 5 minutes.

1

The event is generated for host 1 because the criteria for it is satisfied 11 times.

Policy selection criteria: status Equals 401

Group by: city

Hostname: host 1 (static value)

For last: 5 minutes; When minimum count is: 10

The condition is satisfied 11 times for host 1 and city 1 and 20 times for host 1 and city 2 in the last 5 minutes.

2

  • One event is generated for host 1 and city 1 because the criteria for it is satisfied 11 times.
  • One event is generated for host 1 and city 2 after the criteria is satisfied 10 times in the logs. When it is satisfied another 10 times, for the same event, the Repeated count field is updated as 1.

Policy selection criteria: status Equals 401

Group by: hostname, city, and country

Hostname: $.hostname

For last: 5 minutes; When minimum count is: 10

The condition is satisfied 11 times for host 1, city 1, and country 1 and 20 times for host 2, city 2, and country 2 in the last 5 minutes

2

  • One event is generated for host 1 because the criteria for host 1, city 1, and country 1 is satisfied 11 times.
  • One event is generated for host 2 after the criteria is satisfied 10 times for host 2, city 2, and country 2. When it is satisfied another 10 times, for the same event, the Repeated count field is updated as 1.

To enable the cross-launch link from BMC Helix Dashboards

In BMC Helix Dashboards, you can view the events generated in BMC Helix Operations Management. A cross-launch link to BMC Helix Log Analytics is provided from the event (in the Search Parameter field). However, the cross-launch link is disabled by default. 

  1. Add a dashboard panel of type Table (old) and add the alert_launch_params and _modified_time fields to it.
    For more information, see Configuring dashboards, panels, and queries.

  2. To enable the cross-launch link to BMC Helix Log Analytics:
    1. Go to Dashboards > Manage.
    2. Edit the panel that you have created for the Log Events class.
    3. Go to Panel > Column Styles > Name pattern, enter alert_launch_params.
    4. (Optional) In Column Header, enter a meaningful name for the column.
      For example, Search parameter.
    5. Enable Render value as link.
    6. From the Type list, select String.
    7. Enable Sanitize HTML.
      SanitizingLinksDashboards.gif
    8. Save the panel.
      The Review Logs link is enabled in the Search parameter column.
      Log Events in Dashboards.png

Related topic

Creating-and-managing-alert-policies-with-REST-APIs

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*