Actions for advanced and time-based enrichment

Actions are conditions that are run sequentially on each event that matches the event selection criteria. 

Actions can be of the following types:

• Actions that help you define conditions to get a subset of matching events. For example, use the Lookup action to retrieve existing events that match the specific criteria.
• Actions that help you define conditions to further enrich matching events. For example, use the If-Then-Else action to process events in a specified way and the Enrich action to enrich event information.

Based on your use case, you can combine various actions to build the refinement, advanced enrichment, and time-based enrichment policy workflows.

Actions can be classified as: 

• Root action, which is the highest action in the workflow structure. This action has no parent. This action can have one or more child actions, but cannot have a sibling action or be repeated in the same workflow. You cannot add more than one root action in the same workflow. Lookup, Unless, Trigger-If, and Timeout are root actions.
• Non-root action, which can occur anywhere in the workflow structure and can be repeated multiple times in the same workflow. All actions other than Lookup, Unless, Trigger-If, and Timeout are non-root actions.   

Common actions

The following actions are common to refinement, advanced enrichment, and time-based enrichment policies:

• Variable
• If-Then-Else
• Enrich

Variable

Use variables to point to static text, event data (values of event slots), results of mathematical functions, and values of previously defined variables. Variables have the following types:

• Local variables: These variables can be used only within an enrichment policy and are prefixed with a dollar sign ($).
• Global variables: These variables can be used to share data across enrichment policies, which means they can be referred to in any other event policy. These variables are prefixed with $GV. Set attributes for these variables in enrichment policies by using the Set global variable check box as shown in the following example.
  Do not select the check box if you want to set only local variables. You can also specify global variables in the event selection criteria while creating event policies and in the notification settings while creating a notification policy.
  

You can reuse variables for building conditions for processing events. You can specify the variable name as the value of these actions. When naming variables, the underscore (_) is the only special character supported in the variable name.

You can specify variables by selecting previously-defined variables from the list of slots. Alternatively, you can specify variables in text strings. For example, you can specify a variable as an input for specific functions, or you can add the variable as a text string value for the Enrich action. You can also add the variable as a text string value for the Variable action. You can convert variable values from String to other data types based on your needs for managing variable values. You can do this by using the following converter functions in advanced enrichment policies:

• StringToInteger
• StringToReal
• Split (String to List of String)

You can also use variables to convert other data types to String. For more information, see Functions-for-advanced-and-time-based-enrichment.

At the time of processing events, the variable name is replaced with the value of the variable.

In the policy workflow, under Variable Settings, you can set one of the following values:

• Text string: Assign a text string containing alphanumeric characters. You can also specify existing variables, refer to existing and incoming event slots.
  For example, $msg_var, $NEW.msg, $OLD.msg.
  When the event policy is applied, the slot and variable names are replaced with appropriate slot values if available from the incoming event.
• Variable: Assign an existing variable’s value to this variable. 
• Function: Assign the function value as the variable value. To understand the supported functions, see Functions-for-advanced-and-time-based-enrichment.
  The function parameters display slots based on the data type of the function parameter as shown: 

  Data type of parameters	Slots available
  String	Displays slots and global variables that have either the String, Number, or Boolean data types.
  List of string	Displays slots and global variables that have the List of String data type.
  Number	Displays slots and global variables that have the Number data type.

• Event Slot: Assign the slot value as the variable value.

To learn how to use the variable action in the policy workflow, see the following topics:

• Example-Check-whether-there-is-a-time-delay-between-events
• Example-Enrich-events-according-to-the-event-hostname-and-event-message

If-Then-Else

The If-Then-Else construct represents a conditional execution. If the condition specified in the If action is evaluated as true, the Then action is run. If the condition specified in the If action is evaluated as false, the Else action is run. The Else action is optional. If you specify multiple classes in the event selection criteria, refer to the following points:

• The event slots present in all the classes in the event selection criteria are displayed for selection in the following sections:
  • If action of an enrichment policy
  • Slot placeholder fields of an enrichment, correlation, and notification policy
• The event slots that are common to the multiple classes are suffixed with the name of the event class in the enrichment, correlation, and notification policy.
• An event slot is not suffixed with the name of the event class if the slot is not present in all the classes that you specify in the event selection criteria.

In a policy workflow, under If Settings, you can define a condition by specifying slots and existing variables. You can specify local variables on both the left and right sides of a condition. However, you can specify global variables on only the right side of a condition. If you specify a condition that consists of an event class, manually specify the internal name of the class in the condition. To know the internal name of a class, export an event. For instructions on exporting an event, see Exporting-events.

With the If-Then-Else construct, you can separate conditions and add more complex workflows for your use case. To do this, you can use multiple branches in a workflow or define a branch within a branch. This feature can help you separate conditions and add more complex workflows to cater to your use case.

To learn how to use the If-Then-Else action in the policy workflow, see Example-Assign-open-events-to-specific-people-and-change-the-event-severity-and-status.

Enrich

Use this action to enrich event data with meaningful user-specified data, data from other events, or results of functions and existing variables.

You can enrich event information by assigning a new value to the selected event slot. Events come with a set of out-of-the-box slots. However, if you want to change a piece of event information that is not already captured by the out-of-the-box slots, it is important that you have existing custom slots. While defining the enrichment conditions, you can select the custom slot to enrich. If you specify multiple classes in the event selection criteria, refer to the following points:

• The event slots present in all the classes in the event selection criteria are displayed for selection in the following sections:
  • Enrich and If action of an enrichment policy
    Enrich action: Event slots are displayed in Slot to enrich and on the Event Slot tab in the action value.
  • Slot placeholder fields of an enrichment, correlation, and notification policy
• The event slots that are common to the multiple classes are suffixed with the name of the event class in the enrichment, correlation, and notification policy.
• An event slot is not suffixed with the name of the event class if the slot is not present in all the classes that you specify in the event selection criteria.

For instructions on specifying multiple classes, see Creating-and-enabling-event-policies.

For example, the following slots are displayed if you specify the Anomaly and Alarm class in the event policy selection criteria:

In the policy workflow, under Enrichment Settings, you can select a slot to enrich, and set one of the following values:

• Text string: Assign a text string containing alphanumeric characters as the slot value. You can specify slot values in multiple lines for the Message and Detailed Message event slots. You can add a line break to move content to a new line by using the <br> tag.
  You can also specify existing variables and refer to existing and incoming event slots.
  For example, $msg_var, $NEW.msg, $OLD.msg.
  When the event policy is applied, the slot and variable names are replaced with appropriate slot values if available from the incoming event.

• Variable: Assign an existing variable’s value as the slot value.

  Important

  While creating a workflow, you can assign existing variables that are defined until then in the current workflow, as long as they are in the same branch of the workflow.

  You can also specify the match slot variables generated by the Data Table Lookup action in this setting. After specifying such variables, the match slot value present in the data table is enriched as the event slot value.

• Function: Assign the function value as the slot value. To understand the supported functions, see Functions-for-advanced-and-time-based-enrichment.
  The functions list displays function based on the data type of the slot that you enrich as shown:
  

  Data type of slots	Return functions available
  String	Displays functions that return either a String, Number, or Boolean value.
  List of string	Displays functions that return a List of string value.
  Number	Displays functions that return a Number value.

  
  The function parameters display slots based on the data type of the function parameter as shown:
  

  Data type of parameters	Slots available
  String	Displays slots and global variables that have either the String, Number, or Boolean data types.
  List of string	Displays slots and global variables that have the List of String data type.
  Number	Displays slots and global variables that have the Number data type.

• Event Slot: Assign another slot's value as the slot value.

To learn how to use the enrich action in the policy workflow, see Example-Enrich-events-according-to-the-device-status.

Actions specific to advanced enrichment

The following actions are specific to the advanced enrichment policy:

• Function
• Lookup
• Unless
• Trigger-If
• Data Table Lookup

Function

These functions are void functions that do not return a value. Therefore, these functions are different from the functions that can be added as the value for Variable and Enrich actions.

For more information, see Functions-for-advanced-and-time-based-enrichment.

To learn how to use the function action in the policy workflow, see Example-Generate-events-to-check-application-availability.

Lookup

Use this action to retrieve information from the event repository of existing events. The resulting events can be used for further processing. This action is a root action and therefore, must always be specified at the top of the workflow before other actions. 

In the policy workflow, under Lookup Settings, you can specify whether you want to lookup one of the following:

• Duplicate events that are associated to custom classes. To do this, select a custom class. 
  By default, duplicate events associated to out-of-the-box classes are dropped. Therefore, you cannot lookup such events even if you select an out-of-the-box class under the Lookup events containing Class list.

  Important

  Suppose you selected the criteria of With duplicate events and in the subsequent action if you selected Update old events followed by another action, for example Enrich action.

  In this scenario, if the events matching the criteria have missing dedup values, all the matching events are updated.

  If you select the criteria of With duplicate events, duplicate events that are not closed are looked up and retrieved based on:

  • <dedup_slots> for event classes containing dedup slots
  • ID for event classes not containing dedup slots

  If you use the Enrich action to enrich slots in the existing event after you select Update old events, the repeat count for the event is auto-incremented.

• Events matching a custom criteria associated to all the classes irrespective of whether it is out-of-the-box or custom.
  For custom criteria, you must specify a condition. This is similar to constructing the event selection criteria.
  By default, this setting does not lookup events that are closed. If you want to lookup closed events, contact BMC Support.

  Important

  We recommend the following points for the custom criteria:

  • Avoid specifying a generic condition to lookup events.
    For example, a generic lookup condition in which you lookup events of only a specific class.
    If you still want to specify a generic condition, make sure that you configure the lookup to fetch only the latest events (Last occured option) to avoid enrichment policy performance issues.
  • Filter events by specifying conditions in the lookup criteria for conditional execution instead of using the If action after the lookup criteria. This configuration improves the event processing time.

The event class value defaults to the class specified under the event selection criteria. If no class is specified, the base EVENT class is considered for the event selection criteria, and therefore, the value defaults to the base class value.

Under the settings, if you:

• Do not change the value: The same class is used to lookup existing events and incoming events. 
• Do change the value: The new class value is used to lookup existing (or old) events and the class specified in the event selection criteria is used to lookup incoming (or new) events. 
  While building the condition, you can specify slots prefixed with $OLD and $NEW. Slots prefixed with ‘$OLD’ refer to slots of existing events and slots prefixed with ‘$NEW’ refer to slots of incoming events. This feature allows you to add conditions to compare old and new slot values.

Additionally, you can define whether you want to consider all events or only the latest events matching the Lookup condition. You can also define the duration for the lookup in seconds. If the duration is not specified, the duration is assumed to be active as long as the policy is active. The lookup will continue, until the policy is disabled or deleted.

After configuring the Lookup conditions, you can configure actions to update new events (or incoming events) and old events (existing events). This action allows you to create a logical branch in the policy workflow. Under each branch, you can add subsequent actions for enrichment or further processing. You can even configure actions to update only new or only old events. If you add the Enrich or variable actions after you configure actions to update only new or only old events, you can specify existing variables and refer to existing and incoming event slots in the Text String tab of the action value. For example, $msg_var, $NEW.msg, $OLD.msg. When the event policy is applied, the slot and variable names are replaced with appropriate slot values if available from the incoming event.

The lookup action can accumulate up to 20000 existing events for a duration of 5 minutes. This limit is the cumulative lookup event count when events arrive in bulk. If the existing events that are looked up exceed 20000, the error log is captured in the _errors slot of the event and the system activates a temporary skip window for a duration of 3 minutes during which the lookup policy configuration and other advanced enrichment policy configurations are skipped. This skip window is applicable at the policy level. After the 3 minutes lapse, the lookup window is activated again. To change these limits, contact BMC Support.

As a tenant administrator, use the BMC Helix Audit Dashboard in BMC Helix Dashboards to view the audit trail for the lookup policy to achieve improved user accountability, compliance with organization policies, and system security. You can audit the lookup policy in the following scenarios:

• The lookup event count exceeds the maximum limit.
• The temporary skip window for the lookup policy expires.

For more information, see Auditing configuration changes in BMC Helix Dashboards.

The following image displays the audit trail for the lookup policy in the BMC Helix Audit Dashboard. Note that the selected resource type is Event Policy Processing. Click the link in the Operation column to view the before and after values for the lookup policy.

To learn how to use the lookup action in the policy workflow, see Example-Drop-duplicate-events-and-update-the-existing-event-with-new-severity.

Unless

This action applies the subsequent action(s) only if the specified query condition does not find a match. The Unless action is similar to Lookup, but the logic is reversed.

This is a root action and therefore, must always be specified at the top of the workflow before other actions. 

Similar to the Lookup action, in the policy workflow, under Unless Settings, you can:

• Specify an event class to query events belonging to that class 
• Specify a condition to query events matching the condition. This is similar to constructing the event selection criteria.

The event class value defaults to the class specified under the event selection criteria. If no class is specified, the base EVENT class is considered for the event selection criteria, and therefore, the value defaults to the base class value.

Under the settings, if you:

• Do not change the value: The same class is used to query existing events and incoming events. 
• Do change the value: The new class value is used to query existing (or old) events and the class specified in the event selection criteria is used to query incoming (or new) events. 
  While building the Where condition, you can specify slots prefixed with $OLD and $NEW. Slots prefixed with ‘$OLD’ refer to slots of existing events and slots prefixed with ‘$NEW’ refer to slots of incoming events. This feature allows you to add conditions to compare old and new slot values.

If the condition specified is not satisfied, subsequent action(s) are run. 

To learn how to use the unless action in the policy workflow, see Example-If-change-event-does-not-exist-drop-the-related-task-event.

Trigger-If

This action triggers the policy workflow of an incoming event or if a specified slot of an existing event is modified. This is a root action and therefore, must always be specified at the top of the workflow before other actions. Unless an event matches the conditions specified in this action, the advanced enrichment process cannot begin. 

In the policy workflow, under the Trigger-If settings, you need to specify:

• The option to trigger the action on existing events in Apply for Existing events only.
  Select this option to trigger the action only on existing events.
  If not selected, the action is triggered on both incoming and existing events.
• The slot that needs to be monitored for changes. Slots that have the Enum, String (for example, Message), and Integer (for example, Repeated) data types are supported. Read-only and hidden slots are not listed but internal slots are listed.
• Conditions to monitor the value. You cannot specify a global variable or a slot name in an incoming event to replace the slot value.
  
  • The new slot value matches the given condition.
  • The previous slot value matches the given condition.

This feature allows you to monitor precisely when the slot value changes from or to a particular value. The slot value can change from:

• Any value to any value (The Trigger specific Criteria and If value was check boxes are not selected).
• Any value to a specific value (Only the Trigger specific Criteria check box is selected).
• A specific value to a specific value (Both The Trigger specific Criteria and If value was check boxes are selected).
  For incoming events, only the Trigger specific Criteria option is considered and the If value was option does not apply.
  The If value was option is considered only if the Apply for Existing events only option is selected.

The action is triggered each time the monitored slot's value changes.

To learn how to use the trigger-if action in the policy workflow, see Example-Close-events-when-the-event-priority-changes.

Data Table Lookup 

This action retrieves information from the data table that matches the event data and stores each column of the matching data row in corresponding variables. The stored variable value can further be used for advanced event enrichment.

The following video (2:45) explains how you can store data table values as variables in advanced enrichment policies.

 https://www.youtube.com/embed/49vIwkTL3WQ?rel=0

In the policy workflow, configure the following settings for this action:

• Lookup data table
  Select the data table from which you want to look up the data. The data table name is displayed in an easily readable format:
  dataTableTag
  dataTableID
  You can add values in the Lookup data table slots as slot names in the following format:
  %<slotName>%
  For example:
  %source_hostname%
  For example:
  Import dynamic enrichment file
  e30b3234-1abf-11ec-bd40-1b8fc5191ded
  You can import data only from existing tables by using this setting. To create data tables and import data from them, we recommend that you use REST APIs and select the newly created data table in this setting. For more information, see Dynamic-enrichment-policy-data-table-management-endpoints-in-the-REST-API.
  After you select the data table, you can preview the data table content.
• Match dynamic value for
  Use this setting to define match slots to be used for matching data in the imported data table with values in the incoming event. Select the slots that you intend to use as match fields in the data table. This setting is like the match settings in dynamic enrichment policies.
  For example, suppose your data table contains the following information:
  • Column A corresponds to the Severity slot.
  • Column B corresponds to the Location slot.

  • Column C corresponds to the Owner (or the assigned user) slot.

    A	B	C
    CRITICAL	Houston	Dave Johnson
    MAJOR	Atlanta	Emily Brown
    MINOR	Seattle	Mike Adams

    For this scenario, you could select the Match slots as the Severity and Location columns. The value in the match slots is stored in variables and can be referenced in the policy workflow by using the following format:
    $DDE.slotName
    Internal slot names are displayed for variables when you reference them in the policy workflow. For example, the internal name for the Message slot is msg. You can reference the variable as $DDE.msg.
    The value in the remaining slots, for example, the Owner column can be referenced in the policy workflow by using the following format:
    $DDE.enrichToken<columnNumber>
    Here,

    • $DDE is a reserved key. If you change this key, you cannot reference the Match and Enrich slots.
    • enrichToken refers to the column values of the data table. You can edit the token name from the UI or REST APIs.
      To learn about editing variable names in the policy workflow by using REST APIs, see the PUT /event_policies/<id> endpoint at Event policy management endpoints in the REST API.

    • <columnNumber> is the column number of the data table that is suffixed to enrichToken. The column number starts with 1.

      Important

      If you specify slot placeholders (%slotName%) in the data table for enrich slots, the incoming event slot value is not enriched in the event when the policy is applied.

      If you want to enrich the event with values from the incoming event by using slot placeholders, use the Event Slot tab in the Enrich action.

• Matching preference
  You can select one of the following data matching preferences. The matching process is like dynamic enrichment policies.
  
  • First Match: Searches each row of the data table sequentially, starting at the top, until it finds the first match.
  • Best Match: Searches each row of the data table sequentially, starting at the top, until it finds the best match according to the following order of matching preference:
    1. exact match
    2. starts with
    3. ends with
    4. contains
    5. regular expressions
    6. any
• Variables for enrich slots
  This setting lists variables for each column in the data table that matches the incoming event value. If you used slot placeholders (%slotName%) in the data table, a variable is created with the resolved value from the incoming event slot value. The values in the enrich slots are stored in variables that have the following format:
  enrichToken<columnNumber>
  These variables are local variables. You can edit the variable name and reorder the variables.
  If you haven't specified Match slots in the Match dynamic value for setting, this setting lists as many variables as the number of columns in the data table. The number of variables that are listed varies according to the value in the Match dynamic value for setting. 

Actions specific to time-based enrichment

Time-based policies execute 30 seconds to a minute after the delayed duration lapses because the scheduler checks every 30 seconds whether the system should execute the time-based policy. The Timeout action is specific to the time-based enrichment policy.

Timeout

Use this action to set a timer after which actions are run on incoming events. This action is a root action and therefore, must always be specified at the top of the workflow before other actions.

In the policy workflow, under Timeout settings, you need to specify:

• The duration
  For the duration, you can specify a delay after which the policy is executed by using one of the following options:
  • Specify a static number in the duration field.
  • Click Select duration slot, and from the slot list, select an event slot or a global variable that has the integer data type.
    This option allows you to specify a dynamic delay based on the slot value of an incoming event.
    If you specify multiple classes in the event selection criteria of a policy, the duration field displays slots that are common to the multiple classes.

• Unit for the duration
  You can specify one of the following units:
  • Seconds
  • Minutes
  • Hours
  • Days

The specified duration is calculated from the time the event arrived (_arrival_time slot).

You can add subsequent actions for enrichment or further processing. After the specified duration lapses, the subsequent actions are run.

To learn how to use the timeout action in the policy workflow, see Example-Set-a-timer-to-run-enrichment-actions.

Example - Closing events automatically

According to your license agreement, open events are closed 90 days after the last modification date. For more information, see BMC Helix Operations Management service. Use event policies to automatically close the events earlier. 

The following are a few scenarios that show how you can close events automatically:

• Create a time-based event policy to close the events that have not been modified in the last 24 hours.
  • Create a time-based event policy to close the events if the event severity is INFO or OK.
• Create an advanced enrichment event policy to close the temporary or custom events that are generated to invoke an event policy.

We will learn the steps to close events automatically by using a time-based event policy if an event has not been modified in the last 24 hours. For this purpose, we will add the following variables and actions to a time-based policy workflow:

• Timeout action
• Variables
  • To store the current time
  • To store the last modified time of the event
  • To store the time difference between the current and last modified time
• If action to verify whether the last modified time is greater than 24 hours
• Enrich action to close the event

Task 1: Add a Timeout action

1. From the Policy Configuration field, select Time Based.
2. Click Timeout.
3. Enter a label; for example, Timeout.
4. Enter 24 as value and select Hours as the unit.
5. Click Apply.

Task 2: Add variables

1. Add a variable to store the current time:
   1. Click Add below and select Variable.
   2. In the Label and Name fields, enter a label and name for the variable; for example, CurrentTime.
   3. In the Value field, click Function, select CurrentTimeStamp as the function, and click Apply.
2. Add another variable to store the last event modified time:
   1. Click Add below and select Variable.
   2. In the Label and Name fields, enter a label and name for the variable; for example, ModifiedTime.
   3. In the Value field, click Event Slot, select Modified as the slot, and click Apply.
3. Add the variable to store the time difference:
   1. Click Add below and select Variable.
   2. In the Label and Name fields, enter a label and name for the variable; for example, TimeDifference.
   3. In the Value field, click Function, and select the function as Math.
   4. Select Minus as Operator, $CurrentTime as Number 1, and $ModifiedTime as Number 2.
   5. Click Apply.

Task 3: Add if action

1. Click Add below and select If.
2. In the Label field, enter a label; for example, TimeDifference is greater than 24 hours.
3. In the If field, enter ( $TimeDifference Greater than 86,400 ).
   Ensure that you convert hours into seconds.
4. Click Apply.

Task 4: Add enrich action to close the event

1. Click Add below and select Enrich.
2. In the Label field, enter a label; for example, ChangeEventStatus.
3. From the Slot to Enrich list, select Status.
4. From the Value list, select Closed.
5. Click Apply.
6. Click Save.
7. Select the Enable Policy check box.
8. Click Save.

The following image shows these time-based configurations in an event policy:

Where to go from here

Now that you understand actions, see Building-a-policy-workflow-for-advanced-and-time-based-enrichment. You can also see workflow examples that show you how to combine various actions to achieve different use cases.