Configuring alarm policies

With alarm policies, you can configure thresholds that help you detect anomalies in the system. Alarms are types of events that are generated when threshold values are violated. You can configure the threshold values. Thresholds define an acceptable value above or below which an alarm is generated. Thresholds can be applied to multiple monitor types on multiple PATROL Agents.

As a tenant administrator, you can create, edit, view, and delete alarm policies. You can also view a list of alarm policy conflicts and an audit trail of all updates made to all alarm policies.

To configure alarm policies

Important: Required permission for the custom restricted-user role

Make sure that the monitor.external_entity_types.view permission is assigned to a custom restricted-user role, so that the associated users can view external entities while configuring alarm policies.

The following video (6:02) illustrates how you configure alarm policies.

Note that the video displays screens from an earlier version, however, the information provided in the video is still relevant to the current version of the product.

https://youtu.be/egs6dodNm1E

To create an alarm policy

While creating an alarm policy, you can view and select metrics for dynamic monitor types. For this, ensure that monitor instances are available for a device on the Devices > Device Details > Monitors tab.

On the Configuration > Alarm Policies page, click Create, and do the following:

Specify a unique name and an optional description.
Add a unique precedence number to the policy.
You can add a custom value in this field, or use the arrows to increase or decrease the value. For more information, see Alarm-policies.
Create the conditions based on which the alarm will be generated.
Constructing the alarm generation conditions
You can define the alarm generation conditions for multiple or all instances.
Start adding a condition by specifying these details:
1. Metric and instance details: The metric and the number of instances for which you want to add this policy.
  Select one of the following options:
  Instance name
  The instance name is case sensitive. Ensure that you use the instance name as displayed in the Device Details page while defining the selection criteria.
  The instance name that you enter in the alarm policy is displayed in the object slot for an alarm event and not in the instancename slot on the Events page.
  All Instances
  Multiple Instances: When you select multiple instances, you can define multiple conditions using parameters such as agent tag name, hostname, instance name, port, etc. You can also use regular expressions while defining multiple instances by using the Matches operator.
  You can also copy the criteria by clicking Copy . The copied criteria can be reused in subsequent policies by pressing Ctrl+V in the selection criteria field.
  
  Preview button
  The Preview button is disabled if you specify the Device Host Name and Instance Name in the selection criteria.
  The Preview button displays only those devices that satisfy the selection criteria.
  Duplicate metrics
  You can create alarm conditions for multiple instances in a single alarm policy. You can use the same metric information for these instances.
  
  Example...
  
  For example, an alarm policy can have the following conditions for the /boot and /tmp instances:
  Agent hostname = XYZ
  Instance name = /boot
  Device name = XYZ
  => If FSCapacity is above 76%, generate a MINOR alarm
  => If FSCapacity is above 86%, generate a MAJOR alarm
  Agent hostname = XYZ
  Instance name = /tmp
  Device name = XYZ
  => If FSCapacity is above 86%, generate a MAJOR alarm
  
  You cannot create multiple policies with duplicate metric information. Policies with duplicate metric information can be added only if you specify different instance types (all and multiple). In this case, the policy set for multiple instances gets precedence.
2. Threshold details and post trigger actions: The threshold value, violation duration, and details about when the generated alarm must be closed eventually.
  Specify if the event must be closed immediately after it is generated or after the metric reaches a normal state and a duration after the violation time period has lapsed. You can also specify that the event must not be closed. Alarm events that are not closed remain open until they are closed manually, the policy is deleted, or the PATROL Agent associated with the alarm is deleted. To change any of the values, click them.
3. Baseline: Specifies whether after the static threshold is breached and the baseline calculation for a metric is violated, only then the alarm is generated.
Example 1: As per the condition defined in the following image (Monitoring solution: Linux; Monitor type: CPU; Metric: Utilization), when the CPU Utilization of all Linux computers whose instance name starts with 'CPU1' crosses the threshold of 75% for a period of 15 minutes, a Major alarm will be generated. After the Utilization returns to a normal state of below 75% and a 15 minutes lapse, the generated alarm is automatically closed.
You can add multiple conditions per metric. To add additional conditions, click Add Condition. Click Duplicate Condition to create another condition with the same details that you can modify later. To add conditions for a different metric, click Add Instance Policy.
If you have multiple conditions with varying threshold values and severity levels, an alarm is generated when the first condition is breached. When the next condition is breached, a new alarm is not generated. Instead, the severity of the first alarm changes.
Suppose you added these conditions:
- If CPU utilization crosses 75% for a period of 15 minutes, generate a Major alarm.
- If CPU utilization crosses 85% for a period of 15 minutes, generate a Critical alarm.
In this scenario, when the CPU utilization of a computer crosses 75% for a period of 15 minutes, a Major alarm is generated. When the CPU utilization of the same computer crosses 85%, the earlier alarm severity changes from Major to Critical. If the CPU utilization returns to 75%, the alarm severity changes from Critical to Major.

Example 2: As per the condition defined in the following image (Monitoring solution: Linux; Monitor type: CPU; Metric: Utilization), when the CPU Utilization of all Linux computers whose agent tag name is production and agent host name begins with clm crosses the threshold of 75% for a period of 15 minutes, a Major alarm will be generated. After the Utilization returns to a normal state of below 75% and a 15 minutes lapse, the generated alarm is automatically closed.

Example 3: The following example uses a regular expression while defining multiple instances. As per the condition defined in the following image (Monitoring solution: Linux; Monitor type: CPU; Metric: Utilization), when the CPU Utilization of all Linux computers whose agent host name matches the regular expression AM-.* and instance name matches the regular expression ^0.*, crosses the threshold of 75% for 15 minutes, a Major alarm will be generated. After the Utilization returns to a normal state of below 75% and a 15 minutes lapse, the generated alarm is automatically closed. The following example explains how the regular expressions defined in the following image helps you to filter the instances based on agent host names and instance names.
- Agent host names
  hostAM-34TEST1
  devAM-PRODSETtest
  M-PSRTESTserver
  As per the agent host name regular expression AM-.*, only 2 host names (hostAM-34TEST1 and devAM-PRODSETtest) are selected from the preceding list.
- Instance names
  04578
  0prodtest
  qa0test
  As per the instance name regular expression ^0.*, only 2 instance names (04578 and 0prodtest) are selected from the preceding list.
Example 4: As per the condition defined in the following image (Monitoring solution: Linux; Monitor type: CPU; Metric: Utilization), when the CPU Utilization of all Linux computers whose instance name starts with 'CPU1' crosses the threshold of 75% for a period of 15 minutes, and it violates the baseline determined for that event, a Major alarm will be generated. After the Utilization returns to a normal state of below 75% and a 15 minutes lapse, the generated alarm is automatically closed.
(Optional) Select Enable Policy.
You can enable or disable the policy any time from the Alarm Policies page.

Save the policy.

Can I create a new policy using an existing policy?

You can create a new policy quickly by copying an existing policy and editing the configurations. For more information, see copying an alarm policy.

To edit an alarm policy

On the Configuration > Alarm Policies page:

Do one of the following:
- Select the policy and click Edit.
- From the Actions menu of a policy, select Edit.
Change the configuration details provided while creating the policy and click Save.

To view conflicting alarm policies

To view a list of the alarm policies that have the same severity defined for the same metric, go to Configuration > Alarm Policies and click Check policy conflicts. A PDF file with the list of conflicting alarm policies is downloaded.

Important: For Chinese (Simplified), use web browsers to view the PDF

If your language is set to Chinese (Simplified), use web browsers such as Microsoft Edge, Mozilla Firefox, or Google Chrome to view the PDF. Do not use Adobe Acrobat Reader to view the PDF. You can also use any other PDF readers.

To copy an alarm policy

On the Configuration > Alarm Policies page:

Click the action menu of the policy that you want to copy and select Copy.
The Create Alarm Policy page is displayed with the configurations of the copied policy.
Modify the configurations according to your requirements to create a new policy quickly.
(Optional) Select Enable Policy.
You can enable or disable the policy any time from the Alarm Policies page.

Save the policy.

To view the list of alarm policies

On the Configuration > Alarm Policies page, view the list of alarm policies.

By default, the policies are sorted by Name. To sort on a different column, click the column heading.

To enable or disable an alarm policy

On the Configuration > Alarm Policies page, do one of the following:

Select the policy and click Enable or Disable.
From the Actions menu of a policy, select Enable or Disable.
Edit the policy and select or clear the Enable Policy check box.

What happens if I enable or disable a policy?

What happens when I disable an enabled policy?
All alarms that are created based on the policy are closed.
What happens when I enable a disabled policy?
New alarms are created based on the policy criteria with new event IDs.

To delete an alarm policy

On the Configuration > Alarm Policies page, do one of the following:

Select one or more policies and click Delete.
From the Actions menu of a policy, select Delete, and click Yes.

What happens to an open alarm?

In the policy post-trigger actions, if you have specified that the alarm must not be closed, such alarm events are automatically closed if they are not updated for more than the period specified in the retention policy. All closed events are automatically deleted from the system as per the retention policy. However, if you delete such a policy or the PATROL Agent associated with such an alarm is deleted, the alarm is automatically closed.

For more information about the retention policy, seeBMC Helix Operations Management service.

To view the audit trail of alarm policies

As a tenant administrator, you can use the BMC Helix Audit dashboard in BMC Helix Dashboards, to view the trail of all changes that were made to alarm policies. The BMC Helix Audit Dashboard provides the audit trail of alarm policies.

Scenario

Tina is a tenant administrator and Sarah is a system administrator at Apex Global. Tina has left on a vacation and she won't be back at work for two more weeks. Sarah has taken up some of Tina's responsibilities during this time. Sarah is looking at some alarm policies in the system and she wants to know when Tina created them and when they were updated. Because Tina is on vacation, how can Sarah obtain this information?

Sarah can log in to BMC Helix Dashboards and use the BMC Helix Audit Dashboard to see a complete audit trail of all alarm policies.

For more information, see Auditing user activities in BMC Helix Dashboards.

The following image displays the audit trail of alarm policies in the BMC Helix Portal Audit dashboard. Note that the selected resource type is Alarm Policy.