This documentation supports an earlier version of BMC Helix Operations Management. Use the Product version picker to select and view the latest version of documentation.

Event deduplication and suppression for filtering unwanted events

Incoming events are processed through a set of out-of-the-box deduplication rules to determine whether the incoming event is a duplicate event or a new event. If it is a new event, it is ingested and displayed on the Events page. However, if the event is a duplicate, it is are deduplicated and dropped. 

As an administrator, you can suppress duplicate events based on custom event selection criteria by configuring a suppression policy as part of an advanced enrichment policy.

Event suppression and event deduplication are performed together in the same stage of event processing. Event deduplication and suppression can help you filter out unwanted and unnecessary events.


Related topics

Event-correlation-for-aggregating-related-events

Managing maintenance windows with blackout policies 

Event-policy-types-and-evaluation-order

Out-of-the-box-event-policies-and-templates

Out-of-the-box event deduplication

During deduplication, events are consolidated or folded into a single event representation based on the first occurred event. 

As new or incoming events are received, they are checked against existing events based on deduplication slot values (for example, source address, alarm ID, and so on). If the incoming event has the same dedup values as an existing event, the incoming event is identified as a duplicate. When an event is identified as a duplicate, new information from that event is used to update the existing event and the new event is dropped. Dropped events are not ingested and therefore not available on the Events page. The product comes with a set of out-of-the-box dedup slots that form the default criteria for event deduplication. For more information about dedup slots, see Slot-facets.

For example, suppose an alarm event occurs at 9:00 A.M. Another alarm event occurs at 9:10 A.M. with same alarm ID (deduplication slot) as the first alarm event, but a different severity.

The following steps describe the deduplication process:

  1. The first occurred alarm event (9:00 A.M. event) is automatically updated with these details:
    • New severity: The first occurred event is updated with the severity of the subsequent event (9:10 A.M. event). 
    • Updated repeat count: The Repeat count in the first occurred event is updated with +1. The repeat count refers to the number of occurrences of similar events. 
    • Updated modified time: The first occurred event is updated with the time when it was updated.
  2. The subsequent event (9:10 A.M. event) is permanently dropped. 

Important

Deduplication occurs only for out-of-the-box event classes. If you want to perform event deduplication for custom classes, you can configure an advanced enrichment policy by:

  1. Performing a Lookup on deduplicate events criteria
  2. Using the DropNewEvent function.

For more information, see details of the Use case: Drop duplicate events and update the existing event with new severity at Examples: Event policies for enrichment, correlation, notification, and suppression.

A small set of dedup slots are defined for particular event classes. However, you can define custom dedup slots in existing classes by using the events/classes API endpoint. For more information, see Managing-events-with-REST-APIs. 


Event suppression based on the event selection criteria

If you want to drop events based on a custom event selection criteria, you need to configure a suppression policy. In a suppression policy, the event selection criteria determines which events are selected for suppression. The selected events are permanently dropped. Dropped events are not ingested and therefore not available on the Events page. Unlike event deduplication, the first occurred event is not updated with the new details of a duplicate event.

For example, suppose you create a suppression policy with the event selection criteria: Class="EVENT" AND Message=page not found

All the events with the message, "page not found", are directly dropped. Note that suppression is applied on incoming events only. Existing events that are already ingested cannot be suppressed by using a suppression policy. 

Event suppression can be very useful for known scenarios of unnecessary event occurrence. 

For example, based on your past experience, suppose you know that events containing a certain message are unnecessary and of no value. You create a suppression policy and provide an event selection criteria to select events containing that specific message. In this scenario, say a new event matching the desired criteria occurs. After three minutes another event occurs; then another after 30 minutes, and again another event occurs the next day, and so on. The suppression policy will keep dropping these events until you disable or delete the policy. 


Event suppression by using advanced enrichment policy

While creating an advanced enrichment policy, you can build a policy workflow based on complex conditions and add a void function to drop new events. For more information, see Functions-for-advanced-and-time-based-enrichment.

Similar to event deduplication and event suppression, dropped events are not ingested and therefore not available on the Events page. 

An advanced enrichment policy works based on the following criteria:

  1. Event selection criteria that determines which events to select for processing (same as suppression policy)
  2. Complex conditions defined in the policy workflow

This capability allows you to add a more granular condition to process events even after defining an event selection criteria. Also, this capability can be very useful for isolated use cases where you want to quickly drop specific events. For more information, see Example-Drop-duplicate-events-and-update-the-existing-event-with-new-severity.

The void function processing happens in the phase of advanced enrichment processing. Event processing happens in phases based on built-in rules. Based on these rules, the policies are evaluated in a certain order. For more information, see Event-policy-types-and-evaluation-order.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*