Managing global thresholds on the TrueSight console


From the Global Thresholds page in the TrueSight console, you can view and manage thresholds for each attribute of every monitor type listed under the TrueSight Infrastructure Management Server. For details about understanding and enabling global thresholds in your view, see the Enabling and accessing global thresholds on TrueSight console.

Best practices


Always make global threshold changes from the TrueSight console for the following reasons:

  • The changes made at the TrueSight console are propagated to all TrueSight Infrastructure Management Server instances in your tenant.
  • Global threshold changes done at the TrueSight Infrastructure Management Server-level are only applied locally and are not propagated to other Infrastructure Management servers.
  • Restarting a TrueSight Infrastructure Management Server after making the global threshold changes locally causes global thresholds from the TrueSight Presentation Server to be reapplied to the Infrastructure Management Server local changes done at the TrueSight Infrastructure Management Server  before a restart are overridden.

In addition, before configuring the global thresholds settings, ensure that the monitor type exists in the TrueSight Infrastructure Manager. Suppose if it does not exist at the time of configuring global thresholds and you added the monitor type later, then either restart the TrueSight Infrastructure Management Server or edit and save the global threshold settings for the monitor type again from the TrueSight console for the settings to be applied.


Notes

  • BMC recommends that you use the TrueSight console to manage and view these threshold configurations. Do not use the operator console to set thresholds.
  • When you configure global thresholds in the * tenant, the thresholds are applied to all the TrueSight Infrastructure Management Servers across all the tenants. However, if you configure the thresholds in a non-* tenant, they are applied only on the TrueSight Infrastructure Management Server added to that particular tenant. The thresholds set by a non-* tenant admin takes precedence over the thresholds set by a * admin and applies for that particular tenant. When a non-* tenant admin deletes  thresholds configured at the tenant-level, the corresponding thresholds configured in the * tenant, if present, are inherited. This inheritance is indicated by an additional information icon on the TrueSight console, as shown in this figure.

Managing absolute, signature, and abnormality thresholds

You can manage the three types of global thresholds:

TypeDescription
Absolute thresholds

Simple and static set of thresholds when configured without baselines. Absolute thresholds are better suited for attributes that indicate a change in status. For example, availability of a host is 0% may indicate a problem with either the host or the network that connects to it.

Absolute thresholds can also be configured with baselines. In this case, the data value has to cross both the threshold value and breach the specified baseline for an event to be generated

Signature thresholds

Dynamic thresholds generated based on a seasonal behaviour. A set of low and high baselines are automatically generated and therefore, users do not need to set a threshold value manually. Because of this, it is a much more scalable approach to managing thresholds.

Abnormality thresholds

Work the same way as the Signature thresholds. However, instead of regular events, abnormality events are generated. The abnormality events are utilized for Probable Cause Analysis. Abnormality thresholds are automatically set (out-of-the-box) on all KPI attributes.

For additional details, see Server thresholds, KPIs, and baselines.

To manage global thresholds

  1. Log on to the TrueSight console.
  2. In the navigation pane, click Configuration and select Global Thresholds.
    The Global Thresholds page is displayed. By default, All is selected on the solutions tree pane.
  3. Select a solution from the tree pane to view additional details.
  4. Expand to view the monitor type from the table.
  5. Select an attribute under the monitor type in the attribute table to manage its global threshold settings.
    By default, only the Basic view containing the Absolute Thresholds section is displayed in the thresholds pane.
  6. Add or edit absolute thresholds.
  7. (Optional) From the Thresholds section, click View and select Advanced to view additional threshold settings.
  8. (Optional) Add or edit signature thresholds.
  9. (Optional) Add or edit abnormalities thresholds.


To add absolute, signature, or abnormality thresholds

  1. Perform the appropriate steps in To manage global thresholds:
    - For absolute thresholds, perform steps 1 through 5.
    - For signature and abnormality thresholds, perform steps 1 through 7.

  2. Select the Enable check box to enable the specific type of thresholds displayed or select any one of the threshold settings. 
     icon appears next to the check box.
  3. Specify the desired threshold values in the other fields to complete the configuration.
  4. (Optional) Select the Generate Predictive Event check box to cause the predictive events to be generated.
  5. (Optional) Set a value for the Start Predicting At field. This value is represented in percentage with minimum and maximum limits set to 20% and 80 % respective. To know more about this, see Basic and Advanced thresholds.

    You can set a value for the Start Predicting At field only if the Generate Predictive Event check box is selected.

  6. (Optional) Clear the Auto Close Event check box to keep an event open after the monitored attribute returns to normal levels. By default Auto Close Event is selected.
  7. Click Save.


To edit absolute, signature, or abnormality thresholds

  1. Perform the appropriate steps in To manage global thresholds:
    - For absolute thresholds, perform steps 1 through 5.
    - For signature and abnormality thresholds, perform steps 1 through 7.

  2. Select the Enable check box to enable the specific type of thresholds displayed or select any one of the threshold settings. 
     icon appears next to the check box.

  3. Specify the desired threshold values in the other fields to complete the configuration.
  4. Click Save.


To delete absolute, signature, or abnormality thresholds

  1. Perform the appropriate steps in To manage global thresholds:
    - For absolute thresholds, perform steps 1 through 5.
    - For signature and abnormality thresholds, perform steps 1 through 7.

  2. Clear the Enable check box to disable the specific type of thresholds displayed. A  icon appears next to the check box.
  3. Click Save.

Basic and Advanced thresholds

If you select the basic option, only the absolute threshold section is displayed. For the advanced option, all threshold sections are displayed. Use the following tables to understand the various fields:


Absolute thresholds are static thresholds set by an administrator. You can specify that a threshold breach occurs when the data value is greater than, less than, greater than or equal to, less than or equal to, or equal to the specified threshold value. When an absolute threshold is breached, it generates an event of type Critical, Major, or Minor based on the specified severity for the configured threshold. You can also enable an absolute threshold with any of the baseline types. Having the baseline in the trigger condition allows a threshold to generate events based on learned behavior.

For example, you can enable an absolute threshold to use the Weekly baseline. An absolute threshold with this configuration would not generate an event until the Weekly baseline had been breached in addition to the absolute threshold value being breached. Combining the absolute threshold with baselines reduces the number of events received by the Infrastructure Management Server. If the absolute threshold is not enabled with a baseline, then an event is generated whenever the threshold value is breached, regardless of the baseline.

Field name Usage
Enable Allows you to set the threshold value for the selected attribute. You must have selected an attribute from the table view.
Severity Severity of the generated event: Minor, Major, or Critical.
Duration (min) Specify the duration, in minutes, of delay from when the threshold is breached to when an event is generated. The same duration is required to close an event, starting with the first data point where the threshold is no longer breached.
Comparison Select a reference point for comparison of the actual measurements with the defined threshold values and is available only for the Absolute Thresholds settings. The supported operators for comparison are ><==>=, and <=. For example, if you select the >= operator for an attribute and set the threshold value to 90%, events are generated when the threshold value is greater than or equal to 90%.
Threshold Value

Specify a threshold value for the data. Data values that breach the threshold value specified can cause an event to be generated.

Outside Baseline Type

Specify the baseline to associate with the threshold. Setting an appropriate baseline helps to reduce false events associated with latest trends.

The following baseline definitions are available:

  • Auto Baseline - The associated Infrastructure Management Server determines the best baseline type for the selected attribute.Selecting this baseline allows the Infrastructure Management server to automatically detect the abnormality in each KPI monitor type attribute.
  • Hourly Baseline - Each hour of the day has a high or low value that is tracked, and this pattern is repeated for each day. An hourly baseline is initialized after 24 hours of data collection has occurred.
  • Daily Baseline - A high or low value is derived from the moving average of each consecutive day. This high or low range is taken from a larger number of data values and, consequently, covers a wider range than the hourly baseline. A daily baseline is initialized after 24 hours of data collection has occurred.
  • Weekly Baseline - This baseline is calculated daily from Monday to Friday. All these days share the same 24-hour baseline. A weekly baseline is initialized after the monitor instance is created and 168 hours of data collection has occurred. 
  • Hourly and Daily Baseline - This baseline is a combination of Hourly and Daily Baselines.
  • All Baselines - This option combines all of the above baselines.
    For details, see Types of thresholds, KPIs, and baselines.

Advantages of Absolute thresholds with outside baseline enabled:

Absolute thresholds with outside baseline are thresholds which combine both absolute thresholds (static threshold value) and dynamic thresholds (which use the baseline) as the threshold. The user can still set the threshold value  with but as additional criteria for alarm generation.

  • Reduce the number of false alarms when compared with using static threshold. 
  • Absolute thresholds with outside baseline allows more control than signature thresholds. 
  • Absolutes with no baselines are for the availability metrics, or other metrics that have discrete, state based values; that is, 0=OK, 1=Down, 2=Admin Down.
  • Absolutes with baselines are for user influenced performance metrics with normalized values (0-100) or with set definitions of values for CPU utilization, memory utilization, process utilization, and so on.  


Generate Predictive Event

Select this check box to generate predictive events or clear this check box to stop generating predictive events.

Start Predicting At

Enabling/Disabling Start Prediction At %

By default this column is not enabled. If you want to enable this, do the following steps:

  1. From the Presentation Server command line, run the following commands:
    tssh properties set show.prediction true
    tssh properties reload
  2. Refresh the Global Thresholds view to see this column enabled.

You can perform the same steps to disable by setting the show.prediction to false.

This value represents the cut-off limit for the monitored value at which the system should start generating the predictive events. Let us consider the following example to understand this better:

Problem statement: For monitoring CPU load, you want to set the absolute threshold value as 80 for generating a critical event, and you want to generate a predictive event when the CPU load reaches 80% of the absolute threshold value set.

The following screen shot represents the absolute threshold setting feature to solve the preceding problem:


80 % of 80 is 64.

When the monitored value reaches 64, the system generates a predictive event to alert the user that the monitored value has reached 80% of the absolute threshold value set. Seeing a predictive event, the user can take any corrective action to avoid the critical event to be generated.

Auto Close Event

Clear this check box to prevent the event from getting closed automatically, when the data value no longer breaches the specified threshold value. This option is useful for events that have a short duration. This option is enabled by default.


Signature thresholds are dynamic thresholds that an administrator can set that use the baseline data collected by Infrastructure Management Severs for a KPI attribute as the threshold value. The baseline itself forms the signature threshold. Because signature thresholds use the learned behavior of the service or device, the signature thresholds change as the service or device attributes change. The longer a signature threshold is in place, the more accurate it becomes.

When a signature threshold is breached, it sends an event to the console. This event indicates the severity of the threshold breach. An administrator can define the severity levels as Critical, Major, or Minor when the threshold is set. Use signature thresholds if you want to signify that this attribute is so important that if it breaches the baseline that you want to generate an event that is not informational. Signature thresholds are useful for performance attributes such as response time, utilization, and errors.

Due to the non-deterministic nature of the performance data, it might take longer than anticipated for a Signature threshold to generate an event.

Note

The difference between Absolute and Signature thresholds is in the percentage of data points that must meet the trigger condition. 100% for Absolute thresholds and just enough for Signature thresholds.


Signature threshold for performance metrics that have no set concept of value is completely dependent on the attribute and the instances (transaction response time, Ping Response time, etc) being monitored.

Example

For example, the ping response time is 900ms. The Signature threshold is considered poor if this response happens in a setup within the same data center having a gigabit switch, but it is considered good if it happens in a setup that extends across the continent.

 

Abnormality thresholds— are high or low thresholds determined by the specified baseline data collected by Infrastructure Management Server for an attribute. Abnormality thresholds are enabled by default for Key Performance Indicator (KPI) attributes. Only abnormality thresholds are enabled by default. All other thresholds must be created manually. If an abnormality threshold is breached, it always generates an event with a severity of Informational type to the console.

For baselines to be generated for an attribute, that attribute must have an active abnormality threshold. An active abnormality threshold means that the threshold exists and is not suppressed. Additionally, the Key Performance Indicator (KPI) mode must be active. The baselines are generated only for those attributes that have an active abnormality threshold and are also KPI attributes. Neither the Absolute thresholds with outside baseline nor the Signature thresholds satisfy these requirements.

Note

 Abnormality thresholds are used only for a probable cause analysis (PCA).

Field name Usage
Enable Allows you to set the threshold value for the selected attribute. You must have selected an attribute from the table view.
Severity

Severity of the generated event, one each for an above and below baselines: Information, Minor, Major, or Critical.

Note

For the abnormality thresholds, only the informational severity type is available. For the signature thresholds, the informational severity type is not available.

Duration (min) Specify the duration, in minutes, of delay from when the threshold is breached to when an event is generated. The same duration is required to close an event, starting with the first data point where the threshold is no longer breached  

Above/Below

A reference point for the following parameters:

  • Above - Above the high baseline
  • Below - Below the low baseline
Outside Baseline Type

 Specify the baseline to associate with the threshold. Setting an appropriate baseline helps to reduce false events associated with latest trends.

The following baseline definitions are available:

  • Auto Baseline - The associated Infrastructure Management Server determines the best baseline type for the selected attribute. Selecting this baseline allows the Infrastructure Management server to automatically detect the abnormality in each KPI monitor type attribute.
  • Hourly Baseline - Each hour of the day has a high or low value that is tracked, and this pattern is repeated for each day. An hourly baseline is initialized after 24 hours of data collection has occurred.
  • Daily Baseline - A high or low value is derived from the moving average of each consecutive day. This high or low range is taken from a larger number of data values and, consequently, covers a wider range than the hourly baseline. A daily baseline is initialized after 24 hours of data collection has occurred.
  • Weekly Baseline - This baseline is calculated daily from Monday to Friday. All these days share the same 24-hour baseline. A weekly baseline is initialized after the monitor instance is created and 168 hours of data collection has occurred. 
  • Hourly and Daily Baseline - This baseline is a combination of Hourly and Daily Baselines.
  • All Baselines - This option combines all of the above baselines.
    For details, see Types of thresholds, KPIs, and baselines.
Threshold Value

Specify a threshold value for the data. Data values that breach the threshold value specified can cause an event to be generated.

Specify this value if you do not want an event to be triggered on trivial conditions. For example, specify a higher threshold value so that thresholds are triggered only if data values are higher else an event is triggered for every trivial condition.

Minimum Sampling Window (min)

Define the minimum amount of time or data points (depending on the attribute) required before an event is generated on a threshold breach. These data points are considered for an intelligent threshold computation and the data points outside this range are usually discarded.

At least five data points are required before an event is generated for any threshold type. The TrueSight server waits until at least for five data points are collected before generating an event, regardless of the value entered into the sampling window. For example, if you set a minimum sampling window to 10 minutes on a specific monitor attribute but the polling rate of that monitor is 5 minutes, 25 minutes must pass before an abnormality is generated. For this reason, BMC recommends using lower polling rates for monitors.

Note

Abnormalities are closed automatically when the number of data points exceeded in the last minimum sampling window size is not considered significant. For example, if six out of seven data points out of range are statistically significant, then as soon as the last exceeded points drop to five out of seven points, the abnormality is closed. By default, even if no explicit global signature threshold is set, the Infrastructure Management Server generates abnormalities for the above baseline conditions.


Absolute Deviation

Defining the deviation is a way of introducing tolerance into the baseline values. If there is a small change in the baseline that is not significant for reporting purposes, the deviation can be set to ignore this small change. The deviation value is additive (not multiplicative) and is given in the same units being measured by the particular monitor attribute (that is #,%, bytes, milliseconds, and so on). The default value for absolute deviation is five units.

For example, if a baseline-low value is set to 40, and a baseline-high value is set to 100, and the standard deviation is 2, a signature threshold event is generated if the data values go above 102 or below 38.

Percent Deviation (%) The percent deviation has the same meaning as the absolute deviation, except that the actual value of the deviation is expressed in percentage of the baseline value rather than an absolute value. The default value for percentage deviation is 5%.


Was this page helpful? Yes No Submitting... Thank you

Comments

  1. Peng Li

    Start Predicting At field not found. 

    Feb 28, 2019 06:12