Synthetic metrics, rules, and events

TrueSight Synthetic Monitor generates events based on a system of metrics and metric rules. The metrics measure the results of a transaction. Metric rules use the metrics to define criteria for generating events on monitored transactions.

Global metric rules are included out of the box, and they apply to all transactions in all applications. You can define your own metric rules for an application, Execution Plan, transaction, and one or more locations.

Synthetic Monitor also includes a less-robust event generation mechanism using SLAs (see Service level agreement (SLA) definitions for more details).

This topic describes the following concepts for Synthetic Monitor:

Metrics

Metrics are measurements of a transaction. Synthetic Monitor includes two types of metrics:

  • Error metrics—Count the number of errors of a certain type that occurred during the transaction. Error types include Availability, Accuracy, and Execution. You cannot define additional error metrics.
  • Performance metrics—Measure latency of the entire transaction or the latency of a specific timer defined in the script.

The following table describes the four basic metrics that are available for all transactions.

Basic metrics

MetricDescription
Performance End-to-end latency of the transaction measured in seconds
Availability 

Number of availability errors in the transaction

Availability errors occur when the application does not respond, or when the application responds with an error that indicates it is unavailable. The number of Availability errors provides information regarding whether the application is running and whether it provides basic responsiveness to client requests.

Accuracy

Number of accuracy errors in the transaction

Accuracy errors are calculated with the assumption that monitored applications are working as designed and that the information transmitted to clients is correct. Functions that can be evaluated to determine accuracy include link checking, content validation, title validation, and response data verification. If a monitoring script contains customized functions that are used to ascertain system accuracy, those functions are factored in as well.

Execution

Number of execution errors in the transaction

An execution error indicates that there were problems with the script execution. It does not indicate anything about the health of the application.

In certain circumstances, Execution errors can happen intermittently. If this occurs, the TEA Agent attempts to run the Execution Plan again. If the retry is successful, no Execution error is reported.

If you have Execution errors configures to be counted as Availability errors (see Reclassifying Synthetic Monitor Execution errors and Accuracy errors as Availability errors for more details), this metric is irrelevant.

In addition to the basic metrics, you can measure page timers and custom timers that are defined in your scripts. Page timer and custom timer metrics are measured in seconds. The Execution Plan must have run at least once for the timers defined for that Execution Plan to be available as metrics. 


Metric rules

A metric rule is a set of definitions for generating events based on the results of monitored transactions. Based on events generated by the metric rules, notifications are issued, the status of the application is updated, and the events can be viewed on the Synthetic Health tab. 

Note

Status in the Application View tab is not updated based on synthetic metric rules. Events for synthetic transactions in the Application View are based on Service level agreement (SLA) definitions.

A synthetic metric rule defines the conditions for violations and for generating events:

  • A violation occurs when the value of a metric breaches the defined minor or critical threshold. 
  • An event is generated based on all the definitions in a synthetic metric rule.

Global metric rules are included out of the box, and they correspond to the basic metrics: Performance, Availability, Accuracy, and Execution. They apply to all transactions in all applications.

When you create or modify a metric rule, you define the elements that are displayed in the following image of the Create Synthetic Metric Rule page.

Create Synthetic Metric Rule page


The following sections describe the options and setting that make up synthetic metric rules:

Violation frequency

The violation frequency is the number of execution cycles that violate the thresholds out of a total number of execution cycles.

The violation frequency can help you control how frequently you want to generate an event for a set of violations. For information about execution cycles and violation frequency, see Execution cycles and violation frequency later in this topic.

Violation Frequency section of the synthetic metric rule definition


Application, Execution Plans, and transactions

You can define the scope of a synthetic metric rule by defining the application, Execution Plans, and transactions.

  • Application—You can define a metric rule for only one application.
    The out-of-the-box global metric rules (Performance, Availability, Accuracy, and Execution) apply to all applications.
  • Execution Plan and Transaction—For the selected application, you can define a metric rule to apply to one Execution Plan or any Execution Plan. Likewise, you can define a metric rule to apply to one transaction or any transaction.

    The following table describes the results you can expect from using the Any option or by selecting a specific item.

    Execution PlanTransactionResults
    AnyAnyA breach of the defined thresholds by any transaction of any Execution Plan generates an event.
    AnySpecificA breach of the defined thresholds by a specific transaction from any Execution Plan generates an event.
    SpecificAnyA breach of the defined thresholds by any transaction of the selected Execution Plan generates an event.
    SpecificSpecificA breach of the defined thresholds by the selected transaction of the selected Execution Plan generates an event.

Application, Execution Plan, and Transaction section of the synthetic metric rule definition


Metrics and thresholds

After you determine the scope of the synthetic metric rule, you select one or more metrics for the rule to monitor.

    • Metrics—You can select one or more metrics for each metric rule.
      If you selected a specific transaction, you can see the historic statistics for the metric to help you set realistic thresholds for the metric rule.
    • Thresholds—You can define a threshold based on a measured number of errors or a measured amount of time in a single transaction.
      For example, you can set a threshold for an Availabilty metric to 2 errors or for a Performance metric to 10 seconds. If a threshold is breached it is a violation.
      You can define two levels of thresholds: minor, critical, or both. Defined critical thresholds must always be higher than minor thresholds.

Metrics and Thresholds section of the synthetic metric rule definition



Synthetic Monitor continuously measures the results of your synthetic transactions and compiles statistics for each metric. You can use these statistics to set metric thresholds. Statistics are displayed only if a single transaction is selected.

  • Min and Max show the lowest and highest values measured for the metric.
  • Avg shows the average of all measured values for the metric.
  • Count shows the number of executions included in the statistics.

Metric statistics in the Synthetic Metric Rule page


Locations

You can select one or more specific locations, or select a percentage of locations.

If the specific locations have violations of the thresholds, or if more than the set percentage of locations have violations, an event is generated.

Notifications

You can determine who receives notification, when, and if they need any extra information.

Global metric rules

Global metric rules are included out of the box, and they correspond to the basic metrics: Performance, Availability, Accuracy, and Execution. They apply to all transactions in all applications.

You can modify the following default settings:

  • Status: Active
  • Violation frequency: 1 out of 1 cycles—that is, every time there is a violation, an event is generated.
  • Metric and threshold: Minor threshold is enabled. A Minor and Critical threshold value is provided for each rule:

    • Performance
      Minor: 2s
      Critical: 5s

    • Availability
      Minor: 1 error
      Critical: 2 errors

    • Accuracy
      Minor: 1 error
      Critical: 2 errors

    • Execution
      Minor: 1 error
      Critical: 2 errors

  • Notifications: Include Global Email Recipients List is selected

The following options for global metric rules cannot be changed:

  • Any application
  • Any Execution Plan
  • Any transaction
  • Any location

You cannot delete global metric rules.

Activating and deactivating synthetic metric rules

You can deactivate or activate any metric rule, including the global metric rules. Deactivating a metric rule means that no events are generated and no notifications are issued by the deactivated metric rule. If a metric rule is deactivated, and a more general single-metric rule covering the same metric is active, the more general metric rule takes effect. For example, if an Execution Plan-level Availability metric rule is disabled, an application-level metric rule may be applied instead.

Synthetic metric rule hierarchy

Global synthetic metric rules are general rules because they apply to any application, any Execution Plan, or any transaction.

You can define more specific rules, rules that apply to a single application, Execution Plan, or transaction. 

If there is an existing open event in the system (For example, event generated by a generic rule) when a specific rule is created for the same metric, the existing event is closed and new event will be created based on the newly evaluated specific rule.

A specific rule overrides a more general rule. In the following table, each rule is more specific than the one above it. 

Note

Locations are not considered a part of the synthetic metric rule hierarchy.

ApplicationExecution PlanTransaction
AnyAnyAny
App_aAnyAny
App_aEP_aAny
App_aEP_aTx_a

For a specific metric, only the last rule in the table is evaluated.

The following examples present a set of general and specific rules and show which rules are evaluated.

Example 1: Specific application rule to override general rule

RuleApplicationExecution PlanTransactionMetric
AAnyAnyAnyPerformance
BAnyAnyAnyAvailability
CAnyAnyAnyAccuracy
DAnyAnyAnyExecution
EApp_aAnyAnyPerformance

Result: Rule A is not evaluated for the Performance metric on App_a because Rule E overrides it. All other metrics, such as  Availability, Accuracy, Execution for App_a  are evaluated by the general rules B,C and D.

Example 2:Rule with multiple metrics using OR logical operator to override general rules

RuleApplicationExecution PlanTransactionMetric
AAnyAnyAnyPerformance
BAnyAnyAnyAvailability
CAnyAnyAnyAccuracy
DAnyAnyAnyExecution
EApp_aAnyAnyPerformance OR Availability

Result: Rules A and B are not evaluated for the Performance and Availability metrics on App_a because Rule E overrides them. All other metrics, such as  Accuracy and Execution for App_a  are evaluated by the general rules C and D.

Example 3:Rule with multiple metrics using AND logical operator to override general rules

RuleApplicationExecution PlanTransactionMetric
AAnyAnyAnyPerformance
BAnyAnyAnyAvailability
CAnyAnyAnyAccuracy
DAnyAnyAnyExecution
EApp_aAnyAnyPerformance AND Availability

Result: Rules A and B are not evaluated for the Performance and Availability metrics on App_a because Rule E overrides them. All other metrics, such as  Accuracy and Execution for App_a  are evaluated by the general rules C and D.

Example 4:Rule with multiple metrics using AND logical operator and a specific Execution Plan  to override general rules

RuleApplicationExecution PlanTransactionMetric
AAnyAnyAnyPerformance
BAnyAnyAnyAvailability
CAnyAnyAnyAccuracy
DAnyAnyAnyExecution
EApp_aAnyAnyPerformance AND Availability
FApp_aEP_aAnyPerformance

Result: Rule A is not evaluated for the Performance metric on App_a for EP_a because Rule F overrides it.

Rule B is not evaluated for the Availability metric on App_a because Rule E overrides it.

All other metrics, such as  Accuracy and Execution for App_a  are evaluated by the general rules C and D.

Example 5:Rule with multiple metrics and specific Execution Plans to override general rules

RuleApplicationExecution PlanTransactionMetric
AAnyAnyAnyPerformance
BAnyAnyAnyAvailability
CAnyAnyAnyAccuracy
DAnyAnyAnyExecution
EApp_aAnyAnyPerformance AND Availability
FApp_aEP_aAnyPerformance
GApp_aEP_aAnyAvailability

Result: Rules A, B, and E are not evaluated for Performance and Availability metrics on App_a for EP_a because Rules F and G override them. Rule E is evaluated for Performance and Availability metrics for other EPs on App_a. All other metrics, such as  Accuracy and Execution for App_a  are evaluated by the general rules C and D. 

Example 6: Rule with multiple metrics using AND logical operator with specific Execution Plan to override general rules

RuleApplicationExecution PlanTransactionMetric
AAnyAnyAnyPerformance
BAnyAnyAnyAvailability
CAnyAnyAnyAccuracy
DAnyAnyAnyExecution
EApp_aAnyAnyPerformance AND Availability
FApp_aEP_aAnyPerformance
GApp_aEP_aAnyPerformance AND Availability

Result: Rule A is not evaluated for the Performance metric on App_a for EP_a because Rule F overrides it.

Rule B is not evaluated for the Availability metric on App_a for EP_a because Rule G overrides it.

Rule E is not evaluated for the Performance and Availability metrics on App_a for EP_a because Rules F and G override it.

All other metrics, such as  Accuracy and Execution for App_a  are evaluated by the general rules C and D.

Example 7:Rule with multiple metrics and specific Execution Plans and Transaction to override general rules

RuleApplicationExecution PlanTransactionMetric
AAnyAnyAnyPerformance
BAnyAnyAnyAvailability
CAnyAnyAnyAccuracy
DAnyAnyAnyExecution
EApp_aAnyAnyPerformance AND Availability
FApp_aEP_aAnyPerformance
GApp_aEP_aTx_a

Performance OR Availability OR Tx_a

Result: Rules A, B, E, and F are not evaluated for App_a, EP_a, and Tx_a because rule G overrides it. All other metrics, such as  Accuracy and Execution for App_a  are evaluated by the general rules C and D.

Execution cycles and violation frequency

In a synthetic metric rule the violation frequency helps you determine how often you want to generate an event. When the defined metrics breach the thresholds in the defined location or set of locations, do you want to generate an event every time it happens? Or do you want to generate an event only if it happens often? How often? In the violation frequency you can define how often by using execution cycles. An execution cycle comprises a single run of an Execution Plan in all of its defined locations.

The following diagram shows several execution cycles. In each cycle, the Execution Plan runs on all locations. Some of the locations are impacted, that is, some metrics breached the defined thresholds.

Execution cycles

When you set the violation frequency, you enter the total number of consecutive execution cycles that you want to evaluate (up to 60), and out of that number, how many of the execution cycles need to be impacted for an event to be generated.

An event is generated if the number of Impacted Execution Cycles is reached within the number of Total Execution Cycles.

Examples

  • Impacted Execution Cycles = 1 and Total Execution Cycles = 1: An event is generated every time an execution cycle breaches the defined thresholds.

  • Impacted Execution Cycles = 3 and Total Execution Cycles = 3: An event is generated if three execution cycles in a row breach the defined thresholds.
  • Impacted Execution Cycles = 3 and Total Execution Cycles = 6: An event is generated if three out of six execution cycles breach the defined threshold.

What is the difference between an event and a violation?

A violation occurs when the value of a metric breaches the defined minor or critical threshold (as defined in the rule).

An event is generated based on all the definitions in a synthetic metric rule:

  • The metrics breached the defined threshold (according to the AND or OR logical operator).
  • The breaches occurred over the defined number of locations.
  • The breaches occurred at the defined violation frequency.


Service level agreement (SLA) definitions

Synthetic Monitor includes SLA definitions that can generate events based on percentages of executions that have errors.

  • SLAs generate events for the basic metrics: Performance, Availability, Accuracy, and Execution. SLAs cannot be defined for page timers or custom timers. 
  • SLAs generate events based on a percentage of executions that have a particular kind of error within the defined time frame. SLAs do not generate events based on a single execution or a specific number of executions.
  • SLAs can be defined at the application level, and the Execution Plan level only. 
  • No historic data is stored based on the SLAs. 
  • SLAs do not include the ability to generate an event for consecutive occurrences of a threshold breach.
Was this page helpful? Yes No Submitting... Thank you

Comments