History data

The TrueSight Middleware and Transaction Monitor (TMTM) history database stores raw (high-resolution) and summarized (low-resolution) values of attributes, where they are available for querying and generating reports. At 30-minute intervals, the History Service compresses the raw data into summarized data, in accordance with the associated history rule.

TMTM provides a History Template for each Physical Type that provides a list of the attributes associated with that type. In this template, you can select the attributes for which you want history.

As with Event Templates, History Templates are defined for each Physical Type. You then associate each instance with one or more templates for that type. The History Service then subscribes to all TMTM topics checked in the associated templates.

The term "summarization" is the same as "data compression" and "Lo Res." The following terms can also be used (they are used interchangeably throughout these sections): Summarized Data vs Compressed Data vs Lo Res Data.

Similarly, "Hi Res" and "Raw" are the same.

Related topics

Architecture

Defining data rules and templates

Data history fields, parameters, rules and resolution

Database performance recommendations

Accessing the Monitor Console

Defining configuration options in services.cfg

Note

History applies only to Physical Types since Logical Types are collections of physical artifacts and history on the logical would be redundant.

Data compression and history rules

History rules, or data-compression rules, are created for the numerical properties of objects in the physical tree of the Monitor Console.

Example

CurrentGetRate is a property of a queue. Because it is a numeric property, the CurrentGetRate property can be used in a history rule definition.

Compression parameters are the building blocks of the history rule and are accessible for each monitored object property in the History tab view pane.

History rules and templates

A template is a rule holder that contains one rule for attachment to one or more monitored objects.

An object can only be associated with one history template. Many objects can be associated with that template which makes them reusable. Changes to the template are applied to all object associations.

Data resolution

Data resolution is a measure of the number of reported samples taken of the monitored data per unit time. The higher the data resolution, the more storage resources are used and the greater the network load. Resolution of reported data is set by the sample interval property.

Accessing history data

Application Specialists and Administrators access history data from the Monitor Console, and the Monitor Console can obtain history data from the following sources:

TMTM Topic Service

This service provides high-resolution data monitored in real time at the agent. This data is fed to the Monitor Console in real time and stored by the TMTM History Service.
TMTM History Service

This service accepts the high-resolution data from the TMTM Topic Service and stores it as high-resolution and low-resolution data. Stored data is available to the Monitor Console when required for populating reports and charts. High-resolution data is used in trend charts; low-resolution data is used in reports and charts.

Flow of history data

High resolution data is collected at the agent. The TMTM Topic Service passes monitored data on a publish and subscribe basis to the Monitor Console. It also passes the high-resolution data to the TMTM History Service. The TMTM History Service stores the high-resolution data and low-resolution data (summarized from the high-resolution data using history rules) in the database. historyDataFlow

Note

Recording high-resolution data for an object requires that object have a history rule, even though the history rule only specifies which attributes are summarized and does not otherwise affect the high-resolution data.

High-resolution data is stored in the database as it is sampled. Low-resolution data is compressed from the high-resolution data and stored at the same time. If the database stops accepting data, both high- and low-resolution storage stop.

Storage duration of history data

Raw data is stored for 40 days. The 40-day setting is configurable by adjusting the amount_of_hires_to_keep parameter in the services.cfg file.

Summarized data is stored indefinitely. The setting is configurable by adjusting the amount_of_lowres_to_keep parameter in the services.cfg file.

Issues that can affect properties of data

This section describes certain issues that might or might not require your attention.

Time-zone differences: The time stamps on monitored information are set by TMTM Services.

TMTM stores history data using the local time of the TMTM History Service, which affects your reports during daylight saving time changes.
If any of your TMTM components (services, agents, or clients) exist in different time zones, time differences can cause short- and long-term history to appear in an untimely manner, or reports to appear inaccurate. See Accounting for time-zone differences.

History count and total: The CurrentMsgs field is reset every time a channel disconnects, whereas the MsgsTotal field continues to maintain data until it is reset automatically at midnight (by default). This can be configured.
Total resets: Attribute totals are collected at set intervals and are reset (by default) at midnight local machine time every night on the agent computer.