From the Administration tab, click Data Warehouse > Aging configuration. This page allows you to set some important parameters that affect both the efficiency and the disk space needed by the data warehouse. For each table that stores BMC TrueSight Capacity Optimization metrics, you can set how long the data should be kept and whether to partition the table into smaller chunks if needed.
Use the Data Warehouse aging state option to enable or disable aging configuration for the Data Warehouse.
Click Advanced aging configuration to view a page that displays a summary of the aging and partitioning configuration for each data table, including:
Example of the Custom tables aging configuration table
Click on the name of a table to view the detail page for that configuration, where you can edit or delete its settings.
Example of detail page for the SYS_CONF_STAGE table
The Aging parameter defines the number of days that the specific metric value will be kept in the data table, using its timestamp as starting date. It is important to properly tune this parameter, as it can strongly influence the space occupied in each tablespace and the availability of the data for analysis purposes. Following type of tables are available in the aging configuration:
*_CONF_STAGE: these tables are used to temporarily load the ETL data until the data warehouse copies it to the DETAIL tables
*_DATA_DETAIL: these tables are used to store more detailed resolutions
*_DATA_DETAIL_SPLIT: these are internal tables used by near real time warehouse and are not used in analyses, models, etc.
*_DATA_DH: these tables are used to store data at hour resolution
*_DATA_D: these tables are used to store data at day resolution
*_DATA_MDCH: these tables are used to store data for month at day resolution
*_DATA_STAT: these tables are used to store system statistics
For example, let's consider the table SYS_DATA_DETAIL. It holds the system metrics collected at the maximum detail available, given that the duration of the samples is greater or equal to 5 minutes. According to your needs, you can decide to reduce or increase the aging parameter value for this table:
The Aging parameter also regulates the data retention period of BMC TrueSight Capacity Optimization. As data is automatically aggregated at different time levels, when the highest level data (usually, WKLD_DATA_MDCH and SYS_DATA_MDCH) becomes old (i.e. its aging period expires) it is deleted and all information about that period of time is lost.
Moreover, the Aging parameter directly affects the maximum age of the data that can be imported in BMC TrueSight Capacity Optimization, i.e. the latency. The latency is specific to the entity and is automatically set equal to the Aging parameter values of the SYS_DATA_DETAIL and WKLD_DATA_DETAIL tables.
The latency setting limits the maximum age of data imported by an ETL task: if it is older than the aging policies of the SYS_DATA_DETAIL and WKLD_DATA_DETAIL tables it will be discarded by the warehousing engine and thus become useless.
For example, if the latency is set to six months, the warehousing process only takes into account the data present in a stage table whose timestamp is not older than six months. If the data loaded by an ETL task refers to metrics collected before this date, they will been ignored by the warehousing engine and will not be reported.
These settings are necessary because the warehousing process has to deal with a great volume of summarizations; establishing limits prevents the process from becoming unmanageable.
Another very important parameter that can be set in the Aging Configuration page is the partition period, which allows you to improve the performances of the BMC TrueSight Capacity Optimization DWH by using table partitioning. The partition period is expressed in days and defines the time span of each chunk in which the tables will be partitioned.
To tune this parameter you must consider the specific needs of your BMC TrueSight Capacity Optimization deployment, because this setting speeds up the execution of SQL instructions by limiting it to one partition at a time. Partitions allow smaller indices to query, update and drop table rows. This feature is totally transparent to BMC TrueSight Capacity Optimization users, and a good tuning of the parameter for each table can lead to major speed-ups.
For example, if you want to have a data retention period of three months but your users frequently perform analyses that are based on data from the last two weeks, you could set the partition period to 15 days. This way, most of the queries will be executed only on the last partition, which is much smaller than the whole table.
Click Add configuration to define a new aging configuration. The Add aging configuration form has the following fields:
Name of the database table for which you want to set an aging configuration
Name of the database column that contains the data timestamp
The number of days/weeks/months that should pass before the data is considered old and removed
Select this option if you want to partition this table. Selecting this option has the following effect:
A drop-down menu that lets you select the time span for the partition chunk, e.g. year, semester, month, etc.
The name of the tablespace to which the table belongs
How many days in advance the partition should be created
Additional filtering conditions for data deletion that can be used only with unpartitioned tables. For example, in the case of stage tables aging, you can express the additional condition and status>0 to avoid deletion of data not yet processed by the warehousing engine.