Data collected by ETL tasks is not available to the analysis module before it is processed by the warehousing engine.
Data flow reports allow you to keep TrueSight Capacity Optimization imports under control, as the number of rows processed per day is a good indicator of the system's health.
When performing historical imports (for example, if you are planning to bulk load more than two million rows), it is strongly recommended to split the data in smaller chunks, limiting the amount of information processed at one time. This prevents congestion in the Near-Real-Time Warehouse engine.
The data warehouse controls the following activities:
The TrueSight Capacity Optimization Data Warehouse houses the following types of data:
A time series is a sequence of samples or statistics for a certain measurement, each corresponding to a point in time. The TrueSight Capacity Optimization DWH contains both time series samples and statistics, aggregated at different time resolution (hour, day, month). All time series are associated with a measured object, described according to a reference model. The following figure provides a basic illustration of the reference model for measured objects.
The model comprises of the following components:
Each available object/metric has a standard name which adheres to the naming convention defined in the datasets. For a complete listing, refer to TrueSight Capacity Optimization ETL Development Studio (also see Developing custom ETLs), or to the Viewing datasets and metrics by dataset and ETL module page in the Data Warehousing menu of the Administration section.
Each metric has a type which defines the measure unit and how statistics about that data should be collected. A complete listing is as follows:
Generic counter, absolute value
A count of events, absolute number
A frequency, in events/sec
Positive accumulation counter
Configuration data (string)
Negative accumulation counter
Elapsed time, in seconds
Percentage counter, weighted
Peak Percentage counter
Peak frequency, in events/sec
Difference between subsequent samples
Generic counter, absolute value, weighted
Measurement units and formats also use common standards:
from 0 to 1
TrueSight Capacity Optimization uses ETL tasks to import data that is collected by third party applications or logged by the monitored entities themselves. This data, from a TrueSight Capacity Optimization perspective, is persistent because another application takes care of its collection and storage. In a second phase, TrueSight Capacity Optimization accesses this data and imports it into its own data warehouse.
Standard data imported by TrueSight Capacity Optimization is always in the form of a time series, which means that it is described by a value (numeric or textual, for configuration properties) which changes over time.
The Data Mover components are used by TrueSight Capacity Optimization to deal with data which does not respect the two aforementioned properties. The Data Mover is able to access data which is not a time series and store it into TrueSight Capacity Optimization.
The workflow of these components is displayed in the following image.