Viewing the status of the data warehouse

Use the Status page to view the processing status and the volume of data processed by the ETLs. The status includes the following details:

  • Processing status, that includes, the message queue size, message queue age, relations queue size, and relations queue age
  • Volume of data processed

You can use this information to understand how much data is still to be processed and the approximate time required to process the data in the queue. 

To access the Status page, click Administration Data Warehouse > Status.

The following table describes the Status page sections:

SectionDescription
Processing

Indicates the message queue size, message queue age, relations queue size, and relation queue age.

  • Timeseries message ageThe approximate time required to process the earlier timeseries data.
  • Timeseries message queue: The number of messages in the queue for timeseries, that is, the amount of data loaded by the ETL that is yet to be processed. The ETL loads data, that is, packages of information in batches. Each package is known as a message.
  • Relations message age: The approximate time required to process the earlier relations data.
  • Relations message queue: The number of relation messages that need to be processed. One relation message can contain up to 500 lines of information. The connection between a system and it's child entity in the Workspace is defined as a relation.
    For example, if you see two messages in this queue, it could mean that 1000 relations are waiting to be processed.
InformationStatusIndicates
Timeseries message ageGreenThe total processing time is less than 8 hours.
YellowThe total processing time is between 8 to 24 hours.
RedThe total processing time is more than 24 hours.
Relations message ageGreenThe total processing time is less than 8 hours.
YellowThe total processing time is between 8 to 24 hours.
RedThe total processing time is more than 24 hours.

Click the Processing details link to view the progress of the data ingestion process of the ETLs. For each ETL, the summary table shows the following details:

ColumnDescription
ETL ID

A unique identification number.

ETL name

The unique name defined for the ETL.

Start Time

The processing start date and time for the ETL.

Last update

The date and time of the latest execution of the ETL.

Timeseries message queue

Number of timeseries messages waiting in the queue to be processed.

Relations message queue

Number of relations messages waiting in the queue to be processed.

Timeseries processed %

The progress of the messages processed in percentage.

Relations processed %

The progress of the relations messages processed in percentage.

After all the timeseries and relations messages are processed for a specific ETL, the ETL is removed from the summary table.

From the Processing details page, click Back to Warehouse status to view the Status page.

Volume

This page displays values and charts for the following volumes processed by the BMC Helix Capacity Optimization components:

  • Processed today
  • Processed yesterday
  • Average daily processed rows (last 30 days)

The following image shows an example of the the Volume values and charts:

You can quickly gauge the status of your data warehouse from the Volume chart:

  • Compare Loaded Rows with Processed Rows. If loaded rows are much higher than processed rows, the queue age will be increased.
  • Compare Processed Rows with Average daily processed rows (last 30 days).
    • If the processed rows are more, it indicates new data flows.
    • If the processed rows are lower than the average value, it indicates less data flows in the system.

The data table displays the data used to build the chart. To view the data table, click Data Table at the top-right of the chart. The data table is displayed. The data flow table lists the date timestamp and the number of samples imported daily.

Was this page helpful? Yes No Submitting... Thank you

Comments