Skip to Content

Retrieving and adding an ETL status

ETLs must be able to work differentially between one run and another, that is, they must import only new data from the source and append it to older data. The lastcounter object keeps track of such append operations, ensuring that all new data collected is added to existing records. 

 

Related topics

ETL-development-API-constructs

ETL-data-flow-and-the-lastcounter-value

Logging-of-custom-ETLs

The-Database-Interface-method 

The DSStatus object

This object has the capability to preserve a permanent status between different runs of the same ETL; it represents the ETL status with respect to the data source of the ETL. Every time an ETL is run, an instance of DSStatus is created that contains the same value as was present at the end of the previous run.

To retrieve the current status of the ETL using the configuration object, use the following code:

# get the current status
DSStatus status = conf.getStatus();

If an ETL is executed for the first time, the resulting instance will be empty. To check the status, use the following code:

if (status == null){

//create a new status
status = new DSStatus();

//and insert it
conf.insertStatus(status);
}

This is usually done at the beginning of an extract method. After each run, the ETL can modify its own status and save the information that is required for it to work in append mode.

Warning

Note

The insertStatus method is used for adding a new ETL status.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*

BMC Helix Capacity Optimization 20.02