Docs.bmc.com brief outage

This site, docs.bmc.com, will undergo a brief outage of less than ten minutes starting at 5 PM CDT June 12/3:30 AM IST June 13.

ETL life cycle

The development life cycle of a custom ETL consists of the following phases:

  1. Development: The ETL is developed on a workstation.
  2. Testing: The ETL is run in simulation mode in a real environment (on the ETL Engine).
  3. Production: The ETL imports production data.

Related topics

Developing-custom-ETLs

Administering

The ETL development lifecycle
etl_lifecycle.png

Each phase comprises of a different set of activities. The following sections discuss these in detail.

Development

In this phase, the work environment is the ETL Development Kit. Activities in this phase include:

  • Code editing: Code is written locally on the development workstation.
  • Saving and sharing: Code is saved and (optionally) uploaded to the database from where other users can download it for sharing and backup purposes.
  • Debugging: New custom ETLs are run for debugging on the development workstation.

Testing

In this phase, the ETL runs in simulation mode on the ETL engine. It gets and manages real data but does not load any data in the database. Activities in this phase include:

  • Activation: If the ETL is a custom module, it needs to be activated in order to be used in the real environment.
  • Configuration: A new ETL instance is created and configured.
  • Validation: The ETL instance is run in simulation mode to ascertain how it works with real data.

Production

In this phase, the ETL production data runs on the ETL engine. Activities in this phase include:

  • Scheduling: The ETL is scheduled to run automatically.
  • Running: The ETL extracts and loads data.
  • Diagnostics: If errors occur, the ETL generates an alert and the execution log is checked.

Notes

  • The test and production phases are the same for custom and standard ETLs.
  • Activation is not required for standard ETLs (they are active by default).
  • The ETL modules you develop are run in a sandbox to ensure that their execution does not harm the production environment.
  • Development ETLs are run on development workstations and not on the ETL engine.
  • Data produced by a development ETL cannot be loaded into the database.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*