Generic ETL based on a template

You can use an Open ETL template to integrate the following ETLs:

Before you begin

Ensure that you have created the required Open ETL template, for more information see Creating an Open ETL template.

Integration steps

To integrate BMC Helix Continuous Optimization with a generic ETL by using a template, perform the following task:

  1. Navigate to Administration > ETL & SYSTEM TASKS > ETL tasks. 
  2. In the ETL tasks page, click Add > Add ETL under the Last run tab.
    The Add ETL page is displayed.
  3. In the Module selection, select Based on Open ETL template.

  4. From the Select Open ETL template list, select the required Open ETL template.
    All properties of ETL are selected according to the CSV file or SQL query used for creating the Open ETL template. 

    Note

    All properties specific to supported ETLs will be available in only the Advanced mode.

  5. In the Entity catalog tab, select the required option for the Sharing status:
    • Shared entity catalog: Select this option if, for the same entities, data is coming from multiple sources.
      • Sharing with Entity Catalog : Select an entity catalog from the drop-down list.
    • Private entity catalog: Select this option if, for the same entity, data is coming from a single source.

  6. Depending on the template selected, select the required tab to view the configuration details:

    In the File Location tab, enter the following details.

    Field

    Description

    File location

    Select any one of the following methods to retrieve the CSV file:

    • Local directory: Specify a path on your local machine where the CSV file resides.
    • Windows share: Specify the Windows share path where the CSV file resides.
    • FTP: Specify the FTP path where the CSV file resides.
    • SCP: Specify the SCP path where the CSV file resides.
    • SFTP: Specify the SFTP path where the CSV file resides.

    Directory

    Path of the directory that contains the CSV file.


    In the SQL Query tab, enter the following details.

    Field

    Description

    SQL query hints

    Provides hints for including the mandatory and optional parameters in the SQL extraction query.

    SQL Query

    This tab is only available when a custom database query has to be built in order to get the data from a generic source. In the text area you can type the SQL extraction query. You can select the required parameter and operator from the list and add them to the text area.

    Lastcounter column

    Specify which is the lastcounter column that will be used to identify new data; generally this is set as the timestamp (TS).

    SQL parameters number

    You can add up to 10 custom parameters to the query; select how many you need from the drop-down menu.

  7. Click Save.

Related topic

Generic-extractors-and-parsers-to-integrate-data