Configuring the ETL task

To configure the BMC Helix Continuous Optimization for Mainframes ETL task, perform the following actions:

Related topic

Using

  1. From the Administration tab, on the BMC Helix Continuous Optimization console, navigate to ETL & SYSTEM TASKS > ETL tasks.
    To configure an ETL task, you must first create a run configuration, which is a set of parameters with which the task will be executed. After you create and save all the required parameters, the new ETL task and run configuration are created.

    Important

    For information about BMC Helix Continuous Optimization console, see Getting started in the BMC Helix Continuous Optimization documentation.

  2. Select Add > Add ETL, and click Advanced (scroll to the bottom, if necessary).
    image2022-9-2_19-42-52.png
  3. On the Run Configuration tab, make the following changes:
    1. In the ETL Task name field, type a name for the ETL task.
    2. In the Run configuration name field, type a name for the configuration.
    3. In the Deploy Status field, select either Production or Test.
    4. From the ETL module list, select BMC - Mainframe CDB Extractor.
    5. From the Datasets list, select the datasets that you want to load into the BMC Helix Continuous Optimization Data Warehouse.
      Initially, we recommend that you select all datasets to ensure that the BMC Helix Continuous Optimization for Mainframes view has all the data it needs.
    6. (Optional) Specify a description and log level for the Run configuration.

      image2022-9-2_13-25-8.png

  4. On the Entity catalog tab, in the Sharing status field, select Private Entity Catalog.
    image2022-9-2_13-26-20.png

  5. On the Object relationships tab, make the following changes:
    1. In the Associate new entities to field, select new Domain.

    2. In the New domain field, replace the default name (the name of the ETL) with the name that you want to use.

      image2022-9-2_13-31-12.png

  6. On the Custom properties tab, make the following changes:
    1. In the CDB Server Data Source Name field, enter the name of the ODBC data source as it was specified in the CDB Automator database catalog.
      For more information, see Configuring-the-CDB-Automator.
    2. In the CDB Server address field, specify the complete URL of the Capacity Database Web Services server (such as CDBServer.bmc.com).
    3. In the CDB Server Use SSL field, change the default value of false to true if you want to request a secure connection to the Capacity Database Web Services database using HTTPS protocol.
    4. In the CDB Server Ignore Certificate Errors field, change the default value of false to true if you want the ETL to ignore certificate errors when authenticating the secure connection to the Capacity Database Web Services database.

    5. If the Capacity Database Web Services server requires credentials for web applications access, specify a user name and password in the CDB Server user name and CDB Server password fields.

      Important

      Specify the Capacity Database Web Services server user name and password, and not the underlying relational database management system (RDBMS) user name and the password that are stored with the ODBC data source in the CDB Automator catalog.

    6. If you have changed the default Capacity Database Web Services server port (80) and virtual directory name (BMCCDB) during installation, replace the default values shown in CDB Server port and CDB Virtual Directory name with the values that you specified. 

      Important

      If you have requested a secure connection to the Capacity Database Web Services database, verify that you have specified the correct port value. The default HTTPS port is 443.

    7. In the Time zone offset field, specify the time zone offset for the selected Capacity Database Web Services data source in the format +/-HHMM.

      Important

      All data in a specific Capacity Database Web Services data source is at the same time zone offset. However, different Capacity Database Web Services data sources can have different time zone offsets. If you want to load data into BMC Helix Continuous Optimization from multiple Capacity Database Web Services data sources, you must specify the correct time zone offset for each data source to ensure correct data synchronization.


    8. In the Use Start and End dates from parameters field, select the method for determining the time span for the ETL task: 

      • Select true to use the start and end dates that you specify in the Extractor Start datetime and Extractor Stop datetime fields.

      • Select false to use the BMC Helix Continuous Optimization lastcounter parameter.
        BMC Helix Continuous Optimization uses lastcounter to pass information about the most recently loaded data from one execution of the ETL task to the next. At the end of each successful execution, ETL stores the timestamp of the last record loaded; the next run of the ETL task starts loading data from that timestamp.
        The time span runs from the lastcounter value to the current clock value (the time of ETL task execution). For information about the lastcounter parameter, seeOverview of the ETL workflow in the BMC Helix Continuous Optimization documentation. For information about scheduling the ETL task, see Scheduling the ETL task to run automatically.

      Important

      When you create a new run configuration, the Status object (and its lastcounter parameter) does not exist. The first time you execute a new run configuration, BMC Helix Continuous Optimization creates the Status object and sets the lastcounter value to one day before the current date. Alternatively, you can copy an existing run configuration and change the lastcounter value to the date from which you want to start loading your data. If you select true and use the Extractor Start datetime and Extractor Stop datetime fields, the Status object and the lastcounter value are not affected.

    9. In the MAX number of days to process field, specify the number of days that you want to process in this ETL run.
      The recommended value is 1. Processing too many days in one ETL run can lead to performance issues, a Java heap overflow, or both.
    10. In the Limit on number of Applications and Limit on number of Suites fields, specify how many of each object type you want to extract from the staging Capacity Database.
      The ETL extracts the most active objects of each type. The default value for each field is 100. 
    11. In the Extract Service Class data field, specify whether to extract service class data.
      The default is True (yes).
    12. In the Extract Report Class data field, specify whether to extract report class data.
      The default is True (yes).
    13. In the Extract Storage Controller data field, specify whether to extract storage controller data. 
      The default is False (no).
    14. In the Extract Storage Pool data field, specify whether to extract storage pool data.
      The default is False (no).

    15. In the Extract Sysplex data field, specify whether to extract Sysplex data.
      The default is False (no).

      image2022-9-2_13-27-55.png

  7. On the Loader configuration tab, in the Empty dataset behaviour field, select Ignore.

    image2022-9-2_13-29-10.png

  8. (Optional) On the Scheduling Options tab, specify custom scheduling parameters, if necessary.

  9. Click Save.
    The Run Configuration panel is displayed.

    Important

    If any tab has missing or incomplete information, you remain on the Add ETL panel and that tab is marked with a red asterisk ({}{*}).

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*