Automatically creating ETLs

Using the wizard

Enable the Auto-create ETL feature

To enroll an existing Gateway Server for Auto ETL maintenance:

1. Select Administration > Gateway Manager > Gateway Servers.
2. Open an existing Gateway server. 
3. In the message indicating that the ETL is not enrolled, click here to enroll the ETL. 

For a new Gateway Server, the Auto create ETLs to populate data from this gateway server is selected by default. For details, see Auto-create ETLs using wizard when enrollment is enabled.

Auto-create ETLs using wizard when enrollment is enabled

1. On the WELCOME page, click Next.

2. On the AUTO-CREATE PARAMETERS page, enter the following details:

   • Basic
   • Advanced

   
   

   Number of manager runs per ETL: Type the required manager runs per ETL in the box. Specify the maximum number of scheduled Manager runs each auto-created BMC TrueSight Capacity Optimization VIS parser ETL will process. A separate ETL is created for each group of scheduled Manager runs equal to this value. For example, if the value specified for this field is 5 and the Gateway Server has 20 scheduled Manager runs, four Auto-create ETLs are created to process the information from the runs. The runs are assigned to the ETLs irrespective of the processed by the run. The auto-create ETLs are configured to process all platform data.

   
   

   Manager runs to exclude: Select the required manager runs in the Manager runs to exclude table to be excluded and to add them to the text box. You can also type the name of the required manager runs separated by semi-colon in the box to exclude them.

3. On the ETL PARAMETERS page, set the values for the following properties under each expandable tabs:

   • Basic
   • Advanced

   
   

   
   
4. Click Next.

5. On the SCHEDULE page, enter the following details:

   Field	Description
   Start timestamp	Select the timestamp in hours and minutes to schedule the ETL .
   Schedulers for created ETL tasks	By default, all available schedulers are listed in the Schedulers to use table and are included for the ETL tasks. You can select the required scheduler from the Schedulers to use table, and add them to the Excluded schedulers table using the >> button. You can also select the required scheduler from the Excluded schedulers table and add them to the Schedulers to use table using the << button.

6. Click Next.
7. On the Summary page, click Finish. The auto-create ETLs and chain controller tasks are created and scheduled for execution.
   Auto-create ETLs are named in the following manner: <GatewayServer_gateway-server-name_scheduler-ID_task-sequence-number>. For example GatewayServer_GWS-austin-05_2_3 would be the third auto-create ETL to process GWS-austin-05 Manager runs on scheduler with ID 2.
   The Auto Create chain controllers are named in the same way except that their task-sequence-number is always 0.

Editing Auto-create ETL configuration

1. Select Administration > GATEWAY MANAGER > Gateway Servers.
2. Click the name of the required gateway server.
   The gatewayserverName page is displayed.

3. Click Edit ETL configuration.
   The Edit Auto-create ETL configuration page is displayed.

   Note

   The Edit ETL configuration option is displayed only if the ETL is not enrolled for the corresponding gateway server.

   

4. In the Edit Auto-create ETL configuration page, modify the following details:

   Field	Description
   Select values to use in scheduling ETLs
   Start timestamp	Edit the timestamp in hours and minutes to modify the ETL schedule.
   Schedulers for created ETL tasks	You can modify the Schedulers to use table by using the >> button to add them to the Excluded schedulers table. You can also select the required scheduler from the Excluded schedulers table and add them to the Schedulers to use table using the << button.
   Configure Auto-create ETL creation parameters
   Manager runs to exclude	Select the required manager runs in the Manager runs to exclude table to be excluded and to add them to the text box. You can also type the name of the required manager runs in the text box to exclude them.

5. Click Save.

Auto-create ETL maintenance

The Gateway Server Admin Helper Task, which is configured to run daily, maintains the auto-create ETLs. When Manager runs are scheduled, unscheduled or removed for a Gateway Server that is used in auto-create processing, the Gateway Server Admin Helper Task adjusts the auto-create ETLs as required, during its next execution:

• If a new run is scheduled, and if there is any auto-create ETL that is processing fewer runs than the value specified for Number of gateway Manager runs per ETL in the Gateway ETL auto-create wizard, then the new run is added to that auto-create ETL. If all existing auto-create ETLs for the Gateway Server are processing the maximum number of runs, then a new auto-create ETL is created to process the new run and it is added to the chain controller executing on the scheduler currently processing the least number of Manager runs.
• If a scheduled run is stopped or removed, the Gateway Server Admin Helper task removes the run from the auto-create ETL that was processing the run. If it is the last run being processed by the ETL, the value ##EMPTY is specified for the Manager run list of the ETL, indicating that it will not process any runs.
• If a scheduled Manager run which was being processed by another VIS files parser ETL when the auto-create processing was initiated for a Gateway Server ceases to be processed by any ETL, the Gateway Server Admin Helper task adds it to the auto-create processing, in the same way as a new Manager run.

The Gateway Server Admin Helper Task does not override any user modifications or deletions of auto-create ETLs.

For more information on the Gateway Server Admin Helper Task, see Creating-and-running-data-feed-health-reports.