ETL common configuration properties
Basic properties that are displayed by default in the Add ETL page are the most common properties that you can set for an ETL. You can choose to leave the default selections for each property as is. Advanced properties are for advanced users and scenarios only. You do not need to set or modify these properties unless you want to change the way the ETL works.
|ETL task name
|By default, this field is populated based on the selected ETL module. You can specify a different name for the ETL Task. Duplicate names are allowed.
|Run configuration name
|The default name is already filled out. This field is used to differentiate between various configurations you can specify for the ETL task. You can then run the ETL task based on the configuration.
|You can select Production or Test to mark the ETL tasks. For example, you can start by marking the task as Test and change it to Production after you have seen that the results are as expected.
|(Optional) Enter a brief description for this ETL.
|Select how much detail you want the ETL log to show. The log includes Error, Warning and Info type of log information.
Log levels 5 and 10 are typically used for debugging or troubleshooting ETL issues. Using a log level of 5 is general practice, however, you may choose level 10 to get a high level of detail while troubleshooting.
|Execute in simulation mode
|Select Yes if you want to to validate the connectivity between the ETL engine and the target, and to ensure that the ETL does not have any other configuration issues. This option is useful while testing a new ETL task.
Ensure that the ETL module is selected.
|Displays a short description of the ETL module.
|Set the sharing status by selecting one of the following options:
|Associate new entities to
Specify the domain where you want to add the entities created by the ETL. You can select an existing domain or create a new one.
Note: By default, a new domain is created for each ETL, with the same name of the extractor module. As the ETL is created, a new hierarchy rule with the same name of the ETL task is automatically created, with status "active," if you update the ETL specifying a different domain, the hierarchy rule is updated automatically.
Select any one of the following options:
|(Optional) Select a task group by which to classify this ETL.
|Running on scheduler
|Select a scheduler for running the ETL. For cloud ETLs, use the scheduler that is preconfigured in Helix. For on-premises ETLs, use the scheduler that runs on the Remote ETL Engine.
|Maximum execution time before warning
|Indicate the number of hours, minutes or days for which to execute the ETL before generating warnings or alerts, if any.
|Select the frequency for ETL execution. Available options are:
|Start timestamp: hour\minute
(Applies to Predefined frequency)
|Indicate the HH:MM start timestamp to add to the ETL execution running on a Predefined frequency.
|Custom start timestamp
|Select a yyyy-mm-dd hh:mm timestamp to add to the ETL execution running on a Custom frequency.
Enables you to select or deselect metric groups for which data will be populated Available datasets. You cannot include additional datasets to the run configuration of the ETL.
|Metric profile selection
Select any one:
For more information, see Adding and managing metric profiles.
|Levels up to
The metric level defines the amount of metric imported into Continuous Optimization. If you increase the level, additional load is added to the ingestion, while decreasing the metric level reduces the number of imported metrics.
Choose the metric level to apply on selected metrics:
|List of properties
Additional properties can be specified for this ETL that act as user inputs during execution. You can specify values for these properties either at this time, or from the "You can manually edit ETL properties from this page" link that is displayed for the ETL in view mode.
|Empty dataset behavior
|Choose one of the following actions if the loader encounters an empty dataset:
|ETL log file name
|Name of the file that contains the ETL execution log; the default value is:
|Maximum number of rows for CSV output
|A number which limits the size of the output files.
|CSV loader output file name
|Name of the file generated by the CSV loader; the default value is:
Continuous Optimization loader output file name
Name of the file generated by the BMC Helix Continuous Optimization loader; the default value is:
Select the level of detail:
|Remove domain suffix from datasource name (Only for systems)
|If set to True, the domain name is removed from the data source name. For example,
server.domain.com will be saved as
|Leave domain suffix to system name (Only for systems)
|(Only for systems) If set to True, the domain name is maintained in the system name. For example:
server.domain.com will be saved as such.
|Update grouping object definition (Only for systems)
|If set to True, the ETL will be allowed to update the grouping object definition for a metric loaded by an ETL.
|Skip entity creation (Only for ETL tasks sharing lookup with other tasks)
If set to True, this ETL does not create an entity, and discards data from its data source for entities not found in Continuous Optimization. It uses one of the other ETLs that share lookup to create the new entity.
|Specify a value to execute the task only during particular hours within the day. For example, 0 – 23 or 1,3,5 – 12.
|Day of week mask
|Select the days so that the task can be executed only during the selected days of the week. To avoid setting this filter, do not select any option for this field.
|Day of month mask
|Specify a value to execute the task only during particular days within a month. For example, 5, 9, 18, 27 – 31.
|Apply mask validation
|By default this property is set to True. Set it to False if you want to disable the preceding Scheduling options that you specified. Setting it to False is useful if you want to temporarily turn off the mask validation without removing any values.
|Execute after time
|Specify a value in the hours:minutes format (for example, 05:00 or 16:00) to wait before the task must be executed. This means that once the task is scheduled, the task execution starts only after the specified time passes.
|Select one of the following options: