Troubleshooting ETLs used to import the custom structure data

BMC Helix Continuous Optimizationsupports custom structure tables for versatile data hosting. These tables are also referred to as buffer tables and are designed to accommodate generic records with custom attributes. For details, see Collecting data for custom structure tables.

You need to configure and run one of the following ETLs to import data into the custom structure table:

Validate that the ETL ran without errors

Navigate to Administration > ETL & System tasks and check the Last exit status of the ETL task.

Understanding ETL run statuses

When running an ETL in simulation or production mode, the Last exit column may display:

Status	Description
OK	The ETL executed without errors in simulation mode.
WARNING	Minor issues detected. Check the ETL log for details.
ERROR	The ETL did not complete. Fix configuration or data issues, then re-run.

If the status is Warning or Error, open the log file for additional information about issues related to packaging data into a message file.

When the ETL finishes packing the data message, it is sent from the remote scheduler to be processed by the ETL Data Sender task. This system task runs at scheduled intervals, so data will not be populated until the next scheduled run. The task is named: ETL Data Sender task for Remote Scheduler on <hostname>. If it shows a warning or error, view the log file for further information on any issues sending the message file.

After simulation completes successfully, switch the ETL to production mode to load data into the custom structure table.

Review the diagnostic alerts

Navigate to Administration > System > Diagnostic Alerts.
Review the list of diagnostic, failure, error, and warning messages generated and logged by BMC Helix Continuous Optimization. Diagnostic alerts for ETLs are listed under the ETL task ID in the Subcomponent column. For example: ~ taskid:800~.
Clear the Group by component and subcomponent option to ungroup the diagnostic alerts and view multiple issues separately for the same task.
Expand the Message column to view the full details of the error.

Identify the failing stage

To isolate where the ETL is failing, classify the issue using the following four-stage model:

Stage	Description
Collect	Source data extraction from the input file or database.
Transfer	Moving the message file to the ETL Engine.
Process	Parsing and preparing the data.
Import	Loading the data into the custom structure table.

Identifying the failing stage helps narrow down configuration or system-level problems and speeds up resolution.

Data and schema requirements

Ensure that the target custom structure table uses supported data types and formats:

VARCHAR columns must be within the supported character limit (1–4096 characters).
NUMERIC columns must use valid precision and scale values.
DATE fields must match accepted formats (for example, dd/MM/yyyy or yyyyMMdd).

Incorrect column limits or incompatible formats may cause the ETL to raise warnings or skip records during import.

Common causes of failures

The target buffer table does not exist. The ETL cannot create it automatically.
The wrong ETL module was used. For example, using a CSV parser when a CST-specific configuration is needed.
Configuration properties are placed under incorrect sections (Basic vs. Advanced) or include unsupported fields.
Datatype or length mismatches. For example, VARCHAR values exceeding the supported limit of 4096 characters.
Transfer or import issues where the packaged message is not uploaded or processed as expected by the system tasks.

Known limitations

The ETL does not automatically create the custom structure (buffer) table. Ensure the table exists before running the ETL.
The following properties available in other products or older documentation are not supported in this context and must not appear in the configuration:
- Metric profile
- Levels up to
- Advanced > Collection level
- Advanced > Metric
- Reduce priority
- Update grouping object definition
- ETL log file name
- CSV loader output file name
- Detail mode
The valid values for Empty dataset behavior are Warn and Ignore.
Custom structure table VARCHAR columns support up to 4096 characters.

Information to collect for Support

If the issue persists after performing the troubleshooting steps, collect the following before opening a Support case:

The ETL task log from Administration > ETL & System tasks (Last run log).
A Diagnostic alerts log bundle using the Collect logs option.
The ETL's Run configuration export (Basic + Advanced properties).
A copy of the source file or representative sample used in the failing run.
The DDL (table schema) of the custom structure table.
The timestamp of the ETL attempt and the corresponding ETL Data Sender task run.