Gathering ETL logs

This topic contains the following sections: 

• • To export ETL run logs
  • To gather ETL log files for advanced debugging
    • Advanced debugging

To export ETL run logs

To export run logs for ETLs from the Helix Capacity Optimization Console, perform the following task:

1. Navigate to Administration > ETL & SYSTEM TASKS > System tasks.
2. Find the ETL you want to gather logs from and click Click to view details  on the corresponding row.
   
   Accessing the ETL log files - View details icon
   
   The links to download the log files are at the bottom of the page that opens, containing the ETL logs for the last execution.
   • The .out file contains the standard output log of the ETL process.
   • The .err file contains the standard error log of the ETL process.
3. To view the logs of a previous execution, click the left arrow button at the bottom of the page.
   
   Accessing the log files for previous executions of the ETL
   
    You can also reach the log page by clicking the name of the ETL and selecting the Show log link at the top of the page.
   
   Accessing the ETL log files - Show log link
   

To gather ETL log files for advanced debugging

To view the logs of the load phase of the ETL, scroll to the bottom of the page.

Three different sets of files are produced for every imported dataset:

• Oracle .ctl files: One file is created for each dataset imported by the ETL, containing data to be loaded by the ETL.
• Oracle .ctl.log files: One log file is created for each corresponding .ctl file, containing logs specific to the load phase of the dataset.
• Debug CSV file: It is a file without extension, containing the first 1000 rows of the real dataset that will be loaded.

Advanced debugging

If you got any warnings for bad data, you can also look for files named as your .ctl files but with a .bad extension (for example, L1S322658.bad). A .bad file is generated for every .ctl file that contains bad data. These files cannot be downloaded directly from the user interface; to retrieve them, access the $CPITBASE/etl/output folder on the server that is running the ETL, using ssh.

Accessing the ETL log files - Advanced debugging

It is possible to edit the ETL run configuration and specify a custom limit of rows (default: 1000) that are saved in the debug CSV file.

Changing the default number of rows in the CSV output

For information about how to send this information to Customer Support, see Support-information.