Creating and managing custom ETLs
You can use the ETL Development Kit (EDK), which is a command line utility, for creating and managing custom ETLs. You can create a custom ETL module in Perl or Java, and it can be a simple extractor, database extractor, or parser. The utility can be used on Windows and Linux systems.
Using the EDK, you can achieve the following goals:
- Create custom ETL modules in Perl or Java programming language.
- Modify existing ETL packages and repackage them.
- Activate or deactivate ETL packages.
The ETL Development kit requires Java 11 (Ensure that you set the PATH environment variable).
To create and modify custom ETL packages
The following procedure illustrates how to create a database extractor in Java.
To deactivate custom ETL packages
When the deployed ETL package is no longer needed, you can deactivate it using the EDK. You can reactivate it when needed.
After deactivation, the custom module is removed from the list of ETL modules that are available to the administrators for creating a new ETL instance. Any ETL instance that was created using this custom module is still displayed in the list of ETLs, but you won't be able to run the ETL until you activate it again.
Navigate to the <project_path>/output folder.
Run the following command to deactivate the package:
.\coetl.cmd --deactivate <project_path>\output\<project_name>.etl.pkg
./coetl.sh --deactivate <project_path>/output/<project_name>.etl.pkg
.\coetl.cmd --deactivate development\MonitoringDatabaseETLProject\output\ETLProject.etl.pkg
./coetl.sh --deactivate development/MonitoringDatabaseETLProject/output/ETLProject.etl.pkg
Similar to the activation command, if you rename the credentials.key file or save it in a folder different from the EDK root folder, specify the API key while deactivating the package.