Overview of ETLs
You can use the ETLs that are available in BMC Helix Continuous Optimization to collect the infrastructure data of your environment. The ETLs are configurable modules that are based on the Extract, Transform, and Load (ETL) technology. They are also known as connectors. They connect to the data sources and collect data. The collected data is transferred to the BMC Helix Continuous Optimization database. This data is used for analyzing and optimizing the capacity of your infrastructure.
Depending on the operating systems and technologies, different ETLs are available for data collection.
Multiple ETLs can access the same data sources to collect different types of data. For example, one ETL might collect the performance data of servers and another might collect the asset numbers of the same servers. BMC Helix Continuous Optimization provides a mechanism called as shared lookup to let multiple ETLs reference the same entities.
For more information about shared lookup, see Understanding-entity-identification-and-lookup.
Types of ETLs
Depending on how ETLs collect data from data sources, they are classified as extractors and parsers. Based on the frequency of data collection, ETLs are also classified as service ETLs and batch ETLs. Service ETLs continuously collect data from data sources. You can configure the frequency of loading the collected data into the BMC Helix Continuous Optimization database. The VMware vCenter and RHEV ETLs are service ETLs.
Batch ETLs collect data at scheduled intervals from the source. You can configure them to run once in an hour, a day, or a week. These ETLs collect data and load it into the database for the entire interval in one run. All the public cloud provider ETLs are batch ETLs.
Although batch ETLs collect data less frequently than service ETLs, the granularity of data (data points per hour) collected by both types of ETLs might be the same.
ETLs are also classified as follows:
Classification | Description |
|---|---|
Out-of-the-box ETLs | These ETLs are available with the product. You can start using them for data collection after installing the product. For information about out-of-the-box ETLs, see Collecting-data. These ETLs also include the ETLs that are provided by these BMC partners: Sentry and Moviri. These ETLs are supported by BMC as per BMC support policies. For more information, see Out-of-the-box-ETLs, Sentry ETLs, and Moviri ETLs. |
Custom ETLs | These ETLs are created by customers, BMC Professional Services, and BMC deployment partners. These ETL do not require an additional license entitlement. BMC Helix Continuous Optimization provides a Software Development Kit (SDK) that includes an Integrated Development Environment (IDE) for creating custom ETLs. For more information, see Developing-custom-ETLs. The BMC Helix Continuous Optimization SDK is supported by BMC; however, custom ETLs are not supported by BMC as specified on the Customization Policy page of the BMC Support website. |
Community-based ETLs | These ETLs (formerly known as "template") are not distributed with the product and are not supported by BMC (as mentioned on the Customization Policy page of the BMC Support website). They do not require an additional license entitlement and are provided for free. You can download these ETLs from BMC Marketplace. When you access BMC Marketplace, ensure that you select Free as price filtering criteria to view the list of community-based ETLs. |
Project-based ETLs | These ETLs are available only through a professional services engagement and are supported based on the engagement terms. You can access these ETLs on the BMC Marketplace. |