Importing ETL configuration by using API

Use the Import configuration APIs to obtain entity catalogs, schedulers, external database connections, metric profiles, task groups, and to create an ETL instance.

Before you begin

Ensure that the following requirements are met:

The appropriate read or write activity is assigned to your user role to access the API. The user account must have the following permissions (through user roles):
- READ: capacity_optimization.admin.etls_read or capacity_optimization.admin.admin_section_read
- WRITE: capacity_optimization.custom_etl.edit or capacity_optimization.admin.admin_section_edit

For information about the user roles and permissions, see Default-user-roles-and-permissions.

Valid user credentials or a bearer token is available for authentication. For more information, see Access-and-authentication-for-the-REST-API.

Get all entity catalogs

GET/opt/api/v1/backend/etls/entitycatalogs/

Retrieves all entity catalogs.

Request URL

https://<host_name>/opt/api/v1/backend/etls/entitycatalogs/

Example request URL

https://hostname.bmc.com/opt/api/v1/backend/etls/entitycatalogs/

Request Header

Content-Type: application/json
Authorization: Bearer <jwttoken>

Sample response

[
{
   "name": "Default Entity Catalog",
   "description": "The default entity catalog",
   "entity_catalog_id": 2
}
]

Get all schedulers

GET/opt/api/v1/backend/schedulers/

Retrieves all schedulers.

Request URL

https://<host_name>/opt/api/v1/backend/schedulers/

Example request URL

https://hostname.bmc.com/opt/api/v1/backend/schedulers/

Request Header

Content-Type: application/json
Authorization: Bearer <jwttoken>

Sample response

[
{
   "name": "Generic Scheduler",
   "hostname": "Helix",
   "status": "OK",
   "scheduler_id": 0
},
{
   "name": "Remote Scheduler on clm-aus-uy4poo.bmc.com",
   "hostname": "clm-aus-uy4poo.bmc.com",
   "status": "OK",
   "scheduler_id": 1
}
]

Get all external db connections

GET/opt/api/v1/backend/etls/dbconnections/

Retrieves all external database connections.

Request URL

https://<host_name>/opt/api/v1/backend/etls/dbconnections/

Example request URL

https://hostname.bmc.com/opt/api/v1/backend/etls/dbconnections/

Request Header

Content-Type: application/json
Authorization: Bearer <jwttoken>

Sample response

[
{
   "db_connection_id": 1,
   "name": "CHF 2002",
   "type": "DB:PG"
},
{
   "db_connection_id": 45,
   "name": "sql_db_event_etl",
   "type": "DB:PG"
}
]

Get metric profiles

GET/opt/api/v1/backend/etls/metricprofiles/

Retrieves the metric profiles..

Request URL

https://<host_name>/opt/api/v1/backend/etls/metricprofiles/

Example request URL

https://hostname.bmc.com/opt/api/v1/backend/etls/metricprofiles/

Request Header

Content-Type: application/json
Authorization: Bearer <jwttoken>

Sample response

[
{
   "name": "Global profile",
   "metric_profile_id": 0
},
{
   "name": "Compatibility profile",
   "metric_profile_id": 1
}
]

Get task groups

GET/opt/api/v1/backend/etls/taskgroups/

Retrieves the task groups.

Request URL

https://<host_name>/opt/api/v1/backend/etls/taskgroups/

Example request URL

https://hostname.bmc.com/opt/api/v1/backend/etls/taskgroups/

Request Header

Content-Type: application/json
Authorization: Bearer <jwttoken>

Sample response

[
{
   "name": "Gateway Server ETL",
   "task_group_id": 1
},
{
   "name": "Capacity Agent",
   "task_group_id": 2
}
]

Get domain ID

To get the domain ID, use the POST/opt/api/v1/catalog/search search REST API listed on the Continuous-Optimization-REST-API-endpoints page.

Configuring object relationships

You can specify the domain to which you want to add the entities created by the ETL.

New domain

To create a new domain and associate entities to it, specify the following:

name of the domain
root domain ID (Keep the value as 0 so that the new entities are a child of Domain, Services and Application.)

Request body

"object_relationships": {
   "domain_mode": "NEW",
   "domain_name" : "Name of the new domain to create",
   "parent_domain_id": 0
},

Existing domain

To use an existing domain and associate entities to it, specify the root domain ID (ID of the destination domain).

Request body

"object_relationships": {
"domain_mode": "EXISTING",
"domain_id": "51931"
},

Configuring entity lookup

Private entity catalog

Configure the ETL using private lookup if this is the only ETL that extracts data from the given set of resources. For private lookup, use:

Request body

"entity_catalog": {
"sharing_status": "PRIVATE",
},

Shared entity catalog

You can also configure an ETL to share the entity catalog of another ETL. For shared lookup, use:

Request body

  "entity_catalog": {
   "sharing_status": "SHARED",
   "entity_catalog_id": 2
},

Create an ETL instance with shared lookup and new destination domain

post/opt/api/v1/backend/etls/instance

Creates an ETL instance with shared lookup and a new destination domain.

Request URL

https://<host_name>/opt/api/v1/backend/etls/instance

Example request URL

https://hostname.bmc.com/opt/api/v1/backend/etls/instance

Request Header

Content-Type: application/json
Authorization: Bearer <jwttoken>

Important

In encryption_passphrase, you must use the same passphrase that was used to export the ETL configuration.

Request body

{
"encryption_passphrase": "<password>",
"scheduling": {
   "scheduler_id": 1,
   "period_sec": 86400,
   "start_date": "2022-08-01T00:00:00Z"
},
"etl": {
   "instance_name": "New name of the ETL",
   "module_name": "com.bmc.bco.aws.extractor.DMAWSExtractorE",
   "module_description": "Amazon Web Services - AWS API Extractor",
   "module_id": 151
},
"entity_catalog": {
   "sharing_status": "SHARED",
   "entity_catalog_id": 2
},
"object_relationships": {
   "domain_mode": "NEW",
   "domain_name" : "Name of the new domain to create",
   "parent_domain_id": 0
},
"properties": {
   "extract.access.key.id": "AKIAJNKJWEC2TDWDG56A",
   "extract.access.key.secret.password.encrypted": "FdxnkeLXO8x7/MAHbVaHEqx6GG5j1heFWoFgQjYiawrHz/icYcdDov9BV5V3u2vest99aJ3rGzVSoYO8Vl8v1HcDs70=",
   "extract.aws.account.options": "SINGLE",
   "extract.aws.us.gov.cloud": "false",
   "extract.database.iscustom": "true",
   "extract.default.region": "us-east-1",
   "extract.module": "com.bmc.bco.aws.extractor.DMAWSExtractorE",
   "extract.prop.bs.tag.key": "Service",
   "general.dataset.idlist": "1;59;103;7;118;30",
   "general.log.level": "10",
   "general.module.selection": "DATASOURCE",
   "general.options.onemptydataset": "WARN",
   "general.simulation": "false",
   "loader.lookup.remove.domain": "false",
   "loader.sysnm.leave.domain": "false",
   "mask.ignorevalidation": "false"
}
}

Create an ETL instance with private lookup and use an existing domain

post/opt/api/v1/backend/etls/instance

Creates an ETL instance with a private lookup and uses an existing domain for the ETL.

Request URL

https://<host_name>/opt/api/v1/backend/etls/instance

Example request URL

https://hostname.bmc.com/opt/api/v1/backend/etls/instance

Request Header

Content-Type: application/json
Authorization: Bearer <jwttoken>

Important

In encryption_passphrase, you must use the same passphrase that was used to export the ETL configuration.

Request body

{
"encryption_passphrase": "<password>",
"scheduling": {
   "scheduler_id": 1,
   "period_sec": 86400,
   "start_date": "2022-08-01T00:00:00Z"
},
"etl": {
   "instance_name": "AWS TETS TO DELEEEEEEEEEE",
   "module_name": "com.bmc.bco.aws.extractor.DMAWSExtractorE",
   "module_description": "Amazon Web Services - AWS API Extractor",
   "module_id": 151
},
"entity_catalog": {
   "sharing_status": "PRIVATE"
},
"object_relationships": {
   "domain_mode": "EXISTING",
   "domain_id": "129112"
},
"task_group_id": 5,
"properties": {
   "extract.access.key.id": "AKIAJNKJWEC2TDWDG56A",
   "extract.access.key.secret.password.encrypted": "FdxnkeLXO8x7/MAHbVaHEqx6GG5j1heFWoFgQjYiawrHz/icYcdDov9BV5V3u2vest99aJ3rGzVSoYO8Vl8v1HcDs70=",
   "extract.aws.account.options": "SINGLE",
   "extract.aws.us.gov.cloud": "false",
   "extract.database.iscustom": "true",
   "extract.default.region": "us-east-1",
   "extract.module": "com.bmc.bco.aws.extractor.DMAWSExtractorE",
   "extract.prop.bs.tag.key": "Service",
   "general.dataset.idlist": "1;59;103;7;118;30",
   "general.log.level": "10",
   "general.module.selection": "DATASOURCE",
   "general.options.onemptydataset": "WARN",
   "general.simulation": "false",
   "loader.lookup.remove.domain": "false",
   "loader.sysnm.leave.domain": "false",
   "mask.ignorevalidation": "false"
}
}

Response

HTTP code	Message	Description
200	OK	Request succeeded
500		Error

Importing ETL configuration by using API

Before you begin

Get all entity catalogs

Sample response

Get all schedulers

Sample response

Get all external db connections

Sample response

Get metric profiles

Sample response

Get task groups

Sample response

Get domain ID

Configuring object relationships

New domain

Existing domain

Configuring entity lookup

Private entity catalog

Shared entity catalog

Create an ETL instance with shared lookup and new destination domain

Create an ETL instance with private lookup and use an existing domain

Response

BMC Helix Continuous Optimization 24.2

On this page