In TrueSight Capacity Optimization, integration with OpenStack is supported through the OpenStack API Extractor Service that brings in OpenStack entities into TrueSight Capacity Optimization.
The OpenStack API Extractor Service supports only VMware and KVM virtualizations.
The TrueSight Capacity Optimization integration with the OpenStack API Extractor Service supports OpenStack versions Liberty, Mitaka, and Newton.
TrueSight Capacity Optimization supports OpenStack v3 API for the Identity service, only in OpenStack Newton deployments.
The ETL now supports data collection by using Nova APIs. To use Nova APIs, apply hotfix CHF#14 or later to TrueSight Capacity Optimization, and configure the ETL. You can download the hotfix from
this location
.
For information about integrating this ETL service with TrueSight Capacity Optimization, see:
Note
The following ports need to be opened for the OpenStack - OpenStack API Extractor Service:
If Ceilometer collection is enabled, open the port: Ceilometer Service URL (default: 8777)
To integrate TrueSight Capacity Optimization with the OpenStack - OpenStack API Extractor Service, perform the following steps:
In the Add ETL page, set values for the following properties under each expandable tab.
Note
Basic properties are displayed by default in the Add ETL page. These are the most common properties that you can set for an ETL, and it is acceptable to leave the default selections for each as is.
Property | Description |
---|---|
Run configuration |
|
ETL task name | By default, this field is populated based on the selected ETL module. You can specify a different name for the ETL Task. Duplicate names are allowed. |
Run configuration name | The default name is already filled out. This field is used to differentiate between various configurations you can specify for the ETL task. You can then run the ETL task based on that. |
Deploy status |
You can select Production or Test to mark the ETL tasks. For example, you can start by marking the task as Test and change it to Production after you have seen that the results are as expected. |
Description | (Optional) Enter a brief description for this ETL. |
Log level | Select how much detail you want the ETL log to show. The log includes Error, Warning and Info type of log information.
Note Log levels 5 and 10 are typically used for debugging or troubleshooting ETL issues. Using a log level of 5 is general practice, however, you may choose level 10 to get a high level of detail while troubleshooting. |
Execute in simulation mode | Select Yes if you want to to validate the connectivity between the ETL engine and the target, and to ensure that the ETL does not have any other configuration issues. When set to Yes, the ETL will not store actual data into the data warehouse. This option is useful while testing a new ETL task. |
Module selection | Ensure that the Based on datasource option is selected. Note If you select Based on Open ETL template, BMC TrueSight Capacity Optimization is integrated with a Generic extractor based on the selected Open ETL template. For more information, see Generic ETL based on a template. |
ETL module (BMC recommends that you make this selection before the previous properties) |
Select OpenStack - OpenStack API Extractor Service. |
Module description | This property is a short description of the module, with a link in the user interface that points you to the technical document for this ETL. |
Entity catalog |
|
Sharing status | Set the sharing status by selecting one of the following options:
|
Object relationships |
|
Associate new entities to | Specify the domain where you want to add the entities created by the ETL. You can select an existing domain or create a new one. Note By default, a new domain is created for each ETL, with the same name of the extractor module. As the ETL is created, a new hierarchy rule with the same name of the ETL task is automatically created, with status "active," if you update the ETL specifying a different domain, the hierarchy rule is updated automatically. Select any one of the following options:
|
OpenStack configuration |
|
Identity service URL | Provide the URL for the OpenStack Identity Service. The Identity Service is used to obtain tokens for accessing cloud services. If the identity service has an HTTPS URL, then the ETL communicates securely over HTTPS. |
Username | Provide the user name to access the Identity Service. This authenticated user must have an admin or ro_admin (read-only admin) role assigned for the specified tenant. The specified user must have read access to all OpenStack resources and endpoints. |
Password required | Select this option if a password is required to access the service. |
Password | If a password is required to access the Identity Service, enter the password. |
Tenant name | Enter the name of the Keystone tenant. The username specified above should have an admin or ro_admin role for the specified tenant. |
Cloud name |
(Optional) Type a name. If you do not enter a name, a default name is created using "Cloud_" as a prefix and the OpenStack identity endpoint (host name, IP address) as the suffix. |
Domain name | Specify a domain name to collect data for or use the default domain. |
Ceilometer data collection |
|
In addition to collecting data for standard metrics, you can configure data collection for Ceilometer metrics for specific platforms. To enable use of Ceilometer API to extract metrics for specific platforms, configure one of the following options: |
|
KVM Hypervisor | Select one of the following options to collect data from the KVM Hypervisor resources:
If you select the Ceilometer option, see OpenStack Clouds - Support for Ceilometer for more details. |
VMware Hypervisor | Select one of the following options to collect data from the VMware Hypervisor resources:
If you select the Ceilometer option, see OpenStack Clouds - Support for Ceilometer for more details. |
ETL task properties |
|
Task group | (Optional) Select a task group by which to classify this ETL. |
Running on scheduler | Select the scheduler over which you want to run the ETL. The type of schedulers available are:
|
Maximum execution time before warning | Indicate the number of hours, minutes or days for which to execute the ETL before generating warnings or alerts, if any. |
Frequency | Select the frequency for ETL execution. Available options are:
|
Start timestamp: hour\minute (Applies to Predefined frequency) |
Indicate the HH:MM start timestamp to add to the ETL execution running on a Predefined frequency. |
Custom start timestamp | Select a yyyy-mm-dd hh:mm timestamp to add to the ETL execution running on a Custom frequency. |
Note
To view or configure Advanced properties, click Advanced. You do not need to set or modify these properties unless you want to change the way the ETL works. These properties are for advanced users and scenarios only.
Property | Description |
---|---|
Run configuration |
|
Datasets | Enables you to select or clear metric groups for which data will be populated. The OpenStack connector allows you to choose only from the given list of datasets; you cannot include additional datasets to the run configuration of the ETL.
|
Collection level |
|
Metric profile selection | Select any one:
For more information, see Adding and modifying metric profiles. |
Levels up to | The metric level defines the amount of metric imported into the data warehouse. If you increase the level, additional load is added to the data warehouse, while decreasing the metric level reduces the number of imported metrics. Choose the metric level to apply on selected metrics:
For more information, see Aging Class mapping. |
Metric filter |
|
Metric list for <selected> datasets | Click Edit and select the metrics that will be loaded for each dataset that you selected under Run configuration > Datasets. If no metric is specified, all metrics will be loaded. |
OpenStack configuration |
|
Role name | Specify the role name that is assigned to the user to access the OpenStack resources. |
Additional properties |
|
List of properties | Specify additional properties for this ETL that act as user input during execution. You can specify values for these properties either at this time, or from the "You can manually edit ETL properties from this page" link that is displayed for the ETL in view mode.
|
Loader configuration |
|
Empty dataset behavior | Choose one of the following actions if the loader encounters an empty dataset:
|
ETL log file name | Enter the name of the file that contains the ETL execution log; the default value is: %BASE/log/%AYEAR%AMONTH%ADAY%AHOUR%MINUTE%TASKID |
Maximum number of rows for CSV output | Enter a number that limits the size of the output files. |
CSV loader output file name | Enter the name of the file generated by the CSV loader; the default value is: %BASE/output/%DSNAME%AYEAR%AMONTH%ADAY%AHOUR%ZPROG%DSID%TASKID |
Capacity Optimization loader output file name | Enter the name of the file generated by the BMC TrueSight Capacity Optimization loader; the default value is: %BASE/output/%DSNAME%AYEAR%AMONTH%ADAY%AHOUR%ZPROG%DSID%TASKID |
Detail mode | Select the level of detail:
For more information, see Accessing data using public views and Sizing and scalability considerations. |
Reduce priority | Select either Normal or High. |
Remove domain suffix from datasource name | (Only for systems) If set to True, the domain name is removed from the data source name. For example, server.domain.com will be saved as server . |
Leave domain suffix to system name | (Only for systems) If set to True, the domain name is maintained in the system name. For example: server.domain.com will be saved as such. |
Update grouping object definition | If set to True, the ETL will be allowed to update the grouping object definition for a metric loaded by an ETL. |
Skip entity creation | (Only for ETL tasks sharing lookup with other tasks) If set to True, this ETL does not create an entity, and discards data from its data source for entities not found in BMC Capacity Optimization. It uses one of the other ETLs that share lookup to create the new entity. |
Scheduling options |
|
Hour mask | Specify a value to execute the task only during particular hours within the day. For example, 0 – 23 or 1,3,5 – 12. |
Day of week mask | Select the days so that the task is executed only during the selected days of the week. To avoid setting this filter, do not select any option for this field. |
Day of month mask | Specify a value to execute the task only during particular days within a month. For example, 5, 9, 18, 27 – 31. |
Apply mask validation | By default this property is set to True. Set it to False if you want to disable the preceding Scheduling options that you specified. Setting it to False is useful if you want to temporarily turn off the mask validation without removing any values. |
Execute after time | Specify a value in the hours:minutes format (for example, 05:00 or 16:00) to wait before the task must be executed. This option means that after the task is scheduled, the task execution starts only after the specified time passes. |
Enqueueable | Select one of the following options:
|
Click Save.
You are returned to the Last run tab under the ETL tasks page.
The OpenStack API Extractor Service defines multiple lookup fields for coordinating with other connectors. The following table lists the sequence of field sets for strong lookup and weak lookup.
Entity type (Click to navigate to section) | Strong lookup fields | Weak lookup fields |
---|---|---|
Cloud - OpenStack | OPENSTACK_GUID | - |
Region - OpenStack | OPENSTACK_GUID | - |
Availability Zone - OpenStack | AZ_NAME&&PARENT_OS_GUID | - |
Host Aggregate - OpenStack | NAME&&PARENT_OS_GUID | - |
Virtual Host - KVM | HOSTNAME | NAME |
Virtual Host - VMware | HOSTNAME | NAME |
Virtual Cluster - VMware | NAME&&VMW_VMREF | NAME |
Virtual Machine - KVM | UUID | NAME |
Virtual Machine - VMware | HOSTNAME | NAME |
Tenant - OpenStack | OPENSTACK_GUID | - |
Nova memory metrics are collected only for the current hour when the OpenStack - OpenStack API Extractor Service is run for the first time.
Metric | Source type | Source info | Description |
---|---|---|---|
CLOUD_NAME | NA | NA | Name of the cloud. If a user-defined name exists, that name is displayed. Otherwise, the default name is displayed which is composed of a prefix "Cloud_" and the OpenStack identity endpoint (host name, ip adders) as suffix. |
HOST_NUM | API | NA | Number of hosts. |
Configuration metrics | Performance metrics |
---|---|
|
|
Metric | Source type | Source info | Description |
---|---|---|---|
CLOUD_NAME | - | - | Name of the cloud. Any name you specify is prefixed with "Cloud_" and the OpenStack identity endpoint (host name, ip adders) is added as suffix. |
HOST_NUM | API | - | Number of hosts. |
Derived metrics (values from the region that belong to the cloud):
Configuration metrics | Performance metrics |
---|---|
|
|
Metric | Source type | Source info | Description |
---|---|---|---|
CLOUD_NAME | - | - | Name of the cloud. If a user-defined name exists, that name is displayed. Otherwise, the default name is displayed, which is composed of a prefix "Cloud_" and the OpenStack identity endpoint (host name, ip adders) as suffix. |
REGION_NAME | API | - | Name of the region read from the OpenStack API. |
HOST_NUM | API | - | Number of hosts. |
Configuration metrics | Performance metrics |
---|---|
|
|
Note
TOTAL_DSTORE_SIZE, TOTAL_DSTORE_USED and TOTAL_DSTORE_UTIL metrics are available only when the OpenStack ETL Service is shared with Gateway Server or VMWare vCenter ETLs.
Metric | Source type | Source info | Description |
---|---|---|---|
CLOUD_NAME | - | - | Name of the cloud. If a user-defined name exists, that name is displayed. Otherwise, the default name is displayed, which is composed of a prefix "Cloud_" and the OpenStack identity endpoint (host name, ip adders) as suffix. |
REGION_NAME | API | - | Name of the region read from the OpenStack API. |
AVAILABILITY_ZONE | API | /v2.1/<tenant_id>/os-aggregates - aggregates.metadata.availability_zone | Availability Zone read from the OpenStack API. |
HOST_NUM | API | - | Number of hosts. |
Configuration metrics | Performance metrics |
---|---|
|
|
Note
TOTAL_DSTORE_SIZE, TOTAL_DSTORE_USED and TOTAL_DSTORE_UTIL metrics are available only when the OpenStack ETL Service is shared with Gateway Server or VMWare vCenter ETLs.
Metric | Source type | Source info | Description |
---|---|---|---|
**CPU_NUM | API | /v2.1/<tenant_id>/os-hypervisors/detail -> hypervisors.cpu_info.topology.cores | Hypervisor number of CPU cores configured in OpenStack. |
**VCPU_NUM | API | /v2.1/<tenant_id>/os-hypervisors/detail -> hypervisors.vcpus_used | Hypervisor number of Virtual CPUs configured in OpenStack. |
**GM_NUM | API | /v2.1/<tenant_id>/os-hypervisors/detail -> hypervisors.running_vms | Hypervisor number of virtual manhines configured in OpenStack. |
**TOTAL_REAL_MEM | API | /v2.1/<tenant_id>/os-hypervisors/detail -> hypervisors.memory_mb | Hypervisor total real memory configured in OpenStack. |
**MEM_CONSUMED (for VMware) | API | /v2.1/<tenant_id>/os-hypervisors/detail -> hypervisors.memory_mb_use | Hypervisor total memory consumed in OpenStack for VMware. |
**MEM_USED (for KVM) | API | /v2.1/<tenant_id>/os-hypervisors/detail -> hypervisors.memory_mb_use | Hypervisor total memory used in OpenStack (for KVM). |
**MEM_FREE | API | /v2.1/<tenant_id>/os-hypervisors/detail -> hypervisors.free_ram_mb | Hypervisor total memory available in OpenStack. |
TOTAL_EPHEMERAL_DISK_SIZE | API | /v2.1/<tenant_id>/os-hypervisors/detail -> hypervisors.local_gb | Hypervisor ephemeral disk size configured in OpenStack. |
TOTAL_EPHEMERAL_DISK_PROVISIONED | API | /v2.1/<tenant_id>/os-hypervisors/detail -> hypervisors.local_gb_used | Hypervisor ephemeral disk size provisioned in OpenStack. |
**VCPU_ON_NUM | API | /v2/<tenant_id>/os-hypervisors/detail - hypervisors.vcpus | Hypervisor number of powered on virtual CPUs configured in OpenStack. |
**GM_ON_NUM | API | Derived | Hypervisor number of powered on virtual machines configured in OpenStack. |
**MEM_UTIL | API | Derivded | Hypervisor memory utilization configured in OpenStack. |
** - This metric is collected when the Ceilometer collection is enabled durig the ETL configuration. The metric can also be collected by using the Nova APIs. To use Nova APIs, apply hotfix CHF#14 or later to TrueSight Capacity Optimization and configure the ETL. Alternatively, you can fetch this metric from either the Gateway Server extractor (for KVM) or VMware vCenter extractor (for VMware). |
Metric | Source type | Source info | Description |
---|---|---|---|
AVAILABILITY_ZONE | API | /v2.1/<tenant_id>/servers/<server_id>- server.OS-EXT-AZ:availability_zone | Availability Zone read from OpenStack API. |
INSTANCE_NAME | API | /v2.1/<tenant_id>/servers/<server_id> - server.name | Name given to the server in OpenStack. |
**CPU_NUM | API | /v2.1/<tenant_id>/flavors/<server.flavor.id> - flavor.vcpus | Requested number of CPU specified on OpenStack flavor associated to the server. |
**DISK_SIZE | API | /v2.1/<tenant_id>/flavors/<server.flavor.id> - flavor.disk | Requested disk size specified on OpenStack flavor associated to the server. |
**MEM | API | /v2.1/<tenant_id>/flavors/<server.flavor.id> - flavor.ram | Requested memory specified on OpenStack flavor associated to the server. |
**TOTAL_REAL_MEM | API | /v2.1/<tenant_id>/flavors/<server.flavor.id> - flavor.ram | Requested memory specified on OpenStack flavor associated to the server. |
REQUESTED_INSTANCE_TYPE | API | /v2.1/<tenant_id>/flavors/<server.flavor.id> - flavor.name | Flavor name. |
** - This metric is collected when the Ceilometer collection is enabled durig the ETL configuration. The metric can also be collected by using the Nova APIs. To use Nova APIs, apply hotfix CHF#14 or later to TrueSight Capacity Optimization and configure the ETL. Alternatively, you can fetch this metric from either the Gateway Server extractor (for KVM) or VMware vCenter extractor (for VMware). |
Metric | Source type | Source info | Description |
---|---|---|---|
APP_CLOUD_NAME | - | - | Name of the Keystone cloud. The name is available on the Keystone server in the OS_CLOUD_NAME variable of the stack.sh file. |
APP_CPU_NUM_QUOTA | API | /v2.1/<tenant_id>/os-quota-sets/<curr_tenant_id> - quota_class_set.core | Maximum number of CPUs that can be assigned to the Service, Application or Tenant. |
APP_INSTANCE_QUOTA | |||
APP_MEM_QUOTA | API | /v2.1/<tenant_id>/os-quota-sets/<curr_tenant_id> - quota_class_set.ram | Maximum amount of Memory that can be assigned to the Service, Application or Tenant. |
APP_ENVIRONMENT |
This section lists all the API calls performed by the OpenStack API Extractor Service on the identity and compute services.
Task | v2.0 API | v3 API |
---|---|---|
Authenticate and generate a token | POST /v2.0/tokens (configured username, password, and tenant name are passed in the request body) | POST /v3/tokens (configured username, password, and tenant name are passed in the request body) |
Get list of tenants/projects | GET /v2.0/tenants | GET /v3/projects |
Task | API |
---|---|
List of aggregates | GET /v2.1/{tenant_id}/os-aggregates |
List of hypervisors | GET /v2.1/{tenant_id}/os-hypervisors/detail |
List of servers for a hypervisor | GET /v2.1/{tenant_id}/os-hypervisors/{hypervisor_hostname}/servers |
Details for a specific server | GET /v2.1/{tenant_id}/servers/{server_id} |
List of details for flavors | GET /v2.1/{tenant_id}/flavors/detail |
For more information, refer to the OpenStack API documentation website.
OpenStack Clouds - Support for Ceilometer
4 Comments
Michael Paska
Sanjay Pendse
Stefan Antelmann
I think it is worth to make some addition to the Username:
From:
Provide the username to access the Identity Service. This authenticated user must have an admin or ro_admin (read-only admin) role assigned for the specified tenant.
To:
Provide the username to access the Identity Service. This authenticated user must have an admin or ro_admin (read-only admin) role assigned for the specified tenant.
The user defined in the ETL requires read access to all OpenStack resources and all ENDPOINTS.
Bipin Inamdar