This topic describes how to integrate TrueSight Capacity Optimization with VMWare vCloud Director to extract vCloud infrastructure data. The ETL imports infrastructure data from VMware vCloud Director.
In the vCloud Infrastructure ETL, the following entities are fetched by Provider vDCs and Organization vDCs:
For more information, refer to the following sections:
To integrate TrueSight Capacity Optimization with the vCloud infrastructure extractor, perform the following task:
In the Add ETL page, set values for the following properties under each expandable tab.
Note
Basic properties are displayed by default in the Add ETL page. These are the most common properties that you can set for an ETL, and it is acceptable to leave the default selections for each as is.
Property | Description |
---|---|
Run configuration |
|
ETL task name | By default, this field is populated based on the selected ETL module. You can specify a different name for the ETL Task. Duplicate names are allowed. |
Run configuration name | Default name is already filled out for you. This field is used to differentiate different configurations you can specify for the ETL task. You can then run the ETL task based on it. |
Deploy status | You can select Production or Test to mark the ETL tasks. For example, you can start by marking the task as Test and change it to Production after you have seen that you are getting what you wanted. |
Description | (Optional) Enter a brief description for this ETL. |
Log level | Select how detailed you want the ETL log to be. The log includes Error, Warning and Info type of log information.
Note: Log levels 5 and 10 are typically used for debugging or troubleshooting ETL issues. Using a log level of 5 is general practice, however, you may choose level 10 to get a high level of detail while troubleshooting. |
Execute in simulation mode | Select Yes if you want to to validate the connectivity between the ETL engine and the target, and to ensure that the ETL does not have any other configuration issues. When set to Yes, the ETL will not store actual data into the data warehouse. This option is useful while testing a new ETL task. |
Module selection | Ensure that the Based on datasource option is selected. Note If you select Based on Open ETL template, TrueSight Capacity Optimization is integrated with a Generic extractor based on the selected Open ETL template. For more information, see Generic ETL based on a template. |
ETL module | Select VMware - vCloud Infrastructure extractor. |
Module description | A link in the user interface that points you to this technical document for the ETL. |
Entity catalog |
|
Sharing status | Select any one:
|
Object relationships |
|
Associate new entities to | Specify the domain where you want to add the entities created by the ETL. You can select an existing domain or create a new one. Additional Information By default, a new domain is created for each ETL, with the same name as that of the extractor module. As the ETL is created, a new hierarchy rule with the same name of the ETL task is automatically created, with status Active. If you update the ETL by specifying a different domain, the hierarchy rule is updated automatically. Select any one of the following options:
|
vCloud Director ETL configuration |
|
vCloud Director server URL | Enter the vCloud Director server URL. For example, https://abc.example.com/ |
Username | Type the username for a vCloud Director Administrator user. |
Password required | Select either Yes or No. |
Password | Enter the password for the credentials provided. |
ETL task properties |
|
Task group | Select a task group to classify this ETL into. it is not necessary to group it into a task group. |
Running on scheduler | Select the scheduler over which you want to run the ETL. The type of schedulers available are:
|
Maximum execution time before warning | The number of hours, minutes or days to execute the ETL for before generating warnings or alerts, if any. |
Frequency | Select the frequency for ETL execution. Available options are:
|
Start timestamp: hour\minute (Applies to Predefined frequency) | The HH:MM start timestamp to add to the ETL execution running on a Predefined frequency. |
Custom start timestamp | Select a yyyy-mm-dd hh:mm timestamp to add to the ETL execution running on a Custom frequency. |
Note
To view or configure Advanced properties, click Advanced. You do not need to set or modify these properties unless you want to change the way the ETL works. These properties are for advanced users and scenarios only.
Property | Description |
---|---|
Run configuration |
|
Datasets | Enables you to select or deselect metric groups for which data will be populated Available datasets. The OpenStack connector allows you to choose only from the given list of datasets, and you cannot include additional datasets to the run configuration of the ETL.
|
Collection level |
|
Metric profile selection | Select any one:
For more information, see Adding and modifying metric profiles. |
Levels up to | The metric level defines the amount of metric imported into the data warehouse. If you increase the level, additional load is added to the data warehouse while decreasing the metric level reduces the number of imported metrics. Choose the metric level to apply on selected metrics:
For more information, see Aging Class mapping. |
Metric filter |
|
Metric list <for selected dataset> | Click Edit and select the metrics that will be loaded for each dataset that you selected under Run configuration > Datasets. If no metric is specified, all metrics will be loaded. |
Additional properties |
|
List of properties | Additional properties can be specified for this ETL that act as user inputs during execution. You can specify values for these properties either at this time, or from the "You can manually edit ETL properties from this page" link that is displayed for the ETL in view mode.
|
Loader configuration |
|
Empty dataset behavior | Choose one of the following actions if the loader encounters an empty dataset:
|
ETL log file name | Name of the file that contains the ETL execution log; the default value is:%BASE/log/%AYEAR%AMONTH%ADAY%AHOUR%MINUTE%TASKID |
Maximum number of rows for CSV output | A number which limits the size of the output files. |
CSV loader output file name | Name of the file generated by the CSV loader; the default value is:%BASE/output/%DSNAME%AYEAR%AMONTH%ADAY%AHOUR%ZPROG%DSID%TASKID |
Capacity Optimization loader output file name | Name of the file generated by the TrueSight Capacity Optimization loader; the default value is:%BASE/output/%DSNAME%AYEAR%AMONTH%ADAY%AHOUR%ZPROG%DSID%TASKID |
Detail mode | Select the level of detail:
For more information, see Accessing data using public views and Sizing and scalability considerations. |
Reduce priority | Select either Normal or High. |
Remove domain suffix from datasource name | (Only for systems) If set to True, the domain name is removed from the data source name. For example, server.domain.com will be saved as server . |
Leave domain suffix to system name | (Only for systems) If set to True, the domain name is maintained in the system name. For example: server.domain.com will be saved as such. |
Update grouping object definition | If set to True, the ETL will be allowed to update the grouping object definition for a metric loaded by an ETL. |
Skip entity creation | (Only for ETL tasks sharing lookup with other tasks) If set to True, this ETL does not create an entity, and discards data from its data source for entities not found in the product. It uses one of the other ETLs that share lookup to create the new entity. |
Scheduling options |
|
Hour mask | Specify a value to execute the task only during particular hours within the day. For example, 0 – 23 or 1,3,5 – 12. |
Day of week mask | Select the days so that the task can be executed only during the selected days of the week. To avoid setting this filter, do not select any option for this field. |
Day of month mask | Specify a value to execute the task only during particular days within a month. For example, 5, 9, 18, 27 – 31. |
Apply mask validation | By default this property is set to True. Set it to False if you want to disable the preceding Scheduling options that you specified. Setting it to False is useful if you want to temporarily turn off the mask validation without removing any values. |
Execute after time | Specify a value in the hours:minutes format (for example, 05:00 or 16:00) to wait before the task must be executed. This means that once the task is scheduled, the task execution starts only after the specified time passes. |
Enqueueable | Select one of the following options:
|
The following TrueSight Capacity Optimization metrics are populated by the VMware - vCloud Director - Cloud infrastructure extractor ETL.
Viewing assistance
To toggle full-screen mode (recommended) for this topic, type f. To hide the navigation tree on the left, type [.
Legend:
C: Configuration metric
T: Time Series metric
Entity | TrueSight Capacity Optimization metric | Type* | Dataset | Description | vCloud Metric |
---|---|---|---|---|---|
vCloud | VCLOUD_NAME | C | SYSVMW | Name of the vCloud system | - |
CPU_ALLOCATION | T | SYSVIRGLB | Number of Units that have been allocated to all orgvDCs in the vCloud. | - | |
CPU_TOTAL_MHZ | C | SYSCNF | Total CPU available for vCloud. | - | |
CPU_UTILMHZ | T | SYSGLB | Sum of CPU used by all child Provider vDCs | - | |
CPU_UTIL | T | SYSGLB | Percentage (%) utilization, ratio of CPU_UTILMHZ / CPU_TOTAL_MHZ . | - | |
MEM_ALLOCATION | T | SYSVIRGLB | Number of units that have been allocated to consumers that is all orgvDCs in the vCloud. | - | |
TOTAL_REAL_MEM | C | SYSCNF | Total memory available for PvDC | - | |
MEM_USED | T | SYSGLB | Sum of memory used by all child Org vDCs. | - | |
MEM_UTIL | T | SYSGLB | Percentage (%) utilization, ratio of MEM_USED/TOTAL_REAL_MEM . | - | |
DSTORE_ALLOCATION | T | SYSVIRGLB | Number of Units that have been allocated to consumers that is all orgvDCs in the vCloud. | - | |
TOTAL_DSTORE_SIZE | C | SYSVIRDS | Total storage available for vCloud. | - | |
TOTAL_DSTORE_USED | T | SYSVIRDS | Storage used | - | |
TOTAL_DSTORE_UTIL | T | SYSVIRDS | Percentage (%) utilization, ratio of TOTAL_DSTORE_USED / TOTAL_DSTORE_SIZE. | - | |
Provider vDC | VCLOUD_NAME | C | SYSVMW | Name of the vCloud system. | - |
PROVIDERVDC_NAME | C | SYSVMW | Name of the Provider vDC. | QueryResultVMWProviderVdcRecordType.name | |
CPU_ALLOCATION | T | SYSVIRGLB | Number of Units that have been allocated to all orgVDCs under Provider vDC. | QueryResultVMWProviderVdcRecordType.cpuAllocationMhz | |
CPU_TOTAL_MHZ | C | SYSCNF | Total CPU available for PvDC. | QueryResultVMWProviderVdcRecordType.cpuLimitMhz | |
CPU_UTILMHZ | T | SYSGLB | Sum of CPU reserved by all child Org vDCs. | QueryResultVMWProviderVdcRecordType.cpuUsedMhz | |
CPU_UTIL | T | SYSGLB | Percentage (%) utilization, ratio of CPU_UTILMHZ / CPU_TOTAL_MHZ. | QueryResultVMWProviderVdcRecordType.cpuUsedMhz / QueryResultVMWProviderVdcRecordType.cpuLimitMhz) * 100 | |
MEM_ALLOCATION | T | SYSVIRGLB | Number of Units that have been allocated to all orgVDCs under Provider vDC. | QueryResultVMWProviderVdcRecordType.memoryAllocationMB | |
TOTAL_REAL_MEM | C | SYSCNF | Total memory available for PvDC. | QueryResultVMWProviderVdcRecordType.memoryLimitMB | |
MEM_USED | T | SYSGLB | Sum of memory reserved by all child Org vDCs. | QueryResultVMWProviderVdcRecordType.memoryUsedMB | |
MEM_UTIL | T | SYSGLB | Percentage (%) utilization, ratio of MEM_USED/TOTAL_REAL_MEM . | (QueryResultVMWProviderVdcRecordType.memoryUsedMB/ QueryResultVMWProviderVdcRecordType.memoryLimitMB) * 100 | |
DSTORE_ALLOCATION | T | SYSVIRGLB | Number of units that have been allocated to consumers that is all orgvDCs under Provider vDC. | QueryResultVMWProviderVdcRecordType.storageAllocationMB | |
TOTAL_DSTORE_SIZE | C | SYSVIRDS | Total storage available for PvDC. | QueryResultVMWProviderVdcRecordType.storageLimitMB | |
TOTAL_DSTORE_USED | T | SYSVIRDS | Storage Used | QueryResultVMWProviderVdcRecordType.storageUsedMB | |
TOTAL_DSTORE_UTIL | T | SYSVIRDS | Percentage (%) utilization, ratio of TOTAL_DSTORE_USED / TOTAL_DSTORE_SIZE. | (QueryResultVMWProviderVdcRecordType.storageUsedMB/ QueryResultVMWProviderVdcRecordType.storageLimitMB) * 100 | |
HA_STATUS | C | SYSVMW | Read-only indicator. True if compute capacity is highly available. | VMWProviderVdcType.RootComputeCapacityType.IsHA | |
VDC_STATUS | C | SYSVMW | Status of vCloud Virtual Datacenter | QueryResultVMWProviderVdcRecordType.isEnabled | |
Org vDC | VCLOUD_NAME | C | SYSVMW | Name of the vCloud system | - |
PROVIDERVDC_NAME | C | SYSVMW | Name of the Provider vDC | QueryResultVMWProviderVdcRecordType.name | |
ORGVDC_NAME | C | SYSVMW | Organization vDC Name | QueryResultAdminVdcRecordType .name | |
CPU_LIMIT_MHZ | C | SYSVIRGLB | Maximum amount of CPU available to the virtual machines running within this organization vDC (taken from the supporting provider vDC).
| QueryResultAdminVdcRecordType.cpuLimitMhz | |
CPU_RESERVED_MHZ | C | SYSVIRGLB | Guaranteed Number of Units that have been allocated to org vDC. This is allocation multiplied by the Resource Guaranteed CPU. | QueryResultAdminVdcRecordType.cpuAllocationMhz * AdminVdcType.ResourceGuaranteedCpu | |
CPU_UTILMHZ | T | SYSGLB | CPU Used | QueryResultAdminVdcRecordType.cpuUsedMhz | |
CPU_UTIL | T | SYSGLB | Percentage (%) utilization, ratio of CPU_UTILMHZ / CPU_LIMIT_MHZ | (QueryResultAdminVdcRecordType.cpuUsedMhz/ QueryResultAdminVdcRecordType.cpuLimitMhz) * 100 | |
MEM_LIMIT | C | SYSVIRGLB | Maximum amount of memory available to the virtual machines running within this organization vDC (taken from the supporting provider vDC).
| QueryResultAdminVdcRecordType.memoryLimitMB | |
MEM_RESERVED | C | SYSVIRGLB | Guaranteed Number of Units that have been allocated to org vDC. This is allocation multiplied by the Resource Guaranteed memory. | QueryResultAdminVdcRecordType.memoryAllocationMB * AdminVdcType.ResourceGuaranteedMemory | |
MEM_USED | T | SYSGLB | Memory Used | QueryResultAdminVdcRecordType.memoryUsedMB | |
MEM_UTIL | T | SYSGLB | Percentage (%) utilization, ratio of MEM_USED/MEM_LIMIT. | (QueryResultAdminVdcRecordType.memoryUsedMB / QueryResultAdminVdcRecordType.memoryLimitMB) * 100 | |
DSTORE_LIMIT | C | SYSVIRDS | The maximum amount of Storage available to the virtual machines running within this organization vDC (taken from the supporting provider vDC). | QueryResultAdminVdcRecordType.storageLimitMB | |
TOTAL_DSTORE_SIZE | C | SYSVIRDS | Maximum amount of storage available to the virtual machines running within this organization vDC (taken from the supporting provider vDC). | QueryResultAdminVdcRecordType.storageLimitMB | |
TOTAL_DSTORE_USED | T | SYSVIRDS | Amount of storage used by vAPPs, VMs, templates, media files under this Org vDC. | QueryResultAdminVdcRecordType.storageUsedMB | |
TOTAL_DSTORE_UTIL | T | SYSVIRDS | Percentage (%) utilization, ratio of TOTAL_DSTORE_USED / TOTAL_DSTORE_SIZE | (QueryResultAdminVdcRecordType.storageUsedMB / QueryResultAdminVdcRecordType.storageLimitMB) * 100 | |
ALLOCATION_MODEL | C | SYSVMW | Allocation model used by this vDC. One of:
| AdminVdcType.AllocationModel | |
CPU_COMPUTED_LIMIT_OPT | C | SYSVIRGLB | Maximum optimistic amount of CPU the virtual machines running within this organization vDC can avail.
| - | |
MEM_COMPUTED_LIMIT_OPT | C | SYSVIRGLB | Maximum optimistic amount of memory the virtual machines running within this organization vDC can avail.
| - | |
DSTORE_COMPUTED_LIMIT_OPT | C | SYSVIRDS | Maximum optimistic amount of storage the virtual machines running within this organization vDC can avail.
| - | |
VDC_STATUS | C | SYSVMW | Status of vCloud Virtual Datacenter | QueryResultAdminVdcRecordType.isEnabled |
The following TrueSight Capacity Optimization metrics are populated by the vCloud Aggregation Hierarchy Rule
Note
For vCloud Capacity Visibility, the aggregation hierarchy rule executes automatically from the hierarchy manager task.
Entity | TrueSight Capacity Optimization metric | Dataset | Description |
---|---|---|---|
vCloud | HOST_NUM | SYSCNF | Number of hosts attached to vCloud. |
GM_NUM | SYSVIRGLB | Total number of guest machines. | |
GM_ON_NUM | SYSVIRGLB | Total number of guest machines which are turned on. | |
VCPU_NUM | SYSVIRGLB | Total number of virtual CPUs. | |
VCPU_ON_NUM | SYSVIRGLB | Total number of virtual CPUs for VMs that are turned on. | |
Provider vDC | HOST_NUM | SYSCNF | Number of hosts attached to Provider vDC. |
GM_NUM | SYSVIRGLB | Total number of guest machines. | |
GM_ON_NUM | SYSVIRGLB | Total number of guest machines which are turned on. | |
VCPU_NUM | SYSVIRGLB | Total number of virtual CPUs. | |
VCPU_ON_NUM | SYSVIRGLB | Total number of virtual CPUs for VMs that are turned on. | |
Org vDC | GM_NUM | SYSVIRGLB | Total number of guest machines. |
GM_ON_NUM | SYSVIRGLB | Total number of guest machines which are turned on. | |
VCPU_NUM | SYSVIRGLB | Total number of Virtual CPUs. | |
VCPU_ON_NUM | SYSVIRGLB | Total number of virtual CPUs for VMs that are turned on. |
The vCD defines multiple lookup fields for coordinating with other connectors. The following table lists the sequence of field sets for strong lookup and weak lookup.
Entity type | Strong lookup fields | Weak lookup fields |
---|---|---|
vCloud | HOSTNAME | NAME |
Provider vDC | GUID | NAME |
Org vDC | GUID | NAME |
Resource Pool |
| PARENT_VCNAME&&NAME |
Host |
| NAME |
Datastore |
| NAME |
Dataset reference for ETL tasks
1 Comment
Stefan Antelmann