Understanding the Capacity-Aware Placement Advice service
The Capacity-Aware Placement Advice (CAPA) service in TrueSight Capacity Optimization provides real-time recommendations for placement of new workloads on your cloud when provisioning it with BMC Cloud Lifecycle Management. These recommendations are based on available and forecasted capacity.
You will achieve higher utilization and better performance from your cloud compared to traditional round-robin methods if you use the CAPA service.
When combined with BMC Cloud Lifecycle Management, you can continuously optimize your cloud for performance and utilization by optimally adjusting and allocating new workloads according to compute and storage requirements based on CAPA. The service ensures provisioning success by checking capacity availability of Virtual hosts and Virtual clusters. It also performs virtual compute pool placement, and optimizes datacenters over many provisioning requests.
To get a better understanding of key concepts involved in the working of the CAPA service, refer to the following sections in this topic:
Workflow of the CAPA service
- A tenant/user requests a cloud service.
- The BMC Cloud Lifecycle Management Service Governor applies policies to place the service in a container.
- The BMC Cloud Lifecycle Management Resource Manager calls the CAPA API in TrueSight Capacity Optimization before making a placement.
- TrueSight Capacity Optimization computes, recommends and returns the best placement advice to BMC Cloud Lifecycle Management.
- The BMC Cloud Lifecycle Management Service Governor applies policies to place the service.
The actual flow in CAPA can be depicted as:
BMC Cloud Lifecycle Management server > BMC Capacity Management Provider > TrueSight Capacity Optimization > Actual Computation of placement advice > BMC Capacity Management Provider > BMC Cloud Lifecycle Management server
Across a high volume of unpredictable service provisioning requests, BMC Cloud Lifecycle Management and CAPA work together to keep utilization of the infrastructure optimized.
Working of CAPA from the TrueSight Capacity Optimization perspective
CAPA in TrueSight Capacity Optimization provides placement advice based on collected performance data.
The CAPA algorithm considers the following parameters:
- Anticipated resource demand of VM memory, CPU, and disk space.
- Actual utilization and available capacity of virtual hosts and datastores.
- Simulated effect of recent provisioning actions that are not yet reflected in utilization data.
Working of CAPA from the BMC Cloud Lifecycle Management perspective
- CAPA in TrueSight Capacity Optimization chooses between resources selected by the Service Governor.
- The BMC Cloud Lifecycle Management component responsible for making this choice in TrueSight Capacity Optimization is the Capacity Management Provider.
- The Capacity Management Provider runs within BMC Cloud Lifecycle Management via the Platform Manager, and communicates with TrueSight Capacity Optimization via the RESTful API.
Inputs to CAPA
CAPA takes the following as inputs:
- A candidate VM to be created, and:
- ComputePoolPlacementAdvice: A list of target compute pools.
- VirtualHostPlacementAdvice: A list of target hosts.
- VirtualClusterPlacementAdvice: A list of target clusters.
- VirtualResourcePoolPlacementAdvice: list of target virtual resource pools
- OR, a candidate virtual disk to be created, and:
- VirtualDatastorePlacementAdvice: A list of target datastores.
In the VirtualResourcePoolPlacementAdvice call, CAPA supports only a single parent-child level of resource pools. In other words, all resource pools used for CAPA are assumed to be either a root resource pool, or immediate children of the root resource pool in a cluster or host.
The CAPA service can also take as input a candidate virtual disk and a set of target datastores, and output answers as to whether there is enough capacity for the virtual disk, and which datastore should be used to place it.
The most common case of the CAPA service is where BMC Cloud Lifecycle Management is the caller, and VMware vSphere is the underlying infrastructure.
Outputs of CAPA
CAPA produces the following as outputs:
- Ascertain if there is enough capacity on any of the targets listed in inputs?
- YES: Which target should you use? CAPA responds: Use target X because it has the most capacity available.
- NO: Stop trying to provision.
- Not aware of any of these targets. Fall back to default random placement.
- Close call. Fall back to default random placement.
At this point, the Service Governor has already applied its policies and chosen a list of targets. CAPA must choose only between these targets.
What happens when in CAPA
Frequency of event (When)
One time, during installation
The BMC Cloud Lifecycle Management Infrastructure Extractor and Placement Advice Preparation Tasks are configured, deployed and put in the scheduled BMC Cloud Lifecycle Management Placement Advice Chain. A cloud infrastructure domain is also created in the TrueSight Capacity Optimization workspace to accommodate extracted cloud entities.
Every 5 minutes, by default
The BMC Cloud Lifecycle Management Placement Advice Preparation Task verifies if there is any change in cloud topology through the BMC Cloud Lifecycle Management Infrastructure Extractor, and asks the CAPA backend to create or update structures in the internal cache for the corresponding resources.
Every 30 minutes, by default
The vCenter Extractor Service updates data consumption for cloud entities.
Every time a BMC Cloud Lifecycle Management user requests a service
The BMC Cloud Lifecycle Management Provisioning Chain calls the BMC Capacity Management provider, which in turn calls the CAPA service for placement advice. The BMC Cloud Lifecycle Management Provisioning Chain provisions the SOI.
Correlating TrueSight Capacity Optimization with BMC Cloud Lifecycle Management containers
TrueSight Capacity Optimization correlates BMC Cloud Lifecycle Management containers with infrastructure.
- The BMC Cloud Lifecycle Management Infrastructure Extractor obtains cloud container topology (cloud, pool, pod) and creates corresponding entities in the TrueSight Capacity Optimization workspace.
- The vCenter Extractor Service retrieves resource consumption information of vSphere Infrastructure entities and (sharing the lookup with BMC Cloud Lifecycle Management Infrastructure Extractor) correlates them with BMC Cloud Lifecycle Management containers.
Sequence of CAPA calls
The following is the sequence of CAPA calls followed in various cases:
Results of CAPA calls
The results of CAPA calls can be one of following:
Successful – sufficient capacity
There is enough capacity on at least one target, and BMC Cloud Lifecycle Management should use a particular target, specified by CAPA, for provisioning.
Successful – insufficient capacity
There is not enough capacity on any of the targets. BMC Cloud Lifecycle Management should refuse to provision the VM.
CAPA was not able to provide an answer, either because it could not identify the targets mentioned in the call to determine their capacity and utilization, or because of some configuration error. BMC Cloud Lifecycle Management should fall back on its default algorithm to select a target for provisioning.
Estimating the resource usage of candidates
The CAPA service receives configuration data for the candidate VM from BMC Cloud Lifecycle Management for the following metrics:
- Virtual CPU number
- Total Real Memory
TrueSight Capacity Optimization estimates utilization by using these metrics. The estimation can be controlled by using parameter values that can be set on the Placement Advice Preparation Task.
Estimating the available capacity of clusters, hosts, and compute pools of clusters and hosts
The available capacity for targets is obtained by considering CPU and memory utilizations for the following requests from BMC Cloud Lifecycle Management:
- ComputePoolPlacementAdvice for compute pools of clusters or hosts
- Available CPU capacity is obtained by comparing CPU Utilization in MHz (CPU_UTIL_MHZ) with Total Clock Frequency in MHz (CPU_TOTAL_MHz)
- Memory capacity available is obtained by comparing Consumed Memory (MEM_CONSUMED) with Real Memory (TOTAL_REAL_MEM)
Estimating the available capacity of compute pools of resource pools
For those ComputePoolPlacementAdvice calls whose targets are compute pools of resource pools, the following metrics are used:
- The same utilization metrics are used as for hosts and clusters – CPU_UTILMHZ and MEM_CONSUMED.
- For total capacity: Capacities are added up for the contained resource pools as follows:
- For CPU: Total CPU_COMPUTED_LIMIT_PESS of the contained resource pools is taken.
- For memory: Total MEM_COMPUTED_LIMIT_PESS of the contained resource pools is taken.
The above calculations ensure that we do not over-commit the underlying resources that back the resource pools. The PESS metrics are "pessimistic" in that they count only the total of the reserved resource of the resource pool and the proportionally allocated unreserved resource available in the underlying host or cluster.
Estimating the available capacity of VMware resource pools
For the VirtualResourcePoolPlacementAdvice call, the following metrics are used:
- For CPU and memory usage: The same utilization metrics are used as for hosts and clusters – CPU_UTIL_MHZ and MEM_CONSUMED.
- For the total capacity:
- Unlimited_VRP_capacity = Min ( reserved ( VRP ) + unreserved ( cluster ), capacity ( cluster ) – usage ( cluster ) + usage ( VRP ) )
- VRP_capacity = min ( Unlimited_VRP_capacity, limit ( VRP ) )
where limit can be CPU_LIMIT or MEMORY_LIMIT
If the VRP is “unlimited”, then there is no limit set, and effectively Unlimited_VRP_capacity is used.
Performing datastore placement
The VirtualDatastorePlacementAdvice is a separate call made by BMC Cloud Lifecycle Management in order to find out whether there is enough datastore space for a virtual disk of a VM, and where to place the virtual disk.
How CAPA places the vDisks of all the VMs of an SOI
The VDS Placement call is made once for every vdisk, usually with the same set of target datastores. CAPA tries to put all the vdisks for all VMs of the same SOI, on the same datastore. For each call, CAPA selects a datastore based on maximum available capacity. If this datastore would accommodate only some of the vdisks but not all, then CAPA tries to place the remaining disks on another datastore, and so on. If all the CAPA calls for the SOI (including the compute as well as the datastore placement calls) succeed, then CLM will proceed with the provisioning action.
The VDS Placement calls for different vdisks of the same SOI do not necessarily pass in the same set of target datastores. CAPA remembers the datastore that was decided for one vdisk, and if that same datastore is passed in as a target on subsequent calls for the same SOI, then it tries to use the same datastore again if possible.
How CAPA places the vdisks of all the VMs of multiple SOIs
CAPA might get calls for multiple SOI vdisks interleaved. Each vdisk placement decision is made independently, based on the maximum available capacity. CAPA remembers placement decisions made earlier so that it can take into account the anticipated space needed for vdisks whose placement has already been decided. Still, CAPA tries to place the vdisks of a single SOI on the same datastore if possible. It does not try to place vdisks for different SOIs on the same datastore, but it does not try to prevent such a placement.
Parameters to modify Datastore Placement behavior
For the candidate virtual disk to be placed, CAPA uses the following parameters by default:
- Candidate default disk size (if not available)
- Disk PPA multiplier (applied to disk size for simulation)
Normally the above parameters are not used by TrueSight Capacity Optimization, because the VirtualDatastorePlacementAdvice call specifies the size of the virtual disk. To understand what these parameters mean and when they might need to be modified, see Affinity settings.
Metrics used for Datastore Placement
For this call, CAPA uses the following Datastore metrics to compute available capacity on the targets:
- Datastore free space (DSTORE_FREE)
- Datastore size (DSTORE_SIZE)
The following threshold is applied (settable, again, as described below):
- Datastore space used threshold
Viewing results of CAPA calls
The following sections discuss how and where you can view results of CAPA calls.
Viewing logs of the CAPA service
For details on each call, including the arguments (candidate and targets) and the results, see the CAPA service log located under Administration tab > Data hub > API Providers - CAPA service.
For help on how to access the Data hub administration GUI, see Data hub administration.
Keywords used in log entries
The following table lists all keywords used in the CAPA log entries.
Indicates the Service Offering Instance (SOI) identifier in BMC Cloud Lifecycle Management
Indicates the type of CAPA call. It can be one of the following:
Indicates the number of CPUs and memory configured for the VM being provisioned.
Indicates the list of target resources (hosts, clusters, resource pools) passed into the call. A tool-tip over each target shows its CPU and memory consumption as last seen by TrueSight Capacity Optimization.
Identifies the target that was selected by the CAPA call.
It is the result of the CAPA call. Normally, the result is that the CAPA service returns the target to be used by BMC Cloud Lifecycle Management for provisioning.
Error statuses in CAPA
The CAPA service has several error statuses. A few of these are discussed in the following table.
Error status code
This is a generic issue in TrueSight Capacity Optimization, check in Datahub logs
Insufficient Capacity error
None of the targets of a call is able to accommodate the input VM (using “real” data). This status is treated as a prohibition by BMC Cloud Lifecycle Management; it will fail the provisioning action due to insufficient capacity.
Insufficient Capacity based on PPA error
None of the targets of a call is able to accommodate the input VM if PPA simulations are considered, but at least one has enough resources if PPA simulations are not considered. BMC Cloud Lifecycle Management does not treat this as a prohibition; it will still select the returned target for provisioning.
No Valid targets are included in a call
This can be due to a failure in the lookup or to missing configuration metrics. BMC Cloud Lifecycle Management treats this as a problem with CAPA and falls back to its default method for placement.
For more information on error codes, see Error codes and Return Status (error) codes returned by the Capacity-Aware Placement Advice service.
Model preparation in TrueSight Capacity Optimization
The Placement Advice Preparation Task:
- Creates placement advice models
- Keeps models aligned with the cloud infrastructure
Here are some key points about the Placement Advice Model:
- It is optimized for speed and suitable for real time execution
- Uses a 24-hour profile for compute resources, CPU and Memory
- Simulates pending provisioning actions that are not yet in performance data
- One model is created for each pool in the cloud
Every model in TrueSight Capacity Optimization includes elements of the matching cloud entity in BMC Cloud Lifecycle Management. For example, cluster compute pool models contain clusters, host compute pool models contain hosts, and so on.
Turnaround time of the Placement Advice model
If you are creating available capacity by moving away workloads, or by adding CPU or memory, then you don't need to do anything special for CAPA to start using the new capacity.
To understand how quickly CAPA will react, and to try to make it react faster if you need to, see Placement Internals used by the Placement Advice model.
There are many parameters listed in the Placement Advice configuration tab of the Placement Advice Preparation Task. Usually you are not expected to modify these. If you would like to change the behavior of CAPA in general (for example, to make it more aggressive to increase consolidation ratios), then the following are relevant:
Target CPU threshold
Stop placing new VMs when target reaches this utilization. This parameter applies to both host and cluster targets. (Do not use the parameter named Target cluster-level CPU threshold)
Datastore space used threshold
Stop placing new candidate virtual disks at this level of space utilization.
Target memory threshold
Stop placing candidate VMs on the target at this level of memory utilization. This parameter applies to both host and cluster targets.
The parameters in the following table do modify the behavior of CAPA when initially placing the candidate, but this effect wears off when the actual target performance becomes known to TrueSight Capacity Optimization. So, these will make a difference only for the last candidates placed before thresholds are exceeded – not very useful for increasing consolidation ratios.
Candidate default CPU
Assume this CPU speed allocated for the VMs to be created
Candidate default disk size
Assume this size if BMC Cloud Lifecycle Management does not know the specified size of vDisk
Candidate default CPU utilization
Assume this percentage for VMs to be created
Candidate default memory utilization
Assume this percentage for VMs to be created
If you modify any of these parameters, verify that the Placement Advice Preparation task is running every 5 minutes as it should, so that it will pick up the new values.
Placement internals used by the Placement Advice model
This section discusses the following:
- Placement Advice Model
- Pending Provisioning Action (PPA) simulations
- Response of the model to changing conditions
Placement Advice Model
CAPA uses a Placement Advice Model in TrueSight Capacity Optimization to compute its answers. Targets of every single call are sorted according to their amount of free resources and they are chosen according to this sorting, so that targets with more available capacity are preferred.
In case of calls on clusters (VirtualClusterPlacementAdvice), the evaluation of the feasibility of the placement is done at host level (considering all the hosts contained in all the clusters), so that there is at least one host in the chosen cluster that has the required capacity.
Consumption data used to compute placement are organized in Day Profiles, for CAPA. Therefore, for every target and resource, 24 samples (corresponding to all hourly slots of a day profile: e.g. 0.00, 1:00, …, 23:00) are considered. For CAPA, the model is predefined to use the last 24-hour period as the time filter for the analysis period.
Placement of the input VM of the call is evaluated on every sample of the target. If the VM configuration coming from the placement call is a single value, then the algorithm is equivalent to considering the maximum value of utilization for the target.
Pending Provisioning Action (PPA) simulations
As TrueSight Capacity Optimization gets performance data only periodically, the utilization data it has available when provisioning a VM can often be somewhat old. As many service offering instances are being provisioned rapidly, it is possible that recent provisioning actions are not yet reflected in the utilization data.
CAPA is designed to take into account the impact of these PPAs, and simulates their effect on top of the utilization data available from the targets on which they are provisioned. For the compute placement calls, the PPA is a simulation of the CPU and memory usage of the VM being provisioned. For the datastore placement calls, the PPA is a simulation of the disk space of the virtual disk being provisioned.
CAPA maintains these PPA simulations on the corresponding targets in its models and expires them after the performance data is expected to appear.
The time period for which these PPA simulations are maintained can be adjusted using the "Scenario PPA Time Validity" option in the General configuration tab of the run configuration of the Placement Advice preparation task.
There is usually no need to change this time period. Its only purpose is to distribute placement correctly in the face of a flood of provisioning requests. If you are finding many BCO_PAS_ERR112 warnings (Insufficient Capacity based on PPA), then the problem is that none of the targets have enough capacity, or that the candidate size estimates are too large. The latter can be adjusted via the candidate size parameters above.
If you are running in an environment with a slow utilization feed (for example, more than two hours) and find that PPA simulations are expiring too soon, then it might help to increase this value.
Response of the model to changing conditions
The choice of a 24 hour analysis period provides a good balance between an aggressive response to sudden increased utilization and a more cautious response to sudden decreased utilization. Thus, if you move existing workloads away from the targets, or remove them completely, the new available capacity on the targets will automatically be utilized by CAPA in a cautious manner.
It could take up to 24 hours for the peak utilization to be forgotten by the model. To make the model temporarily respond faster, you can decrease the analysis period by using the Placement Advice Preparation task option (see the General Configuration tab). Remember to change it back to 24 hours after the peak has passed, otherwise CAPA will become very aggressive and increase the chances of provisioning failures due to lack of capacity.
If you add CPU or memory to one or more targets, then the response of CAPA to this increased capacity is much quicker. Recall that the target CPU and memory utilization is being measured using actual CPU MHz consumed and actual memory GB consumed, against the available resources. As soon as TrueSight Capacity Optimization detects the new capacity, the behavior of CAPA will change to take into account the new capacity.
If you have access to all the features of TrueSight Capacity Optimization, then you can view the actual target utilization values used by the model. Find the target and create a quick analysis using a "last 24 hours" time filter with hourly resolution in the console workspace.
Where to go from here
To know what you need to do before you can start using CAPA, see Configuring BMC Cloud Lifecycle Management before using CAPA.
To know more about working with and using CAPA, see Working with and using CAPA.