Troubleshooting Virtual Guest Jobs

This topic contains the following sections:

• • Overview
  • Virtual Guest Job Execution
  • Increasing the VGJ timeout
  • Disable SOI Rollback on failure
  • Virtual Guest Job errors caused by problematic templates
  • VM Deployment to VirtualHost targets

Overview

A Virtual Guest Job (VGJ) creates the virtual machine (VM) in the Virtual Center and enrolls it in BMC BladeLogic Server Automation.

VM creation includes both cloning and customizing the VM in the Virtual Center. The VGJ fails if an issue occurs during the cloning or customization phase.

You can find error information about failures in VGJs performed by BMC Cloud Lifecycle Management in the following locations:

• In Cloud Portal, access Advanced Error Information to find error details.
• In csm.log, search for the error reported in Advanced Error Information.
   Verify that the error in csm.log is relevant to the failure of the service offering instance (SOI) in question by comparing the name of the SOI in csm.log or matching the time the error occurred.
• On the Virtual Center, access the specific virtual machine in question, and then access Tasks and Events.
• On the Virtual Center server, access bl-vmware.log. (Default Location: C:\Program Files\BMC Software\BladeLogic\8.1\RSCD\vmware\log\bl-vmware.log.) In BMC Server Automation 8.2 and later versions, there is no longer a bl-vmware.log file. All of the  logs related to VMware are written to files named blcoserver_.log. (Default Location: C:\Program Files\BMC Software\BladeLogic\RSCD.) Check the latest logs based on time stamp.
• Virtual Guest Job logs from the BMC Server Automation Console — Select the VGJ from the CSM_Virtual_Guest_Jobs folder, right-click on the job, and select Show Results.
• Application Server log — By default, the BMC Cloud Lifecycle Maintenance Install Planner creates two Application Server instances named config_deployment and job_deployment. The VGJ logs are written to the job_deployment instance. For example: C:\Program Files\BMC Software\BladeLogic\8.1\Operations Manager\NSH\br\job_deployment_hostname.log.

Virtual Guest Job Execution

When the VGJ starts in BMC Server Automation, BMC Cloud Lifecycle Management polls the status every minute.

For example:

The status poll is controlled by these properties in the providers.json file:

• BBSA_VG_JOB_STATUS_POLLS
   The number of times BMC Cloud Lifecycle Management checks to determine if a BMC Server Automation VGJ is complete before timing out. The default value is 60. It can be increased if necessary.
• BBSA_VG_JOB_STATUS_POLL_INTERVAL_SECONDS
   The number of seconds between polls for the VGJ status. The default value is 60. It can be increased if necessary.

When a VGJ fails in BMC Server Automation, the job status polling is completed and csm.log captures the error reported in BMC Server Automation logs.

Increasing the VGJ timeout

When you request multiple VMs in a SOI, the Virtual Center might require more time to complete the cloning, depending on the performance of Virtual Center. Additional time might cause the VGJ to time out and fail. The default value is 60 minutes.

By default, the property Dclone-timeout=3600000 is set to one hour in the vmware.properties file on VC Server.

Default Location:

BMC Server Automation versions earlier than 8.2 — VC Server\\C:\Program Files\BMC Software\BladeLogic\8.1\RSCD\vmware\vmware.properties

BMC Server Automation version 8.2 and later — RSCD_HOME\daal\Implementation\BMC_VMware_VirtualInfrastructureManager_win64\win64\vmware.properties

Increase the value of the property to the time required, such as two hours. In that case the value of the property would be Dclone-timeout=7200000 (milliseconds). Re-start the Virtual Center agent after modifying the value.

Also ensure the relevant properties are modified in BMC Cloud Lifecycle Management as well. Set the following properties to the same amount of time:

BBSA_VG_JOB_STATUS_POLLS
BBSA_VG_JOB_STATUS_POLL_INTERVAL_SECONDS

Disable SOI Rollback on failure

When there is a failure while running any BMC Cloud Lifecycle Management jobs in BMC Server Automation, BMC Cloud Lifecycle Management triggers the de-provision workflow to decommission the VM and free up its associated resources.

In certain situations, you may need to retain the target VM in order to troubleshoot a failure.

To disable the SOI rollback, modify the providers.json file:

• Set enableSOIRollback to false 
• Set BBSA_ENABLE_SOI_ROLLBACK to false.

Restart the CSM service for the settings to take effect.

Virtual Guest Job errors caused by problematic templates

The following error indicates an issue with the template:

Mostly, VGJ issues are specific to the template being used. Try using a new template to quickly isolate the issue.

VM Deployment to VirtualHost targets

When deploying VirtualMachines to VirtualHost targets, verify the associated VirtualCluster has the DRS option turned off. VM deployment to VirtualHosts is not supported on DRS-enabled VirtualClusters.

The following error is generated while provisioning:

As a workaround, you can target the cluster instead of deploying VMs on a specific host. In other words, the compute pools created in BMC Cloud Lifecycle Management should be based on clusters instead of ESX hosts.