Troubleshooting continuous compliance for servers issues

TrueSight Server Automation

Perform the following checks to isolate the issue.

• Check whether ITSM Approval is configured for the Compliance or Audit Job. In the TrueSight Server Automation Console, select Configuration > Approval Configuration.
• Check whether a compliance or audit discrepancy was found. In the Jobs folder, select a Compliance Job or an Audit Job, right-click, and select Show Results.
• Check to see if the interaction with the change management system was successful by reviewing the job schedule log, which is displayed under the job until the job starts running.
• Check the job schedule log by completing the following tasks in the TrueSight Server Automation Console:
  • Select the job from the Jobs folder.
  • Right-click and select Open.
  • Click the Schedule tab.
  • Review the job schedule log to see:
    • when the job has been approved and is waiting to run
    • if the attempt to schedule the job has failed (for example, failed to create or update the change ticket)
  • After the job starts executing after approval, the information in the job schedule log is rolled into job log.
• View the job schedules and note the value of ChangeID, which shows the ID of the Change Request created, if available.
• Check the Job status:
  • Waiting for approval
  • Request approved – Waiting for schedule
  • Request approved – Executed (success or failure)
• After the change request is approved in BMC Remedy ITSM, TrueSight Orchestration triggers the remediation job in TrueSight Server Automation. Check the Remediation Job Results view to see if the job ran. From the Jobs folder, navigate to a job, right-click and select Show Results. Check to see if there were errors in the job run.
• Check to see if the Remediation Job sent results back to BMC Remedy ITSM.

TrueSight Orchestration

Perform the following checks to verify that the modules and adapters are loaded and working.

• Verify that the TrueSight Orchestration modules and adapters are configured and loaded. For more information, see the Troubleshooting topic in the TrueSight Orchestration online documentation.
• Verify that the TrueSight Orchestration monitor adapters are able to read alerts from BMC Remedy ITSM.
• Verify that the TrueSight Orchestration actor adapter is able to connect to BMC Remedy ITSM. For more information, see Verifying-the-connection-to-BMC-Remedy-ITSM.
• Verify that the TrueSight Orchestration for TrueSight Server Automation adapter is able to run BLCLI commands. For more information, see Verifying-the-connection-to-TrueSight-Server-Automation.
• Verify that the TrueSight Orchestration SNMP Monitor adapter is able to recent alerts/traps from TrueSight Server Automation.

Perform the following checks to see if the ticket was created.

• Query for the change request ticket number identified in the TrueSight Server Automation Job, if available. If the Change ID is not available in the Job, query for Change Requests based on the name of the Change Template used to create change requests. This action checks to see if the change request was created but the Change ID was not propagated back to the TrueSight Server Automation Job.
• If the change request is not found, TrueSight Orchestration might have encountered an error creating the change request.
• If the change request is found, check the value of Status and Next Approval Phase to see the current status of the change request:
  • Status = Scheduled, and Approval Status = Review – The change request has been submitted but has not gone through the approval process. Verify the change request is assigned appropriately. If the change request is undergoing approval, investigate the approval process in BMC Remedy ITSM.
  • Status = Implementation in Progress, and Approval Status = Close Down Approval  – The change request has been approved.  This is the status shown when the job is waiting to run, or is waiting for the update that the job has run. 
  • Status  = Completed or Closed.  The job has been run in TrueSight Server Automation, and details of the job should be in the work details.
• Perform the following queries:
  • Query on Alert Events to see if the notifications by BMC Remedy ITSM on the changes in the state of the change request were sent to TrueSight Orchestration.
  • Query for all Alert Events to be read by TrueSight Orchestration using the following: User = <BMC Remedy user that was used to configure the BMC Remedy Monitor adapter>.
  • Query for all Alert Events on a particular Change Request/Incident by using the following: 'Alert Text' LIKE "%<Change ID or Incident ID>%"
  • The value of the Read field indicates whether the alert has been received by TrueSight Orchestration. 'Read' = X means the alert has not been read by TrueSight Orchestration. 

TrueSight Server Automation

Perform the following checks to isolate the issue:

• Check the Job Schedule to see if the job was created with approval. See Executing-a-job-with-BMC-Remedy-ITSM-approval in the TrueSight Server Automation documentation.
• Note the values of status, approval type and change parameters specified. If the status is Waiting for Approval, it means TrueSight Server Automation has sent the request to BMC Remedy ITSM to create the change request and is waiting for the response that it was created and approved. See Status types for change management approval jobs in the TrueSight Server Automation documentation.
• If the status is Scheduled, it means that although the change request was created and associated and it has been approved, but is awaiting the scheduled time.
• Check whether there is a value of ChangeID in the job properties, which will indicate whether the change request has been created and associated.
• Check to see if the job completed, and if so then view the results. From the Jobs folder, navigate to a job, right-click and select Show Results.

TrueSight Orchestration

Perform the following checks to verify modules and adapters are loaded and working:

• Verify that the TrueSight Orchestration modules and adapters are configured and loaded. For more information, see Troubleshooting in the TrueSight Orchestration online documentation.
• Verify that the TrueSight Orchestration monitor adapters are able to read alerts from BMC Remedy ITSM.
• Verify that the TrueSight Orchestration actor adapter is able to connect to BMC Remedy ITSM. For more information, see Verifying-the-connection-to-BMC-Remedy-ITSM.
• Verify that the TrueSight Orchestration SNMP Monitor adapter is able to recent alerts/traps from TrueSight Server Automation.

Perform the following checks to see if the change request was created or updated.

• Query for the change request ticket number identified in the job, if available. If the change ID is not known, query based on the Change Template categorization that is used to create change requests. This query will show you if the change request was created, but the ID was not propagated back to the TrueSight Server Automation job.
• Verify the incident or change request is related to the CI specified in the template.
• Check the Approval Status of the change request to see if it will go through the approval process.
• Verify the categorization of the change request requires approval. See Selecting job types to which to apply job options for Change Automation in the TrueSight Server Automation documentation.
• Check the Approvers page of the change request to see whom is it waiting for approval.
• Check the value of status of the change request:
  • Pending Approval --The change request requires approval, and is awaiting for it
  • Planning – The change request has been approved, or does not require approval. This status indicates that the Job is waiting to run, or BMC Remedy ITSM is waiting for the update that the job has run.
  • Resolved or Closed – The change request has been approved (if necessary), the job has run, and the change request has been updated with the results of the job and closed.
• Perform the following queries:
  • Query on Alert Events to see if the notifications by BMC Remedy ITSM on the changes in the state of the change request were sent to TrueSight Orchestration.
  • Query for all Alert Events to be read by TrueSight Orchestration using the following: User = <BMC Remedy user that was used to configure the BMC Remedy Monitor adapter>.
  • Query for all Alert Events on a particular Change Request/Incident by using the following: 'Alert Text' LIKE "%<Change ID or Incident ID>%"
  • The value of the Read field indicates whether the alert has been received by TrueSight Orchestration. 'Read' = X means the alert has not been read by TrueSight Orchestration.

TrueSight Server Automation

TrueSight Orchestration

Perform the following checks to verify that the modules and adapters are loaded and working.

• Verify that the TrueSight Orchestration modules and adapters are configured and loaded. For more information, see the Troubleshooting in the TrueSight Orchestration online documentation.
• Verify that the TrueSight Orchestration monitor adapters are able to read alerts from BMC Remedy ITSM.
• Verify that the TrueSight Orchestration actor adapter is able to connect to BMC Remedy ITSM. For more information, see Verifying-the-connection-to-BMC-Remedy-ITSM.
• Verify TrueSight Orchestration BladeLogic actor adapter is able to connect to TrueSight Server Automation. For more information, see Verifying-the-connection-to-TrueSight-Server-Automation.

Perform the following checks:

• View the incident in Incident Management. Navigate to the Related Items page and verify there is a related ComputerSystem. Note the name of the ComputerSystem. Verify that the incident is open.
• Verify an alert was sent from Incident Management to TrueSight Orchestration by issuing the following query on the Alert Events form: where 'Text' Like AOSIE:<Incident number>%
• Note: Alerts are periodically purged after they are read by TrueSight Orchestration, so they will not collect here indefinitely.