Using Orchestration actions to enable triage and remediation of events



Supported with version 11.3.02 and later

The capabilties of integrating TrueSight Presentation Server with TrueSight Orchestration and initiating Orchestration actions from events are available only with TrueSight Presentation Server 11.3.02 and later.

TrueSight Orchestration is licensed separately and not bundled with TrueSight Operations Management. You must download and install it separately. For more information, see Downloading the installation files for TrueSight Orchestration.

TrueSight Orchestration enables you to automate known remediation scenarios. So, when an event occurs on the TrueSight console, you can easily initiate an Orchestration action from that event. If the Orchestration action is successfully initiated, the remediation steps are run.

Incoming events displayed on the Monitoring > Events page can be of many types. Some events are important and actionable, others are informational, and still others contribute noise. Some of the important and actionable events are recurring events with known remediation steps. Such events can be remediated quickly and easily by running Orchestration actions to reduce time, errors, and delays associated with manual methods. You can also run Orchestration actions to perform triage activities only. By default, TrueSight Presentation Server uses predefined context-based event selection to display out-of-the-box Orchestration actions for relevant events only. 

Note that Orchestration actions can be run for PATROL events and alarm events only. To be able to access the Orchestration actions, Presentation Server must be integrated with TrueSight Orchestration and a set of prerequisites must be met on both Presentation Server and TrueSight Orchestration.

If you want to remediate events for use cases other than the ones covered by the out-of-the-box Orchestration actions, you need to configure custom Orchestration actions. 

Note

On the TrueSight console, Orchestration action names are displayed in English only.



Related topics

Monitoring and managing events


End-to-end process overview

The following image depicts the end-to-end process involved when you initiate an Orchestration action.


The process starts when an IT operator on the TrueSight console launches an Orchestration action for a particular event. Event data is sent to TrueSight Orchestration. Based on the use case, a triage action is triggered, which verifies the validity of the event. Note that based on the use case triage may not always be required.

Next, an incident is created. By default, BMC Service Resolution is configured to perform incident management. However, you can manually configure TrueSight Orchestration to perform incident management. To configure TrueSight Orchestration for incident management, you need to change some settings on TrueSight Orchestration, in the BMC-SA-Event_Orchestration_Configuration module configuration, under the Specifics > BMC_TrueSight configuration group. For more information, see  Configuring modules in the Event Orchestration runbook.

Then, a change request is created and the workflow waits for the change to be approved. By default, change management is already enabled through TrueSight Orchestration. Note that both change and incident management may not be required based on the use case for which you want to initiate an Orchestration action.

If the remediation action is defined, the remediation action is run, which performs the corrective action on the target server where the problem has occurred. After the remediation is complete, the validation actions are run to ensure that remediation is successful. If a change request was created earlier, it is updated with the latest status and the incident is resolved. Furthermore, TrueSight Infrastructure Management detects that the condition has returned to normal and subsequently closes the event.

Each step of this orchestration process is configurable, such as whether to perform a triage only action, or a combination of triage and remediation or only remediation, and so on. TrueSight Presentation Server provides you with out-of-the box Orchestration actions to perform triage and remediation for service down use case and only triage for device down use case. If you want to run Orchestration actions for any other use case, you need to perform a set of configurations on TrueSight Orchestration. For more information, see Configuring Orchestration actions for custom use cases.

At each stage of the process, related events are associated with the event from which the Orchestration action was run and are displayed under the Remote Action Result tab. 

Requirements for Presentation Server

To be able to initiate Orchestration actions from events, ensure that the following requirements are already met:

#Requirement
1

On Administration > Components, add a TrueSight Orchestration component and a TrueSight Infrastructure Management component.

For more information, see Adding and editing components.

Notes:

  • Ensure that the user that you provide for the integration is part of the AoAdmin group on Remedy Single Sign-On.
  • Ensure that TrueSight Orchestration Platform 8.2.00 and later is already installed and the same details are given while registering the component.
2On the Administration > Components page, ensure that the TrueSight Orchestration component status is Connected.
3

Ensure that you have the permission to view and initiate Orchestration actions from events. This permission is governed by the Allow Orchestrator Actions permission on the Roles page.

4If you plan to use BMC Service Resolution for incident management, ensure that Presentation Server 11.3.02 is integrated with BMC Service Resolution.

Requirements for TrueSight Orchestration

To be able to initiate Orchestration actions from events, ensure that the following requirements are already met:

#Requirement
1

Ensure that TrueSight Orchestration Platform version 8.2 or later is already installed.

For more information, see Installing the TrueSight Orchestration Platform.

2

Ensure that Event Orchestration runbook version 20.19.01 is already installed on the TrueSight Orchestration Platform.

You can choose to configure content either during the installation or after the installation manually.

  • For information about installing the Event Orchestration runbook, see Installing the Event Orchestration runbook.
  • For information about configuring content post installation manually, see Configuring content in the Event Orchestration runbook.
3Ensure that TrueSight Orchestration is already integrated with BMC Remedy IT Service Management for performing the change management process.

Out-of-the-box Orchestration actions

By default, you can initiate Orchestration actions for the following use cases:

Use caseOut-of-the-box Orchestration actionDescription
Service downRestart ServiceInitiate this Orchestration action to triage and remediate service down problems.
Device downCheck Host ConnectionInitiate this Orchestration action to triage device down problems.

Note

The out-of-the-box Orchestration actions are enabled for Windows and Linux operating systems only.

To initiate an Orchestration action from an event

An Orchestration action can be initiated for one event at a time.

  1. Go to the Monitoring > Events page.
  2. Click the action menu of the desired event and select Launch Orchestration Actions.
  3. Select an Orchestration action from the list displayed, and then click Launch.
    A status message indicating whether the action initiation was successful is displayed at the top of the page.

Examples of out-of-the-box Orchestration actions

The following examples describe the high-level process involved when you run the out-of-the-box Orchestration actions.

High-level process flow for the service down use case

 Click to see details

Scenario: Suppose you see the following event on TrueSight console indicating that the BAO-REPO service is down on the Windows platform.

In this scenario, you can directly run the out-of-the-box Restart Service Orchestration action and remediate the problem. 

High-level process flow: When you initiate the Restart Service Orchestration action for the BAO-REPO event, the TrueSight Orchestration Process Event workflow is run and the following steps are performed.

  1. TrueSight Orchestration performs triage to determine whether the event is valid and the BAO-REPO service is actually down.

  2. After a successful triage, TrueSight Orchestration checks if an incident is created for this event and performs one of the following actions:
    • If the incident is already created via BMC Service Resolution: TrueSight Orchestration updates the incident with the latest status.
    • If the incident is not created: If TrueSight Orchestration is configured for incident management, then TrueSight Orchestration creates an incident.
  3. TrueSight Orchestration creates a change request, the task related to the change, and associates the change with the incident. 
  4. The change request is then sent to the change approver for approval. 
  5. After the change is approved, TrueSight Orchestration starts the BAO-REPO service on the target server (remediation action).
  6. After the remediation is complete, TrueSight Orchestration validates that the BAO-REPO service is up and running.
  7. Finally, TrueSight Orchestration sends an event to TrueSight console indicating the successful remediation of the BAO-REPO service. Also, the change request is closed and the incident is resolved. 

  • Related events: At every step, related events are are logged in relation to the main event from which the Orchestration action was run. These events are displayed in the Event Details page as remote action results. To view remote action results, from the event action menu, select View Remote Action Results. Alternatively, click the icon displayed in the event message or on the event toolbar.
  • Informational events: At every step, an Information event indicating the status is sent to the TrueSight console. You can view these events by selecting the Information quick filter at the top of the page.

Related events indicating the status (under the Remote Action Result tab)

High-level process flow for the device down use case

 Click to see details

Scenario: Suppose you see the following event on TrueSight console indicating that a particular machine is down in your environment.

In this scenario, before remediating the problem, you can run the out-of-the-box Check Host Connection Orchestration action to triage if the computer is still down.

High-level process flow: When you initiate the Check Host Connection Orchestration action for the device down event, the TrueSight Orchestration Process Event workflow is run and the following steps are performed.

  1. TrueSight Orchestration performs triage to determine whether the event is valid and the machine is actually down.

  2. After a successful triage, TrueSight Orchestration checks if an incident is created for this event and performs one of the following actions:
    • If the incident is already created via BMC Service Resolution: TrueSight Orchestration updates the incident with the latest status.
    • If the incident is not created: If TrueSight Orchestration is configured for incident management, then TrueSight Orchestration creates an incident.
  3. Finally, TrueSight Orchestration sends an event to the TrueSight console indicating that the triage is successful. 

The IT operator assigned to the event must ensure that the device is restarted and the incident is closed. After monitoring restarts on the device, the event is automatically closed.

After you run the Orchestration action, you can track the status of the action by looking at:

  • Related events: At every step, related events are are logged in relation to the main event from which the Orchestration action was run. These events are displayed in the Event Details page as remote action results. To view remote action results, from the event action menu, select View Remote Action Results. Alternatively, click the icon displayed in the event message or on the event toolbar.
  • Informational events: At every step, an Information event indicating the status is sent to the TrueSight console. You can view these events by selecting the Information quick filter at the top of the page.

Related events indicating the status (under the Remote Action Result tab)


Configuring custom Orchestration actions

If you want to initiate Orchestration actions for use cases other than the out-of-box use cases, then you need to perform additional configurations.

The following table summarizes the configuration steps required for configuring a custom Orchestration action and enabling it on relevant events. 

StepProduct involvedTask
0

Before you begin: Ensure that the requirements for TrueSight Orchestration and requirements for Presentation Server are already met.

1TrueSight Orchestration

Add a custom use case to enable Orchestration actions

(Required to configure a custom Orchestration action for the custom use case.)

2TrueSight Presentation Server

Configure event selection for running custom Orchestration actions

(Required only if you want to enable the custom Orchestration action on relevant events. Skipping this step will result in the custom Orchestration action getting displayed on all the events irrespective of context.)

Example scenario for a custom use case

Suppose you see an event on the TrueSight console indicating that more than 75% memory is getting utilized on a particular computer.

You want to perform triage to see the top 10 processes that are consuming the maximum memory on the affected computer. Also, you want the custom Orchestration action to be enabled on Linux events with the memory parameter value greater than 75%. To understand the end-to-end configurations required for enabling this custom Orchestration action, see Example of configurations required for a custom use case.

Was this page helpful? Yes No Submitting... Thank you

Comments