Framework processing

The basic Triage and Remediation workflow occurs in the following stages:

1. The incoming event that triggered the workflow is received.
2. Relevant data is extracted from the event, and the configuration items are extracted from the module configuration data. The event can be enriched with the following information:
   • Configuration item information from the BMC Atrium CMDB, if the configuration item is retrieved

   • PATROL annotation, if the event is a BMC PATROL event

     Note

     The PATROL annotation subworkflow applies to the OS Disk Space Full, ESX Host Not Responding, DB Tablespace Full, and Failed Backup and Recovery workflows.

3. The target system on which the workflow actions are performed is identified, and its connection details are verified.
4. The predefined triage actions are run.
5. An incident is created or an existing incident is updated. The incident information is associated with the event.
6. For remediation, a change request is opened if this is a normal change type where approval is required. If this is an emergency change, the change is approved automatically.
7. The remediation actions, defined by tasks associated with the change, are run on the target system. The event receives the corresponding incident and change updates. The event notes and task workinfo are updated with remediation data.
8. The incident and the change are closed when the event is closed, and the event is updated accordingly.

The processing of the event, incident, change, and their related subworkflows are common to the framework of every workflow. What distinguishes each workflow are its triage and remediation actions. If you create a new workflow, you need to specify the triage and remediation actions. The framework supplies the fundamental, common processes.

The following figure depicts the initial stages that are common to every workflow of the framework:

Initial stages of the framework

Processing incoming events

The framework takes the incoming event and extracts from it the identifier value and other data. It then applies the appropriate event mappings based on the identifier value. (See the default event mappings file on Event_Management). As output, the framework populates the event concept fields with the extracted data and appends a log file to the event.

Getting host connection information (AutoPilot Credentials Store)

After extracting event data, the framework workflow proceeds next to extract connection information for the host system. This process is performed by the Get Host Connection Details subworkflow of the AutoPilot Credentials Store. It takes as input the host name (mc_host ) and extracts the data for the host from the Datacenter or Databases group.

The AutoPilot Credentials Store module supports encrypted data. You have the option of storing password credentials in encrypted form. Secured credential values are displayed as asterisks (*****) in the *Item Details Value of the module configuration. These secure credentials are also transmitted in an encrypted format.

Data center

The Data center is a subset of a secure credentials container. It consists of secure authentication type items instead of XML elements. The Datacenter contains one default domain, which is labeled with a name that begins with the word Domain. The default domain captures host names that are passed to it without a domain name. It contains other domain categories (each labeled with the keyword Domain ) for one or more IP addresses.

Each domain category can contain multiple host definitions, as illustrated in the following figure:

AutoPilot Credentials Store: Datacenter hierarchy



Each domain specifies a different grouping of IP addresses: for example, abc.com, xyz.com, abc.gov, and so forth. The default domain category (Domain_Name = default) can contain hostnames without specified domain names. It can also contain an IP address for a host. (See IP address for the host node under the default domain.) 

When adding a domain to the list, you must use the keyword Domain at the beginning of the string: for example, Domain_BMC.

Each domain category contains one or more hosts, which are designated as Host 1, Host 2, Host 3, and so on. Each host is defined by the following details:

• Host name (required)
   Each host that you add must start with the keyword Host.
  You can use the asterisk "*" as the host name value to denote hosts that share common login credentials within the domain. (See also Asterisk (*) character and domain search.)
• User name (required)
   Enter the user name by using the Static Value type in the Item Details pane.
• Password (required)
   You can encrypt the by password using the new Secure type.
  
• Invocation_Mechanism (required)
   Valid values include SSH, Telnet, windows-command, and command-line.
• Timeout (optional)
   The Timeout entry is an optional detail item. It applies to the SSH, Telnet, or windows-command adapter. The default value is 60 seconds. You can modify the timeout value to better accommodate your environment. For example, if your network response time is slow, you can increase the default value so that the connection does not time out.

Asterisk (*) character and domain search

The asterisk (*) character refers to a common set of credentials – user name, password, and invocation mechanism – that are used to access host systems in a particular domain. BMC recommends that every domain contain a host entry with a common set of credentials denoted by the asterisk (*) character.

The AutoPilot Credentials Store uses the asterisk (*) as a default credentials match when the domain search cannot find a matching specified hostname in the domain. Do not remove the default asterisk (*) entry from any domain. If the default credentials marked by the asterisk (*) are not available, then the workflow continues to try to log on to the specified host system, but the login fails because the workflow does not find a match in the domain.

The domain search works differently based on how the hostname is specified. For example, if the host name is a fully qualified domain name, the search looks in the domain category with the corresponding domain name. If it finds a host name match, then it uses the selected host name's definition. If the search does not find a matching hostname, then it selects the credentials of the hostname designated by the asterisk in the domain category.

If the host is designated by a hostname only, then the search looks in the default domain category. If it finds a matching hostname, then it uses the selected hostname's definition. If the search does not find a matching host name, then it selects the credentials of the hostname designated by the asterisk in the default domain category.

If the host is designated by an IP address, then the search also looks in the default domain category. If it finds a matching IP address, then it uses the selected definition of IP address. If the search does not find a matching IP address, then it selects the credentials of the hostname designated by the asterisk in the default domain category.

If the host name is a fully qualified domain name, but the domain is not defined in the AutoPilot Credentials Store, then the search looks to match the hostname in the default domain category. If it does not find a match, then it selects the credentials of the host name designated by the asterisk.

IP address for the host node under the default domain

The IP_Address field is an optional item detail that you can specify under a host node entry of the default domain group. (The default domain group contains host node entries that do not have associated domain names.) The IP_Address value is valid only for host node entries in which the hostname value is well defined, not where the hostname is an asterisk value (*).

The Hostname field is used as a lookup value. In a default domain group where the host node entry has both a well-defined hostname value and an IP_Address value, the IP_Address value is sent as the return value in the connection details. In the context of the default domain group, in which host node entries do not have specified domain names, the IP_Address value provides more certainty to the adapter connection.

In the following example, the Hostname field with the value pun-esx-mgc13 is also identified by the IP_Address field with the value 132.255.30.44 under host node 2 of the default domain.


If the host system cannot be identified by the hostname pun-esx-mgc13, then the specified IP address 132.255.30.44 is used.

When you include an IP address entry under a host node, the connection details that are generated for the host entry return the IP address value for the hostname element. The following XML example shows the IP address value (highlighted in bold, red text) in the hostname element of the connection details:

Databases

The Databases group is designed to support the DB Tablespace Full workflow. Like the Datacenter, the Databases group is subdivided into domains. The domains contain host systems and databases that reside on each host within the domain. The following figure displays this hierarchy:

Databases group in the AutoPilot Credentials Store



In the Databases group, the Host group name contains the following attributes:

• Hostname (required)
   Each host that you add must start with the keyword Host.
  You can use the asterisk "*" as the hostname value to denote hosts that share common login credentials within the domain. (See also Asterisk (*) character and domain search.)
• Port (required)
   This is the port number through which you connect to the database.
• Subprotocol
   The subprotocol specifies the communications subprotocol that the JDBC driver uses when it connects to different data sources. The default is oracle:thin.
• Driver
   The driver specifies the JDBC driver that is used to connect directly to Oracle. The default is oracle.jdbc.driver.OracleDriver.
• Invocation_Mechanism
   The invocation mechanism refers to the command used to invoke the method of certain data types in Oracle. The default command is sql.
  
  When specifying the Database group name, you must preface the name with Database._ The Database group contains the following item details:
• DB_Name
   This is the name of the database that you wish to access. You can use the asterisk "*" as the DB_Name value to denote databases that share common login credentials within the domain. (See also Asterisk (*) character and domain search.)
• User name
   This user name is the database username.
• Password
   This is the database password. You can encrypt the password through the Secure type credential.