Writing an extractor for a CMDB - Object Relationships dataset

The Configuration Management DataBase (CMDB) is a repository that describes all the components of an information system.

Types of data inside the CMDB can be:

• Server configuration data for each server. For example: Number of CPUs, hardware model.
• Service catalog and component tree for each service.
• Low level details of each component and level relationships between components. For example: A virtual machine using a specific database instance.
• Other information: Asset management data, service level thresholds.

Version 2.x onwards, TrueSight Capacity Optimization is able to integrate with CMDBs to gather server configuration data (dataset SYSCNF).

Version 3.x onwards, the integration of Service Catalog and component tree of services is also available (new dataset OBJREL). Other enhancements include:

• New services (domains) are automatically created.
• New systems (or workloads) are automatically created and associated with the domain that uses them.
• When a system/workload is moved to another service, the change is synchronized into TrueSight Capacity Optimization.

The following sections describe the CMDB integration in detail:

• • Phases of synchronization
  • Typical usage scenarios
  • Object relationship rules
  • Lookup notes
  • Transactions

Phases of synchronization

Synchronization is achieved in two phases:

1. An ETL populates the OBJREL (Object Relationships) dataset.
2. A hierarchy rule, based on the Object Relationships template in the Hierarchy Manager task synchronizes the TrueSight Capacity Optimization tree.

Typical usage scenarios

If the catalog is fully owned by CMDB, the TrueSight Capacity Optimization tree exactly reflects the CMDB. For testing purposes, before the real import, it is possible to use the Simulation Mode of the Hierarchy Manager or import the CMDB catalog into a testing subtree.

If the catalog is only partially owned by CMDB, or it is owned by multiple CMDBs, it is possible to synchronize the CMDB catalog to specific TrueSight Capacity Optimization subtrees by specifying multiple rules. For example, there could be an ETL for each service or group of services, or an ETL for each CMDB source (if multiple).

Object relationship rules

In the rule definition, it is possible to set:

• The ROOT domain (where the imported catalog is saved) can be 'All Domains' (the default root of the navigation tree) or a specific domain, to limit the synchronization to a subtree.
• One or more data sources (ETLs) to process.
• The lookup mode for domains, systems and workloads. See Lookup notes.
  
  Defining object relationship rules
  

Building a CMDB integration

To build a CMDB integration, two components are needed:

• An ETL extractor or parser module that populates the dataset OBJREL.
• A hierarchy rule that activates the data processing and selects where to put processed data.

The OBJREL dataset has two working modes:

• CHANGE mode
• ASSERT mode

When populating the dataset, you can specify the objects to import and their relationships. You must define objects before& defining their relationships; then, you can define the following types of relationships:

• Between applications: ENTCATNM = APP, ENTCATNMPARENT = APP
• Between systems: ENTCATNM = SYS, ENTCATNMPARENT = SYS
• Between workloads: ENTCATNM = WKLD, ENTCATNMPARENT = WKLD
• Between systems and applications: ENTCATNM = SYS, ENTCATNMPARENT = APP
• Between workloads and applications: ENTCATNM = WKLD, ENTCATNMPARENT = APP

CHANGE mode

When the OBJREL dataset operates in CHANGE mode, the CMDB is aware of any changes as they occur; it stores delta information for each change and can be queried to extract all changes from a from a given point in time. The ETL only extracts changed data from the CMDB. You can specify the following Change types:

• ADD: Add objects and object relationships
• REMOVE: Remove objects and remove object relationships
• UPDATE: Update one or more of the following fields: name, description, location, type

ASSERT mode

When the OBJREL dataset operates in ASSERT mode, the CMDB contains only the full current status, so at each import all CMDB data is read. At each import, the full CMDB catalog is compared with previous imports and changes are automatically detected, evaluated and applied.

Lookup notes

The OBJREL dataset contains the identifiers used in the data source, whose names begin with the ds_ prefix. They are used to ensure the right lookup when any label is changed in TrueSight Capacity Optimization; the lookup activity is performed by the Hierarchy Manager.

Because the tree synchronization should also take into account manual changes, an additional option is available to drive the lookup. The following options determine the lookup-by-name mode when a CMDB object is not found in the ETL lookup tables, but in TrueSight Capacity Optimization there is an object with a name that perfectly matches the CMDB name:

• Exclusive: The rule considers only objects inserted by this specific ETL; objects that were manually inserted into TrueSight Capacity Optimization can be recreated by the ETL, and no name lookup is performed.
• Subtree: The rule first considers objects inserted by this specific ETL, then performs a name lookup over all objects belonging to the subtree that this rule manages.
• All tree: The rule first considers objects inserted by this specific ETL, then performs a name lookup over all objects belonging to the entire TrueSight Capacity Optimization tree.

This option can be specified in the Hierarchy rule and can be differentiated for systems, workloads, and domains.

Transactions

Each import performed by an ETL is identified as a transaction, with the following characteristics:

• Rule application works in a transactional way.
• Each transaction is associated with its expected row count.
• Transaction processing is aborted in case of incomplete data.
• Data consistency is granted against broken record imports.
• Each transaction is associated with a data source (ETL).
• Each data source has its own scope and can have its own lookup table to match existing TrueSight Capacity Optimization objects.

User operations

User modifications in TrueSight Capacity Optimization are not overwritten by CMDB imports.

Change actions are made considering only changes to the OBJREL dataset (if in CHANGE mode) or the differences between subsequent transactions (if in ASSERT mode). If no change is made in the sources, no change is made in TrueSight Capacity Optimization.

Examples

The following examples describe how to populate the OBJREL dataset and the actions performed by the Hierarchy Manager.

OBJREL dataset in CHANGE mode

OBJREL content imported on 10/05/2008:

These rows mean:

1. A new application Portal (CMDB-ID = 12) has to be added.
2. A new application WS001 (CMDB-ID = 5) has to be added.
3. The Portal application belongs to the ROOTAPP (see below).
4. The system WS001 belongs to the application Portal.

The _ROOTAPP_ placeholder means that "Portal" has to be appended to the root application specified by the rule settings. It identifies the main TrueSight Capacity Optimization tree or a specific subtree, depending on the rule settings.

In this example, two relationships are declared:

• A relationship between the Portal application and the ROOTAPP application.
• A relationship between the system WS001 and the application Portal.

Before you can define relationships between objects, you must explicitly add them: that is why there are four rows instead of two. If the objects were already added during a previous import, there is no need to add them again; but if you try to define relationships between objects that have never been explicitly added, they will not be created.

OBJREL dataset in ASSERT mode

In ASSERT mode, changes are detected as differences between an import and the previous one.

For example, consider the following two imports:

1. Data imported on 11/05/2008 in OBJREL:

2. Data imported on 12/05/2008 in OBJREL:

When the Hierarchy Manager rule runs on 13/05/2008, it will detect the differences between the two imports and apply them. In the example, the following actions will be performed:

• "System 1", that does not appear in the second import, must be removed; because the Hierarchy Manager does not actually remove objects from TrueSight Capacity Optimization, but only removes the relationships between objects, "System 1" will not be removed.
• The relationship between "System 1" and the domain "WebApp", that does not appear in the second import, will be removed.
• A new system named "System 2" will be added.
• "System 2" will be assigned to the domain "WebApp".
• If "System 2" does not appear in the system lookup table of the ETL, a new lookup entry that matches "System 2" with the newly created system in TrueSight Capacity Optimization will be added.

As in CHANGE mode, you must define objects before defining their relationships. In the following example (incorrect definition), the system "System 2" will not be added to the domain "WebApp" because the row that defines the relationship is before the row that defines the object: