Understanding the Identify and Merge activities in CMDB

The reconciliation process of BMC CMDB compares data from multiple data providers and aims to create a single complete and accurate production dataset. This production dataset with reliable data is also called the 'golden dataset'. This production dataset becomes a source of reference for other applications such as ITSM, for various ITIL processes and activities.

You can choose several methods for starting a reconciliation job, including manual, scheduled, continuous jobs, API, or a run process workflow.

The reconciliation engine performs the following important reconciliation activities:

• Identifies CIs that are the same entity in two or more datasets.
• Merges CI attributes from a source dataset to a production dataset to create the most comprehensive information in a single configuration item (CI).

Reconciliation is also used for the following activities:

• The compare activity compares instances in two datasets and either produces a report or executes a workflow based on the comparison results.
  Renaming a dataset does not change the DatasetId, so all reconciliation definitions that include the dataset still work with the new name. 
• The copy activity copies instances from one dataset to another.
• The delete activity deletes instances from one or more datasets. 
  This activity does not delete the dataset itself. 
• The purge activity deletes instances that have been marked as deleted from one or more datasets.
• The execute activity executes a reconciliation job. 

Reconciliation aims to eliminate duplicated data, retain only relevant information, and make sure that the data is correct and complete. 

Structure of a reconciliation job

The reconciliation job is a container for reconciliation activities, and each activity consists of different components. The primary activities are identification and merging. A reconciliation job can have one or more activities, each of which defines one or more datasets and rules for that activity. In addition, you can use a qualification set to restrict the instances participating in a reconciliation activity.

Jobs can use standard or customized rules. Standard rules use defaults for identify and merge activities and automate the creation of reconciliation jobs. You can also create custom jobs that include different activities.

Identification activities to match instances

For example, you can set an identification rule that the names of two different CIs from different datasets should be equal to Computer_1. When the rule finds a match, those instances are tagged with the same reconciliation ID. The reconciliation ID from the target dataset is copied into the source dataset.

These two CIs are considered as different instances of the same item when they have the same reconciliation ID. After CIs are recognized as different instances of the same item, they are now ready for merging based on which dataset is considered to have the most reliable information.

In another example, a rule intended to identify computer system instances might specify that the IP addresses of all be equal. When the rules find a match, it tags the matching instances with the same reconciliation identity.

You can also manually identify instances in an Identify activity.

Merge activities to merge datasets into a reconciled target dataset

Consider a merge operation involving data from three different datasets. Each attribute from each dataset is given a different precedence value based on how reliable that dataset is considered. The higher the value, the higher the priority that attribute from that dataset has over the others. Finally the data that is added to the production dataset has the most reliable data merged from all sources. This data is the production data that other applications can access for various ITIL processes and activities.

You give an overall precedence value to each data set, but you can override that value for particular classes and attributes in each data set. Whichever data set has the highest precedence value for a given attribute has its value for that attribute placed in the target data set. A precedence value for a class also applies to its subclasses unless the subclasses have their own precedence values.

Default merge settings

Some of the default merge settings are:

Ignore if Null- Determines whether a NULL value in the highest-precedence source dataset is merged to the target dataset.

Explicit Precedence- Specifies whether to require explicit Precedence entries for attributes or to apply the class precedence to attributes that have no assigned precedence.

Merge Order- Set Merge Order to By class in separate transactions, which is the fastest processing option, if your reconciliation job runs in a batch processing window, and performance is a concern.

When an instance is added or updated in a data set, BMC Atrium CMDB sets the ReconciliationMergeStatus attribute of that instance to Ready to Merge. When you set Merge Order to By class in separate transactions, the merge activity considers only those instances that have been identified and for which the ReconciliationMergeStatus attribute is set to Ready to Merge. After merging a CI, the reconciliation engine updates the value of the ReconciliationMergeStatus attribute from Ready To Merge to Merge Done.