CMDB synchronization

CMDB synchronization provides a configurable mechanism to keep data in the BMC CMDB (CMDB) synchronized with information discovered by BMC Discovery. You set up connections to CMDBs individually, enabling you to configure specific filters and blackout windows for each. You can also enable or disable and pause or resume synchronization on individual connections, or globally.

BMC Discovery maintains an authoritative model of the dataset of each configured CMDB connection, in its datastore. When a CMDB sync connection is created, BMC Discovery creates a shadow copy of the data that the corresponding CMDB dataset should hold and sends that to the CMDB. When data is modified as the result of discoveries, deletions or manual syncs, these changes are updated in the shadow copy and propagated to the CMDB, without the need to first check the contents of the actual CMDB, meaning that only changes are sent to the CMDB.

Occasionally, the model stored in the CMDB dataset becomes out of step with the shadow copy for a CMDB connection and requires resynchronization. For example, if CMDB tools have been used to modify the dataset. In this case, when updates are written to non-existent nodes, instance errors are raised. When BMC Discovery registers these instance errors it attempts to fix them automatically, but sometimes a resynchronization is recommended.In other cases such as after upgrade from a 10.1 appliance or the creation of a new CMDB sync connection to a non-empty dataset, a resynchronization is mandatory, and CMDB sync is disabled until the resynchronization is complete.

Supported CMDB versions

Data model mapping

The BMC Discovery data model (Discovery model) is different from the Common Data Model (CDM) used in the CMDB, so the synchronization mechanism is responsible for transforming the required information from one data model to the other. The Discovery model is known as the source model, and the CDM is referred to as the target model.

Both models are graph models, meaning that they represent data as nodes connected to each other with relationships. Synchronization operates on portions of the full graph known as subgraphs. A subgraph contains a root node (such as a host computer), plus all the related nodes that belong to it (such as network interface information, software information, and so on), referred to as its components. Some nodes can be shared, meaning that they belong to more than one subgraph. For example, the node representing a subnet is shared by all the host computers on that subnet.

The mapping between the Discovery model and the CDM is defined in syncmapping definitions within TPL files.

The Default CDM Mapping provides a comprehensive mapping from the Discovery model to a best-practices CDM model. If you have added custom nodes to Discovery they are not exported automatically by the default CDM mapping, instead, you will have to create additional mapping definitions within TPL files.

Synchronization stages

Synchronization occurs in the following stages:

1. A root source node is chosen for synchronization. The root node kinds are Host, MFPart, NetworkDevice, Printer, SNMPManagedDevice, Storage System, and CloudService corresponding to the main kinds of device that can be discovered. The list of possible root nodes is fixed and cannot be altered in the Syncmappings.
2. The source subgraph is populated by retrieving all the relevant data from the Discovery datastore.

   • The BMC Discovery filter sub page for a CMDB connection can be used to specify which source nodes will be synced.

3. The source subgraph is transformed into a target subgraph, centered around a target root CI.
   • The CMDB filter sub page for a CMDB connection can be used to specify which CIs will be synced, that correspond to the nodes allowed through by the Discovery filter.
4. The target subgraph is compared with the corresponding subgraph stored in the shadow copy stored in the Discovery datastore.

5. The updates identified are made in batches to the CMDB. Once an update has been made to the CMDB (by creating, updating and (soft) deleting CIs and relationships), it is also made to the local shadow copy for the connection. The next batch is then processed in the same way until no more updates are required.

   Note

   If any changes are made to the class definitions in the CMDB (for example, adding classes or attributes), you cannot synchronize to them until the BMC Discovery services have been restarted. After the system restarts, the class definitions are read and all customized classes and attributes are available for CMDB synchronization.

Data Segmentation

BMC CMDB separates input data from different data sources into separate datasets. Each data source owns its own dataset, and nothing else should write to that dataset. Each BMC Discovery instance must have its own dataset. The default dataset name is BMC.ADDM.

BMC Helix Remedyforce does not have separate datasets. Instead, it identifies the origins of CIs and relationships using the Source Name attribute. Each BMC Discovery instance must use a unique Source Name. The default is "BMC Discovery".

Resynchronization

Occasionally the model stored in the CMDB dataset becomes out of step with the shadow copy for a CMDB connection and require resynchronization. For example, if CMDB tools have been used to modify the data. In this case, when updates are written to non-existent CIs, instance errors are raised. In the BMC CMDB, the Discovery dataset is not intended to be modified (for instance to create Business Services); the Asset dataset should be updated instead. However, when BMC Discovery registers these instance errors, it attempts to resolve the situation automatically, but if it occurs repeatedly a resynchronization is recommended. In other cases, such as after upgrading from version 10.1, or the creation of a new CMDB sync connection to a non-empty dataset, a resynchronization is mandatory, and CMDB sync is disabled until the resynchronization is complete.
Resynchronization must assess the contents of the CMDB dataset against the authoritative version stored in the Discovery shadow copy for the CMDB connection and send the appropriate updates to the CMDB. The stages in a resynchronization are:

1. Preparation – Prepare Resync performs the following actions, without changing any data in the CMDB:
   
   1. Reads identities of all CIs and relationships from the CMDB dataset (in BMC CMDB) or with the expected Source Name (in BMC Helix Remedyforce)
   2. Transforms all BMC Discovery data into the local shadow copy.
   3. Compares the data read from the CMDB with the updated local shadow copy.
2. Commit – Commit Resync performs the following actions in the CMDB:
   
   1. Creates CIs and relationships missing from the CMDB.
   2. Marks deleted any CIs or relationships present in the CMDB but not in the local shadow copy.
   3. Updates all CIs and relationships to ensure their attributes match those in the local shadow copy.

There are two types of resync:

• Complete Resync – in which the Commit Phase runs to completion, and may take a long time.  
• Incremental Resync – in which the Commit Phase is performed incrementally as root nodes are synchronized.

You can choose which type of resync to use on the Resync tab of the CMDB Sync page. You can also choose to prepare the resync only, in which case the resync is prepared on the BMC Discovery side; no updates are made to the CMDB. Using prepare only you can examine the changes that CMDB Sync will make to the CMDB before performing the commit.

Where a resynchronization is interrupted, you can resume resynchronization from the point at which it was interrupted.

In the following situations a resynchronization is mandatory, and CMDB sync is disabled until the resynchronization is complete.

Connection to a populated CMDB

When you add a new sync connection, its shadow copy is created empty. When the connection is created, the credentials and CMDB version are checked. The first check also determines whether the CMDB dataset is populated (in BMC CMDB) or if there are existing CIs with the chosen Source Name (in BMC Helix Remedyforce). If it is, a resynchronization is required to populate the shadow copy with discovered information and sync that with the CMDB. If the dataset and shadow copy are both empty then no resynchronization is required.

For a dataset used in a new connection to be considered empty, it must be totally empty. It must not even contain any CIs that have MarkAsDeleted=Yes. If you connect BMC Discovery to a dataset that you believe to be empty, but BMC Discovery reports that it is not empty, it is likely that there are some marked-deleted CIs present. You must purge the marked-deleted CIs, or create a brand new dataset.

Failed or cancelled resynchronization

If a resynchronization fails or was cancelled, a resynchronization is required before the connection can be resumed.

Changed connection details

If you change the CMDB connection details for an existing connection, you may need to perform a resynchronization, even if the system cannot always determine that itself. In the following cases, you must resynchronize:

• If the CMDB address is changed to a different CMDB. (Whereas changing to use a different address that corresponds to the same actual CMDB does not require a resync.)
• If the BMC CMDB dataset is changed
• If the BMC Helix Remedyforce user name is changed in a way that means a different Remedyforce instance is used
• If the BMC Helix Remedyforce Source Name is changed

Initial configuration

The following steps are required to set up CMDB synchronization:

• Preparing BMC CMDB for synchronization
• Setting up a CMDB synchronization connection

To perform synchronization

Synchronization can be configured to run in a continuous mode, meaning that as soon as BMC Discovery completes its scan of a device, the data corresponding to it is queued for synchronization with the CMDB. Alternatively, synchronization can be launched manually for one or more devices from within the browsing interface.

• Managing CMDB synchronization
• Batch and individual synchronization

To prevent synchronization at certain times

Typically, synchronization can occur at any time. However, synchronization can place a substantial load on the CMDB, so it might be necessary to prevent synchronization from occurring at times the CMDB will be used heavily. This can be achieved by configuring Configuring CMDB Sync blackout windows.

To filter synchronization data

By default, all CIs and relationships in the Default CDM Mapping for all devices are synchronized to the CMDB. Filtering enables you to synchronize a subset of the data. After each of the first three synchronization stages, the data can be filtered to affect the data that finally reaches the CMDB. See CMDB sync performance notes and Filtering nodes and CI types for CMDB Sync for more information.

Root node deletion synchronization failure

When a non-fatal error occurs during synchronization with the CMDB, the root node is re-queued so the synchronization is attempted again. A root node is re-queued up to three times, after which it is dropped and synchronization is not automatically attempted again. When the outage is resolved, subsequent discovery runs trigger continuous synchronization and re-submit root node creations and updates. Root node deletions, however, are not automatically re-submitted, leading to orphaned graphs in the CMDB dataset.

Manual workaround

When the network outage is resolved, perform a resynchronization to ensure the CMDB contents match the data in BMC Discovery. Alternatively, to synchronize a selection of the missed deletions, run the report “Aged-out Hosts and other devices that failed last CMDB sync”, found in Explore > Reports > BMC Discovery Deployment. Select one or more nodes from the list, and choose Actions > Sync to CMDB.

CMDB class definition changes

If you change any of the CMDB class definitions, for example, using the CMDB Class Manager utility, you must perform a CMDB resynchronization.