Managing reconciliation

Reconciliation of data helps you to avoid adding duplicate configuration items (CIs) and assets to your Remedyforce CMDB.

• • Overview of reconciliation
  • Reconciliation process
  • Known limitations
  • Best practices
  • Creating reconciliation rules
  • Reconciliation log
  • Configuring reconciliation log settings
  • Viewing reconciliation report
  • Verifying reconciliation status
  • Related topics

Overview of reconciliation

You can use the reconciliation feature to achieve the following goals:

• Merge information from different data sources: With multiple data sources to add CIs and assets to your CMDB, you might get different data related to one record. If you enable and configure reconciliation, you can merge information about a CI or asset. Therefore, reconciliation enables you to make your Remedyforce CMDB data consistent.
• Avoid duplication of information: With multiple data sources, you risk data duplication. 
• View reconciliation report: You can view reconciliation report to find out records that successfully reconciled and records that failed to reconcile so that you can take corrective measures.

By default, reconciliation is disabled. To apply reconciliation, you need to create reconciliation rules to identify duplicate CIs or assets of a class. If a new record matches an existing record, all fields of the new and the existing record are compared. If the new record has an additional or changed information in one or more fields, those field values are updated in the existing record. If multiple matching records are found within CMDB, neither the existing records are updated nor the new record is inserted. An error is displayed in your Data Loader or Pentaho logs.

The following table describes the conditions when values in a matching record are updated based on the values in the existing records:

The following figure shows an overview of the reconciliation process: 

Reconciliation process

Reconciliation rules are applied under the following conditions:

• When new records are imported from any external source.
  
• When you create a new record or edit an existing record and the fields that you edit are present in the active reconciliation rule.

When you import records or edit those fields of an existing record that are present in the active reconciliation rule, BMC Remedyforce first verifies whether the Reconciliation feature is enabled. If reconciliation is enabled in your Salesforce organization, BMC Remedyforce validates the data against various conditions before inserting or updating a record in Remedyforce CMDB. The following figure shows all conditions that are used for data validation.

Known limitations

The Reconciliation feature has the following known limitations:

• When you are importing records from Pentaho packages or Data Loader, results of successfully updated records are shown with failure or error records.
• If the reconciliation process is ongoing and an update on at least one record occurs during an import, a temporary record is created and removed. This record is created with Mark As Deleted = checked and Inactive = checked and then removed within a few minutes, so that it never shows up in the Remedyforce CMDB viewer or any CI lookup within BMC Remedyforce.
• If you create a reconciliation rule on a Rule Based Asset (RBA) class, you can have a duplicate records, one in RBA and the other in the CI class on which RBA is created.

Best practices

We recommends the following best practices for the Reconciliation feature:

• For faster search of records in your CMDB, we recommend that you get the custom indexes added to the fields that you are using in your reconciliation rules. To add custom indexes on a field, contact Salesforce support. For more information, see Salesforce Help.
• We recommend that you mark the fields used in a reconciliation rule as Required. Records with blank fields are imported to Remedyforce CMDB but reconciliation is not applied.

• While importing records by using Data Loader, Pentaho Packages, or any other third-party tool, ensure that you import records in a batch of no more than 20 records. All the out-of-the-box Pentaho packages provided by Remedyforce to import records from various products have been updated to import only 20 records in a batch. We recommend that you use the latest Pentaho packages to import records or configure the recommended batch size in the packages that you are using.

  Steps to configure batch size in Pentaho

  1. From the KJB file, open the required KTR.
  2. In the KTR file, open the Upsert steps.
  3. In the Batch Size field, enter 20.
  4. Click OK.

  Steps to configure batch size in Data Loader

  1. In Data Loader, click Settings and then select Settings.
  2. In the Batch Size field, enter 20.
  3. Click OK.

• The Source field of type Picklist is added to the Base Element object. When you import records, the field value is set appropriately. For example, if you imported a record from BMC Atrium CMDB by using the Pentaho package provided out of the box, the field value is set to Atrium. This field is not available on the Instance Editor by default. You can use this field in reports. Ensure that you assign Edit permission for this field to your profile.

Creating reconciliation rules

To create a reconciliation rule for a class, you must configure fields of the class on which duplicate records of that class are identified. For example, to uniquely identify a record of the Computer System class, you can use a combination of Serial # and System OID fields.

The following section provides more information about reconciliation rules and when these rules are applied.

• Considerations for creating reconciliation rules
• Before you begin
• To create a reconciliation rule

Considerations for creating reconciliation rules

Consider the following behavior before you create a reconciliation rule.

Before you begin

You have large number of records (more than 200,000) in a class and you are enabling reconciliation for the class by creating a reconciliation rule. When a new record is created for the class or an existing record is updated, you might receive an error. To avoid the error, enable custom indexing on the fields used in the reconciliation rule for the class. To enable custom indexing on a field, you must raise a case with Salesforce. To resolve your case, Salesforce would require information, such as SOQL with bind values as mentioned here. For your reference, see Salesforce Help.

To create a reconciliation rule

1. Click the Remedyforce Administration tab.
2. On the Home page, click the Configure CMDB 2.0 tile, and from the menu, select Reconciliation Rules.
3. On the Reconciliation Rules page, select the Reconciliation Service slider.
   Selecting this slider ensures that whenever a new record is imported, a record is added through the Instance Editor, or an existing record is updated, reconciliation rules are triggered. This slider enables reconciliation on all classes.
4. Click New.
5. In the Rule Name field, enter a unique name for the rule.
   For example, Computer System (Serial Number or Asset #).
6. From the CMDB Class Type and Class Name lists, select the type of class and the class on which you are creating a rule.
   In the Class Name list, only the active, Rule Based classes are available. Abstract and hidden classes are not available.
7. Select the fields based on which you want to identify duplicate records.
   Only the fields of the selected class and the classes higher in the hierarchy (up to Base Element) are displayed. If you have added custom fields to the field set of the selected class or any class in its hierarchy, those custom fields are also shown. 

8. In the Filter Logic field, to form a unique value by using the selected fields, specify the logical operators.
   By default, AND is specified.

   Best practice
   We recommend that you use such fields in a rule whose value would be unique and do not use fields with lots of text, such as Description.
   You can use all the fields in any order in the Filter Logic. For example, you can use 2 AND (1 OR 3).
   

9. Click Save. 

   Note

   By default, records marked as deleted are considered in reconciliation. To exclude these records from reconciliation, update the value of the CMDB_ReconcileMarkAsDeleted custom setting to False.

Reconciliation log

You can configure BMC Remedyforce to generate a reconciliation log that captures the status of records that are imported from sources such as Remedyforce Discovery, Pentaho, or from other sources. You can capture the status of both records - records that reconciled successfully or records that failed to reconcile. The reconciliation log is appended for each import source and records older than 30 days are automatically deleted. 

Configuring reconciliation log settings

You can configure the following settings for the reconciliation log:

• Generate a log for records that failed to reconcile for a reason such as duplicate records. This setting is selected by default.
• Generate a log for records that successfully reconciled and resulted in the database update.
• Delete the log records that are older than 30 days. This setting is selected by default and the time for deletion is not configurable.

Before you begin

• Ensure that reconciliation feature is enabled in the organization.
• Enure that one or more reconciliation rules are configured.

To configure the reconciliation log settings

1. Click the Remedyforce Administration tab.
2. On the Home page, click the Configure CMDB 2.0 tile
3. From the menu, select Reconciliation Rules.
4. Click (Log icon).
5. Perform the following actions:
   • To generate the failure logs for reconciliation, select the Enable Failure Log check box.
   • To generate the success logs for reconciliation, select the Enable Success Log check box.
   • To delete the logs that are older than 30 days, select the Clear Logs older than 30 days checkbox.
6. Click Apply.

Viewing reconciliation report

You can view a reconciliation report to check the status of the records that successfully reconciled or failed to reconcile. The report shows the following information:

• For each record, the report shows the source from which the record is imported.
• For each successfully reconciled record, the Description field shows the details of the record that is updated in the database and the Configuration Item / Asset field shows the unique ID of the record.
• For each record that failed to reconcile, the Description field shows the reason why the reconciliation failed and the details of duplicate recorders that caused the reconciliation failure and the Configuration Item / Asset field is blank.

Before you begin

• Ensure that reconciliation feature is enabled in the organization.
• At least one set of records is imported and reconciled. Otherwise, a blank report is shown.
• Ensure that you have enabled log generation for reconciled records.

To view the reconciliation report

1. Click the Remedyforce Administration tab.
2. On the Home page, click the Configure CMDB 2.0 tile.
3. From the menu, select Reconciliation Rules.
4. Click Reports (icon). The CMDB log page is displayed.
5. Enter the report options to run the report.
6. Click Run Report > Run Report Now.

Verifying reconciliation status

You can view the field update history on the Remedyforce CMDB> Details > Auditing tab.

 The following table describes where can you view the results of reconciliation based on the product that you have used to import records to Remedyforce CMDB.

Related topics

01 enhancements

General-settings-for-Remedyforce