Importing data from BMC Helix Discovery

BMC Discovery, earlier known as ADDM, automates data center asset discovery and application dependency mapping to provide a holistic view of all your data center assets, multi-cloud services, and their relationships.

With the enhanced integration between Remedyforce and BMC Discovery/BMC Helix Discovery, devices, components, services and relationships that are discovered in BMC Discovery or BMC Helix Discovery can be automatically imported into and synchronized with Remedyforce CMDB. This integration no longer requires Pentaho. The configuration is done within the solutions and the data mappings and transfers are achieved with APIs. 

For more information about the enhanced integration, refer the following topics:

Prerequisites 

To use this feature, you must have the following minimum versions of Remedyforce and BMC Discovery: 

• Remedyforce 20.18.01
• BMC Discovery 11.3 

 How does the integration work 

BMC Discovery uses multiple REST API/Bulk API calls to your Salesforce organization to retrieve and push data. Each Salesforce organization has a 24 hour limit of batches and API calls that it can make. Therefore, BMC advises to schedule the data sync at such a time when the API consumption by the Discovery integration does not affect other API consuming functionalities. For information about the limits that you have based on your license, see  Salesforce Help . 

The following diagram shows integration architecture of BMC Helix Remedyforce with BMC Discovery.

The following diagram shows integration architecture of BMC Helix Remedyforce with BMC Helix Discovery.

Connection security 

A secure connection is required from a Salesforce.com application to an external application. To establish a secure connection, you need to create a Connected App. A Connected App can connect to Salesforce.com over Identity and Data APIs. Connected App uses the standard OAuth 2.0 protocol to authenticate, provide Single Sign-On, and acquire access tokens for use with Salesforce APIs. For more information about creating a Connected App, see Creating a Connected App in Remedyforce.

Advantages of Enhanced Integration 

The Enhanced Integration feature provides the following advantages:

• Prior to the Remedyforce Winter 18 release, only 6 out of 28 classes were imported. To import any additional classes, Pentaho jobs and mappings had to be created, that is, customization was required. With the Enhanced Integration feature, all 28 classes can be imported without any required customization.
• Provides a more complete view of one’s datacenter that empowers Remedyforce users across a range of roles to make more informed decisions across all Service Management processes. Following are some examples:
  • Change Management – By providing a more complete view of the datacenter, the Change Manager will be able to perform more comprehensive change impact analysis.
  • Incident Management – The Remedyforce Agent can field a ticket, identify the server and link it to the ticket enabling the team performing the analysis to have more details on that server to better handle the incident ultimately reducing MTTR (mean time to recovery) with accurate data for root cause analysis. The solution can also help prioritize and route tickets based upon an applications business impact.
  • Configuration Management – Many organizations define standard configurations to reduce complexity and management, while also mitigating risk. With a more complete view of one’s datacenter, the Configuration Manager can verify and identify devices not in compliance. This also benefits the Security Team.
  • Asset Management – The Remedyforce Asset Manager can see all servers in use in the datacenter to help determine availability.
  • Software Asset Management – With a catalog focused on datacenter software, the integration can classify, normalize, and import datacenter applications to help the Software Asset Manager understand what exists in their datacenter.
  • Security – The solution supports a range of security activities including documenting inventory for compliance requirements, identifying servers that could be back door entry points and understanding the business impact of a threat and which assets may be affected. 
  • Simplified setup and management, moving from customization to configuration by providing new configuration pages and access to logging. Administrators no longer need to write or modify code.

Considerations 

• The synchronization will override the mapped fields.
• At the time of resycnching, the following actions are performed:

• • CIs and relationships missing from the dataset are created.
  • Any CIs or relationships present in the dataset, but not in the local shadow copy are marked as deleted.
  • All CIs and relationships are updated to ensure their attributes match those in the local shadow copy.

• The non-mapped fields are not overridden.
• BMC recommends that you use the Enhanced Integration feature on your sandbox first because the existing data is modified. Therefore, before using the feature on production environment, ensure that your data is consistent in the sandbox environment.
• Mapped fields should be included in the fieldsets of the respective classes.

• Ensure that API names of custom fields that you create are not same as any Remedyforce custom fields.
  

• Connect a single appliance of BMC Discovery with a production or sandbox organization to ensure successful data synchronization.  

If you were using the Pentaho  packages earlier and now moving to this new integration, then you must understand the following requirements:

• Ensure that the records that you imported using the Pentaho packages have source field in Base Element object set to BMC Discovery, otherwise, BMC Discovery will not consider those records while syncing and it results in duplicate records.
• Instances in Remedyforce CMDB where source is not BMC Discovery are not impacted by the import.
• The final records in Remedyforce CMDB will be the same as the shadow copy in BMC Discovery. For more information about the shadow copy, see  CMDB synchronization .
• Once resync happens when you use the Enhanced Integration feature, the unique CI source is updated and you cannot switch back to Pentaho packages. For more information about resynching, see  Resynchronizing a CMDB connection .

Setting up the integration 

To use the feature, you need to perform the following tasks:

1. Create a connected application in Remedyforce and obtain the following values from the connected application:
   • Consumer key 
   • Consumer secret
2. Ensure that the user whose credentials will be used to set up the CMDB synchronization connection has at minimum the following permissions:

   • Access to the BMCServiceDesk.CMDBAPI Apex class.

   • Access to the Connected app.

   • Read and write permissions on the Base Element and Base Relationship objects.

   • Read and edit access on all the fields of Base Element and Base Relationship objects.

   • Read permission on the CMDB Class object. 

3. Set up a CMDB synchronization connection in BMC Discovery.

Creating a Connected App in Remedyforce 

1. Go to Remedyforce > Setup.
2. In the Search box type Apps.
3. Based on whether you are on Salesforce Classic or Salesforce Lightning, perform the following action.
   • For Salesforce Classic, from the search result, click Build > Create > Apps > Connected Apps > New.
   • For Salesforce Lightning, from the search result, click Apps > App Manager > New Connected App.
4. On the New Connected App page, enter the required values. For more information, see Connected App parameters.
5. Click Save.

6. Click Continue.
   You will be redirected to your Connected App page.

7. Note your consumer key in the Consumer Key field.

8. Note your consumer secret in the Consumer Secret field by clicking Click to reveal.

Connected App parameters  

The following table lists the fields that you need to configure for creating the Connected App. For more information about the fields, see  Salesforce Help

Setting user permissions for a Connected App 

You can manage a Connected App to set permissions such as session level policy or IP Relaxation policy to determine whether a user’s access to the app is restricted by IP ranges. For more information about the policies that you can configure, see  Salesforce Help . Perform the following following steps to manage the Connected App.

1. Go to Remedyforce > Setup.
2. In the Search box type Apps.
3. Based on whether you are on Salesforce Classic or Salesforce Lightning, perform the following action.
   • For Salesforce Classic, from the search result, click Build > Create > Apps > Connected Apps.
   • For Salesforce Lightning, from the search result, click Apps > App Manager.
4. Based on whether you are on Salesforce Classic or Salesforce Lightning, perform the following action.
   
   • For Salesforce Classic, click Manage next to the Connected App that you want to manage.
   • For Salesforce Lightning, click Manage in the the drop down of the Connected App that you want to manage.
5. On the Connected App Detail page, click Edit Policies.
6. Under the OAuth policies section, select the permissions that you want to provide by clicking the Permitted Users field.
7. In the IP Relaxation field, select the IP restrictions that you want to enforce. The default value for this field is Enforce IP restrictions, which limits this app to a specified range. Change the value to Relax IP restrictions or ensure that the IP range restrictions are valid. For more information about IP restrictions, see  Salesforce Help
8. Click Save. 

Setting up a CMDB synchronization connection in BMC Discovery 

Before you begin

• Ensure that you have obtained the consumer key and consumer secret values from your Connected App as documented in the section Creating a Connected App in Remedyforce . You will need to enter these values in the Consumer Key and Consumer Secret fields respectively when you are setting up a CMDB synchronization connection.
• Ensure that the Reconciliation feature is enabled to avoid duplicate records.
• If you are migrating from the old to the new integration, ensure that you have referred the considerations.

To set up the CMDB synchronization connection

1. Log in to BMC Discovery.
2. From the main menu, click the Administration icon .
   The Administration page displays.
3. From the Model  section, click  CMDB Sync . 
   If no CMDB Sync has been configured, a banner displays stating that no CMDB Sync connections are configured.
4. Click Add CMDB Connection to add and configure a new connection.
5. Enter the required details on the Add CMDB Connection page. For more information about the connection details, see Connection details.
6. Click Test Connection to verify that the new connection can be reached and the credentials are valid. Testing the connection also retrieves the list of companies from the CMDB for connections other than Remedyforce.

Connection details 

Enter the following details on the Add CMDB Connection page.

Connection name

Options for Remedyforce

Common options

Managing the synchronization connection 

Support for custom class 

If you have created a custom class or a custom relationship, you can create a mapping by adding Patterns for a custom class support. To create Patterns, the Pattern Language, TPL, is used to describe applications, products and other real-world entities that have been modeled in BMC Discovery. For more information about the language and how to add Patterns, see  The Pattern Language TPL .

To map custom fields

You need to set the following properties in the SysProperties object to support custom classes and relationships. After you have made the required changes, check the entire value in an online JSON parser for validation. 

1. Go to All Tabs.
2. Go to SysProperties tab.

3. If the list view does not include Property and Value fields, edit the list view and include these fields in the list view.

   Note

   If the user does not have access to the SysProperties tab, you will have to grant the access from profile or permission set.

4. The following table lists the properties under SysProperties tab. After you make the required changes in the properties, click Save.

To remove mapping of OOTB and custom fields

1. Go to All Tabs.
2. Go to SysProperties tab.

3. If the list view does not include Property and Value fields, edit the list view and include these fields in the list view.

   Note

   If the user does not have access to the SysProperties tab, you will have to grant the access from profile or permission set.

4. After you make the required changes in the properties, click Save.

   Note

   Ensure that if you are removing the last row from the BMCDiscoveryFieldsMap property, you do not put a comma before the '}'.

Discovering cloud instances

BMC Discovery and BMC Helix Discovery discover the tags provided in BMC Helix Remdyforce and identify the relationship of these tags with the cloud instances. To support this functionality, the following fields and the picklist option are added in BMC Helix Remedyforce:

Best practices 

• For integration, create a user with restricted permissions. For more information about permissions, see Setting up the integration.

• Schedule the synchronization during weekends so as not to consume API requests during the business days.

Salesforce batch size limitation

Salesforce limits the number of daily batch API calls to 10,000 batches per rolling 24-hour period. As Discovery to Remedyforce integration leverages this batch logic driven by Discovery, if this limit is reached within a rolling 24-hour period, the Discovery integration stops running until API calls are freed up.

To avoid this situation, we recommend the following best practices:

• For a first time sync (Resync option) set a batch size of 100 and turn off Reconciliation if any existing records are not in the initial load. Reconciliation can be turned on once the resync is completed. Depending on the number of records to be synced, you might need to run the initial load across multiple days. For example, if you have a 1,200,000 records with a batch size of 100, the process will run for a couple of days as in a best case scenario only 1,000,000 (100*10000) records will get synced in a day due to the Salesforce 24-hour period batch limit.
• For continuous resynching (Continuous Sync option), Discovery keeps Remedyforce CMDB in sync by sending updates as and when a CI is discovered or updated. Such updates consume batches. Considering CIs getting updated on daily basis and the batch size limit, adjust the Discovery scan frequency and the number of devices to be scanned (based on IP Range and so on). Ensure that no more than 6000 host machines are scanned within 24-hour period. 
  
• Discovery sends updates only if any attribute of the CI gets modified and is mapped for an update. If you add any additional fields to the mapping, avoid attributes of less relevance.
  
• For example, if the Last Scan Date field is added to the mapping, batches will be consumed to update Last Scan Date even though no other attribute has been modified.
  
• Blackout period from Discovery can be leveraged to control the volume of data to be updated in Remedyforce CMDB on daily basis so as not to breach the Salesforce 24-hour batch limit. Define your blackout period in Discovery in a way so as not to breach the Salesforce 24-hour batch limit.
  
• We recommend that you test your production data in a sandbox environment to ensure the proper configuration.

Related topics

CMDB synchronization

CDM Mapping for Host

Troubleshooting issues when integrating with BMC Discovery