Common Data Model, Customizations, and Datasets - Best practices


The most important part of the BMC Helix CMDB, and what really set's it apart from other solutions, is its data model and how customizable it is.

The Common Data Model (CDM) consists of three major elements: BaseElement, BaseRelationships, and all sub-classes of each of these two main elements.

For example, a Computer System CI consists of a BaseElement, System, and ComputerSystem. Each of these three CIs has their own specific data elements, as shown in the following image, and generally will make this easier for the database to update specific records by using write calls. This ultimately leads to fewer blocked SQL queries to the database, leading to better performance in busy environments.

Sub-classes - 1.png


This modular approach also has benefits to extend the data model. For example, ComputerSystem is broken down into three sub categories: Mainframe, Printer, and StorageSubsystem.

But, someone might want to create a additional sub category like SuperComputer and you might want to track a attribute like application so you can track options like weather forecasting or electronic design, as shown in the following image:

Sub-classes - 2.png

This attribute would live in it's own form in the database, as not all computer systems would need to have this information recorded. Again, this cuts down on the amount of work you might need to do like recreating all of the same attributes that already exist in computer system, and reduces the storage requirements in the database, amount of data to write per change, and allows for more granular control of your data.

With this level of granularity, it is easy to get into trouble if changes aren't well thought out.

The first point of reference should always be the current CDM for your product version. For more information about the CDM, see Learning-about-the-common-data-model.

The most common issue customers have when extending the CDM is creating new (custom) attributes for data that already has a adequate attributes defined. A common example where a customer might create a redundant field are for the people relationships. Customers might create a Supported By attribute on Base Element instead of using the People/Support Group relationship. This might cause performance issues, data integrity issues, and issues with integrations such as TrueSight Operations Management.

So, a firm understanding of the existing data model is critical to extending your CMDB to capture data that might be relevant to your organization.

CDM and relationships

One of the most powerful aspects of the CDM are the relationships that can be used between the different CI classes. This data model is a real reflection of how a live IT Environment would work, many individual devices and components working together to create a single system that achieves a common purpose. Each of these items are unique and configurable in each environment and use case.

Scenarios

The following examples show how connected and specific the relationships are between CIs.

Basic relationships between CIs

Consider a simple example of an email service as shown in the following image:

CDM_Example 1.png

This model can include many components, such as a service CI, software product, computer system, network card, IP endpoints, and other network equipment. Each of these components are critical to the working of the entire system. If any of these components fail, the service is degraded.

This example can be expanded further.

First, Apache JAMES is a java based application, so a second product will need to be added for Java. This will require a dependent relationship, as Apache JAMES requires a version of Java to operate.

Next would be a Processor. It would be hard for a computer system to process any emails without a CPU.

In addition, an email system would need a networking system, including a IP Endpoint. This would also have unique relationships such as Hosted Access Point, that would need to be considered.

The final view of this expanded example, might look something like the following image:

CDM_Example 2.png

Because the email service depends only on Apache JAMES running, that is the only direct relationship. Everything else on the system, including OpenJDK, CentOS 8, the Processor, and IP Endpoint are only present to keep Apache JAMES operational. This model helps us better understand how our CIs interact with each other.

Relationships to show impact between CIs

In the following example, we consider a complex scenario where we provide some failover.

Consider there are two systems that have a level of redundancy. This allows for the email service to continue to operate if Server001 was rebooted for a regular, scheduled maintenance.

But notice in this example, there is still a single point of failure - the newly added Switch001 is a network switch that both computer systems are utilizing. If this switch was rebooted for maintenance, then neither computer system would be able to communicate with the organizations network, causing an outage of the email service.

With proper impact relationships configured, this would allow for applications, like Change Management, to properly simulate the impact of a change to determine what services would go offline in case of a change.

The following image shows that Server002 is configured differently:

CDM_Example 3.png

Instead of having an Intel processor, it has an AMD processor. In addition, it uses a different operating system and has a different IP endpoint. This is done to demonstrate how a dynamic environment might still deliver a similar service to the customers.


Extending the CDM

You can extend your CMDB through the CMDB Portal by using the Class Manager UI.

Warning

Never make changes to CMDB objects through AR System Developer Studio, and never make changes to the Asset Management forms to include new objects to Asset Management.

To make changes, first open the CMDB Portal by navigating to the following URL for your system:

http(s)://<arserver-restapi-ip>:8008/cmdb/index.html

To learn more about navigating and using the CMDB Class Manager, see the following documentation:

One of the common issues when extending the CDM is creating new classes and attributes within the BMC.CORE namespace. We recommend you create your own namespace instead of adding your custom attributes to the BMC.CORE namespace. This prevents your extensions from being overwritten by new classes during upgrades. The following image shows a new class being created with a custom namespace:

custom namespace.png

The following image shows a new attribute being created with a custom namespace:

Custom namespace attribute.png

As a rule, you can create namespaces in the format <COMPANY.PURPOSE> naming convention, and then add each extension under that namespace until the project is complete.

Additionally, if you create a custom class in CMDB and you want to use that class in Asset Management, create a corresponding record on the AST:ClassAttributes form by using the Sync Asset UI with CMDB mechanism.

Updating BMC Helix ITSM: Asset management with CMDB changes

Most users will interface with CMDB data by using Asset Management, which is not the same as CMDB, but Asset Management needs to match all of the CMDB data. After the changes are made to the CDM by using the Class Manager application, the next step is to ensure your changes are available to users in Asset Management. This is achieved by using an Asset Management mechanism called Sync Asset UI with CMDB.

To learn more about this mechanism, see Synchronizing the BMC Asset Inventory forms with BMC CMDB in the Asset Management online documentation.

When you use the Sync Asset UI with CMDB function, consider the following points:

  • If you have a custom view on any of the AST forms, create a backup of the view before running this function.
    If not, the view might be overwritten or corrupted during exporting the view definition process. For more information about exporting view definitions, see Exporting object definitions, views, and applications.
  • If you create a new custom class, the initial user interface (UI) for Asset Management is inherited from the AST:BaseElement form. You can then customize this form UI by using Developer Studio and hiding, showing, and moving fields to places that work for you.
  • Certain fields are not fully synchronized and might require additional work.
    As an example, Enum fields (selection fields) do not have correct alias values and must be populated.
  • The Sync Asset UI with CMDB function creates duplicate fields if it finds a field with the same name in the Asset Management form and in the CMDB form.
    It creates the duplicate fields with identical names but with different field IDs. Hence, do not create fields on the Asset forms directly.
  • The Sync Asset UI with CMDB can cause a momentary disruption in certain operations such as notifications being processed from the Application Pending form.
    It is not uncommon for notifications to be up to 30 minutes late due to this function.
  • You might encounter some issues with certain version with PWA views when performing a Sync Asset UI with CMDB operation.
    These issues might be due to missing parent field IDs.

Retaining custom classes after an upgrade

There are additional steps for extending the CDM if you want these changes to last through an upgrade. To ensure that your custom classes are retained through an upgrade, Synchronizing the BMC Asset Inventory forms with BMC CMDB. 

Datasets in BMC Helix CMDB

One of the core components of the CDM are the fields that allow for the characterization of CIs. None of these are more important than the DatasetId (400127400) field. A dataset is a logical grouping of data in BMC Helix CMDB and is used to capture data from different data sources. Each source of data for your CMDB should store the data in a separate dataset, as shown in the following image:

CMDB data flow.png

You can create, delete, and change the permissions of datasets by reviewing the following documentation:

Warning

Do not delete a dataset that has been used for anything at anytime. Deleting a dataset is only allowed when it was created unintentionally, and was never used to store any data.

CMDB comes with several datasets by default. The first dataset is the golden or production dataset. By default that is the BMC Asset (BMC.ASSET) dataset. This dataset is treated as the Single Source of Truth for your CIs in your organization. All other datasets are nothing more than a staging area for the data that the organization's other tools collect about information in the environment. The following table shows lists of some of the common datasets and the recommended purpose:

Dataset Name

Dataset ID

Installed By

Purpose

BMC Asset

BMC.ASSET

BMC Helix CMDB

This is the production dataset that you treat as your single source of reference and that you use to make business decisions.

BMC Sample

BMC.SAMPLE

BMC Helix CMDB

This is a staging dataset and a safe place to do testing.

BMC.ADDM

BMC.ADDM

BMC Helix CMDB

This dataset stores the CIs and relationships from BMC Discovery.

BMC Atrium Explorer Asset

BMC.AE.ASSET

BMC Helix CMDB

This is a staging dataset.

When a user edits a CI in BMC.ASSET dataset (production dataset), CMDB creates a sandbox for that user. For example, if your user name is Chris, then BMC.AE.SB.Chris dataset is created for internal use. When the user promotes the CI changes, the changes are copied to BMC.ASSET dataset and the entries are deleted from this internal dataset.

BMC Atrium Explorer

BMC.AE

BMC Helix CMDB

This dataset reconciles data from sandbox datasets to BMC.ASSET. Atrium Explorer - Identification and Merge reconciliation job of this dataset is used to reconcile data from sandbox datasets to BMC.ASSET.

BMC.SANDBOX.DSM

BMC.SANDBOX.DSM

BMC Helix CMDB

When a relationship is created by using Dynamic Service Modeling (DSM) based on a given qualification, the relationship is first created it in the BMC.SANDBOX.DSM dataset. The relationship is later reconciled to the BMC.ASSET dataset.

BMC.ASSET.SANDBOX

BMC.ASSET.SANDBOX

BMC Helix ITSM: Asset Management

If BMC Asset Management is configured with a sandbox dataset, CIs that you manually create or modify, flow through the sandbox dataset. Otherwise, CIs go directly into the production dataset.

BMC Configuration Import

BMC.IMPORT.CONFIG

BMC BladeLogic Client Automation

Imports CIs and relationships from the BMC BladeLogic Client Automation database for reconciliation.

Important

The very first rule of the production dataset is not to update the production dataset directly.

Data found in the production (golden) datasets should be the result of both, the normalization and reconciliation processes and not manual or automated data entry. All of the other datasets should only have their data updated via the tool that is scanning for this information.

For example, the BMC.ADDM dataset should only be updated by BMC Discovery and not by another tool or manually, by a user. Here are some additional things to keep in mind with each dataset:

  • Your organization should thoroughly document each dataset in your environment and clearly state the purpose of the dataset and the tool that is uses that dataset.
  • Except for the production dataset, each dataset should have a normalization and a reconciliation job created for it.
    The production dataset should never have an automated job run against it.
  • As a rule you should only merge one source dataset into the production dataset at a time.
    This helps cut down on the performance impact of reading and writing to the same record multiple times. A way to achieve this is to create a single, intermediate dataset that is used as a pre-production dataset.

Datasets and how they affect Asset Management

While all of the components of the CMDB are important, most users will never interact with the CMDB directly. Instead, they'll interact with CMDB through ITSM applications such as Asset Management, Change Management, and others.

In addition, one of the key, but often overlooked, datasets is the BMC.ASSET.SANDBOX dataset, otherwise known as the "Sandbox" dataset. This is the dataset used when users are in Asset Management. Data is read from the BMC.ASSET dataset, but updates are written to BMC.ASSET.SANDBOX and then Normalization and Reconciliation can process the records as per normal process.

For all of this to take place, Asset Management needs some initial configuration. To learn more details about configuring Asset Management, see Configuring Asset Management settings.

As a best practice, each ITSM instance should be configured as shown in the following table:

Setting

Value

Action on CI Submit

None

Dataset Name

BMC.ASSET

Dataset Access

Writable by Client Only

Sandbox Dataset Name

BMC.ASSET.SANDBOX

Sandbox Enabled

Yes

Sandbox Job Calls

Inline

Sandbox Job Calls for People

Inline

Server Date Format

This should match the format used by the server OS

Default Company

This should be your parent Company

The key element here is calling out the production dataset and sandbox dataset that Asset Management is going to use. Setting these values are critical to data integrity. Without this configuration, the risk of duplicate CIs, bad data integrity, and additional issues with other products is significant.

Some things to keep in mind about the sandbox are:

  • Do not allow users to directly update the golden dataset for any reason. Users often tend to enter data incorrectly by either misspelling words, by using abbreviations instead of spelling out whole words, or just omitting key information, it is best to send the data through all of the normal processing mechanisms to ensure data integrity.
  • Using a sandbox dataset will require that the normalization and reconciliation processes are run for each modification made to the production dataset and the changes will be visible only after these processes are completed in the Sandbox, keeping data integrity intact. If the changed data is not consistent with the rules and standards defined for these processes, your changes might be rejected and will not propagate to the production dataset. A CMDB administrator should review and remediate these failed CIs by using their normal review and remediation process. For more details, see Normalization-Best-Practices and Reconciliation-Best-Practices.
  • For a mature CMDB the Unapproved Handling for a Normalization job should be set to Reject on the NE:DatasetOptions form for the sandbox dataset. This will force aliases to be created for product catalog entries and then enforce the data integrity rules for your organization.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*