Normalization - Best Practices
Normalization of data makes sure that all CIs are in the standard format that every business unit consumes that data in the same way.
For example, a product that has a model name Latitude might not appear on a report or search for a model name with Dell Latitude, even though it is the same product. Normalization helps standardizing such values. It validates approved and normalized products.
Watch the following video (16:21) for an overview of configuring normalization job:
Scenarios
To better understand the normalization process, see the following example of different CIs that are brought into CMDB through different sources and that go through the normalization process:
Source: Atrium Integrator
Dataset: ORG.AI.Spreadsheet.Import
Consider the example of one CI from the dataset ORG.AI.Spreadsheet.Import that was imported by using a spreadsheet that Atrium Integrator collected. The data in the spreadsheet was collected from an export of some procurement receipts.
During the normalization process, this CI would go through the following steps:
- Because normalization is carried out for a class, the dataset is evaluated for all CIs in the same class that have one of the following values as normalization status:
- Not Normalized
- Normalization Failed
- Modified After Last Normalization
- Null
- Each CI is the evaluated individually against the Product Catalog data.
In this instance, there is no product catalog entry found. - If there is no mapping found in the product catalog, the Product Catalog Alias listings are evaluated to determine if there is an alternate value that maps to a known Product Catalog entry. In this instance, no such alias mapping exists.
- Finally, the CI is updated as 'Normalization Status' = Normalization Failed and Not Approved because it does not know of this record.
Source: BMC Discovery
Dataset: BMC.ADDM
Consider an example of a CI from the BMC ADDM dataset. This dataset is created by an automated discovery product during a routine scan of the CIs. This datasource is consistent with the Product Catalog. During the normalization process, this CI goes through the following steps:
- Because normalization is carried out for a class, the dataset is evaluated for all CIs in the same class that have one of the following values as normalization status:
- Not Normalized
- Normalization Failed
- Modified After Last Normalization
- Null
- Each CI is the evaluated individually against the Product Catalog data.
In this instance, a match is found. - The record is automatically updated as 'Normalization Status' = Normalized and Approved because the record matches a Product Catalog entry.
Source: Manual entry
Dataset: BMC.ASSET.SANDBOX
Consider the example of a CI that was manually created by Asset Users. This CI was newly created during a support ticket. During the normalization process, this CI goes through the following steps:
- Because normalization is carried out for a class, the dataset is evaluated for all CIs in the same class that have one of the following values as normalization status:
- Not Normalized
- Normalization Failed
- Modified After Last Normalization
- Null
- Each CI is the evaluated individually against the Product Catalog data.
In this instance, there is no product catalog entry found. - If there is no mapping found in the product catalog, the Product Catalog Alias listings are evaluated to determine if there is an alternate value that maps to a known Product Catalog entry.
In this instance, there is a match. - The normalization engine then updates the CI details with the mapped data, and is evaluated again against the product catalog.
- Finally, the record is updated as 'Normalization Status' = Normalized and Approved because it was able to find an alias mapping and update the record.
The following image represents the normalization process based on these three examples:
Creating a Normalization Job
To create a normalization job, in the CMDB Portal, select Jobs > Manage Normalization and click the Create Job button on the Normalization page, as shown in the following image:
This will allow the creation of a batch job that will run on a recurring schedule allowing for the evaluation of all records. The following image shows the settings that are available when creating a normalization job:
The following table provides details about the job settings to configure when creating a normalization job:
Setting | Value | Additional Information |
Job Name | Provide a meaningful name. Usually this is described as the source dataset's job. | |
Dataset | Select the dataset that will be normalized with this job. | |
Allow New Product Catalog Entry | Select this setting to create new product catalog entries. If this is selected, when the job runs it creates product catalog entries that are missing for records. This is useful if you have an already normalized dataset like when you first import CIs into your CMDB. This shouldn't be used for regular Normalization Jobs. | |
Allow Unapproved CIs | Select this setting to normalize CIs that do not have approvals in the product catalog. Warning: This shouldn't be used for regular normalization jobs. | |
Suite Rollup | Select this setting to define suites and their products, and to identify an instance as a suite, a suite component, or a standalone application. This setting allows more accurate management of product suites and individual licenses. | |
Relationship Name | Select this setting to replace the existing relationship names with the BMC recommended or custom names that are based on the CI classes in the relationship. This setting is applicable to relationship classes only. | |
Instance Level Permissions | Select this setting to set the row-level permissions on CIs as defined by the CI classes and additional qualifiers of the instance attributes. | |
Impact | Select this setting to set the impact attributes for CIs based on the out-of-the-box or custom impact rules for the relationship class, and the source and destination classes to which the CI relates. The impact normalization feature is applied to relationship classes only. | |
Custom | Select this setting to create custom rules when the out-of-the-box Normalization features do not meet your requirements for normalizing data. You can create a custom rule for CI and relationship classes to perform a required action. |
After you save the normalization job, a batch level job is created for the defined dataset.
Normalization job schedules
To learn more about how to schedule your datasets, see the following table:
Use Case | Schedule |
|---|---|
Bulk Data Loads - Large numbers of records like Company expansion or major system overhaul | No Schedule, ad-hoc Only |
Incremental Data Loads - Small number of records like a monthly shipment of new workstations | Recurrence Only |
Regular Data Loads - Usually coming from automated tools, like Discovery. | Recurrence Only |
CMDB Explorer Modifications - Individual modifications made by users updating CIs | Continuous |
Asset Management Modifications - Individual Modifications made by users updating Assets | Use default Sandbox Job |
Configuring dataset normalization rules
Each dataset can be configured independently of the others. This allows for more granular control of your data.
To modify the configurations, open the CMDB Portal, select Configurations > Manage Normalization Rules > Dataset Configurations.
The types of changes that can be made here are as seen below:
The different settings that can be changed are outlined in the below table with the recommended values:
Setting | Value | Additional Information |
Normalization Type | CTI Only | Name and CTI Lookup: Normalizes the product name, model, manufacturer name, CTI (Category, Type, and Item) attributes, and categories. (Default) CTI Only: Normalizes only the CTI categories for a CI, but the Model and ManufacturerName attributes are not normalized. |
Enable Normalization | ✅️ | (Default) Selected: Normalization is enabled for the selected dataset. |
Inline Normalization | Selected: Normalizes data as it is created or modified. For more information about configuring inline normalization. Inline - Normalization is done as soon as a CI is created. | |
Allow Unapproved CIs | ✅️ | (Default) Selected: If the product is unapproved in the Product Catalog, the Normalization Engine normalizes the CI and sets its NormalizationStatus attribute to Normalized and Not Approved. |
Allow New Product Catalog Entry | Selected: If no entry for a CI is found in the Product Catalog, then a new product entry is created in the Product Catalog, using the attributes of the discovered product. The created Product Catalog entry is flagged as Approved. You can review and change the status of these entries in the Product Catalog. | |
Preserve Auto Impact Edit | Preserve Manual Edits | Apply Rule Always: Overwrites the manual edits that you make to the HasImpact and ImpactDirection attributes of the relationship instances with the values that are defined in the best-practice impact normalization rules. (Default) Preserve Manual Edits: Retains any manual edits that you do to the HasImpact and ImpactDirection attributes of the relationship instances, instead of reapplying the values that are defined in the best-practice impact normalization rules. |
Version Rollup | Rolls the product versions (the MarketVersion attribute) to the highest version with which the licenses are associated. This feature provides more accurate version information for managing licenses. | |
Instance | Sets the row-level and attribute-level permissions for CIs as you define them. | |
Relationship | Replaces the existing names for relationships with the BMC recommended or custom names, based on the CI classes in the relationship. | |
Auto Impact | Sets the HasImpact, ImpactDirection, ImpactPropagationModel, and ImpactWeight attributes on the relationship classes based on the defined rules. | |
Custom | Sets attribute values of CIs and relationships based on the custom rules that you have defined. | |
Suite Rollup | Defines suites and their products, and identifies an instance as a suite, a suite component, or a standalone application. This feature enables an accurate management of suite and individual licenses. |
Configuring Class Normalization Rules
Normalization can also be configured for classes as well. This allows for even more granular control of your data.
To make a change, open CMDB Portal, select Configurations > Manage Normalization Rules > Class Configurations, as shown in the following figure:
The classes that you want to normalize, must be listed in the Class Configurations screen.
The default classes included with CMDB are already configured appropriately. In the following images with an example of Computer System and Product classes, each of the fields listed in the respective configurations with a check mark must have a value for the normalization process to complete:
In the examples, you can see that BMC Product class also requires a PatchNumber value, and BMC ComputerSystem class doesn't require them.
Reviewing Normalization Failures
A important part of normalization is monitoring the results. When reporting on any CI, always use the specific class form to evaluate the CI record. Reports themselves can be run against BaseElement, but remediation or evaluation needs to be done by using classes such as Product and ComputerSystem.
The following scenario might need to be evaluated:
When evaluating failed normalizations, create a report on the BMC.CORE:BMC_BaseElement form using the following qualification:
This report will be used to confirm records that need attention from your CMDB administrator by either adding a Product Catalog entry, alias, or modifying data on the record. Each CI will need to be evaluated individually and corrected.
Frequently Asked Questions
See also: BEST FAQ on BMC CMDB Suite - Normalization on BMC Community.
Q. What best practices are useful for planning Normalization?
A. See Planning-data-normalization-and-the-Product-Catalog
Q. What best practices ensure the successful operation of Normalization?
A. See Preparing for Normalization
Q. How can I learn more about how Normalization works?
A. See Normalization-process
Q. Where can I find a walk though example of Normalization?
A. See BMC Atrium CMDB - How to Normalize data successfully on BMC Community.
Q. How do I improve the performance of Normalization?
A. See Normalization-best-practices-to-achieve-improved-performance
Q. How do I resolve ARERR [8753] Error in plugin : BMC.FILTERAPI.NORM.ENGINE?
A. See BMC Atrium Core CMDB:Normalization: ARERR [8753] Error in plugin : BMC.FILTERAPI.NORM.ENGINE on BMC Community.
Q. How do I resolve ERROR 120531?
A. See Atrium CMDB - Getting an ERROR 120531 during Normalization Jobs (Batch mode) on BMC Community.
Q. How do I troubleshoot Normalization Issues?
A: See BMC CMDB Suite - Troubleshooting Normalization Issues with CMDB Normalization Engine on BMC Community.