Batch benchmarking

Batch benchmarking refers to load testing a series of calls to a server without a graphical user interface. BMC Atrium CMDB batch processing is an example of batch benchmarking. Batch benchmarking can reveal the throughput of processing thousands, and even millions, of items through your servers.

This topic includes the following information:

Batch processing

CMDB batch processing includes the following steps:

Onboarding or updating configuration items (CIs) — Onboarding simulates the loading of new CIs into a source dataset in the CMDB. BMC recommends that you use off-peak hours for onboarding a large number of CIs. Smaller quantities of CIs (for example, 5,000 CIs and relationships, or CiRs) can be onboarded with an online transaction processing (OLTP) load. Updating CIs refers to some CIs being modified in the source dataset.
Normalizing CIs — The normalization process ensures that product names and categorization are consistent across different datasets and from different data providers.
Reconciling CIs — The reconciliation process compares data from different data sources and creates one complete and correct production dataset. The reconciliation consists of the identification and merge activities. Identification recognizes class instances that are the same entity in multiple datasets. Merge consolidates CI attributes from a source dataset to a production dataset.

In batch processing, you can run normalization and reconciliation jobs in either batch or continuous mode. Batch jobs process all outstanding CIs at once and stop when those are finished. Continuous jobs keep running and poll at specific intervals for work to be done. Continuous jobs do not end until they are manually stopped.

Onboarding is usually done with batch jobs because a large bulk of CIs needs to be processed. In a production environment, batch or continuous jobs might be run, depending on how many CIs need processing. If CIs are trickling in, a continuous job might be the better option. If you have a small set to process (for example, fewer than 10,000 CIs), a batch job is sufficient.

Determining the data source and testing volume

CiRs for onboarding can come from a variety of sources, such as BMC Atrium Discovery and Dependency Mapping and BMC Atrium Integrator. To ensure that data represents your company, acquire data from one of these production sources. Make a copy of the data to facilitate retesting.

From these sources, determine the data model and CI attributes used. A typical data model includes the ComputerSystem CI as the parent. The children are the parts found in a computer system, such as the software represented by the Product CI.

A destination dataset can be empty or populated with CiRs. First-time onboarding usually means that the CMDB is nearly empty. Determine what testing volume best describes your situation.

Setting up the test environment

After obtaining a volume source CMDB, set it up in its own test environment that contains a source application, an AR System server, and BMC Atrium CMDB. Isolate the test environment from other traffic to achieve repeatable tests.

Executing the test

To ensure performance can be measured for onboarding, normalization, and reconciliation, perform each of those steps individually.

When creating a CMDB batch processing checklist, include the following items:

Restore the database to baseline data
Verify that no other activities are on the system
Set basic system and software configuration
Load CiRs into the source dataset
Verify that all CiRs have been loaded
Optional: Back up the database for repeating normalization and reconciliation tests
Start a normalization batch job
Measure normalization throughput
Optional: Back up the database for repeating reconciliation tests only
Start a reconciliation batch job that includes identification and merge
Measure reconciliation throughput

When you onboard CiRs (load new CiRs into a source dataset in the CMDB), the source dataset should be empty. During onboarding, record the start and end times and number of CIs and relationships processed.

The BMC.CORE:BMC_BaseElement and BMC.CORE:BMC_BaseRelationship forms can be queried by the dataset name to determine how many CIs and relationships were loaded.

After a successful CiR load, back up the database. You can then rerun normalization and reconciliation without having to load CiRs again.

Normalization requires that you create a normalization job that starts and stops normalizing on your source dataset. In this case, run a batch normalization job so that it processes all un-normalized CIs and stop the job when those are finished. Record the start and end times and the number of CIs processed. Obtain this information from the BMC Atrium Console where the normalization job was started. Roll your mouse over the normalization job to display the information.

After a successful normalization, back up the database again. You can then rerun reconciliation without having to load CiRs and normalize again.

Reconciliation requires that you create a reconciliation job that identifies and merges all CiRs on your source dataset to the destination dataset. This reconciliation job is usually BMC.ASSET. You can either create separate reconciliation jobs to do identification and merge separately, or create one reconciliation job to do both. Separating the identification and merge enables you to observe the behavior of each more thoroughly. Separating or combining reconciliation activities gives you throughput information on each activity.

A batch reconciliation job starts and stops reconciling when CiRs are processed. Roll your mouse over the reconciliation job to display the start and end times and the number of CiRs processed.

For each step, measure the system resources used, such as the CPU and memory on the AR System server and the database. The BMC Atrium Console shows the normalization and reconciliation job progress, information about start and end times, and the number of CiRs processed.

Analysis and reporting

In CMDB batch processing, throughput can be calculated by CiRs per second (CiR/s). The throughput formula is as follows:

Total number of CiRs/Total time to process (seconds) = Throughput

Calculate throughput for onboarding, normalizing, reconciliation identification, and reconciliation merge.

Use the steps for online benchmarking reporting to present batch benchmarking results.