Evaluating volume with production or synthetic data

Depending on your benchmarking goals, you can use real production data or synthetic data in the test database.

Using production data in the test database provides the following benefits:

• Produces realistic results for benchmarking current production
• Shows performance changes from old to new applications
• Determines whether an upgrade will work for your organization
• Validates synthetic data

However, you can generate synthetic data to demonstrate benchmark scalability. You must use synthetic data when production data is unavailable.

This topic includes the following information:

Using production data

Using production data requires knowledge of what is in the production database. To use production data, take a snapshot of the production database and use the snapshot for all tests.

When using production data, use the following guidelines:

• Determine users with assigned tickets
• Determine users with proper permissions
• Modify user passwords
• Determine search criteria
• Back up the test production database

Determine users with assigned tickets

If Update Incident or Update Change is involved in your transactions, tickets must be updated. Only users assigned to work on open tickets can perform updates — you must know who these users are.

For example, if you query the HPD:Help Desk form by Assignee and the Assigned, In Progress, or Pending status and then group the results by Assignee name, you have the names of users who have open tickets. This query also shows the number of tickets that each user has, which can be important, depending on your update scenario. If the update is to close the ticket, after the ticket is closed, it is no longer available. However, if the update is simply to add a work log entry, having unique entries to update might not be as important.

For change tickets, any user who belongs to the Infrastructure Change User group can modify any change tickets.

Determine users with proper permissions

Aside from updates, transactions must have appropriate permissions to run correctly. An AR System administrator user who is also an application administrator must query the People and People Permission Group forms.

The following table shows the minimum application permissions required for the most common actions.

For example, query the CTM:People Permission Groups by the Permission Group field for the application permission group values in the table.

Application permissions use a superseding model in the following order:

1. Incident Master
2. Incident User
3. Incident Submitter
4. Incident Viewer

To find users, start with users who have the most permissions. Because Incident Master has all incident permissions, it can perform Search, Create, and Update Incident transactions. Systematic querying from top to bottom yields the proper users that can be divided for each transaction. Some users can be members of more than one Incident Management permission group. Also, users can have not only Incident Management permissions but also other application permissions.

The following example shows a systematic query for a test that consists of only Incident Management and Infrastructure Change transactions. Excluding Infrastructure Change permission groups prevents double use of users in Incident and Change transactions. Excluding users that were found with assigned tickets also eliminates double use.

1. Query for Incident Master permissions, excluding the Infrastructure Change permission group and users with assigned tickets.
   1. Select users for the following transactions in this order: Update Incident, Create Incident, Search Incident.
   2. If the number of users is not enough for all Search, Create, and Update Incident transactions, go to step 2.
2. Query for Incident User permissions, excluding Incident Master and Infrastructure Change permission groups and users with assigned tickets.
   1. Select users for the following transactions in this order: Update Incident, Create Incident, Search Incident.
   2. If the number of users is not enough for all Search, Create, and Update Incident transactions, go to step 3.
3. Query for Incident Submitter permissions, excluding Incident Master, Incident User, and Infrastructure Change permission groups.
   1. Select users for the following transactions in this order: Update Incident, Create Incident, Search Incident.
   2. If the number of users is not enough for all Search, Create, and Update Incident transactions, go to step 4.
4. Query for Incident Viewer permissions, excluding Incident Master, Incident User, Incident Submitter, and Infrastructure Change permission groups.
   1. Select users for the following transactions in this order: Update Incident, Create Incident, Search Incident.
   2. If the number of users is not enough for your workload, select users for the combination of Search and Update Incident transactions, adjust the workload, or add permissions to existing users.

Modify user passwords

After users have been defined, modifying the user passwords to one common value facilitates executing load tests.

Modify passwords for a known set of users through a simple program or through SQL by modifying the Password field of the User form.

Determine search criteria

For searches involving incidents and changes, select a search criterion that returns enough results to be relevant (for example, 300 entries). Use a search criterion that returns a range of results instead of random results. Searches can return thousands of records if Max Entries Returned By GetList, an AR System configuration parameter, is unlimited.

Back up the test production database

Back up the test production database after you finish modifying data. A backup database enables you to restore your test database to its initial state and to use it on other systems.

Using synthetic data

Synthetic data is built from the ground up, so test designers are familiar with each detail. This familiarity enables them to better control test scenarios.

When generating synthetic data, use the following guidelines:

• Plan data creation to support test scenarios
• Determine optimal database size
• Determine optimal row entry size
• Vary data generation order
• Back up the test database

Plan data creation to support test scenarios

While planning test scenarios, determine the data required to perform the actions. For example, BMC Remedy IT Service Management applications have many types of users, including end users, support users, managers, and administrators. These users must belong to a company organization or department and have specific roles. In Incident Management, end users usually select options from a menu of incident categories for which they can submit tickets. The menu options must be created. Consider these data requirements during the design process.

Determine optimal database size

Total database size is important. Benchmarking goals define the maturity of the database. If you are benchmarking for future growth, ensure that the database has a healthy amount of data. Some database tables might have one million rows. A database with only 5,000 rows might yield different results. Match the database size to your goals.

Determine optimal row entry size

Row size affects test results. Row size can affect response times over the network — small entries require fewer bits to be transmitted than large entries. Consider the average row size that supports your goals. For example, a table that has only three of thirty columns filled is a small entry.

Vary data generation order

When generating data, ensure that you vary the generation order. A synthetic test database follows a pattern because its data is generated in bulk, as opposed to data in a production database. Creating entries nonsequentially for a particular user is more realistic.

Also, ensure that the data distribution is correct. For example, when incident tickets are generated, the status of most tickets should be Closed; only a few should be Open.

Back up the test database

Back up the test database after you finish generating data. A backup database enables you to restore your test database to its initial state and to use it on other systems.