Smart IT test methodology

This topic provides the following information:

• • Test case
  • Workload
  • Data volume
  • Smart IT Support User configuration - Restricted Supports Users Access

Test case

For Smart IT Key Actions, End User response time with 100ms latency was also captured during 2100 concurrent user test simulation.

The following table lists key actions for which end-user response time was captured:

Workload

For test cases that required a workload simulation, a mixed workload was added to the system to simulate a common workload scenario for Smart IT customers. The mix consisted of workloads from the following products and features:

• Helix Smart IT
• Helix BMC Digital Workplace
• Helix ITSM
• BMC Email Engine (from Smart IT, BMC Digital Workplace, and BMC Helix ITSM workloads)
• BMC Atrium CMDB batch jobs, Normalization Engine, and Reconciliation Engine

The nominal workload environment was defined by the distribution of concurrent users and transaction rates among the test scenarios. This workload was used as the baseline for consistent benchmarking of the performance and scalability of the BMC Helix solutions over time.

The workload from BMC Atrium CMDB batch jobs was executed during a 1-hour simulation. The BMC Atrium CMDB batch jobs created configuration items (CIs) that were normalized, reconciled, and merged into a BMC.Asset dataset.

Nominal BMC Helix Application workload distribution

The workload was split for Smart IT, BMC Digital Workplace, and BMC Helix ITSM applications. Roughly 98% of the user workload was for Smart IT and BMC Digital Workplace; the remaining 2% was for BMC Helix ITSM. Roughly 80% of the transaction workload was for Helix Smart IT and BMC Helix Digital Workplace, and the remaining 20% was for BMC Helix ITSM. 

The following table describes the general transaction and user percentage breakdown by application type.

The workload was further split to distinguish from a universal client (a computer browser) and a mobile client (for example, a tablet). The details of this split are in the tables that follow.

The following table describes the workload split for Smart IT universal client:

The following table describes the workload split for Smart IT mobile client:

The following table describes the workload split for BMC Helix Digital Workplace universal client:

The following table describes the workload split for BMC Helix ITSM:

The following tables list the projected number of executions after 1 hour for 2100 users using Smart IT, BMC DWP IT, and BMC Helix ITSM

The following table describes the projected executions for Smart IT universal client:

The following table describes the projected executions for Smart IT mobile client:

The following table describes the projected executions for BMC Digital Workplace universal client:

The following table describes the projected executions for BMC Helix ITSM:

BMC Service Level Management and email notification workload distribution

Email notifications were sent in the following instances:

• When an incident or a service request was created
• When an incident, a work order, a task, or a knowledge article was updated

BMC Service Level Management targets were also triggered under similar conditions. The following table lists the number of email notifications generated and BMC Service Level Management targets matched for each created incident and service request, and for each updated incident, work order, and task. This workload was executed automatically on the BMC Helix AR System server.

CI normalization and reconciliation workload distribution

During the 1-hour simulation, 7,500 CIs were generated, normalized, and reconciled every 10 minutes. 10% of the CIs were newly created, while the other 90% were updated.

Simulating the workload of 2100 concurrent users

In this test, the following workloads were used:

• Smart IT, BMC Digital Workplace, and BMC Helix ITSM
• 7,500 CIs for the Normalization Engine and Reconciliation Engine

The results are displayed in the following table. Actual entries were created during a 2100 concurrent-user load.

Data volume

The following table summarizes the foundation data and application data inserted into the BMC Remedy AR System database prior to starting the tests:

Smart IT Support User configuration - Restricted Supports Users Access

For these Restricted user Subquery algorithm for Row-Level security field was selected.

The Subquery algorithm for Row-Level security (RLS) field improves performance (response time) by filtering database content based on user-specific roles. 

Subquery algorithm can be used on a form where performance issue is observed while fetching data from RLS-enabled fields. For example, querying the database might take more time for users other than administrators. In this case, using the Subquery algorithm fetches the search results faster.

Steps to identify forms and change algorithm

1. Capture API-SQL Logs for the use case with the high response time (Steps to Enable Logs)
2. Get the list of longest running API Forms.
3. For the Longest running form/s from Row Level Security Fields panel on the Definitions tab in the Remedy Developer studio select Algorithm as “Subquery” (RLS Subquery Enhancement)

Note:

• Changing the RLS algorithm to 'Subquery' for all the forms is not recommended as it might have negative performance impact.
• For View and Union forms, change algorithm for all the regular forms participating in View/Union.
• For UNION forms after changing algorithm, go-to “SHR:Union_ConfigurationConsole” form, select Union form name from “Implementation Area” drop down and Click on “Build/Rebuild Database Union Structure”, to complete migration of Union Form
• For Join Forms, change algorithm for Join Forms, changing algorithm for participating form is not required.
• For BMC.Core* forms make above mentioned changes only in “Base Development Mode”

The following table has the list of form for which Row Level Algorithm was changed from Default to SubQuery in Test environment and S-Table was enabled: