Test environment setup
To conduct BMC Helix solution benchmark, the BMC benchmarking team deployed the following applications to a customer like environment in the BMC performance benchmarking lab.
- BMC Helix IT Service Management (BMC Helix ITSM)
- BMC Helix Helix Dashboard
- BMC Helix Service Request Management
- BMC Helix Knowledge Management
- BMC Helix Single-Sign-On
- BMC Helix Atrium Configuration Management Database (BMC Helix CMDB)
- BMC Helix Smart IT
- BMC Helix Progressive web application (PWA)
- BMC Helix LiveChat
- BMC Helix Digital Workplace
- BMC Helix Business Workflow
- BMC Helix ChatOps
- BMC Helix ChatBot
The test environment setup in BMC performance benchmarking lab comprised the following components:
- An F5 load balancer
- Two pods each of RSSO, Platform FTS, User and SR, DWP tomcat, Catalog ITSM plugin, MidTier, Smart IT , Virtual Chat Plugin and Server, Openfire
- One MS Exchange server
BMC benchmarking team collected end-user response times, server response times, and server resource utilization metrics in the following test scenarios:
| | | Running BMC CMDB processes in continuous mode |
|---|
| | | |
The BMC benchmarking team collected end user response times for the following key actions:
Use Cases
| |
|---|
BMC Helix IT Service Management | - Log on to the home page
- open incident in New mode
- Modify incident to resolve
- Create incident (redisplay after submitting)
- Create Incident via REST API
- Update Incident Via Email
- Open change in New mode
- Create change
- Open Request Entry console
- Create a service request with six questions with a mapping
- Search Knowledge for from articles
- View Overview console for assigned to All My Groups
|
BMC Helix Service Request Management | - View Service Category
- Create New with 6-Question Mapping
- View SR and Add Activity Note
- Search BY Keyword and Open KB Article
|
| - Normalization Continuous Job
- Reconciliation Continuous Job
|
| - Incident Dashboard
- Change Dashboard
- Channel Dashboard
- BWF Dashboard
- DWP Request Overview Dashboard
|
BMC Helix Smart IT - iPad Use Cases | - Incident - Update Status and Priority
- WorkOrder - Update Status and Priority
- Task - Update Status
- Global Search - Asset And KA
- Incident - Create New
- Change - Approve
- Change - Update Status
- Asset - Update
|
BMC Helix Smart IT - Universal Use Cases | - Incident - Update Status and Priority
- WorkOrder - Update Status and Priority
- Service Request - Create New
- Incident And Task- Update Status and Priority
- Incident - Create New
- Global Search - Asset And KA
- Broadcast - Create New
- Change Request - Update And Create Task
- Change Request - Create New
- Knowledge Article - Create New
- Knowledge Article - Update content and status
- Global Search - Change Request
- Change Request - Update Status
- Asset - Update
- Asset - Bulk Update
- Global Search - Release
- Release - Create New
- Dashboard - Apply All filter except 1
- Live Chat - Accept Chat, Create Incident and close the Chat
- Change Request - Collision Detection
- Change Request - Impact Analysis
- WorkOrder - Create New
- Search request catalog items from Smart Recorder
- Create Service Request on behalf of user using DWP Catalog
- View DWPA request in Smart Recorder
- Chat with an Support user Using OpenFire
|
| - Get incident details
- Search similar tickets
- Search for Knowledge
- Update record (cognitive)
- Swarm command (cognitive)
|
BMC Helix Digital Workplace Catalog | - Browse Catalog
- Create SRM Request (Imported SRD)
- Create SB Request
- Create SB Bundle Request
- My Activity - Approve Request
- Search (Service Search, SRM, etc.)
- Search Catalog
- Open Knowledge Article
- Request BWF Request
- Check status of Request
- Admin Service Creation
|
BMC Helix Business Workplace | - BMC Helix Business Workflow test case steps
- Create Case with template
- Create Case via Quick Case
- Search Case
- Update Case Status and Add Task
- View Task and Add Activity Notes
- Create Case via Email
- Search Knowledge Workspace
- Update Case via Email
- Approve Case
|
| - Submit request via ChatBot
- Search Knowledge Article
- Display My Request Status
|
Please refer to the section Measurement of response times for timing measurement details.
Workload
For the solution benchmark, a mixed workload was added to the system to simulate various workload scenarios. The mixed workload comprised the following:
- The workload from BMC Helix ITSM and Integration Users
- The workload from BMC Helix Dashboard users
- The workload from BMC Helix CMDB batch jobs, Normalization Engine, and Reconciliation Engine
- The workload from BMC Helix Smart IT iPad and Universal Console users
- The workload from BMC Helix Digital Workplace Advance Users
- The workload from BMC Helix Business workflow Users
- The workload from BMC Helix ChatBot users
- The workload from BMC Helix ChatOps Users
The nominal workload environment was defined by the distribution of concurrent users and transaction rates among the user scenarios. This nominal workload was used as the baseline for benchmarking the performance and scalability of the Remedy solutions consistently over time.
The workload from BMC Service Level Management and BMC Email Engine was based on the Remedy ITSM and BMC Service Request Management workloads. BMC Service Level Management targets and milestones were triggered by specific Remedy ITSM terms and conditions. Email messages were created based on Remedy ITSM, BMC Service Request Management, and BMC Service Level Management workloads.
The workload from BMC CMDB batch jobs was executed during a 1-hour simulation. The BMC CMDB batch jobs created configuration items (CIs) that were normalized, reconciled, and merged into a BMC.Asset data set.
Nominal Remedy online user workload distribution
The following tables describe the workload distribution for BMC Helix Innovation Suite Applications
The following table describes the workload split for BMC Helix ITSM:
Table
Workload split for Remedy ITSM and Integration:
| Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2130 total concurrent users in one hour |
|---|
Search Incident by ID and Modify Incident to Resolve | | | |
| | | |
Search Records for a User via Global Search | | | |
Search Request Record via ID in Global Search | | | |
Open/Refresh Overview Console | | | |
Create Incident with CI (Modify Request after Submit) | | | |
Create Change with Service CI | | | |
| | | |
| | | |
Knowledge Search and View Articles | | | |
| | | |
Smart Reporting - Access Incident Management report and saves it | | | |
Smart Reporting - Access Incident Management Dashboards | | | |
Smart Reporting - Create report on OOTB Incident Management view | | | |
|
Create Incident using Webservice | | | |
Update Incident Via Email | | | |
Create Incident Via REST API | | | |
The following table describes the workload split for BMC Service Request Management:
Table
Workload split for BMC Service Request Management
BMC Service Request Management scenario | Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2100 total concurrent users in one hour |
|---|
View Services in Category and Subcategory | | | |
| | | |
View Service Request Details and Add Activity Log | | | |
| | | |
The following table describes the workload split for BMC Helix Dashboard:
Table
Workload split for BMC Helix Dashboard
BMC Smart Reporting scenario | Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2100 total concurrent users in one hour |
|---|
| | | |
| | | |
| | | |
Business Workflow Dashboard | | | |
DWP Request Overview Dashboard | | | |
The following table describes the workload split for BMC Helix Smart IT iPad:
Table
Workload split for BMC Helix Smart IT iPad
BMC Helix Smart IT iPad scenario | Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2100 total concurrent users in one hour |
|---|
Incident - Update Status and Priority | | | |
WorkOrder - Update Status and Priority | | | |
| | | |
Global Search - Asset And KA | | | |
| | | |
| | | |
| | | |
| | | |
The following table describes the workload split for BMC Helix Smart IT Universal Console:
Table
Workload split for BMC Helix Smart IT Universal Console
BMC Helix Smart IT Universal Console scenario | Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2100 total concurrent users in one hour |
|---|
Incident - Update Status and Priority | | | |
WorkOrder - Update Status and Priority | | | |
Service Request - Create New | | | |
Incident And Task- Update Status and Priority | | | |
| | | |
Global Search - Asset And KA | | | |
| | | |
Change Request - Update And Create Task | | | |
Change Request - Create New | | | |
Knowledge Article - Create New | | | |
Knowledge Article - Update content and status | | | |
Global Search - Change Request | | | |
Change Request - Update Status | | | |
| | | |
| | | |
| | | |
| | | |
Dashboard - Apply All filter except 1 | | | |
Live Chat - Accept Chat, Create Incident and close the Chat | | | |
Change Request - Collision Detection | | | |
Change Request - Impact Analysis | | | |
| | | |
Search request catalog items from Smart Recorder | | | |
Create Service Request on behalf of user using DWP Catalog | | | |
View DWPA request in Smart Recorder | | | |
Chat with an Support user Using OpenFire | | | |
The following table describes the workload split for BMC Helix Digital Workplace Catalog:
Table
Workload split for BMC Helix Digital Workplace Catalog
BMC Helix Digital Workplace Catalog scenario | Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2100 total concurrent users in one hour |
|---|
| | | |
Create SRM Request (Imported SRD) | | | |
| | | |
| | | |
My Activity - Approve Request | | | |
Search (Service Search, SRM, etc.) | | | |
| | | |
The following table describes the workload split for BMC Helix Business Workflow:
Table
Workload split for BMC Helix Business Workflow
BMC Helix Business Workflow scenario | Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2100 total concurrent users in one hour |
|---|
Create Case with template | | | |
Create Case via Quick Case | | | |
| | | |
Update Case Status and Add Task | | | |
View Task and Add Activity Notes | | | |
| | | |
Search Knowledge Workspace | | | |
| | | |
| | | |
The following table describes the workload split for BMC Helix Digital Workplace for BWF Catalogs:
Table
Workload split for BMC Helix Digital Workplace for BWF Catalogs
BMC Helix Digital Workplace for BWF Catalogs scenario | Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2100 total concurrent users in one hour |
|---|
| | | |
| | | |
| | | |
| | | |
| | | |
The following table describes the workload split for BMC Helix ChatBot:
Table
Workload split for BMC Helix ChatBot
BMC Helix ChatBot scenario | Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2100 total concurrent users in one hour |
|---|
Submit request via ChatBot | | | |
| | | |
Display My Request Status | | | |
The following table describes the workload split for BMC Helix ChatOps:
Table
Workload split for BMC Helix ChatOps
BMC Helix ChatOps scenario | Percentage of
total concurrent users | Transaction rate
(per user per hour) | Projected executions
for 2100 total concurrent users in one hour |
|---|
| | | |
| | | |
| | | |
Update record (cognitive) | | | |
Swarm command (cognitive) | | | |
BMC CMDB workload
BMC CMDB processes were schedule to run in continuous mode to normalize and reconcile CIs.
During the 1-hour simulation, for each tenant, a total of 7500 CIs were normalized, and reconciled. The following are the schedule of CI data changes:
- 10 minutes after test kickoff, starting to create 750 new CIs (10% of total)
- 20 minutes after test kickoff, starting to update 2240 CIs (about 30% of total)
- 30 minutes after test kickoff, starting to update another 4510 CIs (about 60% of total)
Online user workload Simulation
User session control
The BMC Helix Applications are user session based applications. A user session begins when a user logs on to BMC Helix Application's home page and ends when a user logs off. Different type of users has different user log on behavior. The following is how user sessions are simulated in solution benchmark testing:
- Respective BMC Helix Innovation Suite Application's users login once and perform a specified number of transactions for a business case before logging off. Their user sessions lasted for entire duration of the simulation.
The load driver and its deployment
To automate workload generation, Silk Performer, a load automation tool, was used to simulate multiple virtual users who concurrently interact with the system under test. Silk Performer simulates user activity by generating HTTP requests and processing HTTP responses. In addition, Silk Performer timer was used to measure server response times for user actions executed in those user scenarios.
In BMC performance benchmarking lab, Silk Performer was deployed in virtual physical machines which had LAN connection with the F5 load balancer in the test environment setup.
