Gathering metrics
This topic identifies the information to gather during benchmark performance tests:
Roundtrip response time
Most scenarios contain many steps, but usually just a few steps are worth measuring and reporting. Roundtrip response time is one metric that affects most users. For example, roundtrip response time can be the time for a user to respond after clicking a button.
When writing your test scenarios, determine the actions that the test measures. The following table shows an example of a set of steps, or actions, in a Create Incident transaction. It also shows whether the actions are repeated and timed.
Actions in an example Search Change by ID transaction
Transaction throughput
Transaction throughput is as important as the number of users. For example, compare the following tests:
- 1,000 users perform 10 transactions per hour, producing 10,000 transactions per hour
- 500 users perform 30 transactions per hour, producing 15,000 transactions per hour
The 1,000-user test has more users, but they are not as active as users in the 500-user test. A system can handle many users concurrently if transaction throughput is minimal. Tracking user count without transaction throughput is meaningless. To determine the total system throughput, track both user count and transaction throughput.
System, network, and database statistics
Use system resource metrics and database statistics to ensure that your test runs at the established throughput.
Metrics for the CPU, memory, network, and disk input/output (I/O) might identify bottlenecks that impact scalability. To observe system behavior in relation to the workload, take system metrics during performance tests. Total system resources, per-process resources, and database statistics gathered during the steady state help identify performance issues. Ensure that gathering this data does not bottleneck the test.