Gathering metrics
This topic identifies the information to gather during benchmark performance tests:
Roundtrip response time
Most scenarios contain many steps, but usually just a few steps are worth measuring and reporting. Roundtrip response time is one metric that affects most users. For example, roundtrip response time can be the time for a user to respond after clicking a button.
When writing your test scenarios, determine the actions that the test measures. The following table shows an example of a set of steps, or actions, in a Create Incident transaction. It also shows whether the actions are repeated and timed.
Actions in an example Search Change by ID transaction
Transaction | Phase | Transaction steps | Repeat? | Timed? |
|---|---|---|---|---|
Search Change By ID | Initialization | Configure load generator settings | No | No |
Not applicable | Transaction | As a support change user, log on to the homepage | No | No |
Not applicable | Not applicable | Click Change Management Console link | No, refresh sometimes | Yes |
Not applicable | Not applicable | Open the Change form in Search mode | No, refresh sometimes | Yes |
Not applicable | Not applicable | Randomly enter a Change entry ID and click Search | Yes | Yes |
Not applicable | Not applicable | Log out | No | No |
Not applicable | End | Close any open files | No | No |
Transaction throughput
Transaction throughput is as important as the number of users. For example, compare the following tests:
- 1,000 users perform 10 transactions per hour, producing 10,000 transactions per hour
- 500 users perform 30 transactions per hour, producing 15,000 transactions per hour
The 1,000-user test has more users, but they are not as active as users in the 500-user test. A system can handle many users concurrently if transaction throughput is minimal. Tracking user count without transaction throughput is meaningless. To determine the total system throughput, track both user count and transaction throughput.
System, network, and database statistics
Use system resource metrics and database statistics to ensure that your test runs at the established throughput.
Metrics for the CPU, memory, network, and disk input/output (I/O) might identify bottlenecks that impact scalability. To observe system behavior in relation to the workload, take system metrics during performance tests. Total system resources, per-process resources, and database statistics gathered during the steady state help identify performance issues. Ensure that gathering this data does not bottleneck the test.
Comments