Benchmarking for a single user

A single-user benchmark is the first step in revealing system performance. From a single-user standpoint, response time is the main factor for determining performance and is the easiest method of benchmarking. However, use a systematic approach to perform repeatable tests.

This topic provides the following information:

Single-user benchmark factors

Consider the factors in this section at the start of every single-user benchmark.

Environment stability

To ensure that your tests are repeatable, use a test environment that has no other activity running while you take response times. Perform your tests when the mid tier usage and AR System server CPU usage are below 10 percent. If the mid tier has just been restarted, ensure that preload finishes running before you start testing. Verify that no antivirus software is running on the systems.

Time of day

When testing in a production environment, ensuring minimal system activity is difficult. In a production environment, the time of day is significant, because the server load varies throughout the workday. Depending on your testing goals, determine the best time of day to conduct your single-user benchmark test. To ensure repeatability, conduct subsequent tests at the same time of day.

Network latency

Network latency describes how long a packet of data takes to go from a client to a server. The farther away a client is from the server, the longer the latency. A common type of network latency is TCP latency, which is measured by the ping tool. Another type of latency is HTTP latency, which is measured by the http-ping tool. Use these tools to measure the latency from the test client to the mid tier. Repeatable tests should have the same latency.

Client environment

Select a test client system that is typical of your company's end-user hardware. If single-user benchmarks are conducted from different locations, make sure each location's hardware is the same.

Different browsers, and different browser versions, can give different performance results. Use a browser and browser version that your company uses and that is compatible with BMC products. For consistent results, test with the same browser and version.

Ensure the test client system has little to no activity during testing.

Browser caching

If your browser has cached mid tier resources, performance results can vary. An uncached browser is the first access to the mid tier. It does not have any images, javascripts, or stylesheets in the browser's cache. These resources have to be downloaded, which lengthens the response time.

A cached browser already has the mid tier resources in its cache. These resources have an expiration date. After the expiration date, a request is made to the mid tier for any updated resources. If there are no updates, the mid tier responds with a 304 Not Modified{ message, which indicates to the browser to read from the cache. If there are updates, the new resource is downloaded.

Best practice

We recommend an expiration date of one day.

Use both cached and uncached methods. To use the uncached method, clean the browser's cache of all cookies and resources. The uncached method is referred to as a First Session type, and the cached method can be either a New Session or In Session type.

Session types

In addition to browser caching, there is the concept of a First Session, a New Session, and an In Session with respect to the mid tier logon session. The following table describes session types for mid tier logon sessions:

Mid tier logon session types

Mid Tier logon session	Tasks
First session	Mid tier is cached Browser is not cached User logs on and opens console x or form Y. The browser is cached for the first time. Record this time. Log off.
New session	Mid tier is cached Browser is cached from the first session access User logs on and opens console x or form Y for the first time within this session. Record this time. For example, a support user logs on in the morning.
In session	Mid tier is cached Browser is cached from the First Session access User has already logged on and opened console x and form Y User reopens console x and form Y from within this session. Record this time. For example, the remainder of the day after a support user logs on in the morning

Response times vary, depending on which session type is measured. In Session produces the fastest response time of the session types, while First Session produces the slowest response times. Record which type of measurements are captured. In Session is the most commonly measured mid tier logon session type because it describes the typical work environment.

Measuring response time

Measure client response time with a stopwatch or HTTP monitoring tool. A stopwatch includes all browser rendering processing time, but it can be inaccurate with different users. HTTP monitoring tools are more accurate, but they do not account for full resource rendering. The accuracy of HTTP monitoring tools outweigh the full rendering. In any case, use the same tool for all tests.

Reporting results

Record your results for the tests in a report, such as the following sample spreadsheet:

Sample spreadsheet for capturing response times