Planning the stress test (Scripting)

Because of the extensive resources required to perform a stress test, comprehensive planning is crucial. Do not proceed with testing until you have a test plan in place that has been approved by everyone, and you have acquired all of the necessary resources.

Create a test plan

Solid planning up front helps you make the best use of your testing product. Be sure your test plan meets the following objectives:

  • States your testing objectives.
  • States the expected results.
  • Specifies who is responsible for each phase of the test.
  • Specifies what monitoring tools will be used.
  • Specifies who will set up the testing environment.
  • Specifies who will restore the testing environment after the test.
  • Specifies the applications that will be used to perform the test.
  • Specifies when the test is complete.
  • Specifies who will analyze the test results.

Take into account that all components of the operating system, hardware, and application will be used in the test. To perform a Db2 stress test, for example, the application must be executed, the CICS or IMS regions must dispatch the work, and MVS must dispatch the CICS or IMS regions. Because any or all of these components could fail as a result of the test, include in your plan the appropriate technical specialists for these areas to help identify failures, any adverse effects on performance, and to resolve problems.

Stress tests can affect other applications executing in the environment. To ensure that results of the stress test can be measured while preserving the integrity of other applications, execute stress tests on evenings and weekends, or at other times when production activity is low.

Sample stress test plan

State your testing objectives

A customer has decided to simulate 800 users to a single CICS region.

Test description

The accounting database will be exercised to determine how much data can be processed before response time is affected. Account add programs ACCT1234 and ACCT1235 will be exercised. The database can handle the current production load of 500 adds per minute, so the stress test begins at this point. This baseline test establishes baseline measurements from which future test results can be tracked. Each subsequent test increases the number of adds per minute by 50 until database response time diminishes to below the acceptable level of 1.0 seconds per transaction.

Number of Performance Test Batch Jobs involved

Prior tests indicate that each execution of the Performance Test unattended batch (UAPB) job can drive 500 transactions per minute, which results in 300 accounting adds per minute. The test begins with two UAPB jobs until a rate of 600 accounting adds per minute has been achieved. At this point, a third UAPB job is added. These UAPB jobs execute on MVSPROD, which removes Performance Test overhead from stress test statistics.

Number of tests

The test begins with two UAPB jobs until a rate of 600 accounting adds per minute is achieved. At this point, a third UAPB job is added. These jobs execute on MVSPROD, which again removes Performance Test overhead.

State the expected test results

Test results are measured during each test execution. Measurements include:

  • MVS paging rate
  • CPU use
  • CCS paging rate
  • OSCORE use
  • number of times a MAX and AMAX are reached
  • transaction response time.

Database measurements include:

  • response time
  • number of deadlocks
  • number of deadly embraces.

Channel busy and channel contention can also be measured.

Specify the applications that will be used to perform the test

The tests will execute in MVSTEST, CICS test region 1 using the ACCTADD database. CICS test region 1 should be configured to handle 1000 active transactions with a maximum of 1020. Storage size and OSCORE should reflect current production definition. Virtual terminal definitions for Performance Test for VTAM should be added to VTAM and CICS test region 1. The ACCTADD database should be defined with enough threads to handle 600 concurrent users.

Specify who is responsible for each phase of the test

Each test begins at 8:00 p.m. and ends no later than 6:00 a.m. on Saturday and Sunday. One hour before and after the test is dedicated to setting up and restoring the testing environment.

Specify who will set up the testing environment and restore it after the test

The following staff is required to attend each test execution:

  • MVS Specialist
  • CICS Specialist
  • VTAM Specialist
  • Accounting Representative
  • DBA Specialist
  • Quality Assurance
  • Test Manager.

Specify when the test is complete

Testing ends when response time drops below 1.0 seconds per transaction, or when 900 accounting adds per minute is achieved.

Set up the test

Establish the testing environment to alleviate bottlenecks and skewing of test results. Take CICS tasks and storage, database threads, and VTAM storage into consideration. Also, because Performance Test for VTAM uses virtual terminals to execute multiple terminal activity, the definitions for these terminals are required in VTAM and in the CICS region. These definitions should reflect the terminal type normally used in production. A database or file restore from a baseline is also required.

To set up a WebSphere MQ test, clear the queues accessed by the CICS region, and ensure that they are not available to applications other than Performance Test for WebSphere MQ.

Overhead

The main objective of stress testing is to load the system with enough data to simulate application performance requirements. Turn off Performance Test parameters, such as comparing results and dubbing, that are not required during stress testing. This reduces overhead and frees Performance Test to adequately stress system components.

Performance Test for VTAM and Performance Test for Mainframe Servers’ APPC functions use the unattended playback batch job EHSBATCH to perform stress testing. Whenever possible, separate EHSBATCH from the environment being tested. This might involve executing EHSBATCH on a different CPU or LPAR. Removing the job from the environment removes any overhead that EHSBATCH incurs in terms of storage and CPU usage. It also allows a separate MVS to dispatch EHSBATCH tasks and manage the presentation of VTAM messages. The resulting analysis provides a true picture of environment performance.

If you are performing a WebSphere MQ test with Performance Test for WebSphere MQ or TCP/IP tests with Performance Test for Mainframe Servers, execute program TCPPBMN. This program normally runs on the same system as the CICS region or Web application being tested. You can run TCPPBMN on a different LPAR to reduce contention for resources.

Priority

MVS dispatches work processed by CPUs based on priority. In most data processing environments, batch jobs have the lowest priority of all work on the machine because they are long-running tasks and use large amounts of resources. Because Performance Test performs stress testing using EHSBATCH, the systems programmer must ensure its dispatching priority is immediately below VTAM.

TCPPBMN needs to be included with EHSBATCH as a named program. Region size should be 0M.

Region Size

Set the region size for EHSBATCH to 0M. This allows MVS to give EHSBATCH as much storage as needed to perform the test.

Multiple EHSBATCH Jobs

If one EHSBATCH job does not simulate enough virtual terminals to stress the system enough, start more EHSBATCH jobs until the number of virtual terminal simulations is adequate.

Multiple TCPPBMN Jobs

Often an MQGROUP statement can be used to simulate a single WebSphere MQ user. If COUNT is used on the MQGROUP statement, then the value associated with COUNT indicates the number of users that TCPPBMN will simulate. For example, COUNT(10) simulates ten users. The maximum number of MQGROUP statements that TCPPBMN can play is approximately 400.

Often a SOCKETS statement can be used to simulate a single TCP/IP user. If the COUNT statement appears on the SOCKETS statement, then the value associated with COUNT indicates the number of users that TCPPBMN will simulate. For example, COUNT(10) simulates ten users. The maximum number of SOCKETS that TCPPBMN can play is approximately 400.

If modifications to the test scripts are required, make the changes before you use the script in a stress test.

Execute the test

When you run one huge stress test that includes all terminals, applications, queues, and databases, the large number of components makes it difficult to determine causes if the test fails. Not knowing the exact cause of the problem could lead to improper decisions based on the stress test results.

Start your stress test with a small number of virtual terminals within one EHSBATCH job. This determines if the test plan executes well and the test environment is set up properly.

Start your WebSphere MQ or TCP/IP stress test with a small number of messages within the TCPPBMN job. Confirm that the test environment is set up properly, and that a single script can be played multiple times. Often, in WebSphere MQ messages, one or more fields make that message unique (for example, the MsgID field in the message descriptor).

Work from Small to Large

As each test completes successfully, incrementally increase the number of virtual terminals/messages or TCP connections and retest. Change the number of simulated users by changing the TERM parameter (for 3270 playbacks) or the COUNT parameter (for WebSphere MQ and TCP/IP playbacks). You may need to start an additional job to obtain increased volume.

Journals

Do not invoke journals. When performing stress tests, the main objective is to lead the system according to requirements. Writing the results of the playback to external files could prevent Performance Test from sending data quickly enough to stress a system component.

Caching

Turn caching on. Caching reduces storage and overhead used by Performance Test and makes storage and CPU cycles available for stress testing.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*