Testing with Code Coverage

This section introduces basic Code Coverage information, including a scenario of how the product might be used at a fictitious company. This section also includes an introduction to the Primary Menu of Code Coverage.

This section provides information about the following topics:

About Code Coverage

Code Coverage is an advanced analysis tool that clearly and easily identifies how much of an application has or has not been executed. Code Coverage helps identify the percent of code executed for single or multiple programs, for one test or for multiple tests run over time, and from a high-level management view down to a single line of code.

Code Coverage works in conjunction with at least one other mainframe Code Debug test and debugging tool. As you perform your testing, Code Coverage keeps track of what parts of your code have been exercised and what parts have not. The results generated by Code Coverage can be displayed, analyzed, and output using Workbench for Eclipse. You can also output detailed reports directly from Code Coverage on the mainframe.

Typical Code Coverage scenario

To better understand how Code Coverage would typically be used, consider the following scenario: the fictitious ACME corporation uses both Code Debug TSO and Code Debug CICS to test and debug its mainframe applications. After making extensive changes to their payroll, inventory, and order entry programs, ACME’s Internet Technology group wants to run a suite of tests to ensure those programs are ready for production.

In the past, there was no way to ensure whether the tests chosen and data used would thoroughly exercise every part of the modified programs. Now that ACME uses Code Coverage, however, it is possible to know precisely how thorough their testing efforts have been and whether more testing is required.

To prepare for Code Coverage testing, the Code Coverage administrator first creates one or more Results Repository data sets (For more information, see to the Repository Utility Screen). These data sets hold Code Coverage data collected during testing. After a repository data set has been allocated, the administrator uses the Setup screen in Code Debug TSO to define two Code Coverage systems, PAYROLL and INVENTORY. These Code Coverage systems make it easier to organize test results. The ACME order entry application runs under CICS, so the Code Coverage administrator uses the CICS Populater and Program Inventory Utility to ensure the modified programs will have their test data collected by Code Coverage. The XVCC transaction is used to create the CICS system name ORDER ENTRY. (See Part 3, Using Code Coverage with Code Debug CICS for more information.)

The ACME staff then performs extensive testing of their modified programs using Code Debug TSO and Code Debug CICS. When the testing completes, the Code Coverage administrator generates comprehensive reports showing counts and percentages of execution, varying the degrees of detail for all the modified programs. The administrator can use Workbench for Eclipse to view graphical representations of the Code Coverage results.

Knowledge gained from these Code Coverage reports can be immediately put to use in refining the ACME test methodology to ensure complete testing and prevent surprises when the modified programs are put into production.

About system flow

Code Coverage provides a feature to identify what elements such as transaction IDs, load modules, and programs have been executed during the course of executing an application. This feature is called System Flow.

System Flow works in conjunction with a mainframe Code Debug test and debugging tool. As you perform your testing, System Flow keeps track of what elements have executed. The results, generated by System Flow reporting, can be displayed on the mainframe directly to your ISPF terminal or sent to a printer.

System Flow collection may be started independently of Code Coverage collection or concurrently with Code Coverage collection. If System Flow collection is started concurrently with Code Coverage collection, both features will record data to the same system name and test ID, and the data will be written under the same Results Repository. Code Coverage and System Flow reporting can then be used independently to produce the desired reports.

Typical system flow scenario

To better understand how to use the System Flow feature of Code Coverage, consider the following scenario: the ACME corporation has just acquired a CICS application as the result of a recent merger with another company. ACME, as we know from the Code Coverage scenario above, uses Code Debug CICS to test and debug its mainframe CICS applications.

ACME would like to understand the flow of the new system it has acquired. Now that ACME uses Code Coverage with the System Flow feature, it is possible to record what transactions, load modules, and programs are called during execution of the newly acquired system without making any changes to the applications.

To prepare for System Flow testing, the administrator can either use existing Code Coverage repositories for System Flow data (Code Coverage and System Flow data can co-exist on the same repository), or can optionally create one or more Code Coverage repositories exclusively for System Flow data. These data sets hold the System Flow data collected during the testing. After a repository data set has been allocated, the administrator uses the setup screen in Code Debug CICS to define System Flow systems and identify which applications to monitor.

The ACME staff then performs extensive testing of their newly acquired application. When testing is complete, the ACME administrator halts the System Flow collection and prints reports showing the transaction IDs, load modules, and programs that have executed as a result of the testing.

Knowledge gained from these System Flow reports can be immediately put to use in understanding the architecture of this new system.

Primary menu

The Code Coverage Primary Menu is shown in the following figure. On the right side of the screen, the ID of the logged-on user, their default data set prefix, and the current system time are displayed.

Code Coverage Primary Menu

Type 0 in the OPTION field and press Enter to specify the desired defaults for job and processing parameters, Code Coverage repository data sets, source listing override data sets, report filters, and optionally the CICS program inventory settings, CICS application load libraries, and CICS source listing data sets. For details, see to Specifying Defaults.

Type 1 in the OPTION field and press Enter to specify the level of Code Coverage reporting to use and various other parameters. For details, see to Generating Code Coverage Reports.

Type 2 in the OPTION field and press Enter to generate System Flow reports. For details, see to Generating System Flow Reports.

Type U in the OPTION field and press Enter to display the Code Coverage UTILITIES menu. From the UTILITIES menu, you can access Code Coverage utilities to create and delete repository data sets, list and delete repository members, export and import those members, and merge the records of two repository data sets. You can also access the CICS Program Inventory utility and print a report detailing Code Coverage release and applied maintenance information. The UTILITIES menu is described in detail in Using Results Repository Utilities.

Type X in the OPTION field and press Enter to exit the Primary Menu.

Using wildcard characters

On any character selection field within the Code Coverage product, the wildcard character asterisk (*) can be used to specify a whole field or part of a field that users wish to exclude when selecting records for comparison. If the selection field is left completely blank, it has the same effect as if a single asterisk were entered. In this case, any data encountered on the file is accepted.

Any data entered followed by an asterisk and trailing blanks selects only those records that begin with that character string. For example ABC* will match any field beginning with ABC including the strings 'ABCDEF', 'ABC12 ' and 'ABC '.

If an asterisk is imbedded within a character string, such as A*B, that specific character would not be compared. This string would match A1B, AxB, and other similarly.
Multiple asterisks can be entered in a field. A**B or A*B* are acceptable entries. The string 'A**B' would match any string with A in the first character and B in the fourth character and blanks in the fifth character onward.

Wildcard characters cannot be used within date or time selection fields.