IAMSRORG — REORG Command

REORG Command Processing

The REORG command is used to create a reorganized copy of an input IAM data set directly into a new IAM data set. This process eliminates the necessity for an intermediary sequential data set, thus resulting in a faster reorganization process with using the original data set as a backup copy. Based on the actual input data set and the various keyword parameters, IAMSRORG uses various optimization techniques to facilitate a fast reorganization process. The optimization techniques available for use during execution of the reorganization process include:

• Bypassing the Index. Through special IAM open processing for the IAMSRORG reorganization process, the index to the file will not be read into storage which will reduce open processing time and minimize storage requirements.
• Block level input I/O. The special block level I/O requests used reduces the interaction when reading the input IAM data set to once per block, as opposed to once per record. This process also reduces data movement as the data is processed by IAMSRORG directly out of the IAM data buffer.
• Avoid data compression overhead by retaining the data compressed state.
• Through special positioning requests. IAMSRORG positions the I/O requests at the physical beginning of the extended area blocks, and reads through those blocks in physical sequence, passing the overflow records to the system SORT utility using the sort input exit.
• In the sort output exit, IAMSRORG merges the sorted overflow records with the data records from the prime area, and writes them out in ascending key sequence to the output file.

Input Data Set

The input data set must be a loaded IAM Enhanced Format data set with either the standard record based overflow or the Prime Related Overflow (PRO) structure. The input IAM data set can be specified with one of the following methods:

• JCL DD card named IAMFILE or
• JCL DD card with another name that is specified by the INFILE keyword or
• By data set name using the INDSNAME keyword parameter.

When specified by INDSNAME, IAMSRORG will dynamically allocate the specified IAM data set. When using dynamic allocation, IAMSRORG enables any warning or error messages from dynamic allocation to be written out to the joblog for the job.

Data Integrity

IAMSRORG endeavors to ensure that the output data set of the reorganization process has preserved the complete data contents of the input file. IAMSRORG will compare the number of records actually read with the current internal statistic for record count, and produce a warning message and a condition code of 4 if they are not equal. The internal statistic can be incorrect if a system failure has occurred, previous close processing failed for some reason, or if the data set is opened for update by another address space. For that reason it is only flagged as a warning unless the REORG process is going to be overwriting the original data set, in which case the process terminates with a return code of 8.

For Share Option 1 or 2 data sets, IAMSRORG will request exclusive use of the internal update ENQ to prevent updates during the reorganization process to preserve the integrity of the data that will be written to the output IAM file. Proper use of the IAMSRORG utility is that the input data set be allocated with DISP=OLD. DISP=OLD is required to ensure proper function of any subsequent renames that may be performed. It is also needed for data sets defined with a share option 3 or 4. IAMSRORG will default to allocating the INDSNAME data set with a default DISP=OLD. Users can specify the optional keyword SHRINPUT (SHR) to allocate it with a DISP=SHR, if it is in use for read only by other jobs and is defined with share option of 1 or 2.

Primary Output Data Set

The primary output data set of the REORG process is an IAM Enhanced Format data set. It is specified by:

• JCL DD card named OUTFILE or
• JCL DD card with another name that is specified by the OUTFILE keyword or
• By the data set name using the OUTDSNAME keyword parameter.

It is a best practice and highly recommended that the output data set be a newly defined and previously unused IAM data set, particularly if the data set is multi-volume. While reuse of an existing data set is usually fine, they are subject to unusual errors particularly if they reside on multiple volumes. Suggestions for best practice use are to define the output data set in a prior step, using the MODEL parameter to ensure that the important data characteristic values are used. Alternatively, provide the name of a non-existing data set with the OUTDSNAME parameter and IAMSRORG will define the output data set automatically based on the attributes of the input data set. Users can alter the STORCLASS, MGMTCLASS, DATACLASS and the volume list with OUTVOLUMES keyword parameters to control allocation of the new data set.

Secondary Output Data Set

An optional secondary output data set of the REORG process is a sequential backup of the input data set. If the secondary output data set is requested, it will default to being a portable copy of the data set, i.e. an export format, with full data set information, compressed data records if applicable, and the hardware compression dictionary if applicable. If you prefer a file with only the user data records that are uncompressed data format can be produced with the specification of the NOEXPORT keyword. Please note that use of the NOEXPORT keyword will result in a less than optimal performance process. The secondary output data set is specified by:

• JCL DD card named SEQFILE or
• JCL DD card with another name that is specified by the SEQFILE keyword or
• By data set name using the SEQDSNAME keyword parameter.

The sequential output data set can be z/EDC compressed, which may result in a significant disk space savings.

Secondary output data set must have space allocated for it, either through the provided DD card or previous allocation. IAMSRORG will not request dynamic allocation to do a new allocation if the data set does not exist. IAMSRORG will set the DCB attributes including record format to variable blocked, and the logical record length as it determines is required. It will let the system determine the block size. This data set can be z/EDC data compressed. which may substantially reduce the amount of disk space utilized for the exported backup copy.

There is a special case for the REORG process if a secondary output data set is specified without a primary output data set. In that circumstance IAMSRORG will do the reorganization in two phases. The first phase will write out to the secondary output data set. The second phase will read from the secondary output data set and copy the data back into the original IAM data set. If any errors occur during the first phase, including a record count mismatch, then the second phase will not be done. Use of the FORCE keyword will allow the process to continue for a mismatch record count.

Sort Considerations

The IAMSRORG program will utilize the installation default sort utility program for REORG and the EXPORT processing except when there is are no extended area containing overflow or Prime Extension (PE) records.

If IAM is using 64-bit virtual buffers and the job is running under z/OS 2.1 or above, then the PGM=SORT64 will be used. Otherwise PGM=SORT will be invoked. You will need to specify the appropriate DD statements necessary to facilitate use of the SORT program. This will vary depending on how your installation has the SORT configured, but in general there may be a requirement for a set of SORTWKnn DD cards for temporary DASD space. If the sort work space is needed and there are no SORTWKnn DD statements, then the SORT will dynamically allocate the necessary SORTWKnn data sets. A SYSOUT DD statement is required for SORT messages. By default there will be no SORT messages produced unless an error in the SORT occurs. Refer to the MGSGPRT parameter for the various SORT print options. The IAMSRORG program will provide the basic control cards that are needed for the SORT. If you want to provide additional parameters, this can be done for DFSORT with the DFSPARM DD card, and for SyncSort with the $ORTPARM DD card.

Estimating Sort Work Space

The amount of DASD space required for the temporary work files of SORTWKnn will depend on the amount of records in the overflow area of the input IAM data set. It is recommended that you let the SORT determine the amount of space needed and the number of work files by letting the SORT dynamically allocate the sort work space. Optionally, the SORTWKnn work data sets can be specified in the JCL. Using the LISTCAT IAMPRINT output or a recent IAMINFO report will help estimating the sort work space. Using one of those reports, add the number of extended overflow blocks and extended PE blocks, then divide by number of blocks per track to calculate the number of tracks of extended area data. Increase that value by 10% (that is, multiply it by 1.1) to get an approximation of the amount of sort work space that will be needed. Divide that by the number of SORTWKnn you want to specify to determine the primary quantity for each work file.

As an example, say the file has 1,164,407 extended overflow blocks and 93 extended PE blocks. So the total is 1,164,500 blocks divided by 4 blocks per track yields 291,125 ‘tracks. Multiplying that by 1.1, results in an estimated 320,238 tracks for sort work space. So for 4 SORTWKnn files, divide that by 4,is 80,060 tracks per sort work file. You can specify that in cylinders by dividing that by 15 and round up to get 5,338 cylinders.

REORG Command Statement

Executes the IAM data set reorganization process. No operands are needed if the default DD names are used in the JCL, unless there is a need for a non-default option to be used.

IAMSRORG REORG Command Operands

REORG Command Operands

The specification for the REORG keywords are as described below.