NGT Recover sort parameters

NGT Recover has several parameters that are set at installation time that affect sorting.

SMCORE option

NGT Recover provides the SMCORE installation option to give you control, when necessary, over the amount of memory that BMCSORT uses during a recovery job.

SMCORE is specified at installation time and is part of the AFR$OPTS options module. You can override SMCORE at run time by using the OPTIONS command. This option contains two parameters, total memory and below-the-line memory. For more information, see SMCORE=(0K,0K).

The following values tell NGT Recover to determine the appropriate amount of memory to use for each sort based on the following criteria:

• Value that you specified for REGION in either your JCL or system exits

• Amount of memory that is available during optimization

• Number of sorts to process

  • Using KSORTSHARE YES, the maximum number of sorts is:

    MAXKSORT + MAXLSORT

  • Using KSORTSHARE NO, the maximum number of sorts is:

    MAXLSORT + (MAXKSORT * MAXLSORT)

  For more information, see MAXLSORT and KSORTSHARE options and MAXKSORT option.

Total memory

The first parameter value of the SMCORE option tells NGT Recover how much total memory, above and below the 16-MB line, that you want BMCSORT to use during a single invocation. You can enter the value as a number kilobytes (using the suffix K), or megabytes (using the suffix M).

BMC recommends that you specify zero (as 0K or 0M) to allow NGT Recover to determine the optimal amount of storage.

In addition to 0 KB or 0 MB, valid values are 4096 KB through 65536 KB (using the suffix K) or 4 MB through 64 MB (using the suffix M). Regardless of whether NGT Recover determines the value for total memory or you specify a value, NGT Recover multiplies this value by the number of required sort processes to get a value for the total memory required for the current job. Depending on the workload and system environment, NGT Recover distributes this total memory among the sort processes for the job.

For example, if you specify 4096 KB and NGT Recover determines that it needs four sort processes for this job, NGT Recover calculates that it needs 16384 KB total memory for the job. If the workload for each sort process is different, NGT Recover invokes BMCSORT for each sort process with varying amounts of memory. Some of these amounts will be lower and some of these amounts will be higher than the 4096 KB that you specified.

The following additional considerations apply to the SMCORE option:

• NGT Recover always attempts to honor the value that you specified. Improper setting could lead to performance degradation and memory shortage. The region size available for your recovery job with the value that you specify for this subparameter can constrain the number of sort processes that NGT Recover could start. Because the region size must include space for buffers and other required structures, the entire region size is not available for sort processing.

• When you allow NGT Recover to optimize total memory, NGT Recover does not use more than the value of your region parameter and does note use more than some reasonable percent of available memory.

• If you specify an SMCORE value that is larger than your region size or the available memory on the system, NGT Recover will never use more than the value of your region size or the available memory on the system.

Below-the-line memory

The second parameter value of the SMCORE option tells NGT Recover how much memory below the 16-MB line that you want BMCSORT to use during a single invocation. You can enter the value as a number kilobytes (using the suffix K), or megabytes (using the suffix M).

BMC recommends that you specify zero (as 0K or 0M) to allow NGT Recover to determine the optimal amount of below-the-line memory to use. In addition to 0 KB or 0 MB, valid values are 256 KB through 4096 KB (using the suffix K) or 1 MB through 4 MB (using the suffix M).

BMCSORT never needs more than 256 KB of memory below the line. Specifying a value greater than this number can limit the number of sort tasks that NGT Recover can start concurrently.

MAXKSORT option

NGT Recover determines the optimal number of sorts executing concurrently, depending on available resources. You can specify the maximum number of concurrent sorts with the MAXKSORT option.

Under normal circumstances, allow NGT Recover to control the number of sorts processed concurrently. If system resources are constrained or other problems arise, you can change the MAXKSORT option to limit the number of sorts that are running concurrently.

The MAXKSORT installation option or the MAXKSORT option in the OPTION command statement determines the maximum number of index key sorts that NGT Recover can run in parallel and the maximum number of indexes (or index partitions) that NGT Recover can rebuild in parallel for each table space. Setting the value of MAXKSORT too low could significantly increase the time required to rebuild indexes. Setting the value of this parameter too high could result in overuse of system resources, which could cause total system performance degradation as well as performance degradation of the recovery job.

For indexes on nonpartitioned spaces, the optimum value for MAXKSORT is the largest number of indexes that will be rebuilt for any table space in the recovery job. For example, if a recovery job rebuilds five indexes for table space A and three indexes for table space B, set MAXKSORT to a minimum value of 5. For this example, if MAXKSORT is set to a minimum of 8, NGT Recover might be able to overlap the index rebuilds for table space A with the recovery of table space B, reducing the elapsed time of the job. If the number of indexes to rebuild is so large that you cannot set MAXKSORT to that number, set MAXKSORT as high as possible. NGT Recover groups indexes with similar key lengths together to form an index group.

NGT Recover displays the index group assigned to each index in message BMC40865I in the Object Summary.

For indexes on partitioned spaces, NGT Recover uses the value of MAXKSORT and attempts to divide partitions into groups to allow parallel unloading of keys. The number of partition groups is balanced with the number of indexes to achieve the largest number of parallel sorts and index rebuilds without increasing the total number of bytes sorted or the size of the largest sort.

As with nonpartitioned indexes on nonpartitioned table spaces, nonpartitioned indexes are placed in index groups. The number of parallel sorts is the number of partitions in a partition group multiplied by the number of index groups. For a partitioning index rebuilt with nonpartitioned indexes, the partitioning index may be assigned to its own index group or may be placed in a group with nonpartitioned indexes. The number of indexes that are built in parallel equals the number of index groups.

For example, assume you have a partitioned table space with 30 partitions and four nonpartitioned indexes with key lengths of 200, 120, 60, and 50. If you specify OPTION MAXKSORT 15, NGT Recover creates three index groups with the 200- and 120-byte keys having their own group and the 60- and 50-byte keys sharing a group. Each partition group contains five partitions, and there are six groups of partitions.

The partitions are assigned to groups based on the estimated number of rows in each partition such that, as much as possible, the partition groups contain an equal number of rows. For this example, there are 15 parallel index key sorts (five partitions in each partition group multiplied by three index groups). Three indexes are built in parallel because there are three index groups.

The maximum value that you can assign to MAXKSORT is determined by the amount of below-the-line and above-the-line memory available in the region in which the NGT Recover job runs. NGT Recover does not allow a value for MAXKSORT that is larger than that which can be supported by the available memory. The formula to determine the maximum value allowed for MAXKSORT is as follows:

a = above-the-line memory / 30 MB + 1

b = (below-the-line memory - 1.5 MB) / 300 KB + 1

MAXKSORT = minimum(a, b)

For example, in a region with 7.5 MB below the line and 720 MB above the line, the maximum value for MAXKSORT is 21:

a = 720 MB / 30 MB + 1

b = 7.5 MB - 1.5 MB / 300 KB + 1

MAXKSORT = minimum(25, 21)

MAXKSORT = 21

You should also consider the value of MAXLSORT when setting the value of MAXKSORT. For more information, see MAXLSORT and KSORTSHARE options and Data sets needed for parallel index sorts and rebuilds.

MAXLSORT and KSORTSHARE options

NGT Recover can run multiple log sorts and parallel MERGE phases. SNAP and RESTORE phases (for inline copies) also run in parallel.

The MAXLSORT option, which you can set in the installation options (MAXLSORT=0) or on the OPTIONS command (MAXLSORT integer), specifies how many log sorts can run in parallel and also determines the number of MERGE/SNAP/RESTORE phases that can run in parallel, regardless of log requirements. In the case of parallel phases, the phases run in subtasks and MAXLSORT sets the number of subtasks.

Use of the MAXLSORT option is dependent on the amount of available memory. BMC suggests that a MAXLSORT value of 10 to 12 is practical. The default value of the MAXLSORT installation option is 0, which allows NGT Recover to determine the number of sorts.

If you specify MAXLSORT 1, you set up your job to run as it would in NGT Recover version 8.1.00 and earlier and turn off parallel log sorts and parallel MERGE/SNAP/RESTORE phases. These phases then run serially in the main task in the order in which they are scheduled by the NGT Recover planning component.

NGT Recover must run some phases serially due to constraints such as

• Sharing a DSNUM ALL or cabinet copy

• Using copies on stacked tape

• Sharing a CHANGE ACCUM file

To honor these constraints, the NGT Recover planning component places the phases into groups. Each phase in such groups runs serially in a subtask. NGT Recover distributes the processing of these groups to balance the workload.

When a table space is recovered and associated indexes are rebuilt in the same step, the MERGE phase extracts keys for the indexes. Since each table space generally uses the maximum number of key sorts specified by MAXKSORT, NGT Recover may not be able to run MERGE phases from different table spaces in parallel because each MERGE requires MAXKSORT key sorts. (For more information about MAXKSORT, see MAXKSORT option.) In this case, you use the KSORTSHARE option, which you can set in the installation options (KSORTSHARE=YES) or on the OPTIONS command (KSORTSHARE), to specify if key sorts are shared among NGT Recover table space recoveries (MERGE phases) running in parallel.

When you specify KSORTSHARE YES, which is the default value, up to MAXKSORT key sorts are shared at execution. Key sorts can only be used by one table space at a time. With KSORTSHARE YES, NGT Recover generates UNLOAD phases to extract keys. However, during execution, if a sufficient number of key sorts is available when the first MERGE phase for a table space is started, the MERGE phase extracts the keys and the UNLOAD phase is skipped.

When you specify KSORTSHARE NO, each MERGE phase has its own set of key sorts and NGT Recover can have up to MAXKSORT * MAXLSORT key sorts active at any given time. Since the number of sorts that can be active in a system is fairly small - usually no more than 30 - a value of NO for this option may severely limit the number of recovery operations that NGT Recover can perform in parallel when index rebuilds are also requested.

If the values that you specify for MAXLSORT and MAXKSORT result in too many sorts, NGT Recover reduces the value of MAXLSORT until the number of sorts can be accommodated. This reduction in MAXKSORT and MAXLSORT is only necessary when you specify KSORTSHARE NO.

RESINV parameter

The NGT Recover RESINV parameter is used to specify the amount of virtual storage to be reserved below the 16-MB line for program-invoked sorts. This parameter is specified at installation time. NGT Recover recommends that this parameter always be specified as RESINV 0K, which is the default. This value allows the sort utility’s installation default to be used.

For more information, see RESINV=0K

SORTNUM and SORTDEVT parameters

The SORTNUM parameter controls the actual number of sort work files that are allocated. The SORTDEVT parameter controls the device type to which the sort work files are allocated. These parameters control where each of the sort work files are allocated when dynamically allocating files. For a summary of information about SORTNUM and SORTDEVT, see the following table.

For more information about SORTDEVT, see SORTDEVT and General information about SORTDEVT and SORTNUM.

For more information about SORTNUM, see SORTNUM and General information about SORTDEVT and SORTNUM.

Parameters for DASD Work Space

Other sort parameters

Further control of the sort is available through the NGT Recover OPTIONS command.

See SORT specification description. The OPTIONS command includes the following options:

• SORTDIAG, which turns diagnostic messages on and off

• SORTDYN, which controls and overrides installation defaults for sort work