IAM/PLEX Record Lock Recovery

Record lock recovery is an optional process that enables IAM/PLEX to re-establish the record locks for recoverable files when an IAM/PLEX instance is restarted after a shut down or failure. This function enhances the data integrity by providing persistent record locking within the IAM/PLEX sphere, which will enable recovery of the work that was in process at the time of the incident prior to any other attempts to update those records through IAM/PLEX. Using a two phase recovery process, as explained below, IAM will re-establish the record locks as they were at the time of failure based on data from both the record lock journal and the IAM/PLEX System Logger journal.

Eligibility requirements for record lock recovery are that, the locks being held are for a recoverable IAM data set, which is defined as being a data set that is using the IAM/PLEX journaling that includes before images. When record lock recovery is active, it will also automatically result in before images for CICS recoverable files to be written in the IAM/PLEX journals. The journal records are necessary for the recovery process.

New users of IAM/PLEX will not be able to take advantage of the IAM/PLEX Record Lock Recovery feature. Therefore, it is recommended that when you set up IAM/PLEX for the first time, you should not add a RLOKJRNL DD statement to your IAMPLEX startup Started Task JCL. After gaining experience with IAM/PLEX usage, advanced features like IAM/PLEX batch journaling with syncpointing and IAM/PLEX Record Lock Recovery can be setup and exploited with a good understanding of how IAM/PLEX performs its functionality.

How Lock Recovery Works

The record lock recovery process retains information about the record locks that are currently held on disk in an IAM data set, that is referred to as the Record Lock Journal. The presence of the Record Lock Journal in the IAM/PLEX JCL is the indication that the record lock recovery process is to be activated. Each IAM/PLEX instance will need to have its own Record Lock Journal data set. To minimize the performance impact, the I/O on the data set utilizes deferred writes which delays the actual physical writing of the record lock status. At periodic intervals, referred to as the record lock checkpoints, all the buffers containing updated record lock data are written out to DASD and the current position in the IAM/PLEX journal is recorded.

The checkpoint interval can be set through the IAM/PLEX parameter LOCKCHKPT, which will specify the checkpoint interval in minutes. The default if the LOCKCHKPT parameter is not provided is 5 minutes. Using shorter times may increase the overhead during regular processing, but decrease the recovery time. Using longer checkpoint times will have the opposite effect.

The actual record lock recovery process is done in two phases while the IAM/PLEX instance is starting up. The record lock recovery must complete before the IAM/PLEX instance will start processing I/O requests. The first phase is to read through the Record Lock Journal data set and establish the record locks based on the information from that data set. If the status indicates that the IAM/PLEX instance had been shut down properly, meaning that the lock recovery journal has all of the lock data, then the recovery is complete.

If the IAM/PLEX instance was not normally shut down, then the record lock recovery process will go into the second phase. The second phase will read the IAM/PLEX system logger journal. Positioning is established at the place in the journal where the last record lock checkpoint had occurred, then proceed forward in the journal from that point. The recovery process will then update the record locks based on the activity indicated by the journal records, which includes updating the lock recovery journal.

It is recommended that users backup the lock recovery journal prior to restarting the IAM/PLEX instance from a failure. This can be done with a copy, a backup with FDRDSF, ABR or similar product, or by an IDCAMS REPRO. If a failure should occur on the restart of the IAM/PLEX instance, one will then be able to restore the lock recovery journal and retry the restart process again, otherwise a cold start may be necessary.

On completion of the record lock recovery, a summary of the activity performed by the lock recovery process will be printed on the RLSLOGDD data set. Then the IAM/PLEX instance will finish initialization and start accepting requests. To find out what record locks have been recovered, one can either use the IAM/PLEX display retained locks command or use the IAMBMON facility under the IAM ISPF panels. If any of the work is not going to be recovered, the associated locks can be released using the IAM/PLEX release locks command.

Allocating the Lock Recovery Journal

To activate the record lock recovery function, a lock recovery journal data set must be preallocated using an IDCAMS DEFINE prior to use for each IAM/PLEX instance that record lock recovery is going to be used. Each instance will only have the record locks for the IAM data sets that it is servicing. Any IAM/PLEX instance that does not directly do I/O to any file, will not need a lock recovery journal. The lock recovery journal data set contains a control record, lock owner records, and record lock records. The data set will be formatted when it is first used, based on the IAM/PLEX parameters of MAXJOBS, MAXLOCKS, and MAXTRANS. Below is an example of the JCL and control cards that are used to define the lock recovery journal data set:

Example of JCL for Lock Recovery Journal, Member LKRCVJRN in ICL

When using the above JCL, users will need to change the data set name and the STORCLAS. Also the space requested should be updated based on the MAXLOCKS value they are using. Please note that one needs to use the next larger value from the table if the MAXLOCKS value chosen exceeds the prior value. For example if MAXLOCKS=133104 is specified, then the space allocation used must be the one for up to 266,239 lock records. All of the other parameters and overrides shown in the example above must be used to provide proper functioning and performance of the lock recovery function. If the value for MAXLOCKS is ever increased, the lock recovery journal should be re-allocated with the required space values indicated.

Initializing the Lock Recovery Journal

After defining the lock recovery journal, users must then make the following changes to the IAM/PLEX JCL:

• Add the RLOKJRNL DD statement to the IAM/PLEX JCL specifying the newly defined data set.
• Add a card in the IAMOVRID member input that specifies the following:
  ACCESS DD=RLOKJRNL,NORLS,JRNAD=NONE

When the IAM/PLEX instance is started, IAM/PLEX will automatically determine that the data set is unloaded and format the data set with the appropriate number of preformatted records. This pre-formatting is completed quickly and will provide performance benefits for the operation of the lock recovery function. If any of the key IAM/PLEX parameters are changed, that is MAXLOCKS, MAXJOBS, or MAXTRANS, then the data set should be re-initialized by specifying the IAM/PLEX parameter of LOCKSTART=COLD which will trigger the reformatting of the data set. If the value for MAXLOCKS has been increased then the data set may need to be deleted and re-defined with the appropriate amount of space.

Normal Operation of Lock Recovery Journal

It is highly recommended that a backup copy of the lock recovery journal be taken whenever the IAM/PLEX has come down, whether it was scheduled or unscheduled. The data set can be backed up with a copy, FDRDSF, ABR or similar type of backup, or by an IDCAMS REPRO. Then if there is a problem on the restart, the lock recovery journal can be restored prior to retrying the restart.

Backups can be taken at other times while the IAM/PLEX address space is running, however, the restored copies can not really be used to restart the IAM/PLEX for other than a cold start (LOCKSTART=COLD). This is because the restored copy is far from being synchronized with the actual state when the IAM/PLEX address space had terminated. A cold start is also likely to be required when starting IAM/PLEX address spaces at disaster recovery sites

There generally should not be any need to reorganize the lock recovery data set. The define parameters and overrides in the example above were carefully chosen along with the way the data set is processed to minimize the need for use of the overflow area. This is particularly true when appropriate values were specified on the IAM/PLEX parameters such that they were sufficient to meet the processing that was done. One can either look at a LISTCAT output or at the IAMINFO report to determine if there are a lot of records in overflow. If there are, then one can check on the maximum locks used from the RLSINFO report that is produced or from the IAMBMON display to make sure that sufficient lock records had been formatted. One should also check that all of the appropriate parameters in the define example above were used. It may be necessary to delete and redefine the data set such that the overflow area usage will be minimized. BMC technical support can provide assistance if needed.