IAM Journal and Recovery
Why Use IAM Journaling ?
The main reason to use the IAM journaling feature is to improve data availability for IAM data sets. This is accomplished because the enhanced recovery capabilities can reduce the frequency of backing up the entire data set and / or eliminate the need to restore IAM data sets when jobs abend or otherwise fail.
If the REDO (after update) images are journaled, rather than backing up large IAM data sets every day, they can be backed up less frequently as once a week. Only the journal containing the updated data would be backed up on a daily basis. By backing up less data, the amount of time a data set is not available for update processing, is reduced. The amount of time saved by not doing full backups each night can be quite substantial. In case of a media failure, the data set is recovered by restoring the data set from the last full backup, then executing a journal forward recovery using IAMJREST or IAMRREST depending on the type of journal being used.
If UNDO (before update) images are journaled then the time to recover after an abend or other failure can be reduced. This is accomplished by backing out the updates and then resuming processing. This avoids having to restore the entire contents of data sets, and potentially rerunning several jobs. The updates made by the failing batch job can be backed out by the IAM provided recovery utility IAMBREST, IAMJREST or IAMRREST. Upon recovery completion the batch job processing can be restarted. If need be, the updates from multiple jobs steps and jobs can be backed out. The backout facility may be able to save a lot of time when recovering from typical job abends, by eliminating the need to restore data sets and rerun an entire sequence of batch jobs. Additional savings are possible by eliminating the need for batch jobs to perform separate backup of critical files prior to starting an update process.
Types of Journals Available
As mentioned previously, there are three different options for the type of journal processing that IAM can use: LOGSTREAM, IAM/RLS and IAM/PLEX, and BASIC. IAM will only invoke one of these journal methods at a time to avoid excess journal processing overhead.
Highlights of each of these functions is shown in the section below:
LOGSTREAM Journals
- Journal data is written to a z/OS System Logger logstream. It can be either the DASD-ONLY or a Coupling Facility type of logstream, dependent on application requirements.
- Multiple files can use the same logstream.
- Data sets must be on SMS managed volumes.
- Specified by LOGSTREAMID, LOGREPLICATE and FRLOG define parameters, or the LOGSTREAM and FRLOGIAM overrides.
- Journal records are in a format that can be utilized for GDPS Active-Active using IBM Info Sphere Data Replication for VSAM (IIDR) and by IBM CICS VSAM Recovery (CICS/VR). CICS/VR has limited support for IAM files. The BATCHBACK command works with the RCDS(NO) parameter, but the RECOVER command fails for an undetermined reason with no error messages.
- Undo and Redo recoveries can be done with the IAMRREST utility.
- Requires IAM Version 9.3 or above.
- Users can setup and enable an IAMLOGVR address space to centralize IAM’s LOGREPLICATE and/or FRLOG logstream write access and the IAMRREST utility’s read access.
IAM/RLS and IAM/PLEX Journals
- Journal data written to either a z/OS System Logger logstream or a set of sequential data sets.
- Multiple IAM files can use same set of journals.
- Used forIAM files processed by either IAM/RLS or IAM/PLEX.
- Specified by the IDCAMS define parameter LOG() in conjunction with IAM/RLS or IAM/PLEX startup parameters, and the IAM JRNAD overrides.
- Journal records are in IAM format for DASD sequential file type journals. Journal records are in IBM Data Replication format for logstream type journals and can be used with IBM data replication products if LOGREPLICATE and LOGSTREAMID are used on the Define of the data set. IAMjournal utilities can also be used and the IAM utilities do not have the IDCAMS DEFINE LOGREPLICATE and LOGSTREAMID requirement.
- Will optionally write journal records for CICS processing.
- Undo and Redo processing done by IAMJREST, with an option for undo being performed automatically with IAMBREST the Dynamic Job Backout utility.
Basic Journal
- Journal data written to a sequential data set, only one IAM data set per journal file, must be named same as the IAM data set appended with .LOG.
- Each IAM file journaled needs it’s own journal data set.
- Specified by LOG() define parameter and IAM JRNAD overrides.
- Journal records are a shortened version of IAM/RLS journal records, and are available to users for their own post or recovery processing.
- Will write journal records when data set is processed by CICS.
- Undo and Redo processing provided by IAMJREST.
IAM does not journal records written for file load processing. Unless otherwise indicated IAM does not journal records for processing done by CICS, as CICS performs it’s own journal processing.
Preparing for Journaling
Journaling does provide excellent recovery capabilities that can improve batch throughput by eliminating time for application data set backups and reducing recovery time when problems occur. There is a cost to journaling, including the extra physical I/O to the journals, storage requirements for journaled data, and some CPU time. To minimize the overhead, only journal the data that you are going to use. If you are going to use journals only for backout purposes, don’t journal the redo records. If you are not going to do backout recoveries, then don’t journal the undo records. The two main decisions, once one has decided to utilize the journaling capabilities are, what records to journal and which journal function to utilize. The list below provides a few thoughts on choosing the appropriate journal method.
Logstream Journals:
- Required for data replication (LOGREPLICATE)
- Best choice for journaling multiple data sets for an application
- Best choice if recovery involves multiple data sets
- IAM/RLS and IAM/PLEX Journals
- Recommended for files used under IAM/RLS or IAM/PLEX
- Necessary for Record Lock Recovery function
- Good choice for multiple data set usage and recovery
Base Journals:
- Good choice if only considering forward recoveries at data set level.
- Good choice if doing journaling for only a few select data sets
If you are journaling for all or most of the update activity on an IAM data set, then you should activate journaling by specification during the file define process. Use the FRLOG, LOGREPLCIATE or LOG parameter to specify the journal records to be written.
RLS Journaling
Further information on the IAM RLS journals is provided in IAM-PLEX-Journaling, IAM/RLS Batch Considerations, and IAM/RLS Data Recovery of the IAM space. IAM PLEX journals are discussed in IAM-PLEX-Journaling These sections include information on setting up the journals, managing the journals, batch syncpoint facility, and recovery using the journals.
Logstream Journals
Logstream journals, as the name indicates, are journals of data set updates that are written to a z/OS System Logger logstream. The logstreams can be either DASD only type or utilize a Coupling Facility structure type when there is a need to be shared by multiple systems. The DASD only logstream is intended for single system use, but within that single system it can have multiple concurrent users. A DASD only logstream can not be used concurrently across multiple systems, however, it can be serially used on other systems. The Coupling Facility logstream can be used concurrently by multiple systems within the same SYSPLEX.
The first requirement to use the logstream journals is that the data set must be on an SMS managed volume. With that criteria met, then the z/OS System Logger logstream is identified by either the LOGSTREAMID IDCAMS Define Cluster parameter or alternatively by the SMS Data Class specified by or automatically selected for the data set being defined. Full information on defining and setting up a logstream is in the IBM space MVS Setting Up a Sysplex.
LOGREPLICATE
When data replication is one of the intended uses of the journal data, then the LOGREPLICATE parameter is specified on the IDCAMS Define Cluster or in the data class that the data set is using. LOGREPLICATE specifies a comprehensive set of records be written to the journal, including the undo records and the redo records. CICS will generate the log replication records for IAM and VSAM files that are used by applications running under CICS using standard CICS File Control services. For non-CICS usage IAM will write the replication records either directly or via the IAMLOGVR address space (see below). This happens automatically when the data set is defined with the LOGREPLICATE attribute, an FRLOG value is specified, and a LOGSTREAMID is included. While the primary intent of these journal records is to facilitate data replication, they are usable for undo or redo recoveries for IAM data sets performed by the IAMRREST recovery utility, or by CICS/VR BATCHBACK command.
Example of DEFINE using LOGREPLICATE
(NAME(IIDRTEST.RPL100.CLUSTER) -
STORCLAS(TSTDATA) -
CYL(10 5) -
OWNER($IAM ) -
LOGREPLICATE -
FRLOG(REDO) -
LOGSTREAMID(IAMLOGR.LOGREP00) -
RECORDSIZE(256 300) KEYS(12,8) -
CISZ(16384) -
FREESPACE(10,10) -
SHAREOPTIONS(2,3))
The actual process of performing the replication is handled by IBM InfoSphere Data Replication for VSAM for z/OS or other replication software product. The product Guide and Reference space documents the instructions on setting up the files for the replication process. The setup for IAM is identical to VSAM, so following the instructions in that space will work for both IAM and VSAM files.
FRLOG
The FRLOG IDCAMS Define Cluster parameter specifies a subset of the records that are generated by LOGREPLICATE to be written out to the logstream. The possible values are UNDO, REDO, ALL or NONE. For non-CICS processing, IAM will generate the type of journal records that are specified by the FRLOG parameter and write them to the Logstream. A distinction between IAM and VSAM, is that IAM will write out the UNDO (before) journal records when requested by the FRLOG ALL or UNDO parameter to the specified Logstream. If CICS/VR BATCHBACK command is used to backout updates to IAM files, LOGREPLICATE must be specified in addition to the FRLOG specification. The table below will show the distinctions.
FRLOG Journal Records Generated in Different Environments
FRLOG(ALL) | FRLOG(ALL) | LOGREPLICATE | LOGREPLICATE | |
---|---|---|---|---|
CICS | REDO | REDO | REDO+UNDO | REDO+UNDO |
BATCH - CICS/VR | REDO | NONE | REDO+UNDO | NONE |
BATCH - IAM | NONE | REDO+UNDO | NONE | REDO+UNDO |
The use of a logstream journal is generally indicated when an IAM file is defined, using the LOGSTREAMID and the FRLOG or LOGREPLICATE parameters. Use of FRLOG, LOGREPLICATE and LOGSTREAMID parameters by CICS or IIDR-VSAM replication requires that the IAM data set be defined with the desired attributes. Shown below is an example of defining an IAM file with UNDO journal image recording being requested.
Example of Defining IAM File with FRLOG(UNDO) and LOGSTREAMID
(NAME(IAMV.JRN160.CLUSTER) -
STORCLAS(TMPDATA) -
CYL(15,5) -
OWNER($IAM ) -
LOGSTREAMID(IAMLOGR.FRLOG00) -
FRLOG(UNDO) -
RECORDSIZE(1000,1300) KEYS(4,8) -
FREESPACE(0,10) -
SHAREOPTIONS(2,3))
Shown below is an example of turning on Logstream journaling for CICS and non-CICS address spaces using LOGREPLICATE and FRLOG on an existing IAM data set.
Example of turning on FRLOG(ALL) and LOGREPLICATE on an existing IAM Data set
//SYSPRINT DD SYSOUT=*
//IAMPRINT DD SYSOUT=*
//IAMOVRID DD *
CREATE DSN=IAMV.MYIAM.TKSDS,LOGREPLICATE,FRLOG=ALL,
LOGSTREAM=LOGSTR.APPLXYZ
/*
//SYSIN DD *
DEFINE CLUSTER -
(NAME(IAMV.MYIAM.TKSDS) -
OWNER($IAM) -
VOL(ANYVOL) RECATALOG )
LISTCAT ENTRIES(IAMV.MYIAM.TKSDS) ALL
/*
IAMLOGVR System Logger Server Address Space
To assist with Logstream management the IAMLOGVR address space can optionally be setup to act as a System Logger Server Address Space. Usage of the IAMLOGVR address space requires that the IAM LOGVR Global Option be enabled. The IAMLOGVR address space makes setting up security access to your Logstreams much simpler. By using IAMLOGVR, you just have to provide Logstream security access to a single started task User ID rather than all the various batch job User IDs that might be used by your production jobs. The IAMLOGVR address space also decreases Logstream Connect and Disconnect overhead because fewer Connects and Disconnects are necessary. Additionally, the IAMLOGVR address space has the ability to buffer newly created journal records while a Logstream error condition is being cleared, such as a Coupling Facility Logstream full condition.
Start the IAMLOGVR address space as part of your system IPL after the IAM VIF is started. IAMLOGVR will open the Logstreams provided via the SYSIN DD at startup time, or via the ADD command after being started. Logstreams can also be removed from the address space’s control via the REMOVE command. Each Logstream specified will cause a subtask to be attached, which will manage that particular Logstream.
The IAMRREST utility will perform restores from an IAMLOGVR managed Logstream by going through the IAMLOGVR address space to retrieve journaled records.
Example of IAMLOGVR Started Task JCL; available in IAMSAMP data set.
//* THIS IS THE IAMLOGVR PROC
//IEFPROC EXEC PGM=IAMLOGVR,
// REGION=0M,DPRTY=(15,15),TIME=1440,MEMLIMIT=4G
//STEPLIB DD DSN=SYS3.IDPIAM.LOAD,DISP=SHR
//SYSLIB DD DSN=SYS3.IDPIAM.LOAD,DISP=SHR
//SYSPRINT DD SYSOUT=*
//SYSABEND DD SYSOUT=*
//SYSIN DD DISP=SHR,DSN=SYS3.IDPIAM.PARMLIB(&LOGSTRL)
Example of IAMLOGVR SYSIN list of managed Logstream IDs
IAMLOGR.ALTGRPXX
IAMLOGR.CICSVR00
IAMLOGVR Operator Commands
The following operator commands can be issued to the IAMLOGVR address space to help manage the Logstreams controlled by the address space.
ADD | The ADD command provides a new logstream name that will cause IAMLOGVR to attach a new subtask to manage the additional Logstream. Example- F IAMLOGVR,ADD,IAMLOGR.NEWSTRNM |
REMOVE | The REMOVE command will cause IAMLOGVR to shutdown the subtask processing the specified Logstream. Example- F IAMLOGVR,REMOVE,IAMLOGR.NEWSTRNM |
CHECKBUFF | The CHECKBUFF command will list the total number of buffers allocated and the number on the available queue. IAMLOGVR starts with 64 Meg of 64-bit storage for buffers which is enough for 2,047 buffers. There is a low threshold of 500 available buffers which if detected will cause another 64 Meg of 64-bit storage to be allocated and added to the buffer pool automatically. Example- F IAMLOGVR,CHECKBUFF |
MOREBUFF | The MOREBUFF command will tell the IAMLOGVR to get another 64 Meg of 64 bit storage for buffers and add them to the available pool. Example- F IAMLOGVR,MOREBUFF |
DUMP | The DUMP command is used to request that the IAMLOGVR address space take an SVC Dump. Example- F IAMLOGVR,DUMP |
SHUTDOWN | The SHUTDOWN command will cause IAMLOGVR to post all the subtasks to disconnect from the Logstreams they are managing and terminate. Once all subtasks terminate the address space will shutdown. Example- F IAMLOGVR,SHUTDOWN |
Using Logstream Journals
The logstream journals are ideal for applications where there are multiple files being updated that would benefit from being recoverable and that are not being used under IAM/RLS or IAM/PLEX. The primary reason is that all of the journal records for the application can be contained in a single logstream, which facilitates a simple single step recovery process for all of the updated data sets by IAMRREST. The backout can be run within the failing job automatically if the job step abends, or as a separate job. The journal records can also be copied to a sequential data set with IAMJUTIL for faster access if subsequent recovery is needed to avoid having to read through a portion of the other data in the logstream.
Example of Backout In an Abending Job
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//IAMINFO DD SYSOUT=*
//SYSIN DD *
RESTORE BACKOUT,JOBNAME=*,STEPNAME=JRN161B,
LOGSTREAMNAME=IAMLOGR.FRLOG00
/*
Logstream Journal Backout
The above example is the JCL to automatically do a backout when the prior step of the job abended. Specification of JOBNAME=* will use the currently executing job name and JES job id, along with the specified step name to identify the journal records to use for the backout process. If the backout is to be done as a separate job, then the JOBNAME parameter would specify the actual job name, and the JOBID parameter would specify the JES job identifier.
The same thing can be done using CICS/VR instead of the IAM provided utility. If the failing job step had real VSAM data sets, then CICS/VR would either recover the real VSAM files automatically, or would need a separate backout to do the VSAM files. This is due to CICS/VR not being able to journal IAM data sets, and therefore doesn’t keep the information it needs to automatically do the recovery.
Example JCL for BACKOUT Using CICS/VR
//STEPLIB DD DSN=SYS3.SDWW520.LOAD,DISP=SHR
// DD DSN=CICSTS52.SDWWLENU,DISP=SHR
//DWWLOAD DD DISP=SHR,DSN=SYS3.SDWW520.LOAD
// DD DISP=SHR,DSN=CICSTS52.SDWWLENU
//*DD DISP=SHR,DSN=EXITS LOAD LIBRARY NAME
//DWWMSG DD SYSOUT=*
//DWWDMSG DD SYSOUT=*
//DWWDUMP DD SYSOUT=*
//DWWPRINT DD SYSOUT=*
//DWWIN DD *
BATCHBACK JOB(JRN161) STEP(JRN161B) RCDS(NO)
MVSLOG NAME(IAMLOGR.FRLOG00)
/*
Logstream Journal Forward Recovery
IAMRREST can also perform a REDO (forward) recovery which applies all of the updates to a restored copy of the file.
Example of JCL to Perform a REDO or Forward Recovery
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//IAMINFO DD SYSOUT=*
//SYSIN DD *
RESTORE FORWARD,JOBNAME=*,STEPNAME=JRN261B,
LOGSTREAMNAME=IAMLOGR.CICSVR00
/*
Copying Journal Records
As mentioned previously, IAMJUTIL can be used to copy the journal records for a specified job and even just a specific job step. The flat file copy of the journal records can be used for subsequent recovery with the IAMRREST program if necessary to have it readily available for recovery or auditing purposes. Below is an example of the JCL to do that, which includes a scan of the output journal for a summary report of the data that it contains.
Example of JCL to Copy Journal Records From Logstream
//SYSPRINT DD SYSOUT=*
//JRNLOUT DD DSN=IAMV.JRN160.JOURNAL,
// STORCLAS=TSTDATA,SPACE=(CYL,(100,10)),DISP=(,CATLG)
//SYSIN DD *
COPY JOBNAME=*,STEPNAME=JRN161B,
LOGSTREAM=IAMLOGR.FRLOG00
SCAN JOURNAL=IAMV.JRN160.JOURNAL
/*
Logstream Journals In Action
As a way to try out all of the above functions, the member EX1088JR example is provided in the example JCL library included with the IAM product, which will perform all of the above examples, including a few other actions. The example JCL will set up three small IAM files, perform some updates, inserts, and deletes, followed by a back out. After the backout is verified, then a redo or forward recovery is performed with the prior backed out updates recovered. The file is again verified to make sure all of the updates were properly re-applied. The job also includes a few utility steps using IAMJUTIL to copy, scan, and print the records. Feel free to experiment with different FRLOG and / or LOGREPLICATE process, splitting the backout into a different job, and using the flag file to provide the input instead of directly from the logstream.
If you are going to be selectively journaling for only specific jobs, you can try that also. In that circumstance, we recommend that data set still be specified with the LOGSTREAMID parameter, but specify FRLOG(NONE) parameter as well. For the job steps that you want to collect the recovery data for, use the IAM ACCESS override FRLOG with the value needed such as if only doing backouts specify FRLOG=UNDO. You can do that with a ACCESS DD=&ALLDD,FRLOG=UNDO which will only be used for IAM files that were defined with the LOGSTREAMID parameter.
Basic Journal Processing
For Enhanced Format IAM data sets without spanned records, IAM offers an optional basic journaling facility along with an associated recovery capability. IAM journaling is activated through IAM Overrides for the data sets that you select. The IAM journaling facility provides a tool to capture the images of records prior to being updated, called before images, and / or images of records after being updated, called after images. Once a journal has been created, it can be used by the provided utility program IAMJREST to recover theIAM data set to a particular point in time. There are two types of recovery possible, forward recovery and backward (or backout) recovery. Forward recovery is done by restoring a backup copy of the affected IAM data set, then applying the after images from the journal data sets to the IAM data set. In other words, the recovery proceeds forward from a particular point in time to an identified point in time, prior to the failure. A backout recovery does the opposite function. Starting with a data set that has been updated by various jobs, it provides a capability to backout the updates from selected jobs. This is accomplished by updating the data set from the log using the before images.
Preparing for Journaling
If you are journaling for all or most of the update activity on the IAM data set, then you should activate journaling by specifying the IAM CREATE JRNAD= override when the file is defined or loaded. The journaling option you select will be retained with the data set, and used on each access to the data set. If you are only planning on using the journaling for a few selected jobs to be able to backout their updates, then the journaling can be specified on the selected jobs steps using the IAM ACCESS JRNAD= override. As operands on the JRNAD keyword, you should specify the types of records that you need to have written to the journal. Valid values are:
- BEFORE for before images,
- AFTER for only updated record images, or
- BOTH to collect BEFORE and AFTER images.
Using BOTH will provide the most flexibility, but will also incur the largest overhead in terms of space required and journal I/O activity. When estimating the space, keep in mind that IAM does not journal during file load or reorganization, so there will be no second copy in the journal of the entire data set.
Managing the Journal Data set
For the use of basic journaling, you are responsible for allocating and managing the journal data sets.
When using IAM journaling for files that are used under CICS, the files must be in a LSR POOL.
The next step is to decide how you want to manage your IAM journal data sets. The IAM Journal data sets must reside on DASD while they are being actively used for journaling. They can reside on either tape or disk when performing the recovery phase. If you are opting for the full journaling, then you will need permanent DASD data sets for the journals. Typically, the journal data sets will be backed up daily using IEBGENER. The reason for using IEBGENER is so that the backup copy of the LOG data set can be used directly as input to the IAMJREST utility. That would not be possible if the data set was backed up by DASD management software products, such as FDR/ABR. The backup copy can take the form of an accumulation data set or as a separate data set each day, perhaps by using a GDG.
To help minimize both the amount of DASD space for the journal and the journal backup time, you will probably want to empty the journal data sets after they are backed up. This can be easily accomplished by running program IEBGENER with an empty input data set. Or, you might decide that it is more critical to minimize the time it takes to set up the recovery. In that case you would want to accumulate the journal data within the journal data set itself, containing all the data since the last full data set backup.
If you are just going to do journaling for batch update jobs to provide a backout recovery capability, you have some different choices. You can either go with a permanently allocated journal data set, or allocate one at the beginning of the batch job stream. In either case, you will most likely want to back up the journal data set at the end of the batch stream. If you are using a permanent journal data set, you will want to empty it prior to or at the beginning of the job stream that is being journaled.
Estimating Journal DASD Space
With the above decisions made, you can next estimate the amount of DASD space that will be needed for your journal data set. To make the estimate, you will need some statistical information that can be found in IAM LISTCAT or IAMINFO reports. These reports should cover the typical length of time that the data will be residing in the journal, whether that is just for a day, a week, or through a batch job stream. The important numbers are the number of inserts updates and deletes for the length of time in question, as this will be used to calculate the number of records that are written to the journal. To estimate the number of records, which we will call R, select one of the following calculations based on the types of records that are being collected:
- For both BEFORE and AFTER Images: R = (2 * (updates + deletes)) + inserts.
- For only BEFORE or AFTER Images: R = updates+deletes+inserts.
The IAM journal will typically use ½ track blocking, which will be 27998 when residing on a 3390 and using variable length records and blocks. The maximum record length is the defined user record size plus 52 bytes for the header information on each journal record. If the maximum record size exceeds the ½ track block size, then a block size of 32760 will be used. To play it absolutely safe, you should estimate your journal space requirements using the maximum record length. However, for files that have a very large maximum record length, for which you know that the actual average record length is considerably smaller, you can use the smaller record length. Select the record size that seems most appropriate, and add 52 bytes to that length for the header information. Divide the expected block size minus four by the record size to get the average number of journal records per block, dropping any fraction. Multiply the result by 30 if using ½ track blocking, or by 15 if the larger block size of 32760 must be used. That will yield the number of journal records per cylinder, which we will call C.
The estimated number of cylinders will be R / C, which is to be rounded up. Or in words, the estimated number of log records divided by the number of records per cylinder yields the number of cylinders. Use this value for the primary space allocation of the journal data set. Then use some fraction in the range of 10% to 25% as the secondary space value for your journal data set allocation.
Example of Journal Space Calculation
As an example, let’s say that we have a file that typically has 15,000 updates, 1,000 inserts, and 500 deletes in a day. Our plan is to collect both before and after images, enabling both forward and backout recovery. The journal will be backed up and then emptied each day after the online system comes down, prior to starting the batch update runs. The estimated number of daily journal records will be:
- R = (2 * (15,000 + 500)) + 1,000 = 32,000.
The maximum defined user record length is 256, so the maximum journal record length is (256 + 52) = 308. The journal is going to reside on a 3390, which has a ½ block size of 27,998. Dividing (27,998-4) by 308 = 90 log records per block. Next, multiply the number of log records per block times the number of blocks per cylinder (which is 30), to come up with 2,790 log records per cylinder.
- C = ((27998-4) / (256 + 52)) * 30 = 2,700.
The number of cylinders required is R / C or 32,000 / 2,700 = 12.
Journal Data Set Name
The name of the journal data set is the data set name (or cluster name) of the IAM data set, with the characters ‘.LOG’ appended to the end. If the length of the data set name exceeds 40 bytes, then the end will be overlaid with the .LOG character string and be 44 bytes long, except when the 40th byte is a '.'. In that case, the data set name will only be 43 bytes long, still ending with the .LOG .
Journal Allocation Concerns
There are a few considerations for doing the actual journal allocation. First, to maximize the recovery potential, and to avoid I/O contention between the journal and the actual data set, the journal data set must be on a different volume. It would be best for the journal data set to be on a device that is on a different channel and controller if possible. Secondly, make sure to pick a volume that has sufficient DASD space available to allow the journal data set to go into extents if necessary. Thirdly, do not specify any DCB characteristics, such as LRECL, BLKSIZE, or RECFM. The IAM journal program will set those values automatically the first time the journal data set is used.
Example A: Defining an IAM file with Journaling and a Log Data set
The example JCL below demonstrates defining an IAM data set with the journaling option requested on the IAMCREATE Override statement.
Example of Defining an IAM Data set with Journaling (EX1088A)
//SYSPRINT DD SYSOUT=*
//JOURNAL DD DSN=MY.IAM.KSD.LOG,
// UNIT=SYSDA,DISP=(,CATLG),
// SPACE=(CYL,(12,6)),VOL=SER=MYVOL2
//IAMOVRI DD *
CREATE DD=&ALLDD,JRNAD=BOTH
/*
//SYSIN DD *
DEFINE CLUSTER -
(NAME(MY.IAM.KSD) -
OWNER($IAM) -
VOLUMES(MYVOL1) -
CYL(60 6) -
RECORDSIZE(100 256) -
KEYS(24 8) -
FREESPACE(5 20) -
SHAREOPTIONS(2 3) -
REUSE )
LISTCAT ENT(MY.IAM.KSD) ALL
/*
Example B: Setting up GDG for Journal Backups
To continue with the above example, the next step is setting up whatever JCL is needed for management of the log data. For the IAM data set being journaled, the backup frequency had been once a day. With the IAM journal being active for this data set, the backup frequency will be changed to once a week. The journal will be backed up daily, to a GDG, and then emptied. As part of the weekly data set backup, we will accumulate the daily journals for the past week, into a single data set on tape, which will be the weekly journal GDG. Step 2 is to set up the GDG for the journal backups, which can be done as shown below:
Defining GDG for Journal Backups (EX1088B)
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DEFINE GENERATIONDATAGROUP -
(NAME(MY.IAM.KSD.DAILY.JOURNAL)
LIMIT(7) SCRATCH)
DEFINE GENERATIONDATAGROUP -
NAME(MY.IAM.KSD.WEEKLY.JOURNAL) -
LIMIT(54))
/*
Example C: Daily Journal Backup Job
The daily journal backup job will run six days a week. This job will not only make a copy of the journal data set to the new daily journal backup GDG, but will also empty the journal file so that the most it will contain is one day’s worth of data. The following example will show how this can be accomplished:
Example of Daily Journal Backup (EX1088C)
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT2 DD DSN=MY.IAM.KSD.DAILY.JOURNAL(+1),
// DISP=(,CATLG),SPACE=(CYL,(12,6)),UNIT=SYSDA,
// DCB=(MY.IAM.KSD.LOG),VOL=SER=MYVOL3
//SYSUT1 DD DSN=MY.IAM.KSD.LOG,DISP=OLD
//EMPTYJRN EXEC PGM=IEBGENER,COND=(0,NE,BKUPJRNL)
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT1 DD DUMMY,DCB=(MY.IAM.KSD.LOG)
//SYSUT2 DD DSN=MY.IAM.KSD.LOG,DISP=OLD
Example D: Weekly Journal Backup
The weekly journal backup job should be run as part of the same job that performs the backup of the base IAM data set. Preferably as the step immediately after the data set backup. If time is critical, the weekly backup of the journal data sets can be done concurrent with the data set backup. This job will copy the previous six daily backups along with any current data in the journal, and subsequently empty the journal data set itself.
Weekly Backup to Tape of Journal Data (EX1088D)
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT2 DD DSN=MY.IAM.KSD.WEEKLY.JOURNAL(+1),
// DISP=(,CATLG),UNIT=CART
//SYSUT1 DD DSN=MY.IAM.KSD.DAILY.JOURNAL(-5),DISP=OLD
// DD DSN=MY.IAM.KSD.DAILY.JOURNAL(-4),DISP=OLD
// DD DSN=MY.IAM.KSD.DAILY.JOURNAL(-3),DISP=OLD
// DD DSN=MY.IAM.KSD.DAILY.JOURNAL(-2),DISP=OLD
// DD DSN=MY.IAM.KSD.DAILY.JOURNAL(-1),DISP=OLD
// DD DSN=MY.IAM.KSD.DAILY.JOURNAL(0),DISP=OLD
// DD DSN=MY.IAM.KSD.LOG,DISP=OLD
//EMPTYJRN EXEC PGM=IEBGENER,COND=(0,NE,WBKUPJRN)
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT1 DD DUMMY,DCB=(MY.IAM.KSD.LOG)
//SYSUT2 DD DSN=MY.IAM.KSD.LOG,DISP=OLD
Journal Recovery Procedures
Now that the journaling is all in place, along with the proper backup procedures, the recovery process from the IAM journal can be examined. Recovery with the journal is provided for by the IAMJREST program. The IAMJREST program uses as input the journal data sets, the base IAM data set, and some control parameters directing its operation. The output of IAMJREST is the recovered IAM data set. This section will provide the basics of using IAMJREST, along with examples. For additional information in IAMJREST, refer to IAM-Journal-Restore-Programs of the space, which has a complete description of the IAMJREST program.
Forward Recovery
There are two different types of recovery procedures that can be performed by IAMJREST. The first type is a forward recovery. A forward recovery will typically be used to recover a data set if the device it resides on is damaged or not operational, or if the data set has been seriously corrupted. A forward recovery is one where first the base data set is restored, then the IAMJREST utility is executed to apply the updates, inserts, and deletes from the journal to the base data set. For a forward recovery to occur, the journal must contain AFTER images for all the jobs that updated the data set. This is accomplished by specifying JRNAD=BOTH or JRNAD=AFTER on the IAM CREATE override statement.
Example E: Forward Recovery with Restore
Shown below is an example of performing a forward recovery. In the example, it is presumed that a failure occurred on the third day after the last backup. There are the two daily backups of the journal file and the current journal file that will be used for the recovery. These three data sets will be input to the recovery procedure. The data set being recovered will first be deleted and defined on a new volume, then it will be reloaded from the backup tape. After the restore, IAMJREST is executed with the three input files to perform the forward recovery. The failure occurred while running the job BATCHUP2. The recovery will be performed for all update jobs that ran up to and including job BATCHUP1. Once any other data sets are recovered, the BATCHUP2 job can be restarted.
Forward Recovery Preceded by a Restore (EX1088E)
//SYSPRINT DD SYSOUT=*
//IAMOVRID DD *
CREATE DD=&ALLDD,JRNAD=BOTH
/*
//SYSIN DD *
DELETE MY.IAM.KSD CLUSTER
IF MAXCC NE 0 THEN -
DELETE MY.IAM.KSD NOSCRATCH
SET MAXCC = 0
END
DEFINE CLUSTER -
(NAME(MY.IAM.KSD) -
OWNER($IAM) -
VOLUMES(NEWVOL) -
CYL(60 6) -
RECORDSIZE(100 256) -
KEYS(24 8) -
FREESPACE(5 20) -
SHAREOPTIONS(2 3) -
REUSE)
LISTCAT ENT(MY.IAM.KSD) ALL
/*
//RESTORE EXEC PGM=IDCAMS,COND=(0,NE)
//SYSPRINT DD SYSOUT=*
//BACKUP DD DSN=MY.IAM.KSD.BACKUP,DISP=0LD
//IAMFILE DD DSN=MY.IAM.KSD,DISP=OLD
//IAMINFO DD SYSOUT=*
//SYSIN DD *
REPRO INFILE(BACKUP) OUTFILE(IAMFILE) REUSE
LISTCAT ENT(MY.IAM.KSD) ALL
/*
//RECOVER EXEC PGM=IAMJREST,REGION=64M,COND=(0,NE)
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(50,10))
//SORTWK02 DD UNIT=SYSDA,SPACE=(CYL,(50,10))
//SORTWK03 DD UNIT=SYSDA,SPACE=(CYL,(50,10))
//IAMFILE DD DSN=MY.IAM.KSD,DISP=OLD
//IAMJRNL DD DSN=MY.IAM.KSD.DAILY.JOURNAL(-1),DISP=OLD
// DD DSN=MY.IAM.KSD.DAILY.JOURNAL(0),DISP=OLD
// DD DSN=MY.IAM.KSD.LOG,DISP=OLD
//IAMINFO DD SYSOUT=*
//SYSIN DD *
RESTORE FORWARD,TOJOB=BATCHUP1,TODATE=2009182
/*
Backout Recovery
The other type of recovery that can be performed is a backout (or backwards) recovery. This type of recovery backs out updates to a data set from the specified jobs or job steps. To perform a backout recovery, the journal must include BEFORE images. BEFORE images are written to the journal when either JRNAD=BOTH or JRNAD=BEFORE have been specified on the IAM overrides. A forward recovery can be done to restore a data set to the same point that can be done with a backout recovery. The backout recovery however, will generally be much faster than a forward recovery. This is because the data set does not have to be restored, and eliminates having to apply updates from potentially several days. Backout recoveries are also ideal if you only need to use IAM journaling to provide recovery for a few jobs or a job stream.
Example F: Backout Recovery
Shown below is an example of a backout recovery. In this recovery, all the updates for job BATCHUPD that was run on June 30, 2009 (2009.181) are removed from the IAM data set.
An Example of a Backout Recovery (EX1088F)
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(50,10))
//SORTWK02 DD UNIT=SYSDA,SPACE=(CYL,(50,10))
//SORTWK03 DD UNIT=SYSDA,SPACE=(CYL,(50,10))
//IAMFILE DD DSN=MY.IAM.KSD,DISP=OLD
//IAMJRNL DD DSN=MY.IAM.KSD.DAILY.JOURNAL(-1),DISP=OLD
// DD DSN=MY.IAM.KSD.DAILY.JOURNAL(0),DISP=OLD
// DD DSN=MY.IAM.KSD.LOG,DISP=OLD
//IAMINFO DD SYSOUT=*
//SYSIN DD *
RESTORE BACKOUT,JOBNAME=BATCHUPD,FROMDATE=2009181
/*
Example G: Identifying the Contents of a Journal
If you are unsure of what jobs or job steps have been journaled, the IAMJUTIL provides the SCAN capability. This command will read the specified journal data set, and print a line identifying each of the job steps that have records on the file. Included with that information is the time and date. This information may be helpful when setting up to run a recovery. The SCAN command also has a DETAIL operand, which provides a way to determine the complete contents of the journal data set, should that be needed. Usually, it is expected that the summary information identifying all the job steps that have updated the data set will be sufficient information.
Example of Identifying Contents of Journal (EX1088G)
//SYSPRINT DD SYSOUT=*
//IAMJRNL DD DSN=MY.IAM.KSD.LOG,DISP=OLD
//SYSIN DD *
SCAN SUMMARY
/*
To nullify LOGREPLICATE, FRLOG and LOGSTREAM parameters
You can see the following example to nullify the LOGREPLICATE, FRLOG and LOGSTREAM parameters.
//SYSPRINT DD SYSOUT=*
//IAMPRINT DD SYSOUT=*
//IAMOVRID DD *
CREATE DSN=<DATASET_NAME>,
NOLOGREPLICATE,FRLOG=NONE,LOGSTREAM=NULL
/*
//SYSIN DD *
DEFINE CLUSTER -
(NAME(<DATASET_NAME>) -
OWNER($IAM) -
VOL(<VOLUME>) RECATALOG )
LISTCAT ENT(<DATASET_NAME>) ALL
/*