Introduction to FDRABR Archiving and Superscratch

ABR

Program FDRABR (Automatic Backup & Recovery (ABR®)) automates the execution of FDR full-volume and data set backups and restores for the purposes of:

• Data availability - Creating backup copies of DASD data sets to protect against physical loss or logical damage.
• Space management - Identifying data sets that do not need to be on DASD (usually based on the last date they were used) and moving them to backups (called “archiving”), from which they can be automatically recalled if needed. Data sets that will never be needed again can be scratched without creating a backup.
• Disaster recovery - Creating backups from which all or part of your DASD data can be quickly recreated at a disaster recovery site.
• Tape Mount Management (TMM) - Maximizing tape volume usage by staging small tape data sets to a DASD buffer, then moving many of them to a tape volume.

These objectives are accomplished by the use of several ABR functions:

• This section describes ABR Archive (DUMP TYPE=ARC) and Superscratch (DUMP TYPE=SCR) that are used for “space management” and “tape mount management”.
• ABR Volume Backups (DUMP TYPE=FDR/ABR/AUTO/DSF) are found in FDRABR-Volume-Backups.
• FDRAPPL Application Backup (DUMP TYPE=APPL) is in FDRAPPL-Application-Backup.

ABR Backups

All the backup functions of ABR share these common characteristics:

• The backups are in standard full-volume (FDR) or data set (DSF) backup format, described in FDR-and-FDRDSF-Introduction. If necessary, data can be restored from ABR backups with FDR or FDRDSF, but ABR automates the restore process.
• Although ABR backups require a separate backup data set for each DASD volume processed, ABR automatically stacks multiple backup data sets on tape, creating multi-file tapes, to make best use of today's high-capacity tape volumes. If necessary, multiple output tape volumes are used. No special JCL is required since ABR handles the file creation internally.
• ABR backups to DASD are also supported. ABR automatically allocates the backup data sets on the backup volumes you specify. DASD backups are sometimes used with Archiving to provide fast recall of archived data, although customers with an Automated Tape Library (ATL) or Virtual Tape System (VTS) can get the same results.
• The output devices (tape and/or DASD) are specified in the ABR batch JCL. However, you only need to identify the output device; ABR automatically names and creates every backup data set and catalogs it if required.
• You can specify in the JCL that two copies of each backup are to be created, even though the input DASD is read only once. When Archiving this is often used to create one short-term copy on DASD for fast recall and one on tape for long-term retention, but it can also be used to create a copy for off site storage. In ABR these are known as COPY1 and COPY2 (see Archive-and-Superscratch-Job-Control-Requirements for details). The ABR utilities FDRTCOPY and FDRTSEL (FDR-and-ABR-Backup-Maintenance) can be used to create COPY2. Since an archived data set exists only in the backup files, two copies are recommended for Archive Backups.
• If multiple output devices are specified in the JCL, ABR automatically uses internal sub-tasking to process more than one DASD volume concurrently.

ABR archive backup

Although the price per megabyte of mainframe DASD storage has dropped considerably, the demand for DASD storage has increased in most installations, so DASD storage remains a significant part of the Data Center budget. DASD space must be used efficiently in order to justify its cost.

However, your DASD volumes may contain data sets that no longer need to be on DASD. In some cases, they are orphan data sets, created by some job or TSO user and never used again. Sometimes, they may be needed again in 6 months or a year, but do not need to be on DASD in the meantime. Sometimes data sets simply don't meet installation standards, such as uncataloged data set

ABR Archive Backup is used for space management, by moving data sets that are not currently required to be online from expensive DASD to less expensive medium such as tape or highly-compressed DASD. The data sets are moved by backing them up with an FDR data set backup, then deleting them from the DASD. They can be recalled to DASD whenever they are needed.

The DASD volumes to be processed by Archive must contain an ABR Model DSCB. See Overview-of-FDRABR-Volume-Backups for details on creating and maintaining the ABR Model DSCB. Archive does not record anything in the ABR Model DSCB, but there is a flag enabling or disabling each volume for archiving; you can protect whole volumes from archiving by disabling the flag, or, if you prefer, you can enable the flag only when you are about to run archiving on a given volume.

The data sets moved from DASD by ABR Archive are recorded in an Archive Control File (ACF). The ACF is a Direct Access (DA) data set that can record several hundred such archived data sets in a single track. There is usually one common ACF for all archived data sets. For each archived data set, the ACF records the ABR backup file which contains the data set (device type, volume serials and file number), the expiration of that backup, and some basic descriptive information about the data set (for example, DSORG and size). If two copies of the backup were made, both copies are recorded in the ACF. Since the ACF is cumulative, the most recent entries are recorded at its end.

You can specify various criteria for selecting the data sets to be archived, such as “not cataloged” or “not used in the last 15 days”. Different criteria can be specified for various data sets (by volume, by data set name mask, or a combination). However, the most commonly used criteria is days since last usage, since this automatically identifies data sets that have not been used recently and are probably candidates for archive. For SMS-managed data sets, you can optionally select data sets based on the attributes of the SMS management class assigned to each data set.

ABR Archive is often used to put one backup copy on DASD and a second on tape. The DASD backups are in a highly compressed format, taking much less room than the original, yet they can be recalled quickly since no tape mount is required. Most installations expire the DASD backup after a short time (15-30 days) and either delete it or move it to tape. If you have an Automated Tape Library (ATL) or Virtual Tape System (VTS), restore from tape backups is almost as fast, so DASD output is probably not needed.

FDRINSTANT

FDRINSTANT is a separately priced member of the FDR family that enhances FDRABR.

ABR archive restore

Data sets that have been archived can easily be restored by an ABR batch job. Control statements specify the data set names that are required. ABR will search the Archive Control File backwards, in order to find the most recent copy of each requested data set. If a data set has been archived more than once, you can specify which copy you want. Multi-volume data sets are handled automatically, restoring all pieces of the data set.

The data from the ACF provides all the information necessary to locate and restore the requested files. You will usually use an option that allows ABR to dynamically allocate the backup files on tape or DASD without JCL. ABR will sort the required backup files by volume serial and file number (for tape) so that each tape is mounted only once, even when many data sets are being restored.

ABR Auto-Recall

When a data set is archived, it can be marked as eligible for Auto-Recall. With Auto-Recall, the catalog entry for an archived data set is left in the z/OS catalog, with a special indicator that indicates that it was archived. When an archived data set is referenced by batch JCL, dynamic allocation, or TSO, the ABR Catalog Locate exit detects the indicator and automatically “recalls” the data set to DASD before it is needed. Auto-recall makes the use of archived data sets transparent to the job or user, other than a short delay in execution while the data set is recalled.

TSO users normally have the option of bypassing the recall, and have a choice of waiting for the recall to complete (foreground) or doing the recall in a separate started task while they do other work (background). TSO users may also add recall requests (and some other ABR requests such as Archive) to a “remote queue” data set that is processed by a batch ABR job.

Tape Mount Management

ABR Archive can be used to implement the IBM concept of Tape Mount Management (TMM) where data sets that are normally on tape are directed to SMS-managed DASD instead, and then moved to tape using ABR. This allows many such data sets to be stacked on tape and uses far fewer tape volumes. IBM has “Redbooks” that describe TMM in detail (using IBM software, of course), but section 70 “SMS” contains additional information on implementing TMM with ABR Archive.

ABR Superscratch

ABR Superscratch operates exactly like ABR Archive, but it does not backup the selected data sets, it simply deletes them. Superscratch can be used to delete data sets that you know will never be needed again (such as temporary data sets). Superscratch does not use the Archive Control File, since there are no backups to record and no way of recovering data sets that were deleted in error (unless you have another backup of the data, such as an ABR Volume Backup or FDRAPPL Application Backup).

Simulation of Superscratch (see below) is recommended, to insure that valuable data sets are not inadvertently scratched.

There is a separate flag in the ABR Model DSCB to enable a volume for Superscratch.

All functions of ABR Archive and Superscratch can be run in simulation mode, allowing you to verify that the correct data will be selected when run for real. Testing with simulation is especially important with Superscratch since an error in control statements may cause irreversible lost of data. Data sets Archived in error can always be restored.

ABR Utilities

ABR includes a utility program, FDRARCH, for maintaining the Archive Control File. Since entries in the ACF are not automatically deleted when they reach their expiration dates, you must at least periodically run the REORG function of FDRARCH to purge those obsolete entries and make room for new ones. FDRARCH documentation starts in Archive-Maintenance-Utility-FDRARCH.

FDRTSEL is a utility that can be used to move or copy ABR Archive backups. Among other things, you can use it to create a COPY2 backup from a COPY1, or to move a backup from DASD to tape. FDRTSEL is described in FDRTSEL-Introduction.

FDRABRP (FDRABRP-Standard-Reporting-Facility) is used for simple reporting on archived data sets, and FDREPORT (FDREPORT-Generalized-Report-Writer) can be used for more sophisticated reporting. The ABR ISPF dialog called SRS (Search, Report, and Services) can also display information on archived data; it is in FDRSRS-Search-Report-and-Services-Dialog.

Tape format and naming conventions

The backup files created by ABR Archive Backups are in standard DSF format. In this format, each backup file can contain data only from one DASD volume. Therefore, if multiple DASD volumes are to be processed, ABR must create multiple backup files on the output tape or DASD. Unless only a single DASD volume is processed in an ABR step, ABR will always create multiple backup files. If the amount of data archived from a given DASD volume is excessive (over 4095 tracks by default), ABR may create multiple backup files, each with a subset of the data sets, in order to improve restore performance.

Since ABR must be able to uniquely name each tape file, and must be able to record the backup file in a way that it can easily be retrieved, ABR uses a special naming convention for the Archive Backup files. The name contains the DASD volume serial, the date of the archive and a uniqueness character in case data is archived multiple times per day, so each ABR Archive Backup from a given DASD volume will have a unique name.

The format is abrindex.Vvvvvvv.bnyydddx where:

abrindex

Is the ABR hi-level index from the FDR Global Options. It is usually “FDRABR”. It can be changed when ABR is installed but must NOT be changed once you start using ABR.

vvvvvv

Is the volume serial of the DASD volume from which the data sets were archived. ABR creates one or more backups file for each DASD volume processed in a given ABR run.

n

Is the copy number. ABR always creates COPY1 and can optionally create COPY2.

yyddd

Is the Julian date of the archive job (5 digits).

b

x

Are qualifiers added to make the name unique when multiple Archive Backups are created for the same DASD volume on the same Julian date. “b” is initially set to “B” while “x” cycles from A to Z and 0 to 9. When “x” reaches its maximum value (9), then “b” resets to “D” through “J” while “x” cycles again. This allows for a maximum of 255 Archive Backups per DASD volume per day. However, this name change occurs only if the name ABR is trying to create is already in the ABR catalog (see below); it may create multiple uncataloged backups with the exact same name.

Examples:

The backup files created by ABR Archive Backup are usually not cataloged. For most such backups, all the information required is stored in the data set records in the Archive Control File. When restoring, information about the backup file to be read is obtained from the ACF so no entry in the ABR catalog is required. There are several exceptions:

• The ACF can only record five tape volumes for the backup file containing a specific data set. If the Archive backup of data from one DASD volume required more than five tape volumes, the backup is cataloged to record the additional volume serials.
• In case tape management catalog control is being used for Archive tapes, the first file created on a tape by a given Archive job is cataloged. Also, any time that the volume serial list of the current backup file is different from the serial list of the previous file (for example, when the backup overflows the current tape volume and a new tape is mounted), that backup is cataloged.
• If EXPDT=99000 (catalog control for many tape management systems) is specified on the TAPEx DD Statement or TAPExx DD Statement, all backup files created on that DD statement are cataloged.
• Archive backup files on DASD are always cataloged.
• The ARCCAT=ALL operand can request that all archive backup files be cataloged.

If cataloging is required, the high-level index chosen (for example, “FDRABR”) must be assigned to a catalog designated for ABR use, called the “ABR catalog”. Since it is used mainly for Volume Backups, it is described in Overview-of-FDRABR-Volume-Backups.

As long as no more than four tape volumes are used for a given Archive Backup file, ABR will record a hardware “block id” in the Archive Control File for each backup file created on a cartridge drive. During restore, this block id is used to invoke a high-speed search at OPEN time to position directly to the beginning of each backup file. This can significantly reduce recall times, especially on high-capacity tapes.

Backup retention and tape management

Depending on the requirements for retention of Archived data sets established by your installation, you may choose to control the retention of ABR Archive Backups in several ways:

• You can let your tape management system expire backups based on a retention period or expiration date. This is known as “date control”. If you are doing Archive Backups to DASD, the retention/expiration is stored in the F1 DSCB of the backup data set, but the backup may not be deleted unless you run ABR Superscratch with the SELECT ABRBKUP statement against those volumes; FDRARCH can also delete expired DASD backup data sets during Archive Control File reorganization. For Archives that are on cloud storage, see the "Note" below.
• You can let ABR decide when to expire backups. When the REORG function of the utility FDRARCH is executed to reorganize an Archive Control File, it will delete obsolete entries, based on the expiration date recorded in the ACF or other criteria you specify. When the last data set entry that points to a given backup file is deleted, FDRARCH will uncatalog the backup file; if it is on DASD, it is also scratched. ABR has no formal interface to any tape management system, but if you use the “catalog control” feature of your tape management system the obsolete tapes are automatically scratched. For Archives that are on cloud storage, see the "Note" below.
• One way to implement the above is to tell FDRARCH to delete the entry for any data set that is no longer cataloged for Auto-recall in the system catalogs. If you Archive GDG generations, old generations are deleted when they “roll-off” from the GDG. Other data sets can be deleted from the Archive Backups simply by uncataloging the data set. This also deletes data sets that have been recalled, even if they were re-archived. When all the data sets on a given backup have been uncataloged and deleted, FDRARCH uncatalogs (and scratches on DASD) the backup.
• You can define ABR Archive as an External Data Manager (EDM) for your tape management system that allows ABR to directly control when tapes are scratched. For some tape management systems, this will reduce the number of records in the tape management database. Details of EDM are in Data-Movement-Between-Different-DASD. 
• If you have no tape management system, you must establish the necessary manual procedures for identifying and expiring ABR backups.

When an Archive Backup is taken, an expiration date is assigned. It can come from several sources, in this order of priority:

• If the RETPD= operand is specified on the ABR DUMP statement, ABR uses that to calculate an expiration date that is assigned to all COPY1 (TAPEx) backups created in that ABR step. If the RETPD2= operand is not specified but COPY2 (TAPExx) backups are also created, the same expiration is assigned to those backups.
• If the RETPD2= operand is specified on the DUMP statement, ABR uses that retention period to calculate an expiration date that is assigned to all COPY2 (TAPExx) backups created in that ABR step.
• If the TAPEx DD Statement or TAPExx DD Statement pointing to the backup tape contains the EXPDT= operand, the date specified is assigned to all backups written to that DD statement. For some tape management systems this value can also be a keyword; for example, EXPDT=99000 indicates “catalog control” to many tape management systems.
• If the DD statement contains the RETPD= operand, z/OS uses that to calculate an expiration date, which ABR assigns to each backup written to that DD statement.
• If none of the above RETPD/EXPDT operands are given, the default retention period of 365 days (1 year) is used to calculate an expiration date which is assigned to each backup.

By whatever means it is calculated, the expiration date is recorded in the Archive Control File record for each data set included in that Archive Backup file. If both COPY1 and COPY2 of a backup are being created, the expiration of each copy is recorded separately in case they are different.

The expiration date is recorded in the tape labels of the backup file and is also recorded by your tape management system, if you have one. If the expiration date is a real date (not a keyword such as 99000) then your tape management system probably returns the tape to scratch status on that date.

Your tape management system usually scratches a backup tape (or multi-volume tape set) when every backup file on the tape has reached its expiration date. Since the backups created on a tape by a given ABR run usually have the same expiration, this is not a problem. However, there are options to place backups with varying expirations on the same tape, so you should beware of creating files on the same tape whose expirations are widely separated, since this may cause the tape to be retained longer than necessary. Since your tape management system and the Archive Control File each store the expiration of a backup file, you must insure that they do not contradict one another; if the expiration is changed in one, it must also be changed in the other.

If you create both COPY1 and COPY2 of your backups, you can use one copy for on site recalls and the other copy for off site recalls at a disaster recovery site. Most tape management systems include vaulting support, allowing you to select tape to be sent off site and returned when no longer required. Most users send COPY2 off site. Since the data set name used by ABR Archive Backup (described earlier) includes the copy number, you should be able to easily send COPY2 backups off site while retaining COPY1 on site.

SMS-managed volumes

On DASD volumes that are managed by SMS, you can choose whether ABR is to select data sets to Scratch or Archive by normal ABR criteria specified on ABR control statements, or by the attributes of the SMS management class associated with the data set. If the management class is chosen, and both SMS-managed and non-SMS volumes are processed in the same ABR run, ABR criteria is used for the non-SMS volumes and ignored for SMS-managed volumes (with some exceptions). A complete explanation of Archive and Superscratch on SMS-managed volumes is in System-Managed-Storage-SMS.

There are advantages and disadvantages to each technique. ABR's Archive and Superscratch selection criteria are generally more flexible and offer more options than SMS, but may require many control statements if varying criteria are to be used for different data sets (usually controlled by data set name). SMS management classes can be associated with data sets at the time they are created as specified by the user or by installation ACS routines, independent of the data set name.

Even on SMS-managed volumes, the decision over whether to create an archive backup on DASD or tape (or both) is still controlled by ABR. However, if you are archiving based on SMS management class attributes, you can optionally use the attributes of the SMS management class associated with each SMS-managed data set to calculate expiration dates for COPY1 on DASD and COPY2 on tape. Details are in Archive Expiration in System-Managed-Storage-SMS. This function allows you to use a global expiration for the tape file (such as EXPDT=99000 for catalog control), while assigning each data set a real expiration date based on its management class. With this option (the SMSEXPIRE= operand), tape management will record a global expiration for the whole backup file, using the rules above, but the ACF entry for each archived data set in that backup will record a specially calculated expiration. When all data sets reach their expirations, FDRARCH will uncatalog the backup file and make it eligible for scratch.

Archive to DASD

ABR Archive Backups can be directed to DASD, instead of tape. This is usually used to provide a quick means of recalling an archived data set if it is needed in the first few days after it was archived; beyond that initial period, recall from tape is used. If you have an Automated Tape Library (ATL) or Virtual Tape System (VTS), you may want to do all of your archiving direct to tape, since the ATL or VTS can quickly mount required tapes and may be nearly as fast as recall from DASD.

Archive-and-Superscratch-Job-Control-Requirements details the JCL requirements for directing Archive Backups to DASD. The TAPEx DD Statement point to one or more DASD volumes that are designated to hold Archive Backups. ABR automatically allocates backup data sets on those DASD volumes, using the naming convention described earlier. These DASD volumes are usually specified using the ABR POOLDISK feature, which allows ABR to monitor the space available on each volume and more efficiently use the DASD.

Backups of data sets to DASD usually take far less DASD space than the original data sets occupied. The original inter-block gaps are eliminated, only used tracks are backed up, and the backup can be compressed using FDR compression (described in Compress Option in FDRABR-Processing-Options-and-Requirements and recommended for backup to DASD).

You can use the FDRTSEL utility (FDRTSEL-Introduction)to move DASD backups to tape. If a DASD backup expires and is not moved to tape, the REORG function of the FDRARCH utility (see FDRARCH-REORG-Statement) deletes and uncatalogs obsolete DASD backup files.

Archive backup execution

To execute ABR and perform Archive Backups, you must create ABR job streams and execute them at appropriate times. Archive-Backup-Examples contains examples of such job streams that you can customize for your installation. If you have an automation or scheduling product in your installation, you may want to use it to schedule daily ABR Archive Backups at appropriate times.

Here are several scenarios for ABR Archive Backups; you can pick the one that best meets your installation's requirements:

• Archive direct to tape - If the TAPEx DD Statement points to tape, then the COPY1 backup is on tape. If you include the optional TAPExx DD Statement, COPY2 is created on tape as well. You probably want to assign a long-term retention to both copies. You might want to send COPY2 to off site storage so that archived data sets can be recalled at a disaster recovery site.
• Archive to DASD only - If the TAPEx DD Statement points to DASD volumes (as described above), ABR allocates the COPY1 backup on DASD. Since the DASD space allocated to ABR is limited, you probably want to specify a fairly short retention period (for example, 7 days, 14 days, 31 days) for the DASD backups. During that period, data sets can be recalled quickly since no tape mount is required. FDRTSEL can be executed (FDRTSEL-Introduction) to move the COPY1 backups to tape with a longer retention as they approach their expiration date on DASD. FDRTSEL can also be used to create a COPY2 on tape from the COPY1 on DASD.
• Archive to DASD and tape - If the TAPEx DD Statement points to DASD and the TAPExx DD Statement points to tape, ABR allocates the COPY1 on DASD and the COPY2 on tape. You probably want to specify a fairly short retention period for COPY1 and longer retention for COPY2 (during recall, ABR automatically switches to COPY2 if COPY1 has reached its expiration date as recorded in the Archive Control File). You may want to execute FDRTSEL to move the COPY1 backups to tape with a longer expiration so that you are not left with a single backup of the archived data.

If you have an Automated Tape Library (ATL) or Virtual Tape System (VTS), you may want to put COPY1 to an ATL or VTS tapes, since they are automatically mounted when required, providing recalls that are almost as fast as recall from DASD.

For data set recovery from Archive Backups, there are several alternatives:

• You can permit users to restore data sets directly by submitting their own RESTORE TYPE=ARC job. This may require mounting backup tapes for each data set required. If FDR security is enabled, users will need UPDATE or ALTER authority to the data sets
• You can allow users to add requests to a remote queue (see details in Remote Queue FDRABR-Processing-Options-and-Requirements). In this case Operations (or your automation software) must submit a RESTORE TYPE=ARC job at regular intervals (for example, every hour, every 4 hours) to process any requests on the queue. If multiple data sets are being restored, this minimizes tape mounts. If FDR security is enabled, users need UPDATE or ALTER authority to the data sets
• You can require that users communicate their data set restore requests directly to someone in the data center, who submits the required RESTORE TYPE=ARC job.
• If the Auto-recall option was in effect when the data set was archived, users can recall archived data sets simply by referencing them from TSO or a batch job. The data set is automatically recalled from the most recent Archive Backup tape on which it appears. Data sets that can be Auto-recalled must all be recorded in a common Archive Control File (ACF), so one ACF is usually used for all Archive Backups.

Archive backup management

Unlike Volume Backups, which are usually kept for a fairly short period of time (so many weeks, or so many generations), Archive Backups may be kept for a long time. The default retention is one year, but many installations make it longer. If you do archiving frequently, you could end up with hundreds or thousands of tape volumes containing active Archive Backups.

The problem is compounded because much of the data on those backups may be obsolete. For data sets that have been deleted from the Archive Control File by FDRARCH (for expiration date, uncataloged, and the rest), pointers to the backups of those data sets no longer exist. However, as long as one archived data set is still recorded pointing to an Archive Backup file, that file is retained. Archive creates multiple files on tape, and may create multi-volume aggregates, and most tape management systems do not scratch any tapes in the aggregate until every file is eligible for scratch.

Therefore, as Archive Backup tapes get older, the chances are that the amount of active recallable data on the tapes will decrease.

This problem is addressed by the ABR utility FDRTSEL, described in FDRTSEL-Introduction. FDRTSEL can copy your old Archive Backup files to new tape. Only the active files on an input tape are copied; for example, if a tape contains 10 files, but the Archive Control File has active records for only three of those files, only those three are copied to the new tape.

FDRTSEL can take files from various input tapes and combine them on one output tape, reducing the amount of tape required.

Normally, FDRTSEL copies the entire contents of a tape file it selects, possibly copying data for data sets that are no longer active in the Archive Control File. If the ARCEDIT option of FDRTSEL is used, it copies only the data belonging to data sets still recorded in the ACF; this reduces the amount of tape required for output even further.

Regular use of FDRTSEL, perhaps monthly, helps keep the number of tape devoted to Archive Backups to a minimum. However, its effectiveness is dependent on your techniques for purging data sets from the ACF. For example, if you do not purge uncataloged data sets or expire based on SMS management class attributes, then all the data in a given backup file always expires on the same day and ARCEDIT is useless.

Archive expiration

FDRABR Archive accepts expiration years from 2028 to 2107 when recording information in the Archive Control File.