Defining IAM Files

Before using an IAM dataset, it must be defined. This define process is identical to what is required for VSAM datasets. During the define process, IAM allocates the DASD space for the dataset, catalogs the dataset, and stores the file attributes within the dataset itself. IAM datasets can be defined by using IDCAMS, through JCL DD cards, or using a variety of methods under TSO, including through the IAM ISPF panels. Many other software products that are used to define VSAM datasets will generally also be able to define IAM datasets. This section will provide information on parameters and examples to define IAM datasets with IDCAMS, through JCL, and under TSO.

How to convert a data set to IAM?

For a dataset to become IAM instead of VSAM, an indication must be provided on the file definition indicating that the file is to be an IAM file. The ways to indicate this are:

• Add the parameter OWNER($IAM) to the IDCAMS DEFINE command.
• Change the dataset name to include the literal $IAM somewhere within the name.
• For SMS managed IAM datasets, use an SMS Data Class or Storage Class with the literal $IAM as part of the class name.

For most datasets, all that is required to implement IAM is to change the file definition using one of the above techniques. Any of the above methods can be specified on an IDCAMS DEFINE. For JCL definition of an IAM dataset, the OWNER parameter is not available, so either the dataset name has to include $IAM or it must be a part of the assigned Data Class or Storage Class name. Most installations select one method as their preferred method, based upon their internal dataset management or accounting requirements. While many installations have chosen the OWNER($IAM) technique, the alternative of placing $IAM within the dataset name has the advantage of making identification of IAM datasets very easy and many installations have chosen this route as well.

Basic define parameters

For any IAM dataset, there is certain basic information that must be provided. This is usually provided through keywords specified either on the IDCAMS DEFINE command, or as JCL. Other sources of these attributes are an SMS Data Class, or from another IAM or VSAM dataset as a model. The basic required information for all types of IAM datasets include the following:

• Dataset Name
• Indication that file is an IAM file, like OWNER($IAM)
• Volumes on which dataset is to reside
• Quantity of DASD space required
• Maximum record size
• Type of dataset (KSDS, ESDS, RRDS, AIX, or PATH)
• Key length and offset (RKP) for KSDS (INDEXED) and AIX type of files

Additional information that can be provided includes free space, share options, expiration date, and Key Label. Several of the other VSAM file attributes can be specified, such as SPEED or RECOVERY, however, they are not relevant to an IAM dataset and will be ignored. Certain attributes unique to IAM can be specified via IAM overrides, which include data compression, IAM file format, and default buffering range. The various unique IAM attributes can also be set as installation defaults in the IAM Global Options Table.

If the IAM allocation encounters any errors, the error messages will appear on the JES job log, with the z/OS allocation messages (SYSMSGS) and on the IDCAMS SYSPRINT, if it is available. Due to the manner in which IDCAMS prints messages on SYSPRINT, the error messages from IAM will precede the DEFINE command. IDCAMS will also print out additional error messages after the DEFINE, performing an analysis on the return codes set by IAM. Whenever possible IAM uses the VSAM return codes that most clearly indicate the actual problem, although that is not always possible. Always refer the IAM and related allocation error messages for the most precise problem determination possible. Refer to IAM-catalog-return-codes for the meaning of the return codes and reason codes returned from IAM after a DEFINE has been processed.

Considerations for defining an IAM data set

In general, one can easily convert a VSAM cluster to IAM, just by modifying the DEFINE, as described above. Because IAM has a different file structure, and is allocated as a non-VSAM dataset or a VSAM Linear dataset for encrypted files, there are some differences between IAM dataset allocations and VSAM allocations that may affect a few of your datasets.

Data set type

Non-encrypted IAM datasets are stored as z/OS non-VSAM type of datasets. They are stored in a DSORG=PS type of file, and can be basic file type, a Large Format Sequential dataset, or an SMS Extended Format dataset. The latter two types are necessary if the file will need to use more than 64K (65,535) tracks on any single volume.

Non-encrypted IAM files default to DSNTYPE=LARGE, except in the following circumstances:

• If the data set is an IAM Compatible format data set, then it can only be a DSNTYPE=BASIC data set, and will default to that value.
• If the Global Option SMSEXTOK is ENABLED (which is the default), data sets will use SMS Extended Format if it goes through the initial SMS ACS routines with that specification.
• Specification of the IAM CREATE Override value DSNTYPE will supersede the above default processing.

DSNTYPE=LARGE data sets are essentially identical to DSNTYPE=BASIC files except that they can exceed 64K tracks per volume, and are so indicated in the VTOC along with an additional byte for the last block pointer.

Pervasively encrypted IAM files are stored as VSAM Linear datasets. IAM reports will display the file format as- ENCRYPTED. Encrypted IAM files must be defined with an SMS Data Class that has Extended Format and Extended Addressability enabled.

EAV support

IAM Version 9.0 and above provides support for IAM data sets to reside on EAV volumes. To make use of EAVs, IAM data sets must either specify the EATTR(OPT) parameter on the DEFINE CLUSTER, be assigned to a Data Class with the EATTR(OPT) parameter specified, or have the IAM Global Option EAV enabled.

Selecting data set type

IAM supports an IAM CREATE Override of DSNTYPE=. Valid values are BASIC, LARGE, or EXT. Specification of this override will supersede any data set type selected within the Data Class or otherwise defaulted to.

BASIC indicates the use of a standard non-VSAM type of data set, LARGE indicates to make the data set a Large Format Sequential data set, and EXT indicates to use SMS Extended Format.

DSNTYPE=BASIC data sets are limited by z/OS to 4,369 cylinders, which is equal to 65,535 tracks per volume.

Using extended format data sets

IAM datasets that are SMS Extended Format data sets can have up to 123 extents per volume. To obtain an Extended Format data set, the DFSMS data class (DATACLASS) must specify “Dataset Name Type” of “EXTENDED REQUIRED or EXTENDED PREFERRED or use the IAM CREATE Override DSNTYPE=EXT, and have a storage class (STORCLASS) with a Sustained Data Rate (SDR) of either blank or 0.

The disadvantage of using Extended Format data sets is that each block has an additional 32 bytes of system data appended to it, so such data sets may use more DASD space than when not in a SMS Extended Format. Because of this, it is recommended that customers use the Large Format Sequential data sets when possible, instead of the SMS Extended Format. Setting the IAM Global Option DISABLE=SMSEXTOK will prevent use of SMS Extended Format data sets, except by an explicit DSNTYPE=EXT IAM CREATE Override, when the data set is defined. This is generally a safe way to automate the use or Large data sets without requiring any JCL or SMS class changes. Users that are planning on setting this option may also want to consider setting the ENABLE=SPACEADJ flag to compensate for DSNTYPE=LARGE data sets being limited to 16 extents per volume.

Because SMS appends 32 bytes of system data to each block, IAM will automatically adjust its block size calculations to consider this requirement; however, the resulting data set may require more DASD space than a non-Extended Format data set. IAM uses BSAM WRITE macros when an Extended Format data set is being loaded, therefore, the EXCP counts for file loads of IAM Extended Format data sets will be the number of blocks written to DASD, not the actual number of EXCPs issued, which will result in higher EXCP counts for file loads of these data sets. BMC recommends that customers use Large Format datasets when possible instead of the SMS Extended format datasets for non-encrypted files.

Space adjustment

To assist users in moving from SMS Extended Format data sets to DSNTYPE=LARGE format, a space allocation adjustment function has been implemented. Use of this feature is entirely optional, and will only be invoked when the SPACEADJIAM Global Option is enabled and the SMSEXTOK is disabled. If the define process encounters an SMS Extended Format data set that it is changing to a DSNTYPE=LARGE, the define process with SPACEADJ enabled will alter both the primary and secondary space amounts by the multiplication factor specified by the Create MAXSECONDARY override or Global Option value. This function is intended to compensate for the 16 extent per volume limitation that DSNTYPE=LARGE has by increasing the amount of space requested per each extent. The secondary reason for this function is that the IAM Secondary Space Adjustment function is limited on SMS managed volumes to one adjustment per job step.

If a data set has had it’s space adjusted by the Space Adjustment function, then it will not be subject to further secondary space adjustment. The maximum amount of the adjustment is 4,369 cylinders which is 64K tracks, except when the NOLIMSEC and the EAV Global options are both enabled, which will remove that limit. An example of how this would work is; with an allocation of 100 cylinders primary, 50 cylinders secondary and the default Create MAXSECONDARY of 10, then the primary would be adjusted to 1000 cylinders, and the secondary would be adjusted to 500 cylinders.

DASD space consideration

IAM data sets typically require 30% to 70% less disk space than your existing VSAM clusters. IAM data sets use DASD space more efficiently. The compressed index and advanced internal structure usually result in about a 20% to 40% reduction of disk space compared to a similar VSAM cluster. IAM's data compression may provide an additional 20% to 50% reduction in disk space for most data sets.

Customers can consider using IAM hardware compression instead of IAM software compression. IAM will automatically generate a hardware compression dictionary using the first several records that are loaded if hardware compression is indicated. Users that are interested in the best possible results from hardware compression should consider building their own customized dictionary for the desired data sets, using the IBM provided REXX EXEC. Information on using that process is described in Using-hardware-compression.

In an effort to conserve disk space and prevent over allocation, IAM releases space that is unused and has not been reserved after the file is initially loaded for non-encrypted data sets. This is done automatically when secondary space is specified. If you want to override IAM's default of releasing the over allocated space, see IAM Override statements. The keyword is RELEASE=NO. After a file has been loaded, a LISTCAT ALL will show you the exact number of tracks an IAM file is using and has allocated. BMC recommends that when converting VSAM files to IAM, initially retain the original VSAM space allocation values. After observing the IAM space requirements, the space allocation can be adjusted if so desired.

Multivolume consideration

For non-encrypted data sets IAM utilizes standard z/OS services to acquire additional DASD space. Because such data sets are non-VSAM, the rules and mechanisms for acquiring additional space for multi-volume data sets are different than VSAM. When IAM needs additional space, it issues the z/OS EOV (End of Volume) service to acquire additional DASD space. For encrypted Linear data sets IAM uses Media Manager Extend invocations to acquire additional DASD space. The only input IAM can provide is a space quantity, by specifying the desired secondary quantity in the JFCB. IAM will attempt adjustments on the secondary quantity, as per the MAXSECONDARY and MULTIVOLUME parameters. There are some circumstances where this secondary value is not honored, particularly for DFSMS managed datasets, in which case the secondary value that was indicated when the data set was originally defined will be used.

The basic rules for IAM data sets are that, the primary space quantity has to be available on the first volume. Then, as new extents are acquired, additional space will be acquired on the current volume, unless the data set has sixteen extents (non-SMS) or 123 extents (SMS) on that volume, or there is insufficient space to satisfy the request. Then z/OS will switch to the next candidate volume. When the next candidate volume is explicitly named, which is typical for data sets not managed by SMS, there must be sufficient space on the next candidate volume for the requested secondary quantity. In other words, z/OS non-VSAM EOV cannot skip over a candidate volume due to insufficient space and go to the next. Attempting to do so will cause a SE37-08, resulting in an IAMW13 File full message.

There are some exceptions to the above described processing. If the file is SMS managed in a guaranteed space Storage Class then the primary allocation is made on each volume when the data set is defined. If the secondary space is zero for non-encrypted files, then once the primary allocation is used, one volume IAM will be switched to the next volume. For encrypted files, under the same multi-volume secondary space is zero circumstance, IAM will assign a secondary space value equal to 25% of the primary space value, because the next volume will not be used until no further extents can be acquired on the current volume.

When all the allocated space is used, attempts to add more data will fail with a file full error. When a secondary quantity is specified then additional extents will be acquired on the current volume, providing there is space to do so, until the data set has reached either the limit of 16 extents on that volume, or has run out of space. IAM will then be switched to the next volume. IAM basic or large type data sets can acquire up to 944 extents across the maximum of 59 volumes. An IAM compatible data set can not exceed a total of 255 extents.

A second exception occurs when the IAM file is a SMS Extended Format data set. The SMS Extended Format datasets can have up to 123 extents per volume. Therefore, IAM data sets that are SMS Extended Format will not be subject to the IAM space adjustment function. The SMS Extended Format data sets are still limited to a maximum of 59 volumes.

A third exception occurs, when the file is defined as multi-volume, not SMS managed and the user has specified a secondary quantity of zero. For systems that have SMS active, IAM will treat this type of allocation like an SMS guaranteed space request. The primary quantity is allocated on each volume when the file is defined. When all the allocated space is used on one volume, then the allocated space on the next volume will be used. Unfortunately, this technique does not work on systems that do not have SMS active. So, for those systems, IAM will set the secondary to be the same as the primary. This usually results in only the primary space being allocated on the first volume, and then potentially multiple extents on the second and subsequent volumes.

Reorganizing, reloading, restoring, or snap copying into a multi-volume IAM data set without deleting and redefining is not recommended, because that can result in some strange space distribution across the volumes, and may result in other problems. For restore or copy, it is recommended that the utility being used, perform the allocation on the target volumes.

The processing done by z/OS EOV when using a previously existing file is different than for new files, and can result in secondary allocations being different than expected. This may also cause problems when restoring an IAM data set from a backup taken prior to the reorganization.

DFSMS support

IAM provides full support for SMS managed IAM data sets. By definition, to be eligible for an SMS managed volume the dataset must be assigned a Storage Class. The DATACLASS, MGMTCLASS, and STORCLASS can be explicitly provided on the DEFINE command or selected by the ACS routines. The Data Class can provide file characteristics for the file being defined including record length, key length, key offset, share options, free space, and others, eliminating the need to specify those values explicitly on the DEFINE. As per SMS rules, the options in the Data Class will be used, unless explicitly overridden on the DEFINE command. IAM files on SMS managed volumes will be cataloged with the class names.

DFSMS ACS routines

A special consideration for IAM files defined with IDCAMS in an SMS environment is that the ACS routines will be invoked twice for the define request of any non-encrypted file. The first time is for the VSAM define process. If the data set is to be a non-encrypted  IAM data set, then the dynamic allocation issued by IAM will result in a second call to the ACS routines as a non-VSAM data set. The exceptions to this are when files are allocated through JCL or files being allocated through the IAM ISPF panels. In the exception cases, IAM calls DADSM directly so only the first VSAM call is performed.

When IAM issues the dynamic allocation request, IAM will specify whatever SMS classes have been assigned to the data set at the point in time that the intercept occurred. IAM will issue the dynamic allocation request with a DDNAME of @#$IAM. This will provide a method for customers to use in their DFSMS ACS routines to identify IAM data set allocations using the &DD read-only variable.

There are a few exceptions to using the DDNAME=@#$IAM in the allocation. These include the following circumstances:

• For non-DFSMS allocations when the VAM option has been enabled.
• For data sets being “defined” through JCL.
• If the job step already has a DD card with the @#$IAM name specified.
• Encrypted files are not converted to non-VSAM. So, there is no dynamic allocation request and DDNAME=@#$IAM is not used.

One of the problems that can occur is, an installation gets established as an SMS Storage class, such as STORCLASS(NONSMS), which users can code to prevent a data set from being SMS managed. That storage class name is nullified on the first pass through the ACS routines, causing the file to be unmanaged and preventing IAM from seeing that original storage class specification. When the ACS routines are called again from the dynamic allocation issued by IAM, because the STORCLAS is null, the ACS routines assign the data set to an SMS managed STORCLAS. This problem is resolved by the use of the IAM STORCLAS Global Option. By setting that option to be the non-managed storage class, (example, NONSMS), IAM will pass that as the STORCLAS on any allocations which do not have one at the time IAM intercepts the request.

Another problem can occur if the ACS routines decide on the second pass that the data set is not to be a DFSMS managed data set. This will cause the dynamic allocation to fail, because the volumes that are passed, are DFSMS managed volumes. To prevent this, the ACS routine must not nullify that STORCLAS for an IAM file that has a STORCLAS specified. The STORCLAS can be changed to a different STORCLAS if desired.

A third consideration for the ACS routines is that some installations have set their ACS routines to perform different actions if some of the classes are already specified on entry to the routine. This is usually due to installations wanting to limit the external use of SMS classes by their users. Because of this, ACS routines with code that checks for the pre-existence of SMS classes, and performing different actions, could result in IAM files being assigned to different classes than expected or desired.

The main point here is that the developer of the ACS routines, particularly the Storage Class routine, must be aware of how IAM file allocations work and must code the routines to achieve their installation's desired results with the above considerations in mind. Establishing an installation standard to use $IAM in the data set name, or as part of a user specified DATACLAS will make it much easier to identify IAM files in the ACS routines.

An alternative to identifying IAM files in the ACS routines is to check for &DD being equal to ‘@#$IAM’ except for encrypted files. The IAM technical support team is available to help review ACS routines, and make suggestions on revising them to meet their objectives forIAM files.

DFSMS ACS routines for encrypted IAM files

Encrypted IAM files require special handling for the Define command request, because they use the VSAM Linear data set format (or ESDS when B=3). The ACS routine is invoked twice during data set definition. The first invocation occurs during the VSAM DEFINE process. IAM is not involved in first ACS pass, as it intercepts after first pass.

IAM behaves as follows:

1. SMS ACS processes the original DEFINE CLUSTER for VSAM. It is the first pass of the ACS routine, and IAM is not involved, because it intercepts after the first pass.
2. IAM intercepts after the first pass of the ACS routine and checks the $IAM string in either DSN, OWNER, or SMS CLASS. If '$IAM' appears anywhere, the request is for defining the IAM file. IAM then checks whether a key label was provided by the DATACLAS, RACF, or the DEFINE statement. Appearance of KEYLABEL implies request is for defining encrypted IAM file.
3. For an encrypted IAM file, IAM invokes IDCAMS call to define a VSAM Linear (or ESDS) VSAM file. It triggers a second pass of the ACS routine, this time as an IAM file.

Unlike dynamic allocation, IAM cannot create the special DDNAME=@#$IAM during an IDCAMS call, so the ACS routine cannot directly identify encrypted IAM files during processing. You can address this by using the ENCDATCLAS global option to override the DATACLAS for encrypted IAM files before the second ACS pass.

You can use the ENCDATCLAS to use a different data class for IAM, the ENCDATCLAS option is useful. For example, for a VSAM encrypted file, you might want to use extended format for IAM Extended addressability. For a VSAM encrypted file, extended format is required, but for an IAM extended extended addressability is required.

In such cases, you might assign a DATACLAS with the Extended Format attribute to all encrypted files during the first ACS pass or in the DEFINE statement. However, this causes IAM encrypted file allocation to fail, because it also requires the Extended Addressability attribute. Specifying a DATACLAS with both attributes in the ENCDATCLAS ensures that IAM encrypted files receive the correct settings during the second ACS pass, overriding the original DATACLAS.

You can also use the ENCDATCLAS option to assign a dummy DATACLAS as an indicator that the data set is an IAM file. This marker is detected during the second ACS pass, allowing the routine to assign the appropriate DATACLAS. When used correctly, the ENCDATCLAS global option can serve as an indicator for encrypted IAM files. To identify non-encrypted IAM files, you can use &DD='@#$IAM'.

Non-specific volume allocation

For data sets that are not managed by DFSMS, IAM offers allocation to non-specific volumes. If you wish to use the IAM non-specific allocation, specify VOLUME(ANYVOL). For multi-volume non-specific allocation, specify VOLUME(ANYVOL ANYV01 ANYV02...). This will result in IAM issuing a non-specific dynamic allocation request for the IAM file. The first volume will be selected by z/OS allocation. Any additional volumes are selected by IAM, which will select volumes from the specified UNIT name that are of the same device type as the first volume selected. IAM builds a list of the eligible volumes, and then selects those volumes that have the largest quantity of available contiguous space. All of the volumes must be mounted as STORAGE to be eligible for selection.

Volume selection is done at the time the data set is defined, and a subsequent LISTCAT will show the volumes selected. The default UNIT name used is SYSDA. To change to a different UNIT, use the IAM CREATE override keyword UNIT=, or change the IAM Global Option WORKUNIT.

When using the IAM non-specific allocation, do not specify UNIQUE. Doing so will cause IDCAMS to attempt to allocate the nonexistent volumes. Similarly, if the MODEL parameter is specified, also specify the SUBALLOCATION parameter, because the MODEL parameter causes the UNIQUE attribute to be assigned.

Customers that are using the CA-ALLOCATE (formerly Sterling Software’s product SAMS also known as VAM) and that have enabled the IAM VAM Global Option cannot use the non-specific volume allocation feature of IAM. They instead should use the pooling and volume selection provided by that product.

This section provides more information about the following topics: