Other Features of IAM
Spanned Record Support
IAM files can contain spanned records. A spanned record is one that is larger than the physical block size of the IAM data set, and will require multiple physical blocks to store it. With IAM, the record size limitation is the amount of data that IAM can fit within 256 data blocks. With a maximum block size of 32760, anIAM file can now have a maximum record length of approximately 8 megabytes. Spanned record support will allow IAM Alternate Index customers to have large numbers of records in a base cluster to have the same alternate key. The spanned record support also eliminates the DASD space waste that occurs on files where the record size requires a physical block size that exceeds the half-track block size that resulted in IAM using a block size of 32760.
Please be aware that, many utilities used to process IAM (and VSAM) files, do not support record sizes of approximately 32760 bytes or larger, including IDCAMS and the various SORT utilities. The IAM file recovery utility program, IAMRECVR, does provide support for these large records.
IAM Index Storage
One of the features of IAM that helps deliver outstanding performance is to have the index to open files reside in virtual storage. The virtual storage used for the index can come out of any one of the following areas:
- 64-bit addressable virtual storage, in users address space
- z/OS Data Space, maximum size = 2 Gigabytes
- 31-bit addressable in users address space, maximum size: available storage < 2 Gigabytes
The default is to use 64-bit addressable storage if it is available. To make this storage available, users will need to specify a MEMLIMIT=nG on their EXEC card unless their installation has set up the exit to automatically allow use of 64-bit virtual storage. The advantage of 64-bit virtual is that, it eliminates the 2 gigabyte size limit that a data space has, and keeps the index out of storage areas typically used by application programs. If 64-bit addressable storage is not available, then IAM will attempt to acquire a z/OS data space to hold the index. If IAM is unable to acquire data space storage, then virtual storage from within the users region will be utilized. The default can be changed either by modifying the INDEXSPACE Global Option or by IAM overrides for the particular job steps.
ESDS Extended Addressability
IAM has support for 8-byte RBA values in the IAM ESDS type of files. This is referred to by IBM as Extended Addressability. VSAM ESDS data sets defined with this DFSMS attribute can exceed 4 gigabytes by using an 8-byte RBA, instead of the normal 4-byte RBA. An IAM 8-byte RBA file is created by specifying the XESDS keyword on the IAM CREATE override, through specification in the DFSMS Data Class, or by setting the IAM Global Option ENABLE=XESDS. Such IAMESDS files do not have to reside on DFSMS managed volumes, as do VSAM ESDS files. Your application programs may not be able to take advantage of the 8-byte RBA values without coding changes. For such situations, you may find that the IAM PSEUDORBA feature, which provides over 4 gigabyte ESDS files with a non-standard 4-byte RBA, is an easy solution.
PSEUDORBA type ESDS files are created through the PSEUDORBA IAM CREATE Override. This enables many applications to exceed the 4-gigabyte size limitation without changing to an 8-byte RBA value. The limitation is that if an application is dependent on the RBA value being identical to the VSAM architecture, then PSEUDORBA cannot be used. The only known application package that has this limitation is SAP.
IAM's Dynamic Tabling
IAM has a feature that can potentially reduce physical I/O's for files that are randomly read. This feature, called IAM's Dynamic Tabling, offers significant performance benefits for some applications. With this feature activated, IAM will table records retrieved randomly from a file in virtual storage, without any programming changes to existing applications. Then, on subsequent random reads, IAM checks to see if the requested record is contained in the table. If the record exists in the table, IAM passes it back to the user, eliminating the I/O to the disk. If the record does not exist in the table it will be read and the record will be moved into the table. Records that are updated will be updated both in the table and on DASD.
IAM Dynamic Tabling virtual storage can come from either the users address space or from an Z/OS data space. The Dynamic Data Space feature will allow for more storage to be devoted to the table, along with improved record and storage management within the table.
To enable this option, use the IAM ACCESS Override statement. The keywords on the override control statement are DYNDS= or DYNCORE= ( See IAM-Overrides). The DYNDS= is specified when the storage is to come from a data space, and is specified in megabytes with values up to 2048. The DYNCORE value is specified in 1024 byte (1K) increments, and the storage will be acquired within the users address space. The example below is an example of using 1 gigabyte (1024 megabytes) of storage in a data space:
ACCESS DD=iamfile,DYNDS=1024
The following example will reserve 200K of storage in extended private (AMODE31) for the Dynamic Table:
ACCESS DD=iamfile,DYNCORE=200
Applications which will benefit the most from Dynamic Tabling are those with high file activity where a subset of records in the file are repeatedly being read, with few ever updated. Small files with high random activity and few updates become in core tables without the need for any programming changes.
IAM's run time IAMINFO report reflects the way an application uses the file. The report includes statistics on requests processed, I/Os to disk, Dynamic Table usage and the number of records retrieved from the Dynamic Table.
DYNAMIC TABLE RETRIEVALS - displays the number of record requests satisfied from IAM's Dynamic Table.
DYNAMIC TABLE RECORDS - displays the maximum number of records in the table.
Mass Sequential Deletes
IAM has incorporated an ability to temporarily logically delete a record from a data block. Then, when that data block is about to be written out to DASD, the records are physically deleted from the data block. This eliminates the overhead of constantly moving records within a data block, as prior records are deleted, only to end up being deleted as well. In the case where every single record is deleted from a data block, this enhancement eliminates the data movement entirely from that process.
Long Key Compression
IAM files that have an index that exceeds 8000 bytes in size are automatically compressed to reduce index storage. Long keys however, experienced less benefit due to the compression technique used. This is resolved with the use of the IAM Long Key Compression function, which now defaults to being used for keys that are 33 bytes or longer. This key compression enhancement may provide a significant reduction in virtual storage, particularly for very large IAM data sets with long keys. For example, a customer had a 33-gigabyte file that had a key length of over 128 bytes. The resulting prime index prior to long key compression required 227 megabytes. With the long key compression, the index size was reduced to 49 megabytes, a 78% reduction in virtual storage requirements.
The IAM Global Option LONGKEYS= specifies the minimum key length to be eligible for long key compression. The default value is 33, which has been reduced from the prior 128.
Storage Pooling
IAM has implemented an alternative storage management technique for handling below the line storage. IAM typically uses a minimum of 4K below the line storage for each opened Enhanced Format file. The storage is required for various I/O control blocks, such as the DCB, IOB’s, channel programs, and JFCB, which still have to be kept below the line. The management of this storage has been divided into three different storage pools. There is a pool for the base required storage for each opened IAM file, containing the DCB, JFCB, and related storage areas. There is a separate pool for the IOB and basic channel program, and a third pool for the sequential chained channel programs. The IOB’s and chained channel program pool entries are acquired and held only for the duration of physical I/O’s, which provides a reduction in the overall long term storage usage because they are now shared at the address space level rather than on a file by file basis.
It is anticipated that the below the line virtual storage usage will be reduced by at least 50% per opened Enhanced Format file, when several files are opened. The use of this feature is controlled by using the IAM Global Option BELOWPOOL=YES, which is the current default value. Although it is not recommended, users can turn this option off if necessary.
Dynamic Region Size Adjustment
As customers have converted files to IAM, they occasionally hit the z/OS default limit of 32 megabytes extended private storage or their REGION size limit. This has in the past necessitated the modification, or in many circumstances writing and supplying an IEFUSI exit. We have developed, and distributed a sample IEFUSI exit, that frequently can be used with minor modifications. This exit is distributed in the ICL (Installation Control Library) provided with IAM.
Because many customers have unexpectedly hit this limitation, IAM has the ability to dynamically increase the extended private region limit. This may be invoked during a file open, or when storage acquisition for a critical storage area fails. When invoked IAM will then attempt to increase the extended private storage limit by the quantity required rounded to 4 megabytes. By default ,IAM will not increase the extended private region to greater than 512 megabytes. This maximum value can be either increased or decreased through the IAM Override facility, or by changing the IAM default in the IAM Global Options Table.
Dynamic region adjustment affects only normal file access; it does not function during a file load process. The value can be changed in the IAM Global Options Table, by using the keyword MAXREGION. See IAM-Global-Option-Change-Facility for information on changing IAM Global Options. The region adjustment can also be controlled with an IAM Override. The IAM ACCESS override keyword is MAXREGION, and it specifies the upper limit for the above the line storage, in megabytes.
Multiple ACB Support
IAM will recognize that an ACB has already been opened for a file, within the same task, and share the index structure and buffers. For applications, or CICS regions, where a file was opened under one DDNAME or ACB for read only, and the other for UPDATE, they occasionally experienced an inability to retrieve an updated data record. This will no longer be the case. This support also reduces the storage requirements for having multiple ACB’s opened to the same file.
A special feature is available when the opened ACB is read only, and an ACB is subsequently being opened for update in the same address space. When that occurs, the overflow index is completely rebuilt, and all the buffers refreshed so that any updates that may have been done by batch jobs will be automatically and immediately available once the update ACB has completed the open process. For online systems that utilize two ACB's, where one is read only and the other update, the update ACB can be closed to allow batch updating. The read-only ACB can remain open for processing, however it may not have access to all of the records updated by the batch jobs unless it is using IAM/RLS or IAM/PLEX. When the batch updating terminates, the update ACB can be reopened, and access to all of the batch updates will be immediately available.
Backup While Open (BWO)
IAM provides support for the Backup While Open (BWO) service interface. IAM will keep the BWO information, consisting of status flags and a time stamp. The BWO callable service will function for IAM files as it does for VSAM files, to retrieve and update the BWO data. Additionally, when BWO data is present in the IAM file control information, that information will be displayed in the IAMPRINT LISTCAT report. This will primarily be of benefit for recovery programs, such as CICS/VR, which will utilize the BWO information.
Simulate File Load
Starting from IAM version 10.0.2, a simulation of loading IAM files without doing most of the actual writes to the output file is supported. This feature provides the ability for a user to determine a large majority of the statistics of an IAM file after a file load, but with only a small IAM file being allocated. For example, after an IDCAMS REPRO load, the file will just contain the IAM control records and none of the actual user data records. IAMINFO report statistics produced by the REPRO will reflect the actual amount of data processed. The resulting file can then be interrogated with IDCAMS LISTCAT DSN() ALL, just like a regular IAM file, and the resulting statistics will display the results as if the file had been normally loaded.
The SIMULATE function is enabled by specifying SIMULATE on an IAM CREATE OVERRIDE statement within the Job Step performing the IAM file load that the SIMULATE request is for. See IAM-CREATE-Override-Statement-Format for CREATE control statement specification details.
SIMULATE can be helpful if you decide to break up a very large IAM file into separate smaller and equal in size IAM files. You can Simulate loading specific key ranges to determine what the key ranges should be to keep the smaller files equal in size without having to allocate all the space that would be normally needed to make that determination.