Component Warehouse

This section describes the function, setup, and maintenance procedures for the ISPW Component Warehouse.

Component Warehouse Overview

The Component Warehouse is a facility to retain copies of ISPW managed components such as source, listings, or load modules. Each of these components may be saved in one of the warehouse datasets and retrieved at a later time. Components are compressed as they are added to the warehouse and decompressed on the way out.

The warehouse runs as part of the Component Transport (CT) address space.

Warehouse Role

The warehouse is fully integrated with the rest of ISPW and is used to retain old versions of components. The warehouse is transparent to the user and controlled by the ISPW administrator.

Warehouse Functions

Within the Application Definition option (M.AD, Levels), there is an entry that specifies the warehouse name and policy to be used. The warehouse name and policy can be specified differently for source objects and generated objects.

When a component is promoted and a copy of that component already exists at the target level, the copy at the target level is pushed into the warehouse (if a warehouse exists for that level). The number of versions of a component that will be retained at a level is determined by the warehouse policy. Any generated objects, such as listings or load modules, may also be pushed into a warehouse based on the definitions for the level.

Components in a warehouse are retrieved as needed by ISPW operations. When components are no longer referenced by ISPW, they are deleted from the warehouse.

Warehouse Elements

Warehouse Manager

Each warehouse consists of one or more datasets. The warehouse manager allocates the datasets and controls access to them.

The warehouse manager runs inside the CT task. It communicates with the CM task to retrieve warehouse definition data and update warehouse storage status information.

Warehouse Datasets

All the warehouse datasets are PDSEs. The datasets are dynamically defined by the warehouse manager as needed. The dataset names, sizes, and locations are defined with the ISPW administration functions.

Objects may be compressed as they are added to the warehouse. The compression option is set in the warehouse policy. The higher level of compression will minimize the amount of disk space used, but require more time and resources to compress. Disabling compression will speed up warehouse operations, but use more disk space.

Storage Policy

Objects are stored in the warehouse as members of PDSE datasets. The number of versions of an object that will be retained in a warehouse is controlled by the warehouse policy defined with the ISPW administration functions.

Warehouse Usage

What should go into Warehouses?

The Component Warehouse allows you to keep backup versions of source and generated objects, such as listings or load modules. The backups can be retained at any level above the bottom test level in the application structure.

The usage of the warehouse facility will depend on your application management structure and backup requirements. At levels where generates can be performed, it may be sufficient to keep backup copies of just the component source. At levels where generates are not permitted, you may want to keep backups of both source and generated objects in the warehouse.

The retention of generated objects in the warehouse will have a significant effect on both warehouse space requirements and the elapsed time to perform promotions.

How many versions should be retained?

The number of versions that will be retained in the warehouse is set in the warehouse policy. The policy can be different for each level of an application.

At lower levels, it may be adequate to retain a small number of versions. At the production level, you may want to keep a larger number of versions, or even all versions.

Versions by Assignment or Level?

The maximum number of versions that will be retained can be set by either the level or the assignment.

If the maximum number of versions is by level, retention is based just on the number of backup versions currently at that level. If the maximum number of versions is by assignment, retention is based on the number of versions of a component at the level that are part of this assignment.

At the production level, you should normally set the maximum versions by level. At lower levels in the application structure, you may want to consider setting the maximum versions by assignment.

This setting and the value of maximum versions will have a large impact on the size of your warehouses.

How many Warehouses?

You can keep all of your backup copies in one large warehouse or spread them across multiple warehouses. Warehouses are split up into multiple datasets, with additional datasets added as required.

You can segregate different applications or groups of applications. You can keep source in a different warehouse than generated components. You can use different warehouses for different levels.

The decision may be dependent on charge back considerations or disaster recovery requirements. You can have as many or as few warehouses as you want.

Warehouse Setup

This procedure assumes that ISPW is already installed and active on your system. The CM and at least one CI and CT task should be running.

Dataset Planning

Before defining any warehouses, you must select a dataset prefix and decide on allocation parameters. All warehouse datasets will be created with the following naming convention:

“Prefix.WarehouseName.StorageID.WPDS”The warehouse name may be 8 characters. The StorageID is 8 characters. This permits a prefix of up to 21 characters. Since the warehouse datasets will be created and accessed by the CT task, make sure that the UserID assigned to the CT task has alter access to this dataset prefix. Since the warehouse datasets are PDSEs, they must be SMS managed. Ensure that any SMS definitions required for the warehouse datasets are in place prior to using the warehouse.

Defining a Warehouse

To define a new warehouse, go into the warehouse option in the maintenance panel (M.WH). A list of all existing warehouses will be displayed. If you are not already in update mode, enter “U” to go into update mode. Enter an “A” to add a new warehouse definition. The warehouse add panel will allow you to enter the name and description of the warehouse, along with a set of values that will be used to create the warehouse datasets. The following figure shows a sample warehouse definition.

Sample Warehouse Definition

Warehouse Policy

To define a new warehouse policy, go into the warehouse policy option in the maintenance panel (M.WP). A list of all existing policies will be displayed. If you are not already in update mode, enter “U” to go into update mode.

Enter an “A” to add a new policy definition. The warehouse add panel will allow you to enter the name and description of the warehouse policy, along with a set of values that will be used to control the versions retained and the compression that will be used.

The following figure shows a sample warehouse policy definition.

Sample Warehouse Policy Definition

Setting Warehouse Usage

Once one or more warehouses have been defined along with one or more warehouse policies, you can enable warehouse usage by applications.

The warehouse usage is set in the application definition level panel (M.AD, Levels). The name of the warehouse and the policy name can be specified for both source and generated components. If no warehouse definition is specified for generated components, only the source will be retained in the warehouse.

The following figure shows a sample application level definition.

Sample Application Level Definition

Ongoing Support and Maintenance

Once set up, the only real maintenance issues involve the warehouse datasets.

Ensure proper and regular backup of all datasets, and ensure that there is enough DASD space available for warehouse expansion.

Make sure that the automatic housekeeping option is enabled in the CT task start parameters. See the Component Transport task installation for a description of the housekeeping option.

HSM Considerations

The CT task does not care when a dataset is migrated or what level it is at. The only issue is the recall time for ML2 vs. ML1. Since the contents of the warehouse datasets are already compressed, migration to level 1 will not save much disk space.

Regarding migration times, ISPW suggests that datasets migrate to level 1 after a few days, then go to level 2 not too long after that (assuming the level 2 recall times are not unreasonable).

When a warehouse dataset is needed, the CT task checks for the dataset status. If the dataset has been migrated, it issues a recall request using the HSM API. When the recall completes, it allocates and opens the dataset.

If the warehouse dataset contains member(s) that need to be deleted in a housekeeping request, it will be recalled, because a member cannot be deleted without bringing it back.

A fallback would cause a recall of the dataset if it contains a component required by the fallback.

Recalls will not affect other CT requests. Since ISPW recalls using the HSM API prior to issuing the dynamic allocation of the dataset, the CT task will not get caught in the serialization that MVS allocation does in an address space.

The CT task can also handle multiple concurrent dataset recalls without hanging up.

If a tape is unavailable, the CT task will fail the request back to CM. Once the tape is available, the request can be re-run, and everything should work (assuming HSM can recall the dataset). Because each HSM request is independent, this would not affect other requests that may be active at the time.