Component Reference
This section describes the Component Reference features within Code Pipeline.
Component Reference Concepts
The Component Reference feature in Code Pipeline encompasses all of the functionality required to gather and store relationships between components. This information is used by:
- Users for documentation, training, and impact analysis
- Code Pipeline as a basis for configuration management.
Note that the Component Reference contains relationships between components.
How is the relationship information gathered?
The Component Reference information is populated either automatically by Code Pipeline or manually by the customer.
Automatically Maintained Reference Information
Code Pipeline provides parsers to gather this information. When promoting a component to the first controlled level, an appropriate parser for the component is executed to gather relationship information. Execution of the parsers is site configurable. New parsers can be added for special technologies, and user exits to cater for site-specific parser complications are supported. During Code Pipeline installation, it is possible to parse component in bulk to enable the initial data to be stored.
If the Code Pipeline External Function Processor Enhancement is fully enabled, when a Code Pipeline for Eclipse user saves a source or include member, or a source or include member is uploaded (for example. as part of a Git Commit), the source file will be parsed. See “Code-Pipeline-External-Function-Process-Enhancement” for more information.
If the Code Pipeline DNA Facility is in use, source members will be parsed when they are generated, and include files will be parsed when they are promoted.
When parsers are executed, the server uses the active Component Reference Rules (M.RR) to resolve reference names to specific components. For more information, see How Component Reference Rules Work.
Manually Maintained Reference Information
By use of Code Pipeline’s External Call Interface, component reference information can be added manually. See the Code Pipeline Interfaces Guide for more information. It is possible to add relationship information once for a component version and have it inherited by all subsequent versions of that component. Alternatively, the user-added relationships could be inserted by the ECI for each component version. (Or it is possible to have a combination of the two.)
What information is gathered?
As well as identifying the specific component referred to, Code Pipeline will determine and store the following:
- Component Category of the referenced component (for example, PGM, INCL, etc.)
- Relationship Type (for example, Static, Dynamic, or Weak)
- Impact Type (text that further defines the relationship)
- Inheritance information (for manually-added relationships).
Data Model
The diagram in the following figure depicts the entities surrounding the Component Reference.
Data Model
Short descriptions of the entities are provided in the following table.
Component Reference Entities
Entity | Description |
|---|---|
Component | Depicts a Component (for example, COBOL program PROG01 in Application PAYR of Stream MAIN). |
Component Version | Depicts a Version of a Component (for example, Version 2 of the above Component). |
Component Version External Name | Any name that this Component can be referenced by. Normally this is the name of the Component itself, but it does not necessarily have to be (for example, aliases may be used). |
Component Version Reference | A list of Components referenced by the Component Version (for example, copybooks, other programs, etc.). |
Component Type | Defines the type of Component and can therefore be used to drive a process specific to that type (for example, COB, ASM, CLST, etc.). |
Application Component Type | Further defines the Component Type’s behavior down to specific Applications. |
Base Component Type | Provides a way of grouping Component Types together for parsing. This is the level at which the parsing routines and exits are defined. |
Component Category | Used to specify the category of the reference. (for example, COB, ASM, and PLI Base Component Types all have a category of PGM). |
Using the Component Reference Information
The Component Reference information can be accessed in the following ways:
- Option A on the Code Pipeline’s Main menu
- Operation AX against a Code Pipeline Task.
See the Code-Pipeline-User-Guide for information on how to use these functions.
Installation Considerations
Overview
Making the Component Reference functional involves the following:
- Install the ISPF skeleton for batch parsing
- Define Component Categories
- Define Base Component Types
- Update Component Type reference data
- Turn on Component Reference for Applications.
Install ISPF Skeleton
Option M.CR file tailors the ISPF skeleton WZZMCR#. This skeleton is found in the base Code Pipeline Samplib dataset and should be installed into the SITE application with any changes being made to it.
Define Component Categories
Component Categories provide the high level definition of Component References. Component Categories that are to be installed as part of the Code Pipeline installation process include those listed in the following table.
Define Component Categories
Component | Description |
|---|---|
INCL | Copybooks, macros, header files, etc. |
JOB | JCL |
PGM | Any program (for example, COBOL, Assembler, PL/I, etc.) |
PROC | JCL Procs |
This supplied set of Component Categories supports the standard 3GL relationships most customers need. New categories can be added as necessary.
Define Base Component Types
This is the level at which the parsers are defined. Base Component Types provide a way of keeping separate the different Component Types that a customer may wish to define in M.CT with the information required for parsing. It is quite possible that a customer may need to define several different Component Types to manage JCL for instance, and all of these can be related to a single Base Component Type of JOB.
The M.BC definition screen appears as shown in the following figure.
M.BC Definition Screen
Command ===> Scroll ===> CSR
List Commands: LOC Locate Entry, U Update Mode
Line Commands: S Select
Type Cat Lang Scanner Description
AMAC INCL ASM WZZPPASM Assembler Macros
ASM PGM ASM WZZPPASM Assembler Programs
COB PGM COB WZZPPCOB Cobol Programs
COPY INCL COB WZZPPCOB Cobol Copy members
INCL INCL PLI WZZPPPLI PLI Include members
JOB JOB JCL WZZPPJCL JCL Jobs
LNK CPGM LNK WZZPPLNK link edit parms
PLI PGM PLI WZZPPPLI PLI Programs
PROC PROC JCL WZZPPJCL JCL Procs
******************************* Bottom of data ********************************
The Type column indicates the Base Component Type (not the Component Type). Note that this is where the Base Component Types are related to Component Categories.
The value in the Scanner column is the physical load module that will be called to scan a component related to this Base Component Type. The scanner names listed in M.BC Definition Screen are supplied as standard with Code Pipeline.
See CT-Component-Type-Definition for more details on this option.
Update Component Type Reference Data
For each main source component type that is to be parsed, the entry in M.CT must include a reference to a valid Base Component Type. Ensure that the field “Base Component Type” for the component type has a valid value in it.
Turn on Component Reference for Applications
For parsing to be enabled for an Application, the Application’s entry in M.AP needs to be updated so that the field “Component Reference” is set to “Y” (as shown in the following figure).
M.AP Screen
Command ===>
Appl (KEY) ==> PLAY
SubAppl ==> PLAY
Generate ==> D (D - DB2 Generate Parms)
Owner ==> USER001
Cross Reference ==> Y (Y/N)
Owning Appl Grp ==>
Sox ==> (Y/N)
Replace ==> (Y/N)
Description ==> TRAINING APPLICATION
Press END to return
See AP-Applications for more details on this option.
PLI Parser Updates
The Code Pipeline PLI parser might not detect some external references in a program due to various factors. The following are some of the reasons for this limitation:
- Complex syntax can mask the names of external functions.
- The DCL and/or CALL statements for external functions exist in code generated by a pre-processor, meaning they do not exist in the source file processed by the PLI parser.
- The DCL and/or CALL statements for external functions are in INCLUDE files, meaning they do not exist in the source file processed by the PLI parser.
The PLI compiler provides an optional feature to generate a file listing all external references. To enable this option, you must specify the compile option XINFO(DEF) and provide a DD card SYSDEFSD with the attributes RECFM=FB and LRECL=128.
The Code Pipeline PLI parser, updated by PTF IWH149A, can scan the SYSDEFSD file to extract external references. By default, the PLI parser performs the following tasks:
- Parses the program source to extract INCLUDE file names, external references, internal procedures, and entry points.
- Registers the include files, entry points, and external references with the CM server.
If the PLI parser identifies the existence of a SYSDEFSD card, it performs the following tasks:
- Parses the source file to extract only INCLUDE file names, internal procedures, and entry points.
- Scans the SYSDEFSD file to extract external reference names.
- Registers the include files, entry points, and external references with the CM server.
To utilize this enhanced functionality, modifications must be made to the skeletons used for PLI program compilation, as program generation is required to create a SYSDEFSD file. The provided sample skeleton, WZU@PLI, has been updated to check whether the ISPF variable XINFO is set to YES in three distinct locations, using )SEL ~XFINFO = YES and the corresponding )ENDSEL statements. If XINFO is set to YES, the generated JCL is modified as follows:
- Includes compile option XINFO(YES) in the compile step.
- Adds DD card SYSDEFSD to the compile step.
- Adds two additional steps after the compile step:
- The first step is to copy SYSDEFSD file contents to SYSOUT.
- The second step is to invoke the parser and include the SYSDEFSD DD card.
To implement these changes, update the PLI generate skeletons used at your site by checking them out into the SITE application and incorporate the three updates mentioned above. The second update includes the text #ALLOTMP, which during the initial Code Pipeline installation would have been replaced with an appropriate UNIT parameter such as UNIT=SYSDA. Review other DD cards in the skeleton, such as SYSUT1, to see what value you currently use and update #ALLOTMP accordingly.
To enable this functionality, determine the best method for setting the ISPF variable XINFO to YES at your site. There are various approaches that you can take, and the following are a few examples:
- Setting it in your generate exit or generate panel.
- Setting it directly in your PLI generate skeleton.
The following sample skeleton is an updated version of WZU@PLI, with the three modified sections highlighted:
)DEFAULT )~?!<|>
)CM ----- DEFAULT CHANGED ===> USE ~ TO INDICATE ISPF VARIABLE -------
)CM ====================== CHANGE LOG ================================
)CM 2023/09/27 BMC ZENG-316737 SUPPORT PARSING OF SYSDEFSD FILE
)CM BT ISPW 2.0
)CM ====================================================================
//****************** BEGIN SKEL WZU@PLI ******************************
//* PL/I COMPILE
//*********************************************************************
//PLI EXEC PGM=IBMZPLI,REGION=128M,
)SEL ~COND ^= ~Z
// COND=(~COND.),
)ENDSEL
// PARM=('A,M,AG,X,LC(60),INC,NEST,OF,MAR(2,72,1),S',
)SEL ~XINFO = YES
// 'XINFO(DEF)',
)ENDSEL
// '~CPARM1.')
)CM -------------------------------------------------------------------
//STEPLIB DD DISP=SHR,DSN=#PLISTEP /* IBMZC PROC */
// DD DISP=SHR,DSN=CEE.SCEERUN /* IBMZC PROC */
// DD DISP=SHR,DSN=CEE.SCEERUN2 /* IBMZC PROC */
//SYSPRINT DD DISP=(MOD,PASS),
// DSN=&&LISTING
//SYSUT1 DD SPACE=(CYL,(5,2)),#ALLOTMP
//SYSUT2 DD SPACE=(CYL,(5,2)),#ALLOTMP
//SYSUT3 DD SPACE=(CYL,(5,2)),#ALLOTMP
)SEL ~XINFO = YES
//SYSDEFSD DD DSN=&&XINFO,DISP=(NEW,PASS),
// DCB=(RECFM=FB,LRECL=128,BLKSIZE=12800),
// SPACE=(TRK,(1,1),RLSE),#ALLOTMP
)ENDSEL
)CM -------------------------------------------------------------------
)SET WZU@DDN = SYSLIB
)IM WZU@IN$M GJC R20
)CM //~WZU@DDN!DD DISP=SHR,DSN=??DSN_NEEDED?? /* P390 */
)CM -------------------------------------------------------------------
)SEL ~INSRCE = ~Z
//SYSIN DD DISP=SHR,
// DSN=~SOURCE(~MEMBWORK)
)ENDSEL
)SEL ~INSRCE ^= ~Z
//SYSIN DD DISP=(OLD,PASS),
// DSN=&&~INSRCE
)ENDSEL
)CM -------------------------------------------------------------------
//SYSLIN DD DSN=&&LOADSET,DISP=(,PASS),
// SPACE=(TRK,(30,15),RLSE),
// #ALLOTMP,
// DCB=(RECFM=FB,LRECL=80,BLKSIZE=3200)
)CM ---------------------------------------------------------
)SEL ~XINFO = YES
//*********************************************************************
//* DESCRIPTION: PRINT SYSDEFSD FILE
//*********************************************************************
//XINFOPRT EXEC PGM=IEBGENER
//SYSIN DD DUMMY
//SYSPRINT DD DUMMY
//SYSUT1 DD DISP=(OLD,PASS),DSN=&&XINFO
//SYSUT2 DD SYSOUT=*
//*********************************************************************
//* PARSE SOURCE AND SYSDEFSD FILE
//*********************************************************************
)SET XP1 = ~WTAPPLID
)SET XP2 = ~WTSUBAPL
)SET XP3 = ~WTSTRM
)SET XP4 = ~WTMTYPE
)SET XP5 = ~WTCLVL
)SET XP6 = ~MEMBWORK
//XINFOPAR EXEC PGM=WZZRCJOB,
// PARM='~WRTCONF/WZZPP ~XP1 ~XP2 ~XP3 ~XP4 ~XP5 ~XP6'
//SYSDEFSD DD DSN=&&XINFO,DISP=(OLD,DELETE)
//SYSPRINT DD SYSOUT=*
)ENDSEL
)CM ---------------------------------------------------------
)CM COMPILE SAVE LISTINGS
)CM ---------------------------------------------------------
)SET LISTSTEP = PLI
)IM WZU@KEEP
//****************** END SKEL WZU@PLI ********************************
Parsing Preprocessor Output
As part of PTF IWH262A, the ability to parse preprocessor output was introduced to Code Pipeline. This preprocessor support works with the existing support for SYSDEFSD functionality and parser user exits. This feature does not change which calls are registered, but instead supports handling cases where the preprocessor inserts additional calls into the code that are not present in the source module.
Enabling Preprocessor Parsing
To enable this feature, the Code Pipeline administrator must add a new parse step to the Generate Skeletons for any Component Types that require preprocessor parsing.
The parse step should execute the program WZZRCJOB, passing the following parameter string:
‘<#Runtime Config>/WZZPP <#Appl ID> <#Subappl> <#Stream Name> <#Component Type> <#Current Level> <#Task name>’
Additionally, a PARSEPRE DD must be specified to indicate the location of the preprocessor output.
Samples:
Below is a sample parse step for processing simple preprocessor output:
// PARM='~WRTCONF/WZZPP ~WTAPPLID ~WTSUBAPL ~WTSTRM ~WTMTYPE ~WTCLVL X
// ~MEMBWORK'
//STEPLIB DD DISP=SHR,DSN=<#AUTHLIB DATASET>
//PARSEPRE DD DISP=SHR,DSN=<#PREPROCESSOR OUTPUT DATASET> (~MEMBWORK)
//SYSUDUMP DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
The preprocessor parsing also supports the additional parsing of a SYSDEFSD file by specifying the SYSDEFSD DD in the same step, a sample of which is shown below:
// PARM='~WRTCONF/WZZPP ~WTAPPLID ~WTSUBAPL ~WTSTRM ~WTMTYPE ~WTCLVL X
// ~MEMBWORK'
//STEPLIB DD DISP=SHR,DSN=<#AUTHLIB DATASET>
//PARSEPRE DD DISP=SHR,DSN=<#PREPROCESSOR OUTPUT DATASET> (~MEMBWORK)
//SYSDEFSD DD DISP=SHR,DSN=<#SYSDEFSD DATASET>(~MEMBWORK)
//SYSUDUMP DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
Parser User Exits
The User Exits provide a mechanism to extend Code Pipeline-supplied parsers so that component references found by Code Pipeline can be altered or ignored, and additional references found by the User Exit can be added.
How are the User Exits called?
User Exits are called from within Code Pipeline and passed the parameters listed in the following table.
Parser User Exits
Parameter | Description |
|---|---|
Control block address | An internal Code Pipeline control block. This contains all of the component reference information for the component being parsed. By using calls to special Code Pipeline functions, the reference information can be manipulated. Reference information consists of:
|
Record area length | Length of the record area that Code Pipeline is passing to the user exit for parsing. |
Record area address | The record area containing the line of code. |
In what language can the User Exits be written?
The User Exits can be written in any LE language, including Assembler, C, and COBOL (390 and COBOL 2)—but not OS/VS COBOL.
Samples
The sample members listed in the following table can be found in the Base Samplib dataset.
Sample Base Samplib Dataset Members
Sample | Description |
|---|---|
WZZPP | COBOL copybook for both the user parser and user parser exit. This is used when writing a parser or parser exit in COBOL. |
$WZZPP | Assembler Macro for both the user parser and user parser exit. This is used when writing a parser or parser exit in Assembler. |
#WZZPP | C Header file for both the user parser and user parser exit. This is used when writing a parser or parser exit in C. |
WZPEXTCO | An example User Exit written in COBOL. |
WZPEXTCL | Linkage statements to link the COBOL example. |
WZPEXTAA | An example User Exit written in Assembler. |
WZPEXTAL | Linkage statements to link the Assembler example. |
How User Exits function
The User Exit is called from Code Pipeline and is passed parameters allowing it to interrogate the record containing the source code. It is also passed the current list of External Names for the component and the list of External References to other components. The User Exit is coded to analyze the line of code passed and can then add to the list of Names and References using specific Code Pipeline calls. The calls to Code Pipeline functions listed in the following table are available.
Available Code Pipeline Function Calls
Function | Description |
|---|---|
WZZPPEXN | As part of the parsing process, the list of External Names can be added to. This function gets a new External Name list element. The new element is added to the front of the list, pointed to by PPB->extrnams. The User Exit will then update that element with the new external name information. On return from this function, the element can be filled in with the external name value. |
WZZPPXRF | As part of the parsing process, the list of External References can be added to. This function gets a new External Reference list element. The new element is added to the front of the list, pointed to by PPB->extrefs. The User Exit will then update that element with the new external reference information. On return from this function, the element can be filled in with the external reference value. |
WZZPPMSG | This function writes a message to the Code Pipeline debug output stream. |
WZZPPMER | This function writes an error message to the Code Pipeline log output stream. |
How to ignore references with the Parser User Exit
Setting the language to an asterisk ( * ) for a type can cause certain references to be incorrectly registered. For example, if you are parsing an ASM module that calls IBM macros and there is a Copybook with the same name as the macro as a Component in Code Pipeline, the Copybook will be registered to the ASM module.
To address this, PTF IWH230A introduced the ability to allow Code Pipeline to ignore certain references for a task when parsing COBOL, Assembler, and PLI source files using the default parsers.
To ignore a reference, the Parser User Exit must set the Delete Indicator flag for that reference. This flag is available in various sample structures provided with the product and is mentioned in the Sample Base Samplib Dataset Members table. Each language supported by the Parser User Exit has an associated flag for both External Names and External References.
The structures and their corresponding flags are listed in the following table. Note that the Parser Language field refers to the language in which the Parser User Exit is written, not the language of the source member being parsed:
Parser Language | Sample Structures | External Names Delete Flag | External References Delete Flag |
|---|---|---|---|
COBOL | WZZPP | PPEXN-DELIND | PPXRF-DELIND |
Assembler | $WZZPP | EXNDELIND | XRFDELING |
C | #WZZPP | delind | delind |
Initially, all flags are set to a blank when control is passed to the User Exit. Setting the flag to any other value will cause any reference with the same name to be ignored for the Source Task being parsed.
This feature allows you to mark the reference to be ignored as either an External Name or External Reference, since the ignore operation is based on the reference name rather than its type.
Below are the sample codes illustrating how to set the flags for a Parser User Exit written in COBOL for parsing COBOL source code.
External Reference Sample
The following example demonstrates adding a reference with the delete indicator set for an included program, where the member name starts at column 13 and is 7 characters long.
MOVE 'INCL' TO PPXRF-REFCAT.
MOVE LINE-LINE (13:7) TO PPXRF-REFNAME.
MOVE 'S' TO PPXRF-RELSHP.
MOVE '5' TO PPXRF-SOURCE.
MOVE 'Y' TO PPXRF-DELIND.
External Name Sample
The following example demonstrates adding an external function name with the delete indicator set. The name starts at column 13 and is 6 characters long.
MOVE 'E' TO PPEXN-EXTNTYPE.
MOVE LINE-LINE (13:6) TO PPEXN-EXTNAME.
MOVE 'Y' TO PPEXN-DELIND.