Filtering historical data (PTFs BQY1950 and BQY2072 applied)
The following options are available for filtering historical data:
Selecting records by a specific date/time range
The first step is to determine the internal date/time values to use as filtering criteria. The prefix.BBSAMP(BBMSMD73) member describes the ORECZTIM field as a 6-byte binary integer as follows:
ORECTIME DS 0XL8 Time stamp in MZZ_ZNTS format
ORECZTIM DS XL6 o Number of elapsed 1/100 X
seconds since the year 1980
ORECZZON DS H o UTC time zone delta X
(+/- 720 minutes) You can use the BBMSRTIM REXX exec to approximate an ORECZTIM date/time value given a local date/time and associated time zone.
The REXX exec accepts two arguments:
- The local date/time specified in the following format: YYYYMMDDHHMMSS.th
- (Optional) The local time zone in hours from UTC in the following format: YYYYMMDDHHMMSS.th offset
The offset variable is an integer representing the offset in hours from UTC. For example, in the U.S. Pacific Standard Time (PST) is –8 hours from UTC; U.S. Eastern Daylight Time (EDT) is –4 hours from UTC. If you do not specify an offset, the default offset value is 0 hours.
Note
You can use fractional hour time zones for locales that require it. For example, ACST is +9.5 from UTC; ACWST is +8.75 from UTC.
You can call the REXX exec to calculate the ORECZTIM value. For example, if you specify date/time values of 20180223080000.00 (8:00 A.M. on February 23, 2018) and 20180223170000.00 (5:00 P.M. on February 23, 2018) in the U.S. Eastern Standard Time zone (–5 hours from UTC), the REXX exec returns the following values:
ex 'BB.MVI63.BBSAMP(BBMSRTIM)' '20180223080000.00 -5'
ORECTIME=x'001C078D8740FED4' ORECZTIM=x'001C078D8740' ORECZZON=x'FED4'
Date/time=20180223080000.00 Zone=-5
ex 'BB.MVI63.BBSAMP(BBMSRTIM)' '20180223170000.00 -5'
orectime='001C07BEF780FED4'
ORECTIME=x'001C07BEF780FED4' ORECZTIM=x'001C07BEF780' ORECZZON=x'FED4'
Date/time=20180223170000.00 Zone=-5 Note
Take care when calculating the time values across a daylight-saving time zone change. If you specify incorrect time or zone values, you might produce overlapping time values.
You can now use these values in the SORT utility to select a subset of extracted history records. The ORECZTIM column compensates for the Record Descriptor Word (RDW, 4 bytes) in the variable length record (RECFM=VB) of the history extract output data set. The ORECZTIM column is located at position 31 of each record (offset X’1E from OREC + 1= X’1F=31). Use the following DFSORT job to extract history written between 8:00 A.M. am and 5:00 P.M. on February 23, 2018.
Note
The following examples assume that you called the REXX exec on a z/OS system with the same leap second offset setting as the system on which the history records were recorded (CVTLSO value in the CVT, z/OS system control block).
Example 1: Extracting data from first shift
//SUBSET1 JOB (),'XXXXXXX',CLASS=A,
// NOTIFY=&SYSUID,MSGCLASS=X,MSGLEVEL=(1,1)
//***************************************************************
//* DELETE OUTPUT DATA SET
//***************************************************************
//DELETE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DELETE BMC.SYSA.MVIP.SHFT1
//***************************************************************
//DFSORT EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=BMC.SYSA.MVIP.HISTDS(-1),DISP=SHR
// DD DSN=BMC.SYSA.MVIP.HISTDS(+0),DISP=SHR
//SORTOUT DD DSN=BMC.SYSA.MVIP.SHFT1,DISP=(,CATLG),
// SPACE=(CYL,(100,50),RLSE),UNIT=SYSDA,
// DCB=*.SORTIN
//SYSIN DD *
INCLUDE COND=(31,6,CH,GE,X'001C078D8740',AND,
31,6,CH,LE,X'001C07BEF780')
OPTION COPY,VLSHRT
// In Example 1, you search the current generations of the history extract GDG for records recorded during the first shift (8:00 A.M. – 5:00P.M.) on Feb 23., 2018 EST. You use the data set creation dates of the input data sets in the system catalog to determine which generations were written on the specified target date.
In Example 2, you call the BBMSRT2Z REXX exec. First, we need to determine the ORECZTIM values for the start and end times for the day:
Determining start and end times
ex 'BB.MVI63.BBSAMP(BBMSRTIM)' '20180705000000.00 -7'
orectime='001C4B66E9C0FE5C'
ORECTIME=x'001C4B66E9C0FE5C' ORECZTIM=x'001C4B66E9C0' ORECZZON=x'FE5C'
Date/time=20180705000000.00 Zone=-7
ex 'BB.MVI63.BBSAMP(BBMSRTIM)' '20180706000000.00 -7'
orectime='001C4BEABFC0FE5C'
ORECTIME=x'001C4BEABFC0FE5C' ORECZTIM=x'001C4BEABFC0' ORECZZON=x'FE5C'
Date/time=20180706000000.00 Zone=-7 In this example, you we are calculating the ORECZTIM values for midnight on July 5, 2018 PDT and July 6, 2018 PDT. You can then insert these values into the JCL to extract a specific day's history using the GDG data sets written between those times as follows:
Example 2
//SUBSET2 JOB (),'XXXXXXX',CLASS=A,
// NOTIFY=&SYSUID,MSGCLASS=X,MSGLEVEL=(1,1)
//***************************************************************
//* DELETE OUTPUT DATA SET
//***************************************************************
//DELETE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DELETE BMC.SYSA.MVMVS.@180705
//***************************************************************
//DFSORT EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=BMC.SYSA.MVMVS.HISTDS(-2),DISP=SHR
// DD DSN=BMC.SYSA.MVMVS.HISTDS(-1),DISP=SHR
// DD DSN=BMC.SYSA.MVMVS.HISTDS(+0),DISP=SHR
//SORTOUT DD DSN=BMC.SYSA.MVMVS.@180705,DISP=(,CATLG),
// SPACE=(CYL,(500,50),RLSE),UNIT=SYSDA,
// DCB=*.SORTIN
//SYSIN DD *
INCLUDE COND=(31,6,CH,GE,X'001C4B66E9C0',AND,
31,6,CH,LT,X'001C4BEABFC0')
OPTION COPY,VLSHRT
// Selecting records by a specific product and product record entry type (RTIN)
The prefix.BBSAMP(BBMSMD73) DSECT member or prefix.BBMAMP(BBMSMD73) DSECT member indicates that the ORECRTIN field is in column 39 (offset X’26 from OREC + 1 = X’27=39) of each history extract record. You can use the Example 3 JCL to extract ORECRTIN=X’01’ record-entry (RE) types from history data extracted from a PAS. Extract ORECOTYP=X’0100’ in column 97 (offset X’60 from OREC + 1=X’61=97) and normal interval set, ORECISID=X’00’ in column 99 (offset X’62 from OREC + 1=X’63=99). Type X’01’ REs contain address space data observed by the PAS normally recorded at 15-minute intervals.
Example 3: Extract MVMVS RE type X’01’ (Address Spaces--ASRE)
//SUBSET3 JOB (),'XXXXXXXX’,CLASS=A,
// NOTIFY=&SYSUID,MSGCLASS=X,MSGLEVEL=(1,1)
//***************************************************************
//* DELETE OUTPUT DATA SET
//***************************************************************
//DELETE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DELETE BMC.SYSA.MVMVS.ASRE
//***************************************************************
//DFSORT EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=BMC.SYSA.MVMVS.HISTDS(+0),DISP=SHR
//SORTOUT DD DSN=BMC.SYSA.MVMVS.ASRE,DISP=(,CATLG),
// SPACE=(CYL,(500,100),RLSE),UNIT=SYSDA,
// DCB=*.SORTIN
//SYSIN DD *
INCLUDE COND=(97,2,CH,EQ,X'0100',AND,
99,1,CH,EQ,X’00’,AND,
39,1,CH,EQ,X'01')
OPTION COPY,VLSHRT
// Selecting records using RE-specific fields in the PACK=N format
PACK=N format data sets are identical to the PACK=Y format data sets except that each record contains only one RE. The PACK=N format accommodates the filtering of records by product-specific RE fields. However, using the PACK=N format might exclude the last segments of logical records, making the ORECFLGS field undependable when you use it to detect the end of segments and logical records.
The prefix.BBSAMP(BBMSMD73) DSECT indicates that the ORECRTIN field is in column 39 (offset X’26 from OREC + 1 = X’27=39) of each history extract record. Use the Example 4 JCL to extract ORECRTIN=X’10’ RE types from history data extracted from a PAS. Extract ORECOTYP=X’0100’ in column 97 (offset X’60 from OREC + 1=X’61=97) and long-term interval set, ORECISID=X’01’ in column 99 (offset X’62 from OREC + 1=X’63=99). Type X’10’ REs contain address space data observed by the PAS in a long-term history interval set. The first and only RE in a PACK=N format data set is in ORECRES (offset X’68 from OREC + 1=X’69=105).
The ADRENAME in the RE X’10’ in the IRSID=X’01’ long-term history data set is offset X’0C’ (12 bytes) from the beginning of the RE (mapped by the ADRE in BBDUCE10 and data map member in the prefix.BBSAMP or prefix.BBMAP data set), and contains the 8-byte address space name. If you take the offset of the ORECRES and add the ADRENAME field offset, the address space is in column 117 (offset 105 + X’C=117).
In Example 4 the job extracts MVMVS RE type X’10’ (from address spaces—ADRE) from a long-term history data set (IRSID=1) for address spaces names which contain the substring PROD. The input history extract data set, BMC.SYSA.MVMVS.HST1DSN(+0), was created using the BBM9MD73 utility program with the PACK=N option to ensure that only one RE is written per output record.
Example 4
//SUBSET4 JOB (),'XXXXXXXX’,CLASS=A,
// NOTIFY=&SYSUID,MSGCLASS=X,MSGLEVEL=(1,1)
//***************************************************************
//* DELETE OUTPUT DATA SET
//***************************************************************
//DELETE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DELETE BMC.SYSA.MVMVS.ADRE.
//***************************************************************
//DFSORT EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=BMC.SYSA.MVMVS.HST1DSN(+0),DISP=SHR
//SORTOUT DD DSN=BMC.SYSA.MVMVS.ADRE,DISP=(,CATLG),
// SPACE=(CYL,(500,100),RLSE),UNIT=SYSDA,
// DCB=*.SORTIN
//SYSIN DD *
INCLUDE COND=(97,2,CH,EQ,X'0100',AND,
99,1,CH,EQ,X’01’,AND,
39,1,CH,EQ,X'10',AND,
117,8,SS,EQ,C’PROD’)
OPTION COPY,VLSHRT
// Selecting records for a specific RE type and a subset of fields (relational project) using the PACK=N format
PACK=N format data sets are identical to the PACK=Y format data sets except that each record contains only one RE. This format accommodates the filtering of records by product-specific RE fields. In Example 5, you select all REs per logical record, we can then use the ORECFLGS field to detect the end of segments and logical records.
The prefix.BBSAMP(BBMSMD73) DSECT indicates that the ORECRTIN field is in column 39 (offset X’26 from OREC + 1 = X’27=39) of each history extract record. You can use the JCL to extract ORECRTIN=X’01’ RE types from history data extracted from a PAS. Extract ORECOTYP=X’0100’ in column 97 (offset X’60 from OREC + 1=X’61=97) and normal interval set, ORECISID=X’00’ in column 99 (offset X’62 from OREC + 1=X’63=99). Type X’01’ REs contain address space data observed by the PAS in normal history interval set. The first and only RE in a PACK=N format data set is in ORECRES (offset X’68 from OREC + 1=X’69=105).
The OREC key containing the interval time in the OREC record header and the Identification Section, Session Count Section, Interval Count Section, and the Session Section of the ASRE RE are copied from the extracted records, reducing the amount of data to only those columns of interest before being exported to analytics tools. The OREC DSECT is defined in the prefix.BBSAMP(BBMSMD73) member and the ASRE is defined in the BBMUCE01 data map member in the prefix.BBSAMP data set or the prefix.BBMAP data set. The ASRE sections are delineated by comments in the BBMUCE01 data map.
The OREC key, ORECGKEY, is located at column 29 (offset X’1C + 1 = X’1D = 29) for a length of 24 bytes. This field includes the ORECZTIM, ORECZZON, ORECRTIN, and ORECSYS subfields.
- If you take the offset of the ORECRES field (X’68) and add the field offset for ASREASID (X’18), the first field in the Identification Section of the ASRE is found at column 129 (X’68 + X’18 + 1 = X’81 = 129). The length of the Identification Section, from ASREASID (X’18) up to but not including ASREFLG1 (X’98’) is X’80 = 128 bytes.
- If you take the offset of the ORECRES field (X’68’) and add the field offset of ASRSLAP (X’1B0), the first field in the Session Counts section of the ASRE is found at column 537 (X’68 + X’1B0 + 1 = X’219 = 537). The length of the Session Counts section of the ASRE from ASRSLAP (X’1B0) up to be not including ASRILAP (X’2A0) is X’F0 = 240 bytes.
- If you take the offset of the ORECRES field (X’68’) and add the field offset of ASRILAP (X’2A0), the first field in the Interval Counts section of the ASRE is found at column 777 (X’68 + X’2A0 + 1 = X’309 = 777). The length of the Interval Counts section of the ASRE from ASRILAP (X’2A0) up to be not including ASRITHNK (X’390) is X’F0 = 240 bytes.
- If you take the offset of the ORECRES field (X’68’) and add the field offset of ASRSWMSG (X’3DC), the first field in the Session section of the ASRE is found at column 1093 (X’68 + X’3DC + 1 = X’445 = 1093). The length of the Interval Counts section of the ASRE from ASRSWMSG (X’3DC) up to be not including ASRDLAP (X’420) is X’44 = 68 bytes.
In Example 5, you extract MVMVS RE type X’01’ (address spaces—ASRE) from a normal interval history data set (IRSID=0) copying only the ORECGKEY field from the OREC header and the Identification section, Session Counts section, Interval Counts section, and the Session section of the ASRE RE. The BBM9MD73 utility program created the input history extract data set, BMC.SYSA.MVMVS.HISTDSN(+0), with the PACK=N option to ensure that only one RE is written per output record.
Note
The first four bytes of the output record contain the record descriptor word (RDW position 1, length of 4 bytes).
Example 5
//SUBSET5 JOB (),'XXXXXXXX’,CLASS=A,
// NOTIFY=&SYSUID,MSGCLASS=X,MSGLEVEL=(1,1)
//***************************************************************
//* DELETE OUTPUT DATA SET
//***************************************************************
//DELETE EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DELETE BMC.SYSA.MVMVS.ASRE.PROJECT
//***************************************************************
//DFSORT EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=BMC.SYSA.MVMVS.HISTDSN(+0),DISP=SHR
//SORTOUT DD DSN=BMC.SYSA.MVMVS.ASRE.PROJECT,DISP=(,CATLG),
// SPACE=(CYL,(500,100),RLSE),UNIT=SYSDA,
// DCB=*.SORTIN
//SYSIN DD *
INCLUDE COND=(97,2,CH,EQ,X'0100',AND,
99,1,CH,EQ,X’00’,AND,
39,1,CH,EQ,X'01')
OUTREC FIELDS=(1,4,29,24,129,128,537,240,777,240,1093,68)
OPTION COPY,VLSHRT
//
Comments
Log in or register to comment.