Analyzing data

Data Solutions’s data analysis function enables you to analysis data in a particular field by comparing the data to its record layout field definition and produce a report detailing whether the data is valid or invalid.

This section gives you practice using Data Solutions to analyze the EMP-HOME-AREA-CODE field in the sample employee file, DASAMP.EMPLOYEE, using a filter to limit the analysis to the area codes specified in the Filter Dataset.

Accessing data analyzer menu (Option 3)

The Data Analyzer function is located on Data Solutions’ Primary Option Menu as function number 3, as shown in the following figure.

File-AID/Data Solutions Primary Option Menu

 File-AID/Data Solutions 21.1  --- Primary Option Menu -------------------------
 OPTION ===> 3
                                                            
   0  Parameters     - Data Solutions Parameters             USERID   - USERID1
   1  Date Aging     - Date Aging Solutions                  DATE     - 20/12/09
   2  Euro Tools     - Currency Conversions                  TIME     - 08:39
   3  Data Analyzer  - Analysis Solutions                    TERMINAL - 3290A
   4  Translator     - Data Translator                       CCSID    - 00037
   5  Generation     - Data Generation Solutions
   6  Field Exits    - Formatted Field Exits
   7  Encryption     - Data Encryption
   8  Replace        - Data Replace
   D  Data Privacy   - Dynamic Privacy Rules
   U  Utilities      - Print and File Override Utilities
   S  Criteria Sets  - Create or modify Criteria sets
   R  XREF           - Create or change record layout cross reference
   T  Tutorial       - Display information about Data Solutions
   X  Exit           - Terminate Data Solutions and return to ISPF

      Product Update History: https://support.compuware.com/s/product-updates   
      Online Technical Support available at:  https://support.compuware.com     
      (c) Copyright 1997-2021 BMC Software, Inc.

From the Data Solutions Primary Option Menu, select Data Solutions option 3 to access the Data Analysis Tools Menu (see the following figure).

Data Analysis Tools Menu

 File-AID ---------------- Data Analysis Tools Menu ----------------------------
 OPTION ===> 3


   1  Date Pattern Analysis - Analyze Date Patterns
   2  Validation            - Validate Data Fields
   3  Data Analysis         - Analyze Data Fields  











      Use END to return to Primary Options Menu

From the Data Analysis Tools Menu, select option 3 to access the Input Dataset Specification screen (see the following figure).

Defining your request

The Input Dataset Specification screen, shown in the following figure, captures:

  • Input dataset
  • Record layout and XREF dataset information
  • Change Criteria usage (and optionally, Change criteria dataset and member name).

Input Dataset Specification screen

File-AID/Data Solutions - Input Dataset Specification -------------------------
COMMAND ===>

Specify Input Dataset Information:
  Dataset name           ===> DASAMP.EMPLOYEE
  Member name            ===>            (Blank or pattern for member lis
  Disposition            ===> OLD        (OLD or SHR)
  I/O exit name          ===>

Specify Record Layout Information:
  Record layout usage    ===> S          (S = Single; X = XREF
  Record layout dataset  ===> DASAMP.LAYOUTS
  Member name            ===> EMPLOYEE   (Blank or pattern for
  XREF dataset name      ===>
  Member name            ===>            (Blank or pattern for member lis  

Specify Criteria Information:
  Use existing criteria  ===> N          (Y = Yes; N = No)
  Criteria dataset name  ===>
  Member name            ===>            (Blank or pattern for member lis  

Specify Business Rules or Field Exits Dataset Information:
  Dataset name ===>
  1. Type DASAMP.EMPLOYEE in the Input Dataset name field.
  2. Specify S in the Record layout usage field.
  3. Type DASAMP.LAYOUTS in the Record layout dataset field.
  4. Type EMPLOYEE in the Member name field.
  5. Press <Enter>.

Specifying data analysis criteria

File-AID/Data Solutions displays the Data Analysis Criteria screen, as shown in the following figure.

Data Analysis Criteria Screen

 Action  Sets  Display  Show  Options  Menu  Help
 -------------------------------------------------------------------------------
 File-AID/Data Solutions ------- Data Analysis Criteria ------------------------
 COMMAND ===>                                                  SCROLL ===> PAGE


 SET  1 OF  1           EMPLOYEE-MASTER-FILE                     GBL = N SEL = N
 ---- FIELD LEVEL/NAME ------- -FORMAT- RO --------- PARAMETERS -------------- +
 ******************************* TOP OF DATA ***********************************
 5 EMP-NUMBER                    5/AN
 5 EMP-LAST-NAME                15/AN
 5 EMP-FIRST-NAME               10/AN
 5 EMP-MID-INIT                  1/AN
 5 FILLER                        2/AN
 5 EMP-TITLE                    30/AN
 5 EMP-PERSONAL-INFO SYNC       23/GRP
   10 EMP-NATL-ID-NUMBER         9/NUM
   10 FILLER                     1/AN
   10 EMP-DATE-OF-BIRTH          6/AN
   10 EMP-DOB-REDEF REDEFINES EMP-DATE-OF-BIRTH
   10 EMP-DOB-REDEF SYNC         6/GRP
     15 EMP-DOB-MM               2/NUM
     15 EMP-DOB-DD               2/NUM
 Valid RO: EX, =, FM  
 Commands: SELect, View

Press <PF8> to scroll forward to the 15 EMP-HOME-AREA-CODE field to select it for analysis.

Data Analysis Criteria Screen

   Action  Sets  Display  Show  Options  Menu  Help
 -------------------------------------------------------------------------------
 File-AID/Data Solutions ------- Data Analysis Criteria ------------------------
 COMMAND ===>                                                  SCROLL ===> PAGE


 SET  1 OF  1           EMPLOYEE-MASTER-FILE                     GBL = N SEL = N
 ---- FIELD LEVEL/NAME ------- -FORMAT- RO --------- PARAMETERS -------------- +
     15 EMP-HOME-AREA-CODE       3/NUM  EX
     15 EMP-HOME-EXCHANGE        3/NUM
     15 EMP-HOME-SUFFIX          4/NUM
 ***************************** BOTTOM OF DATA **********************************










 Valid RO: EX, =, FM  
 Commands: SELect, View
  1. Type EX in the RO field to select the EMP-HOME-AREA-CODE field for data analysis.
  2. Press <Enter>.

Entering data analysis specifications

File-AID/Data Solutions displays the Data Analysis Specification screen, as shown in the following figure.

Data Analysis Specifications Screen

image2021-9-27_12-22-38.png

  1. Specify F in the Occurrences Report field.
  2. Type DASAMP.ANALYSIS.TABLE in the Dataset field (for Option F).
  3. Type 19 in the Location field.
  4. Type PS in the Data Type field.
  5. Type 00005 in the Length field.
  6. Press <PF3>.

File-AID/Data Solutions displays the Data Analysis Criteria screen, as shown in the following figure with the PARAMETERS field displaying the entered data analysis specifications.

Data Analysis Criteria Screen

   Action  Sets  Display  Show  Options  Menu  Help
 -------------------------------------------------------------------------------
 File-AID/Data Solutions ------- Data Analysis Criteria ------------------------
 COMMAND ===>                                                  SCROLL ===> PAGE

 SET  1 OF  1           EMPLOYEE-MASTER-FILE                     GBL = N SEL = N
 ---- FIELD LEVEL/NAME ------- -FORMAT- RO --------- PARAMETERS -------------- +
     15 EMP-HOME-AREA-CODE       3/NUM  EX FLEN(00003);STATS(Y);OCCUR(Y);MAXOC +
     15 EMP-HOME-EXCHANGE        3/NUM
     15 EMP-HOME-SUFFIX          4/NUM
 **************************** BOTTOM OF DATA **********************************











 Valid RO: EX, =, FM  
 Commands: SELect, View

Press <PF3>.

File-AID/Data Solutions displays the Criteria Menu, as shown in the following figure, allowing you to specify further change criteria. For this example, press <Enter> to proceed to the Output Specification screen.

Criteria Menu

 File-AID/Data Solutions ---------- Criteria Menu ---  -------------------------
 OPTION ===>                                                                    
                                                                - Status -      
     0   OPTIONS                - Selection criteria options    default         
     G   GLOBAL                 - GLOBAL Fields                   0 fields      
     1   DATE AGING             - Aging criteria                  0 sets        
     2.1 EURO CONVERSION        - Euro Conversion criteria        0 sets        
     2.2 TRIANGULATION          - Currency Triangulation          0 sets        
     3.1 DATE PATTERN ANALYSIS  - Date Pattern criteria           0 sets        
     3.2 VALIDATION             - Validate criteria               0 sets       
     3.3 DATA ANALYSIS          - Data Analysis criteria          1 sets        
     4   TRANSLATOR             - Data Translator criteria        0 sets       
     5.1 GENERATOR              - Data Generator criteria         0 sets        
     5.2 PATTERN                - Pattern Data Generation         0 sets       
     6   FIELD EXITS            - Utilities criteria              0 sets        
     7   DATA ENCRYPTION        - Data Encryption Criteria        0 sets        
     8   REPLACE                - Data Replace Criteria           0 sets       
                                                                               
 Member list description ===> ______________________________                    
     Long    ===> ______________________________________________________________
 Description ===> ______________________________________________________________
                                                                               
 Commands: VIEW, SAVE, END, CANCEL, ENTER to continue

Press <Enter>.

Specifying output options

File-AID/Data Solutions displays the Output Specification screen, as shown in the following figure. Press <Enter> to proceed with the default Output Processing Options as shown.

Output Specification Screen

File-AID ---------------------- Output Specification --------------------------
 COMMAND ===>

 Input dataset: DFHLLM0.DASAMP.EMPLOYEE  
 Specify Output Processing Option:
   Option             ===> A  (A=ANALYZE; C=COPY; U=UPDATE/PREVIEW)
   Create audit trail ===> N  (Y=Chgd record only; N=No; A=All records in Batch)
   Audit trail prefix ===>                         
 Specify Output Dataset Information (COPY ONLY):
   Dataset name   ===>
   Disposition    ===>                    (OLD or NEW)  
   New VSAM Type  ===>                    (K=KSDS,E=ESDS,R=RRDS,L=LDS,Z=ZFS)
                                          (Blank = Input Dataset Type)
 Specify PDS(E) COPY options:
   Copy empty members from input ===> NO     (Y = Yes; N = No)
   Replace like-named members    ===> YES    (Y = Yes; N = No)
   Rename copied members mask    ===>

 Use ENTER to continue or END to return to Criteria Menu
 Use VIEW command to display criteria summary
 Use SAVE command to write criteria request
 Use CANCEL to restart at Input Specification Panel

Press <Enter>.

File-AID/Data Solutions displays the ANALYZE Execution Options screen, as shown in the following figure. Press <Enter> to proceed with the defaults.

ANALYZE Execution Options Screen

 File-AID ---------------------- ANALYZE Execution Options ---------------------
 COMMAND ===>

 Specify Execution Options:
   Process online or batch    ===> O        (O = Online; B = Batch)
   Maximum invalid fields     ===> ALL      (All or number of invalid fields)
   Display invalid fields     ===> Y        (Y = Yes; N = No)
   Maximum changes            ===> ALL      (All or number of changes)  


 Specify Change Criteria Information:
   Save change criteria       ===> N        (Y = Yes; N = No)
   Change criteria dataset    ===> DASAMP.CHGCRIT
   Member name                ===>          (Blank or pattern for member list)  

 Member description ===> ______________________________
     Long    ===> ______________________________________________________________
 Description ===> ______________________________________________________________


 Use ENTER to execute ANALYZE
 Use VIEW command to display criteria summary
 Use END to return to Output Specification Panel
 Use CANCEL command to restart at Input Specification Panel

Press <Enter>.

File-AID/Data Solutions displays the Data Analysis Report, as shown in the following figure. You may scroll to display the entire report or you may elect to print the report.

Data Analysis Report Screen

Menu  Utilities  Compilers  Help
 -------------------------------------------------------------------------------
 BROWSE    DFHLLM0.DASOLUTN.WF.D00049.T164028         Line 00000000 Col 001 080
 Command ===>                                                  Scroll ===> PAGE
********************************* Top of Data **********************************
SET  1 DATA-====================================================================
SET  1 DATA-DATA ANALYSIS FOR FIELD EMP-HOME-AREA-CODE             RECORDS ANALY
SET  1 DATA-RECORDS ANALYZED __________50
SET  1 DATA-
SET  1 DATA-----STATISTICS REPORT-----------------------------------------------
SET  1 DATA-SIGN = N     DATE PATTERN =
SET  1 DATA- ----COUNT--- ----DESCRIPTION---- ---------------F I E L D  V A L U
SET  1 DATA- __________14 UNIQUE VALUES
SET  1 DATA- __________43 -VALID- VALUES
SET  1 DATA- ___________1         HIGHEST     ______________________909
SET  1 DATA- ___________1         LOWEST      ______________________111
SET  1 DATA- __________10         MOST COMMON ______________________408
SET  1 DATA-                      AVERAGE     ______________________417
SET  1 DATA- __________10         MEDIAN      ______________________408
SET  1 DATA-
SET  1 DATA-----OCCURRENCE REPORT-----------------------------------------------
SET  1 DATA-SIGN = N     DATE PATTERN =
SET  1 DATA- ----COUNT--- ----DESCRIPTION---- ---------------F I E L D  V A L U
SET  1 DATA- __________14 UNIQUE VALUES

Press <PF3>.

Printing the report

File-AID/Data Solutions displays the ANALYZE Print Report screen, as shown in the following figure. Press <Enter> to print the report.

ANALYZE Print Report Screen

 File-AID --------------------  ANALYZE Print Report   -------------------------
 COMMAND ===>




  Print execution report  ===> Y   (Y = Yes; N = No)  

  Instructions:  

  Use ENTER to perform above action and return to the initial screen
  Use END to exit without PRINT

Press <Enter>.

File-AID/Data Solutions displays the Print Parameters screen, as shown in the following figure. Press <Enter> to print the report.

Print Parameters Screen

File-AID -------------------  Print Parameters  -------------------------------
 COMMAND ===>

 Number of lines/page       ===> 55       (0 = Suppress page headings)
 Sysout class               ===> A
 Number of copies           ===> 1  

 Enter One of the Following Optional Destinations:  

    Destination printer     ===>          (Local or remote printer)
    - - - OR - - -
    External JES Node ID    ===>          (Predefined JES Node and symbolic ID
    Target VM/TSO ident     ===>           of intended receiver of output)
    - - - OR - - -
    Sysout writer name      ===>          (Installation assigned output writer)
    - - - OR - - -
                                          (DSORG=PS; RECFM=V; LRECL=125)
    Print dataset name      ===>
    Disposition             ===> OLD      (NEW; SHR; MOD; OLD)
    Volume serial           ===>

  Use ENTER to continue, END to cancel

Press <Enter>.

File-AID/Data Solutions displays the Input dataset specification screen, as shown in the following figure. Your process is complete.

Input Dataset Specification screen

 File-AID ---------------- Input Dataset Specification ------ FUNCTION COMPLETE
 COMMAND ===>

 Specify Input Dataset Information:
   Dataset name           ===> DASAMP.EMPLOYEE
   Member name            ===>            (Blank or pattern for member list)
   Disposition            ===> OLD        (OLD or SHR)  

 Specify Record Layout and XREF Information:
   Record layout usage    ===> S          (S = Single; X = XREF)
   Record layout dataset  ===> DASAMP.LAYOUTS
   Member name            ===> EMPLOYEE   (Blank or pattern for member list)
   XREF dataset name      ===>
   Member name            ===>            (Blank or pattern for member list)  

 Specify Criteria Information:
   Use existing criteria  ===> N          (Y = Yes; N = No)
   Criteria dataset name  ===> DASAMP.CHGCRIT
   Member name            ===>            (Blank or pattern for member list)  

 Specify Business Rules or Field Exits Dataset Information:
   Dataset name ===> DASAMP.EURORATE.FACTORS

Viewing the report

The Data Analysis report, as shown in the following figure, is divided into the following sections:

  • Statistic report that lists the number of unique values, number of valid values, and the range of values found in the selected field
  • Occurrence report that lists each unique value and the number of times each value occurs within the selected field.
  • Invalid data report that reports whether invalid data was found in the selected field. When invalid data is found, Data Solutions lists the number of invalid values and the highest, lowest, and most common invalid value found.

Data Analysis Report

SET  1 DATA-====================================================================================
SET  1 DATA-DATA ANALYSIS FOR FIELD EMP-HOME-AREA-CODE             RECORDS ANALYZED __________50
SET  1 DATA-
SET  1 DATA-----STATISTICS REPORT---------------------------------------------------------------
SET  1 DATA-SIGN = N     DATE PATTERN =
SET  1 DATA- ----COUNT--- ----DESCRIPTION---- ---------------F I E L D  V A L U E---------------
SET  1 DATA- __________14 UNIQUE VALUES
SET  1 DATA- __________43 -VALID- VALUES
SET  1 DATA- ___________1         HIGHEST     ______________________909
SET  1 DATA- ___________1         LOWEST      ______________________111
SET  1 DATA- __________10         MOST COMMON ______________________408
SET  1 DATA-                      AVERAGE     ______________________417
SET  1 DATA- __________10         MEDIAN      ______________________408
SET  1 DATA-
SET  1 DATA-----OCCURRENCE REPORT---------------------------------------------------------------
SET  1 DATA-SIGN = N     DATE PATTERN =
SET  1 DATA- ----COUNT--- ----DESCRIPTION---- ---------------F I E L D  V A L U E---------------
SET  1 DATA- __________14 UNIQUE VALUES
SET  1 DATA- __________43 -VALID- VALUES
SET  1 DATA- __________10 TIME(S)             ______________________408
SET  1 DATA- ___________6 TIME(S)             ______________________312
SET  1 DATA- ___________4 TIME(S)             ______________________215
SET  1 DATA- ___________4 TIME(S)             ______________________404
SET  1 DATA- ___________3 TIME(S)             ______________________212
SET  1 DATA- ___________2 TIME(S)             ______________________415
SET  1 DATA- ___________7 OCCURRENCES BYPASSED, NOT IN FILTER DATASET
SET  1 DATA-
SET  1 DATA-----INVALID DATA REPORT-------------------------------------------------------------
SET  1 DATA- ___________7 INVALID VALUES
SET  1 DATA- ___________7         HIGHEST
SET  1 DATA-                      POS.  1-003
SET  1 DATA-                             ZONE 444
SET  1 DATA-                             NUMR 000
SET  1 DATA- ___________7         LOWEST
SET  1 DATA-                      POS.  1-003
SET  1 DATA-                             ZONE 444
SET  1 DATA-                             NUMR 000
SET  1 DATA- ___________7         MOST COMMON
SET  1 DATA-                      POS.  1-003
SET  1 DATA-                             ZONE 444
SET  1 DATA-                             NUMR 000
SET  1 DATA-
SET  1 DATA-===================================================================================
SET  1 DATA-Records Read 50, Selected 50, Fields Processed 43, Invalid 7

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*