Using the Data Collector Client utility to extract data from the BMC AMI Ops products


Some content is unavailable due to permissions.

You can use the Data Collector Client (DCC) utility to extract data from BMC AMI Ops products via Common REST API. You can configure the utility by specifying various parameters, such as product name, and view name to extract data. The extracted data is stored to your local file system. You can then push the data into analytics tools (For example, Splunk or Elasticsearch) for further processing.  

The DCC utility performs the following tasks:

  • Provides a framework to extract data
  • Extracts data at regular intervals
  • Helps to identify performance issues

Important

The DCC utility is currently supported on Microsoft Windows only. 

Before you begin

  • Make sure that you have CRA installed and running. For more information, see Installing-and-configuring-the-Common-REST-API.
  • Make sure that you have Oracle Java 8 or later installed on your local machine where you want to run the DCC utility.
  • Make sure that you have set the JAVA_HOME environment variable properly.

This topic contains the following sections:

Downloading the DCC utility

You can use the File Transfer Protocol (FTP) to download the utility files from the mainframe system to your windows machine.

Perform the following steps to download the DCC utility:

  1. Go to your local machine where you want to run the DCC utility and open the command prompt.
  2. Enter the following command:

    ftp zosHost
     
    zosHost is the name of the CRA server on which the utility files are located.

  3. Enter your mainframe user name and password.
  4. Enter the following command to navigate to the directory where you want to save the utility files:

    lcd directoryPath

  5. Make sure that you select the binary file transfer mode. To do so, enter the following command:

    bin 
  6. Enter the following command to get the files:
    get fullPath
    fullPath 
    is the path (including the zip file name) where the CRA-Data-Collector-Utility.zip is located. The .zip file is located in the root directory where you have installed CRA. Make sure that you include a full path as shown in the following example:

    get /home/crahome/CRA-Data-Collector-Utility.zipOn successful execution, the CRA-Data-Collector-Utility.zip file gets downloaded to the specified location.

  7. Enter the following command to close the FTP connection:
    bye

Configuring the DCC utility

  1. On your local machine, navigate to the location where you have downloaded the CRA-Data-Collector-Utility.zip file and extract the .zip file to the required directory. 
  2. If you are using a Java version later than 8, open the StartCRADataCollector.bat file and comment the following lines by adding rem before each line as shown in the following sample code:

    rem %JAVA_CMD% -version >result.txt 2>&1
    rem Find "1.8" result.txt >nul
    rem if errorlevel 1 (
    rem ECHO Error: You are using an unsupported JAVA version.
    rem ECHO The program will now exit.
    rem PAUSE
    rem goto exitPGM
    rem )

  3. Open the application.yml file and set up the following parameters.

    Important

    Make sure that you specify the valid parameter values. Specifying invalid values might give unexpected results. 

    Parameter

    Description

    userId

    Valid RACF user ID

    password

    Valid RACF user password

    craProtocol

    Communication protocol that CRA server is set to run with

    Use either HTTP or HTTPS.

    Warning

    Running CRA with HTTP poses a security vulnerability. We recommend running the utility with HTTPS only.

    craHost

    Name of the mainframe host where the CRA server is running

    craPort

    Port number of the CRA server

    dataFilesFolder

    Location where you want the extracted data files to be saved

    Important

    Make sure that the specified location uses forward slashes (/) instead of backward slashes. 

    For example,  dataFilesFolder: "C/CRA/NewFiles/2023-10-12-11-15_Test"

    dataParametersList

    Contains extract data task to request and retrieve data

    Important

    You can add multiple extract data tasks. Make sure that you specify the valid parameter values. For more details, see the example that follows this table. 


    Parameter

    Description

    jobName

    Logical name for data extract task containing request and retrieve product data

    mveServiceName

    (Required) Name of the service (MVE host) from which you want to retrieve data

    productName

    (Required) Name of the product from which you want to retrieve the data

    viewName

    (Required) Name of the view within the product from which you want to retrieve the data

    collectionInterval

    Interval in minutes within which the data is retrieved

    Default value: 5 minutes

    Context

    Context of the server from which you want to retrieve data

    Default value: *

    Server

    Server (SSID) from which you want to retrieve data

    Default value: *

    System

    System (LPAR) from which you want to retrieve data

    Default value: *

    Scope

    Scope for which you want to retrieve data

    Default value: *

    Filter

    Generates a WHERE clause

    Default value: null

    Example: filter APPLID=CMF&SYSNAME=S*

    startRow

    Row from which to start the data retrieval

    Default value: 1

    numberOfRows

    Maximum number of rows to return for the data retrieval

    Default value: 9999             

    dataFilePrefix

    (Optional) Prefix for the data file

    A data file name with prefix takes the following format:

    dataFilePrefix_jobName_yyyyMMddHHmmss

  4. Save your changes. 
Example
configuration:
  userId: "myUserID"
  password: "myPassword"
  craProtocol: "https"
  craHost: "myServer"
  craPort: "8080"
  dataFilesFolder: "C:/CRA/NewFiles/checking/2022-08-15-11-15_Test"
  dataParametersList:
    - jobName: "task_A"
      mveServiceName: "Mv1"
      productName: "mvmvs"
      viewName: "JCPU"
      collectionInterval: 1
      context: "*"
      server: "*"
      system: "*"
      scope: "*"
      filter: ""
      startRow: 1
      numberOfRows: 200
      dataFilePrefix: "PASS1"
    - jobName: "task_B"
      mveServiceName: "Mv2"
      productName: "CMF"
      viewName: "ALARMDIST"
      collectionInterval: 10
      context: "*"
      server: "*"
      system: "*"
      scope: "*"
      filter: ""
      startRow: 1
      numberOfRows: 200
      dataFilePrefix: "PASS2" 



Running the DCC utility

Before you begin

Make sure that you have set the JAVA_HOME environment variable value to your Java home path.

Example

DCC_JAVA_HOME.png

You can run the DCC utility in multiple ways. Use one of the following methods to run the DCC utility according to your preference. 

Important

The DCC utility does not automatically delete any data files that are stored on your local machine. Make sure that you remove the existing data files regularly to avoid slow system performance.

Running the StartCRADataCollector.bat file directly

  1. Navigate to the directory where the utility files are located.
  2. Double-click the StartCRADataCollector.bat file to start the process.
  3. Complete the batch process.

Using the command prompt

  1. On the command prompt, type the following command to navigate to the location where the StartCRADataCollector.bat batch file is stored:
    cd directoryPath
  2. Specify the name of the batch file and press Enter, as shown in the following example:

    C:\cra-utility\CRA-Data-Collector-Utility>StartCRADataCollector.bat

  3. Complete the batch process.

Using the command prompt with inline argument

  1. On the command prompt, type the following command to navigate to the location where the StartCRADataCollector.bat batch file is stored.
    cd
    directoryPath
  2. Specify the name of the batch file with the location of the .yml file location as its argument and press Enter, as shown in the following example: 

    C:\cra-utility\CRA-Data-Collector-Utility>StartCRADataCollector.bat  c:\ymlFullPath\application.yml

  3. Complete the batch process.

Where to go from here

After successful execution, the DCC utility creates requested data files. You can access the data files by navigating to the location that you have specified in the applicaition.yml file. If the DCC utility does not run as expected, an error file (with a prefix Err added to the error file name) is created. 

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*