This documentation supports the 20.08 version of BMC Helix Platform. 
To view an earlier version, select 20.02 from the Product version menu.

Evaluating the cognitive service test results

As an administrator, you can test and evaluate the BMC Helix Cognitive Automation to ensure that the cognitive service correctly auto-categorizes service requests raised by the end users. The test results provide the exact problem area when the data sets do not predict the appropriate categories, so that administrators can rectify the data sets.

For more information about the test metrics, see Leveraging machine learning metrics to improve cognitive service data sets.

Before you begin

After uploading the CSV data set or selecting the application data that you want to test, ensure that you have trained and tested the data set. 

To view the test metrics 

  1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab. 
  2. Click the configuration that you created for training the cognitive service. 
    For example, select My application > Cognitive Training.
  3. Based on the value defined in the Classification-Service-Provider setting, one of the following tabs is displayed:

    ValueTab
    WATSON

    Auto-classification Training and Evaluation - IBM Watson Assistant


    NATIVE

    Auto-classification Training and Evaluation - BMC Native (Google)

  4. From the tab, select Test Results
    The test metrics—accuracy, precision, recall, and F-score are displayed. 

    The following image shows the test metrics for each test data and the CSV file of test results for BMC Native (Google) classification:


  5. In the Results column, select the CSV file corresponding to the data set for which you want to view the test results. 

    The following image is an example of the CSV test results file:


Tip

To delete the test results that are no longer required, you can create a process by using the Delete record element.

To evaluate the cognitive service from the BMC Helix Innovation Studio UI

  1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab. 
  2. Click the configuration that you created for training the cognitive service.
    For example, select My application > Cognitive Training.
  3. Based on the value defined in the Classification-Service-Provider setting, one of the following tabs is displayed:

    ValueTab
    WATSON

    Auto-classification Training and Evaluation - IBM Watson Assistant

    NATIVEAuto-classification Training and Evaluation - BMC Native (Google)
  4. From the tab, select Test Results > Interactive Evaluation
  5. On the Auto-classification Interactive Evaluation page, from the Trained Data Set list, select the data set that you want to evaluate. 
  6. In Text, enter the input text from the data set that you want to categorize.
  7. Click Classify.
    The input text is categorized and the results are displayed on the screen.

To download and rectify the data sets

The CSV data set that you uploaded is split into training data and test data. The test data shows the input texts that were not  categorized as desired.

  1. To download the test results, perform the following steps:
    1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab.
    2. Click the configuration that you created for training the cognitive service.
      For example, select My application > Cognitive Training.
    3. Based on the value defined in the Classification-Service-Provider setting, one of the following tabs is displayed:

      ValueTab
      WATSON

      Auto-classification Training and Evaluation - IBM Watson Assistant

      NATIVEAuto-classification Training and Evaluation - BMC Native (Google)
    4. From the tab, select Test Results.

    5. In the Test Data column, select and download the test data CSV file corresponding to the data set that you want to rectify. 
    6. In the CSV test data file, in the Match column, view the rows that have the value N. 
  2. To rectify the data set, add more input texts for the corresponding categories or remove email addresses, email signatures, salutations, or any other information that is not relevant for categorization. 

Related topics

Leveraging machine learning metrics to improve cognitive service data sets

Types of data sets used to train and test the cognitive service

Testing the chatbot training data to improve predictability

Was this page helpful? Yes No Submitting... Thank you

Comments