Evaluating the cognitive service test results

Before you begin

After uploading the CSV data set or selecting the application data that you want to test, make sure that you have trained and tested the data set. For more information, see Training-and-testing-the-cognitive-service-for-a-custom-application.

To view the test metrics

1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab. 
2. Click the configuration that you created for training the cognitive service. 
   For example, select My application > Cognitive Training.

3. Based on the value defined in the Classification-Service-Provider setting, one of the following tabs is displayed:

   Value	Tab
   WATSON	Auto-classification training and evaluation - IBM Watson Assistant
   NATIVE	Auto-classification training and evaluation - BMC Native (Google)

   For more information about the Classification-Service-Provider setting, see Configuration settings C-D.

4. From the tab, select Test Results. 
   The test metrics—accuracy, precision, recall, and F-score are displayed. 
   The following image shows the test metrics for each test data and the CSV file of test results for BMC Native (Google) classification:
   
   
   
5. In the Results column, select the CSV file corresponding to the data set for which you want to view the test results. 
   The following image is an example of the CSV test results file:
   
   
   

To evaluate the cognitive service from the BMC Helix Innovation Studio UI

1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab. 
2. Click the configuration that you created for training the cognitive service.
   For example, select My application > Cognitive Training.

3. Based on the value defined in the Classification-Service-Provider setting, one of the following tabs is displayed:

   Value	Tab
   WATSON	Auto-classification training and evaluation - IBM Watson Assistant
   NATIVE	Auto-classification training and evaluation - BMC Native (Google)

   For more information about the Classification-Service-Provider setting, see Configuration settings C-D.

4. From the tab, select Test Results > Interactive Evaluation. 
5. On the Auto-classification Interactive Evaluation page, from the Trained Data Set list, select the data set that you want to evaluate. 
6. In Text, enter the input text from the data set that you want to categorize.
7. Click Classify.
   The input text is categorized and the results are displayed on the screen.

To download and rectify the data sets

The CSV data set that you uploaded is split into training data and test data. The test data shows the input texts that were not categorized as desired.

1. To download the test results, perform the following steps:
   1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab.
   2. Click the configuration that you created for training the cognitive service.
      For example, select My application > Cognitive Training.

   3. Based on the value defined in the Classification-Service-Provider setting, one of the following tabs is displayed:

      Value	Tab
      WATSON	Auto-classification training and evaluation - IBM Watson Assistant
      NATIVE	Auto-classification training and evaluation - BMC Native (Google)

      For more information about the Classification-Service-Provider setting, see Configuration settings C-D.

   4. From the tab, select Test Results.
   5. In the Test Data column, select and download the test data CSV file corresponding to the data set that you want to rectify. 
      
   6. In the CSV test data file, in the Match column, view the rows that have the value N. 
2. To rectify the data set, add more input texts for the corresponding categories or remove email addresses, email signatures, salutations, or any other information that is not relevant for categorization.