Testing the chatbot responses by using test cases

As an administrator, create and run test cases to check whether the chatbot application responds to the user queries correctly. By defining a partial expected response, validate whether the chatbot application checks it against the intent and the configured response, ensuring data accuracy.

Before you begin

  • Make sure you have created intents, entities, and dialogs. 
  • If you are using a localized Chatbot Skill, make sure your test data locale matches the workspace locale.
  • Make sure you have created and mapped the Web API connection for Telemetry.

To create test cases

  1. Log in to BMC Helix Innovation Studio and select the BMC Chatbot application.
  2. On the Records tab, select Chatbot Message Monitoring Test Case, and click Edit data.
  3. Click New to create a new record.
  4. In the New Record pane, enter the following details:
    1. From the CHATBOT CONFIGURATION_ID list, select the chatbot configuration.
    2. In the Description field, enter a description for the record.
    3. In the Name field, enter a unique name for the record.
    4. In the Run interval – hours field, select the interval at which this process must run.
    5. In the User query field, type a text string the user might ask the chatbot.
    6. In the Partial expected response field, type the text you expect to get as a response.
      Important: The value you enter in the Partial expected response field is case-sensitive.
      23_03_02_Chatbot_Message_Monitoring_Test_Case_Record.png
      The values for the Created By, Last run, and Status fields are automatically updated.
  5. Click Save.


Alternatively, create views in BMC Helix Innovation Studio to add test cases

To create in-bundle settings for your chatbot application by using the view definitions in BMC Helix Innovation Studio

  1. In BMC Helix Innovation Studio, select the BMC Chatbot.
  2. On the Configurations tab, select New > In-bundle setting. 
    For more information, see Failed to execute the [excerpt-include] macro. Cause: [Error number 2 in 0: No wiki with id [confluencePage:page] could be found]. Click on this message for details.
    .
  3. In the New In-bundle setting window, specify the properties for adding test cases as shown in the following figure:
    23_3_02_Chatbot_inbundle_Add_Test_Cases.png
  4. Click Save.
  5. Repeat step 1 and step 2.
  6. In the New In-bundle setting window, specify the properties for listing the test cases as shown in the following figure:
    23_3_02_Chatbot_inbundle_Test_Cases.png
  7. Click Save.

To create test cases for your chatbot application

  1. In BMC Helix Innovation Studio, navigate to the Administration tab. 
  2. In the Settings pane, select Chatbot Message Monitoring > Add test Cases.
  3. Create a test case by adding details as shown in the following figure:
    23_03_02_Enhancements_Chatbot_Message_Monitoring_Test_Case_List.png
  4. Click + Create Test Case.

To run the test cases

  1. Log in to BMC Helix Innovation Studio and select the BMC Chatbot application.
  2. On the Processes tab, click Manage processes.
  3. From the list of processes, select one of the following processes:
    1. Chatbot Message Action—To check whether the messages to the chatbot are working correctly.
      Specify the chatbot ID, chat ID, and the input text.
    2. Chatbot Message Monitoring Check All—To run all the test cases in a record definition.
    3. Chatbot Message Monitoring Single Check—To run a single test case in a record definition.
      Specify the use case ID and the record definition name.
      23_03_02_Chatbot_Message_Monitoring_Test_Case_Single_Check.png
  4. Click Run.

After the process is run, the status of the test case is displayed on the Chatbot Message Monitoring Test Case list page.

The Chatbot Message Monitoring Log record definition captures the error log information. 

Alternatively, use views in BMC Helix Innovation Studio to run test cases

To validate the test cases

  1. In BMC Helix Innovation Studio, navigate to the Administration tab. 
  2. In the Settings pane, select Chatbot Message Monitoring > Test Cases.
  3. Click Check All Test Cases.
    23_03_02_Enhancements_Chatbot_Message_Monitoring_Test_Case_List_All.png

After the process is run, the status of the test case is displayed on the list page.

To resolve failed test cases

  1. Open a record in the Chatbot Message Monitoring Log record definition to view the error details.
  2. Review the Log short text information.
    23_03_02_Chatbot_Message_Monitoring_Log_Record.png
  3. If necessary, make changes to the intent or the test case, and rerun the test case to validate the responses.







 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*