Tracking and generating chatbot usage reports
BMC Helix Virtual Agent customers can track events such as successful transfer of chat to a live agent and generate reports. To learn about the out-of-the-box-events that can be tracked, see BMC Helix Virtual Agent usage reports.
This document describes the process for tracking custom events and generating reports for these events.
BMC Helix ITSM: Smart Reporting licensing and access
As an application business analyst, you generate the chatbot usage metrics reports in BMC Helix ITSM: Smart Reporting after the administrator has assigned the Event Report Access Role in BMC Helix Innovation Studio. The Event Report Access Role enables access to the BMC Helix ITSM: Smart Reporting dashboard. For more information, see
Enabling access to BMC Helix ITSM: Smart Reporting
.
Before you begin
- Ensure that you have access to the IBM Watson AssistantSkill.
Ensure that you have the BMC Helix Innovation Suite license.
- To view the reports, ensure that the administrator has enabled access to the usage reports. For more information, see
Enabling access to BMC Helix ITSM: Smart Reporting
.
The following table describes the tasks in the process for generating usage reports for a chatbot or a custom application:
Note
The reports are meant for tracking the chatbot usage and only aggregated data is available to generate reports. To create reports for raw events in the Telemetry service, see Creating reports for raw events in the Telemetry service.
Task | Action | Description | Reference |
---|---|---|---|
1 | Create event definition. | Specify the event and its attributes that you want to track. Events are the activities that occur in an application. Event attributes comprise the data related to the event that you want to track. | To create an event definition |
2 | Create event statistics definition. | Specify the frequency at which you want to capture the event statistics. Even statistics are the frequency at which you want to track the event data and to group the data to derive consolidated metrics. | To create an event statistics definition |
3 | Add the Publish Event element in a process or a rule. | Configure the Publish Event element to send the event data to the Telemetry service. | |
4 | Generate the application usage reports. | Go to the BMC BMC Helix ITSM: Smart Reporting dashboard to generate the reports. | To generate reports |
Note
When working in the
Customization layer
, you can only enable or disable the Event definition or the Event Statistics definition.
Task 1: To create an event definition
- Log in to BMC Helix Innovation Studio.
- On the Workspace tab, perform one of the following steps:
- If you want to create an event definition for BMC Helix Virtual Agent, click BMC Chatbot.
- If you want to create an event definition for your chatbot application, select the application in which you have created a chatbot.
- If you want to create an event definition for a custom application, select the application.
- Click Events > New.
On the
Event Properties tab, in the Name field, enter the name of the event.
For example, the Knowledge search event.Note
You cannot rename an event.
- (Optional) If you want to disable the event for tracking, click the toggle key.
- (Optional) If you want to add event attributes, click New Attribute.
By default, all events have the UserName (login name of the user) and CreatedDate (time stamp of the event) attributes. You cannot modify or delete these attributes. - Select the type of attribute.
On the
Event Attribute Properties tab, enter the attribute name without adding spaces.Note
You cannot rename an event attribute.
- Click Save.
Task 2: To create the event statistics definition
- Log in to BMC Helix Innovation Studio.
- On the Workspace tab, perform one of the following steps:
- If you want to create an event statistics definition for BMC Helix Virtual Agent, click BMC Chatbot.
- If you want to create an event statistics definition for your chatbot application, select the application in which you have created a chatbot.
- If you want to create an event statistics definition for a custom application, select the application.
- Click Events Statistics > New.
On the
Event Properties tab, in Name, enter a name for the event statistics definition.For example, for the Knowledge search event, you can add the Knowledge usage statistics.
Note
You cannot rename an Event Statistics definition.
- From the Event list, select the event for which you want to create the event statistics definition.
For example, select the Knowledge search event. - (Optional) If you do not want to collect these statistics for this event, click the toggle key.
(Optional) In the REPORTING section, if you do not want to include the number of event occurrences in the statistics, click the Count Events check box.
To group the attributes so that data is consolidated as a single usage metrics, in the Group by Attributes section, click Add Attribute to Group By and select the attributes from the list.
Note
You can group attributes of only integer, text, or boolean data types.
For example, for the Knowledge Search event and the Knowledge usage statistics, you can group the attributes by Article ID so that the count, average hits, maximum hits, and minimum hits for that article are derived as a single usage metrics report for the Knowledge search event.
In Report By, select the frequency at which you want BMC Helix Virtual Agent to report the usage metrics.
- Click Save.
Task 4: To generate reports
- Log in to BMC Helix Innovation Studio.
- On the Workspace tab, click Reports.
You are redirected to the reporting dashboard. - Select Browse > Browse All.
- In By Folder, click Chatbot Reports and select the report that you want to generate.
For example, click the out-of-the-box report Failed transfer to agent. - (Optional) To export the reports, click
(Optional) To disable event tracking
- Log in to BMC Helix Innovation Studio.
- On the Workspace tab, perform one of the following steps:
- If you want to disable an event for BMC Helix Virtual Agent, click BMC Chatbot.
- If you want to disable an event for your chatbot application, select the application in which you have created a chatbot.
- If you want to disable an event for a custom application, select the application.
- Click Events.
- Click the event definition that you want to disable.
- On the Event Properties tab, click Enabled in the toggle key.
- Click Save.
Troubleshooting report generation or viewing issues
The following table provides the troubleshooting steps if users are unable to generate or view reports:
Scenario | Troubleshooting steps |
---|---|
|
|
| If the Event Statistics definition is deleted, the metrics are not populated and the reports are not generated.
|
Related topics
Tracking application events to generate usage reports
Enabling integration with BMC Helix Digital Workplace Advanced and BMC Helix Business Workflows
Customizing the chatbot actions
Viewing BMC Helix Virtual Agent reporting dashboard
Comments
Log in or register to comment.