This documentation supports the 19.05 version of BMC Helix Chatbot.
To view the documentation for the previous version, select 19.02 from the Product version menu.

Tracking events to generate chatbot usage reports

BMC Helix Chatbot customers can track events such as successful transfer of chat to a live agent and generate reports. The usage metrics help you to understand how end users are using the chatbot application, derive statistics about a successful or unsuccessful event, and so on. You can use the out-of-the-box events and reports, specify new events and generate reports, or enable or disable events for tracking. By default, the user name and event creation date attributes are tracked. You cannot modify or delete these attributes. 

Out-of-the-box events, event statistics, and reports

The following table lists the out-of-the-box events, event statistics, and reports: 

Event name and descriptionEvent attributesEvent Statistics for the Event

Reports and their example

Service Request

(When a service request is successfully submitted from the BMC Helix Chatbot)

  • ServiceName
  • Channel
  • CreatedDate
  • BotID
  • BotName
  • SuccessOrFailure
  • ReasonForFailure
  • Service Request By Bot Id
  • Service Request By Bot Id and Channel
  • Service Request By Bot ID
  • Service Request By Channel

 Example of the Service Request report


Successful transfer to agent

(When the end user's request to transfer the chat to a live agent is successful)

  • User
  • Channel
  • CreatedDate
  • BotID
  • BotName
  • Successful Transfer to Agent By Bot Id
  • Successful Transfer to Agent By Bot Id and Channel
  • Successful Transfer to Agent By Bot Id
  • Successful Transfer to Agent By Channel

 Example of the Successful Transfer to Agent report


Failed transfer to agent

(When the end user's request to transfer the chat to a live agent fails)

  • ReasonForFailure
  • User
  • Channel
  • CreatedDate
  • BotID
  • BotName
  • Failed Transfer to Agent By Bot Id
  • Failed Transfer to Agent By Bot Id and Channel
  • Failed Transfer to Agent By Bot Id
  • Failed Transfer to Agent By Channel
 Example of the Failed Transfer to Agent report


Failed entitlements check

(When entitlements check fails)

  • Itemthatwasbeingchecked
  • User
  • Channel
  • CreatedDate
  • BotID
  • BotName
  • Failed Entitlement Check by Bot Id
  • Failed Entitlement Check By Channel


  • Failed Entitlement Check by Bot Id
  • Failed Entitlement Check By Channel
 Example of the Failed Entitlement Check report

User Feedback

(When user rating and survey comments are successfully entered from the BMC Helix Chatbot)

  • UserName
  • CreatedDate
  • BotID
  • BotName
  • Channel
  • Rating
  • SessionId
  • Comment
  • User Feedback By Bot Id and Channel
  • User Feedback By Bot Id
  • User Feedback By Channel
None

Chat Count by Bot Id report

In addition to the application usage reports, BMC Helix Platform provides a report for number of cogntive service chat conversations by Bot Id. This report helps administrators decide whether to buy more chat capacity, which chatbot is using more capacity, and so on. 

Example of the Chat Count By Bot Id report

A consolidated report for all chatbots in BMC Helix Platform is available in the Telemetry dashboard. For more information, see  Measuring the cognitive service consumption .

BMC Remedy Smart Reporting licensing and access

You can generate the chatbot usage metrics reports in  BMC Remedy Smart Reporting As an administrator, you must enable users to access the BMC Remedy Smart Reporting dashboard by assigning the Event Report Access Role

Before you begin

  • Ensure that you have access to the IBM Watson Assistant Skills.
  • Ensure that you have the BMC Helix Platform license.

  • To view the reports, ensure that the administrator has  enabled access to the usage reports .

Process for tracking events and generating reports

The following table describes the tasks in the process for generating usage reports for a chatbot or a custom application:

TaskActionDescriptionReference
1Create event definition.

Specify the event and its attributes that you want to track.

Events are the activities that occur in an application. Event attributes comprise the data related to the event that you want to track.

To create an event definition
2Create event statistics definition.

Specify the frequency at which you want to capture the event statistics.

Even statistics are the frequency at which you want to track the event data and to group the data to derive consolidated metrics.

To create an event statistics definition
3Add the Publish Event element in a process or a rule.Configure the Publish Event element to send the event data to the Telemetry service.

Storing the application event details to generate usage report

4Generate the application usage reports. Go to the BMC Remedy Smart Reporting dashboard to generate the reports.To generate reports

Note

When working in the Customization layer , you can only enable or disable the Event definition or the Event Statistics definition.

Task 1: To create an event definition

  1. Log in to BMC Helix Innovation Studio.
  2. On the Workspace tab, perform one of the following steps:
    • If you want to create an event definition for BMC Helix Chatbot, click BMC Chatbot.
    • If you want to create an event definition for your chatbot application, select the application in which you have created a chatbot.
    • If you want to create an event definition for a custom application, select the application.  
  3. Click Events > New
  4. On the Event Properties tab, in the Name field, enter the name of the event. 
    For example, the Knowledge search event.

    Note

    You cannot rename an event.

  5. (Optional) If you want to disable the event for tracking, click the toggle key. 
  6. (Optional) If you want to add event attributes, click New Attribute
    By default, all events have the UserName (login name of the user) and CreatedDate (time stamp of the event) attributes. You cannot modify or delete these attributes. 
  7. Select the type of attribute. 
  8. On the  Event Attribute Properties tab, enter the attribute name without adding spaces.

    Note

    You cannot rename an event attribute.

  9. Click Save.

Task 2: To create the event statistics definition

  1. Log in to BMC Helix Innovation Studio.
  2. On the Workspace tab, perform one of the following steps:
    • If you want to create an event statistics definition for BMC Helix Chatbot, click BMC Chatbot.
    • If you want to create an event statistics definition for your chatbot application, select the application in which you have created a chatbot. 
    • If you want to create an event statistics definition for a custom application, select the application.
  3. Click Events Statistics New.
  4. On the Event tab, in Name, enter a name for the event statistics definition. 

    For example, for the Knowledge search event, you can add the Knowledge usage statistics.

    Note

    You cannot rename an Event Statistics definition.

  5. From the Event list, select the event for which you want to create the event statistics definition. 
    For example, select the Knowledge search event.
  6. (Optional) If you do not want to collect these statistics for this event, click the toggle key.
  7. (Optional) In the REPORTING section, if you do not want to include the number of event occurrences in the statistics, click the Count Events check box. 

  8. To group the attributes so that data is consolidated as a single usage metrics, in the Group by Attributes section, click Add Attribute to Group By and select the attributes from the list.

    Note

    You can group attributes of only integer, text, or boolean data types.

    For example, for the Knowledge Search event and the Knowledge usage statistics, you can group the attributes by Article ID so that the count, average hits, maximum hits, and minimum hits for that article are derived as a single usage metrics report for the Knowledge search event.

  9. In Report By, select the frequency at which you want BMC Helix Chatbot to report the usage metrics. 

  10. Click Save. 

 Task 4: To generate reports

  1. Log in to BMC Helix Innovation Studio.
  2. On the Workspace tab, click Reports
    You are redirected to the reporting dashboard. 
  3. Select Browse > Browse All
  4. In By Folder, click Chatbot Reports and select the report that you want to generate. 
    For example, click the out-of-the-box report Failed transfer to agent
  5. (Optional) To export the reports, click 

(Optional) To disable event tracking

  1. Log in to BMC Helix Innovation Studio
  2. On the Workspace tab, perform one of the following steps:
    • If you want to disable an event for BMC Helix Chatbot, click BMC Chatbot.
    • If you want to disable an event for your chatbot application, select the application in which you have created a chatbot. 
    • If you want to disable an event for a custom application, select the application.
  3. Click Events
  4. Click the event definition that you want to disable. 
  5. On the Event Properties tab, click Enabled in the toggle key.
  6. Click Save

Troubleshooting—unable to generate or view reports

The following table provides the troubleshooting steps if users are unable to generate or view reports:

ScenarioTroubleshooting steps
  • Unable to access the Reports link.
  • Unable to view data in the reports for a specific application.
  • Unable to generate custom reports.
  • Ensure that the administrator has assigned the Event Report Access Role to you.
  • If you are an application business analyst, you can generate reports for only those applications that you can access.
  • I am assigned the Event Report Access Role and have access to the application, but I am unable to access the reports.

If the Event Statistics definition is deleted, the metrics are not populated and the reports are not generated.

  1. Log in to BMC Helix Innovation Studio.
  2. On the Workspace tab, click the application in which you created an event and event statistics definition.
  3. To check whether the Event Statistics definition is present, click the Events Statistics tab.

Related topics

Tracking application events to generate usage reports

Enabling integration with BMC Helix Digital Workplace Advanced and BMC Helix Business Workflows

Customizing the chatbot actions

Was this page helpful? Yes No Submitting... Thank you

Comments