Measuring the cognitive service consumption

Administrators can monitor the consumption of the BMC Helix Cognitive Automation by using the Cognitive Consumption dashboard on the BMC Helix Innovation Studio UI. You can also download the consumption reports, configure thresholds, and configure notifications so that you receive an email when the consumption exceeds the specified threshold.

Related topic

Licensing-overview

 

Important

IBM Watson Assistant will be retiring its Tone Analyzer capability by February 2023, a user sentiment detection within BMC Helix Innovation Studio applications. BMC will no longer activate new BMC Helix Innovation Studio customers with IBM Watson's Tone Analyzer capability effective April 13, 2022.

BMC is reviewing options with IBM for potential replacement capabilities for its intelligent user sentiment detection.

You can view the consumption reports of various IBM Watson services provisioned to work with BMC Helix Innovation Studio.

By monitoring the cognitive service consumption, you can get answers to the following questions and make informed decisions:

  • How much of a service am I using?
  • Is there a need to purchase more capacity?

Important

  • If you have subscribed to IBM Watson Plus or IBM Watson Premium plan, a single instance of the service is configured in BMC Helix Innovation Studio. The consumption report shows the usage of the environment that you have currently logged in, for example, QA environment and does not include usage from production and tailoring environments. To get the overall usage of the IBM Watson Assistant instance, contact BMC Support.

  • (For fresh deployment of version 21.3) Raw data is retained for a period of one month only. Therefore, the following reports are populated only if the data available is for more than a month:

    • BMC Helix Cognitive Automation API Call Summary
    • User summary
    • Documents

    Examples:

    • If data is available from July 15 to August 30, the report shows the data from July 15 to August 15.
    • If data is available from July 1 to August 5, the report shows the data from July 1 to July 31.

Reports on the Cognitive Consumption dashboard

Operation

Report name and Time range

Parameter measured

Example

Chat

  • Users (N)—monthly
  • User summary—annual (from activation of the service)

Number of unique users logged in to a chatbot

 

221_Telemetry - Chatbot users.png

  • Conversations (N)—monthly
  • Conversation Summary—annual (from activation of the service)

Number of chat conversations

Important: The BMC Helix Virtual Agent consumption is changed from conversation-based measurement to user-based measurement. If you have already purchased BMC Helix Virtual Agent based on the number of chat conversations, you can continue to view the conversation data, however, you will not see the number of unique users data.

The following actions are considered as a chat conversation:

  • When a chat session times out
  • When an administrator clears the chat session to mark the session complete
  • When a conversation has 50 responses

221_Telemetry - Conversations.png

BMC Helix Cognitive Automation

  • API Calls (N)—monthly
  • API Call Summary—annual (from activation of the service)

Number of API calls made for classification, summarization, or tone analysis.

221_Telemetry - Cognitive Service Automation.png

Cognitive search

  • Documents (N)—monthly
  • Document Summary—annual (from activation of the service)
  • Document Storage (GB)—monthly
  • Document Storage Summary—annual (from activation of the service)

 

  • Number of documents uploaded to a collection in IBM Watson Discovery
  • Size of the documents uploaded to the collection in IBM Watson Discovery

221_Telemetry - Cognitive search.png

Methods for monitoring cognitive service consumption

You can monitor the cognitive service consumption in the following ways:

  • Configure the threshold and email notifications when the cognitive service usage exceeds a specified threshold.
    You can configure the email addresses and three thresholds each for API calls, document storage capacity, and conversation capacity.  For more information about configuring the threshold and email addresses for notifications, see To configure thresholds and email notifications.
  • View the notification history
    The notification history displays the list of notifications that are triggered when the cognitive service usage exceeds the specified threshold. The notification history displays entries only after notifications are triggered. For more information, see To view the notifications history.

 To download the cognitive service usage report from the Cognitive Consumption dashboard

  1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab. 
  2. Select Cognitive and chatbot > Cognitive consumption. 
    The Cognitive Consumption dashboard displays statistics for each operation—chatbot, cognitive search, and cognitive service automation (includes auto-categorization, summarization, tone analysis).
  3. On the Cognitive Consumption page, click Download.
    cognitive_download.png
  4. In the Download CSV file dialog box, perform any of the following actions:
    • Select the predefined time period.

    • Specify a custom time period by selecting Selected time period and then defining a range in Select range field.

      Important

      The Select range field is enabled only when you select Selected time period.

      coginitive_consumption_download.png

  5. In the Download CSV file dialog box, click Download.

The CSV report is downloaded. The following image shows a sample report:

Telemetry - CSV report.png

 To configure thresholds and email notifications for cognitive service usage

You can configure up to three thresholds and 10 email addresses.

  1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab. 
  2. Select Cognitive and chatbot > Cognitive consumption. 
  3. Click the Notifications icon Notifications screenshot.png that corresponds to the operation for which you want to configure the thresholds.
  4. Configure the capacity and threshold.
    1. In the Capacity consumption thresholds section, enter the total capacity. 
      Example: In the Capacity section for Chatbot Users, enter 100. 
    2. In the Threshold 1 (%) field, enter the threshold for when you want to trigger the first notification. 
      Example: 20.  If the total capacity is 100 users, the first notification is triggered when the number of users exceed 20. 
    3. In the Threshold 2 (%) field, enter the threshold for when you want to trigger the second notification.
      Example: 40. If the total capacity is 100 users, then the second notification is triggered when the number of users exceed 40.
    4. In the Threshold 3 (%) field, enter the threshold for when you want to trigger the third notification. 
      Example: 60. If the total capacity is 100 users, then the third notification is triggered when the number of users exceed 60.
  5. Configure the email addresses.
    1. From the Available email notifications recipients list, which displays the email addresses of all the users created in the Foundation data for a tenant, select the email addresses to which you want to send the cognitive service consumption notifications. 
      You can select up to 10 email addresses. 
    2. To add the email addresses to the Selected email notifications recipients list, click Right arrow to add email addresses..png. 
    3. Click Save. 

 To view the notification history

  1. Log in to BMC Helix Innovation Studio and navigate to Administration tab. 
  2. Select Cognitive and chatbot > Cognitive consumption. 
  3. Click the Notifications icon Notifications screenshot.png that corresponds to the operation for which you want to view the notifications history. 
  4. Click Notification history. 

 

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*