Default language.

Auto categorization of incidents

BMC Helix Remedyforce integrates with BMC Helix Platform to provide support for the Auto Categorization feature that provides the following functionalities:

  • Incidents that are created with a blank category can be automatically assigned a category based on the incident description. The categories are identified using calculations based on a confidence score from the cognitive service.
  • Show IT Staff suggested categories based on recommendations from the cognitive service.  Staff users can select one of the suggested categories or override the suggestion made by the cognitive service.

BMC Helix Platform provides a choice between IBM Watson Assistant and BMC Native (Google) as the classification service provider that you can use for auto-categorization. 

If you select IBM Watson Assistant, then BMC will perform the required configurations. For more information about IBM Watson Assistant configuration, see Configuring cognitive service for custom applications by using IBM Watson. 

If you select BMC Native (Google), then you will have to perform the required configurations for BMC Native (Google). For more information about BMC Native (Google) configuration, see Configuring cognitive service for a custom application by using BMC Native (Google) classification.

Note

The Auto Categorization feature is not available for the Standard Layout, Incident Console, and Service Cloud Console.

For more information about how to configure the feature, refer the following sections:

Prerequisites

To use the Auto Categorization feature, the following requirements should be met:

  • Purchase BMC Helix Platform and APIs. Reach out to your Business Relationship Manager or BMC Account Manager to discuss purchasing the BMC Helix Platform as well as the necessary API Calls.
  • Once the environment is provisioned, BMC will share the necessary details, such as follows:
    • BMC Helix Innovation Studio URL 
    • Tenant Admin Username
    • Tenant Admin Password
  • Training data file that you need to create for training the cognitive service.

What is the training data file

To use BMC Helix Platform Cognitive Service for auto-categorization, you must train the cognitive service using a training data file, which is, basically a CSV file.

The training data CSV should have the following format. 

<Description> <Category ID|Category Name| ParentCategory-->Child1-->Child2....-->ChildN>

For example,

a261U000001mA7n|Mobile Network Issue|Network-->Network Issue

Click here to see a sample file.

After you have created the CSV file, it is essential that you scrub the data so that the file contains the most relevant information that can be used by the cognitive service to auto fill or auto suggest the categories.

Preparing the training data file

To prepare the training data file, you have to perform the following tasks:

To download a Data Loader from the Salesforce organization

Log in to the Salesforce organization. 

  1. Go to Setup
  2. In the Quick Search field, type data loader.
    image_AutoCat_SearchDataLoader.png

  3. Download required version of the Data Loader.
    image_Auto_categorization_DataLoaderVersion.png

    Note

    If there are any issues with the installation process, refer to the installation instructions provided on the Data Loader page.

To export incidents data

  1. Launch the Data Loader application.
  2. Click Export.
    image_Auto_categorization_DataLoader.png
  3. Select the Password Authentication option.
  4. Provide organization details, user name, and password.
  5. Provide any of the following login URL based on your organization type:
  6. Click Log in.
    image_Auto_categorization_SalesforceOrgLogIn.png
  7. Once authentication is completed, click Next.
  8. Select the Incident object from the list.
  9. Provide the required file location and file name.
    image_Auto_categorization_SelectDataObject.png
  10. Click Next.

  11. Copy the following query in the Query section.
    SELECT BMCServiceDesk__incidentDescription__c, BMCServiceDesk__FKCategory__c, BMCServiceDesk__FKCategory__r.name, BMCServiceDesk__FKCategory__r.BMCServiceDesk__parentTree__c
    FROM BMCServiceDesk__Incident__c
    WHERE (BMCServiceDesk__isServiceRequest__c = false) and (BMCServiceDesk__Category_ID__c != null)

    • If you want to exclude Open incidents, append the following to the previous query.

    and (BMCServiceDesk__state__c = false )

    • If you want to add date range, append the following to the previous query.

    CreatedDate > 2019-06-21T00:00:00Z and CreatedDate < 2019-07-23T00:00:00Z

    • This query provides a list of closed incidents along with their descriptions, categories, category – parent tree.
    • Service request records and incident records that are without category are excluded from this query result.

    image_Auto_categorization_EditQuery.png

  12. Click Finish.
  13. Click Yes to confirm.
    Export operation is performed and on the completion of the operation, the Operation Finished message pop up is shown.
  14. Click OK.
    You can check exported file at location specified in Step 9.

To create a CSV training file

  1. Open the CSV file with that contains the exported data of incidents. The CVS file will look like the following image.
    image_Auto_categorization_CSVFile.png
  2. If you did not modify the query by adding or removing any field, the CSV file will contain the following columns:
    • BMCSERVICEDESK__INCIDENTDESCRIPTION__C (Incident Description)
    • BMCSERVICEDESK__FKCATEGORY__C (Category ID)
    • BMCSERVICEDESK__FKCATEGORY__R.NAME (Category Name)
    • BMCSERVICEDESK__FKCATEGORY__R.BMCSERVICEDESK__PARENTTREE__C (Category Parent Tree)
  3. If the CSV file has any record where incident description is blank, then remove such a record using excel filter or you can provide a short description for those records.
  4. Provide a temporary name to the cell E1.
  5. Copy the following Excel formula in cell E2 to concatenate the data of columns B, C, D as per the training data file format.
    Excel formula: - =CONCATENATE(B2,"|",C2,IF(D2<>"",CONCAT("|",D2),""))
    Column E will show data as - a261U000001mA7nQAE|Mobile Network Issue|Network Issue-->Network
  6. Copy this formula to all rows by dragging.
  7. Concatenated string is added for all rows as follows.

    image_Auto_categorization_Concat_CSV.png
  8. Copy Column E and paste in Column F as Value by right clicking.
    image_Auto_categorization_CSV_Value.png
    Note: Ensure that you select Paste as Value.
  9. Delete columns B, C, D, and E.
  10. The CSV now shows two columns: A (Description), B (Concatenated string of Category ID|Category Name|Parent Tree)
  11. Remove Header (1st row) of the CSV (A1, B1 cell)
  12. Save file.
    The final CSV should look like the following image.
    image_Auto_categorization_finalCSV.png

Guidelines for preparing the training data

Following are some guidelines to prepare the training data. For more detailed guidelines for preparing the training data, see  and Training and testing the cognitive service for a custom applicationTypes of data sets used to train and test the cognitive service.

  • The descriptions should contain relevant keywords for a particular category.
  • The description should have the phrases most frequently used by clients to submit tickets.
  • As the CSV file can contain only 25,000 records, you can fetch the records from previous months based on the number of records submitted per month. For example, if 5000 records are submitted per month, you can export the last 5 months record, but if only 2000 records are submitted per month, you can export the last 2 years' records.
  • Provide more data around the most frequently used categories.
  • Also, while exporting the records, consider data sets across different peak times in your organization.
  • Category Name should be less than 512 characters.
  • File should not have more than 2 columns.

Configuring the Auto Categorization feature

Configuring the Auto Categorization feature is a three-step process as follows. To perform the actual steps for the process, see To enable Auto Categorization.

image_Auto_categorization_process.png

Step 1: Enabling the Auto Categorization feature

image_Auto_categorization_process_step1.png

Enable Auto Categorization by clicking the button and enter Innovation Studio credentials to establish a connection with Innovation Studio. You also need to provide confidence scores for autofill and suggested categories.

Step 2: Uploading the training data

image_Auto_categorization_process_step2.png

Upload a CSV file that contains the training data and start the training. If any other file type is selected, then appropriate validation checks are carried out. Note that exception is thrown if the training data file that you upload is more than 3.5 MB.

If the selected file is valid, the training data is passed over to Innovation Studio and a success message is shown on the completion of training along with the following information:

  • Details of the uploaded file like file name, file size, owner, and date and time of upload.
  • Test results from the last successful uploaded file is available for download by clicking the file links Training Data, Test Data, and Training Result Data.

Step 3: Starting the Auto Categorization feature

image_Auto_categorization_process_step3.png

Once the training is completed successfully, start the Auto Categorization feature that triggers a background process. This process keeps a watch on newly created incidents and works as follows:

  1. The process runs every one minute and pick up all the newly created incidents in the last two minutes.
  2. Only those incidents are picked whose category information is blank.
  3. For each selected incident, the process picks up the description of the incident and make a call to cognitive auto-classification service for getting cognitive category suggestions.
  4. The category is automatically filled for the incident only if it satisfies the below conditions:
    1. The category confidence score is greater than or equal to administrator set confidence score expectation for applying the category (Autofill confidence score criteria)
      1. If more than one record matches the criteria, the highest confidence score category is selected.
      2. If two categories match with the same confidence score, the first one in the category list (as provided by auto-classification service) is selected
    2. All the existing entitlements, except lookup filters, for the category are honored.

The following image shows the Auto Categorization settings.

image_Auto_categorization.png

To enable Auto Categorization

  1. Go to the Remedyforce Administration tab.
  2. On the Home page, click Integrations > Auto Categorization Settings.
  3. On the Auto Categorization page, click Enable Auto Categorization.
  4. In Step 1, enter the following information:
    • BMC Helix Innovation Studio URL
    • Tenant Admin Username
    • Tenant Admin Password
  5. Select a confidence score for autofill categories.
  6. Select a confidence score for suggested categories.
  7. Click Save.
    If the connection to Innovation Studio is successfully established, the credentials are saved and the Step 2 is enabled.
  8. In Step 2, click Choose File to select a training file that you want to upload.
  9. In the Locale drop down, click the required locale to create the related workspace in Watson.
  10. Adjust the Data Split value between training data and test data.
  11. Click Train And Test.
    The training process starts after clicking this button. The  Refresh icon is displayed that you can click to get the latest training status. It might take several minutes to complete the training depending on the size of the data file. When the training completes successfully, the details of the uploaded file and test results are also displayed.
  12. Click Start.

Related topic

Viewing-and-assigning-a-suggested-category-to-an-incident
Troubleshooting-issues-related-to-auto-categorization-of-incidents