Defining search data sets

After you create the in-bundle configuration UI, you must define the search data sets that you want to use for cognitive search. One search data set maps to one IBM Watson Discovery collection.

You can define the following types of search data sets:

  • BMC Helix Innovation Studio search data set (internal search data set)—This data set includes a record definition with the searchable fields. Data from the searchable fields in the record instances is published to the IBM Watson Discovery collection to make it available for cognitive search.

  • External Search Data Set—This data set exists outside of a BMC Helix Innovation Studio custom application. Examples of external data sets include databases, websites, PDF and Microsoft Word documents, and images you have directly uploaded to IBM Watson Discovery. The Knowledge Management data sets are a type of external search data set. 
    To use an external data set in BMC Helix Virtual Agent or a custom application built on BMC Helix Innovation Studio, first create its collection in IBM Watson Discovery. For more information on the types of data sources that IBM Watson Discovery supports, see Connecting to Data Sources.   
                                  

After you define the search data sets, the corresponding data is available for cognitive search in your application. 

Related topics

Leveraging-cognitive-search-in-your-application

Enabling-cognitive-search-in-an-application

Before you begin

To create a BMC Helix Innovation Studio Search Data Set

  1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab.
  2. Click the in-bundle application configuration that you created for search data sets. 
    For example, if you want to add a search data set in the application My application, click My application > Cognitive Search.
  3. In the Cognitive Search Data Sets section, click New.
  4. From the list, select BMC Helix Innovation Studio Data Set.

  5. In the New Cognitive Search Data Set page, enter the values for the following fields:

    Field

    Description

    Data Set Name

    Name of your Search Data Set.

    Description

    Description that provides details about the Search Data Set.

    Locale

    Locale of your search data set.

    Record Definition Name

    Select the record definition that you want to specify in the search data set

    Confidence Threshold

    Specify a value in the range between 0 and 1 that indicates the confidence level of the data set. The confidence threshold is used as a filter to display the search data set. Only the search data set content with a confidence score greater than or equal to the Confidence Threshold value is returned in the search results.

    Important: If the field is left blank, then the default value (0) is selected.

    For more information about IBM guidelines on assigning confidence values, see How to select a threshold for acting using confidence scores.

    The following image displays the New Cognitive Search Data Set page:
    22_1_New cognitive search dataset page.png

  6. Click Save. 
    The search data set is displayed in the Cognitive Search Data Sets section, as shown in the following image:
    22_1_cognitive search data sets.png
    After the data set is saved, its status changes to Created. This data set is mapped to the IBM Watson Discovery collection.

To create an external Search Data Set or Knowledge Management data set

  1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab.
  2. Click the application configuration that you created for search data sets. For more information, see Creating-an-application-configuration-for-defining-search-data-sets.
    For example, if you want to add a search data set in the application My application, click My application > Cognitive Search.
  3. In the Cognitive Search Data Sets section, click New.
  4. From the list, select External Data Set.

  5. In the New Cognitive Search Data Set dialog box, enter the values for the following fields:

    Field

    Action

    Data Set Name

    Enter the name of your search data set. It is recommended that this is the same name as the corresponding collection in IBM Watson Discovery.

    Description

    Enter a description that provides details about the search data set.

    Locale

    Select the locale of your search data set.

    Collection ID

    Enter the Collection ID of an existing collection in IBM Watson Discovery.

    Important: Make sure that you enter the Collection ID of the external data set. To get the Collection ID, log in to IBM Watson Discovery.

    External Data Set Source

    Select one of the following options:

    • RKM—Select if you want to create a data set for Knowledge Management articles.
    • Other—Select if you want to create a data set for any other external data source, which is not Knowledge Management.

    Confidence Threshold

    Specify a value in the range between 0 and 1 that indicates the confidence level of the data set. The confidence threshold is used as a filter to display the search data set. Only the search data set content with a confidence score greater than or equal to the Confidence Threshold value is returned in the search results.

    Important: If the field is left blank, then the default value (0) is selected.

    For more information about IBM guidelines on assigning confidence values, see How to select a threshold for acting using confidence scores

  6. Click Save.

    The external Search Data Set is displayed in the Cognitive Search Data Sets section.

    The documents within this IBM Watson Discovery collection are included in the cognitive search functionality for this application.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*