Defining search data sets

After you create the in-bundle configuration UI, you must define the search data sets that you want to use for cognitive search. One search data set maps to one IBM Watson Discovery collection.

You can define the following types of search data sets:

• BMC Helix Innovation Studio search data set (internal search data set)—This data set includes a record definition with the searchable fields. Data from the searchable fields in the record instances is published to the IBM Watson Discovery collection to make it available for cognitive search.

• External Search Data Set—This data set exists outside of a BMC Helix Innovation Studio custom application. Examples of external data sets include databases, websites, PDF and Microsoft Word documents, and images you have directly uploaded to IBM Watson Discovery. The Remedy Knowledge Management data sets are a type of external search data set.
  To use an external data set in BMC Helix Virtual Agent or a custom application built on BMC Helix Innovation Studio, first create its collection in IBM Watson Discovery. For more information on the types of data sources that IBM Watson Discovery supports, see Connecting to Data Sources.   
                                

After you define the search data sets, the corresponding data is available for cognitive search in your application. 

Before you begin

• Create the IBM Watson Discovery collection. 
  For more information about using Remedy Knowledge Management articles, see Installing-and-configuring-the-cognitive-search-data-crawler-for-BMC-Helix-ITSM-Knowledge-Management-articles.
  For more information on how to leverage an external database or file system, see Creating-file-system-and-database-collections-for-searching-external-data.
• Create an in-bundle setting in your application. 
• Configure IBM Watson Discovery service.

To create a BMC Helix Innovation Studio Search Data Set

1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab.
2. Click the in-bundle application configuration that you created for search data sets. 
   For example, if you want to add a search data set in the application My application, click My application > Cognitive Search.
3. In the Cognitive Search Data Sets section, click New.
4. From the list, select BMC Helix Innovation Studio Data Set.

5. In the New Cognitive Search Data Set page, enter the values for the following fields:

   Field	Description
   Data Set Name	Name of your Search Data Set.
   Description	Description that provides details about the Search Data Set.
   Locale	Locale of your search data set.
   Record Definition Name	Select the record definition that you want to specify in the search data set
   Confidence Threshold	Specify a value in the range between 0 and 1 that indicates the confidence level of the data set. The confidence threshold is used as a filter to display the search data set. Only the search data set content with a confidence score greater than or equal to the Confidence Threshold value is returned in the search results. Note: If the field is left blank, then the default value (0) is selected. For more information about IBM guidelines on assigning confidence values, see How to select a threshold for acting using confidence scores.

   The following image displays the New Cognitive Search Data Set page:
   

6. Click Save. 
   The search data set is displayed in the Cognitive Search Data Sets section, as shown in the following image:
   
   After the data set is saved, its status changes to Created. This data set is mapped to the IBM Watson Discovery collection.

To create an external Search Data Set or Remedy Knowledge Management data set

1. Log in to BMC Helix Innovation Studio and navigate to the Administration tab.
2. Click the application configuration that you created for search data sets. For more information, see Creating-an-application-configuration-for-defining-search-data-sets.
   For example, if you want to add a search data set in the application My application, click My application > Cognitive Search.
3. In the Cognitive Search Data Sets section, click New.
4. From the list, select External Data Set.

5. In the New Cognitive Search Data Set dialog box, enter the values for the following fields:

   Field	Action
   Data Set Name	Enter the name of your search data set. It is recommended that this is the same name as the corresponding collection in IBM Watson Discovery.
   Description	Enter a description that provides details about the search data set.
   Locale	Select the locale of your search data set.
   Collection ID	Enter the Collection ID of an existing collection in IBM Watson Discovery. Note: Make sure that you enter the Collection ID of the external data set. To get the Collection ID, log in to IBM Watson Discovery.
   External Data Set Source	Select one of the following options: RKM—Select if you want to create a data set for Remedy Knowledge Management articles. Other—Select if you want to create a data set for any other external data source, which is not Remedy Knowledge Management.
   Confidence Threshold	Specify a value in the range between 0 and 1 that indicates the confidence level of the data set. The confidence threshold is used as a filter to display the search data set. Only the search data set content with a confidence score greater than or equal to the Confidence Threshold value is returned in the search results. Note: If the field is left blank, then the default value (0) is selected. For more information about IBM guidelines on assigning confidence values, see How to select a threshold for acting using confidence scores

6. Click Save.

   The external Search Data Set is displayed in the Cognitive Search Data Sets section.

   The documents within this IBM Watson Discovery collection are included in the cognitive search functionality for this application.

Related topics

Leveraging-cognitive-search-in-your-application

Enabling-cognitive-search-in-an-application