Modifying the Knowledge modular tool settings

As an administrator, configure the knowledge modular tools in your AI agents to enable them to retrieve relevant knowledge documents, filter and rank them, and then use them to generate responses to user queries.  The knowledge modular tools help ensure that AI agents respond with accurate, contextually relevant information sourced from your knowledge data sources.

The Knowledge modular tools consist of three tools that form a configurable knowledge‑retrieval pipeline. With these modular tools, you can choose which tools to include in an agent configuration, control the execution order of retrieval and filtering operations, and configure each tool independently with granular settings. Together, they help control how documents are retrieved, filtered, and ranked before an answer is generated.

The knowledge modular tools consist of the following tools:

• (Mandatory) Knowledge Retriever Tool: This tool retrieves documents from OpenSearch based on semantic similarity. This tool is mandatory and always runs first. The tool searches for documents in OpenSearch by understanding the user's intent and analyzing the context of their query, instead of just matching keywords.
• (Optional) Document Grader Tool: This tool evaluates and ranks retrieved documents based on their relevance to the user query. You can configure the grading method and threshold to filter out non-relevant matches.
• (Optional) Feedback Filter Tool: This tool filters documents based on historical user feedback, prioritizing content that has got positive feedback in past interactions.

These tools work together as an independent and composable pipeline. You can include or exclude optional tools, adjust tool configurations, and control the order in which filtering and grading tools run to fine‑tune BMC HelixGPT response quality.

The pre-built knowledge modular tools are available in the HelixGPT Agent Studio, but you cannot modify them directly. However, you can add a new tool of a specific tool type and add the custom configurations to it.

Important consideration for customizing these agents

• The knowledge retriever tool must always run first. Other tools rely on retrieved documents.
•  The feedback filter and context grader tools are optional and independent.
• A very high grading threshold can remove all documents, resulting in no answer.
• Feedback filtering relies on existing feedback data; limited data can affect results.
• Supervisor agents inherit behavior from their sub‑agents unless explicitly overridden.
• Customizations affect only the cloned agent and do not change the behavior of the prebuilt agent.

Before you begin

Make sure you have cloned a pre-built agent, because you can only add a custom tool to a custom agent. The custom agent can be a cloned pre-built agent or a newly created agent. For more information, see Cloning an agent.

To modify the Knowledge Retriever Tool settings

The knowledge retriever tool parameters are mainly defined in the skill configurations rather than in the tool configurations.

1. In the HelixGPT Agent Studio, on the Skills tab, select the application and skill for which you want to modify the configuration parameters for the knowledge retrieval.
2. Go to the Configure tab and add the following parameters in the Configuration box:

   Setting	Description
   knowledgeSearchScope	Defines the scope of the search to answer a user's question in a knowledge Q&A use case. Valid values: ENTERPRISE—The search will be performed on the enterprise knowledge base. WORLD—The search will be performed on the enterprise knowledge base and external articles online.
   lowWaterMarkThreshold	Specifies the minimum relevance threshold for similarity_score_threshold. The value must be between 0 and 1.
   searchType	Defines the type of search the Retriever should perform. Valid values: similarity—returns similar documents similarity_score_threshold—returns similar documents only if they meet the minimum threshold specified by lowWaterMarkThreshold MMR—returns similar and diverse documents based on maximum marginal relevance
   numberOfDocumentsToReturn	Specifies the maximum number of documents to retrieve in response to a query. This parameter helps control the result set size, ensuring that only the most relevant documents are returned based on the query's ranking criteria. Important: For agentic skills, increasing the default value can sometimes affect performance.  We recommend using the default value. If you still experience performance issues, set the value to 3.
   numberOfDocumentsToFetchForMMR	Specifies the maximum number of similar and diverse documents to return based on the MMR algorithm.
   degreeOfDiversity	Defines the level of variation applied when generating or selecting results. A higher value increases diversity, producing a wider range of outputs or responses, while a lower value yields more focused and consistent results. This parameter helps strike a balance between creativity and precision in data processing or response generation.

3. Click Save.

(Optional) To modify the Context Grader Tool settings

1. Add a new tool to your environment. For more information, see Adding tool and toolkit for an AI Agent.
   To the new tool, replicate the same default configurations as the existing pre-built context grader tool.
   Default configuration:

   {
     "grade_threshold": 3,
     "ranking_method": "context_grader_llm"
   }

2. In the Configuration field, add the following configuration parameters and modify the values as per your requirement:

   Setting	Description
   ranking_method	Specifies the method used to evaluate and rank documents. Valid values: none, context_grader_llm, cross_encoder.
   grade_threshold	Defines the minimum score a document must get to pass filtering. Valid values range from 0 to 5.
   grade_prompt	A set of instructions to evaluate the document relevance with the user query. This field is relevant only to the context_grader_llm ranking method. When custom instructions are not specified, default grading prompt instructions are sent to the LLM to evaluate the relevance of each document to the user query. When custom instructions are specified, the custom grading prompt instructions are sent to the LLM instead of the default ones to evaluate document relevance.
   grade_mode	Defines how document grading is done. Either in series, where each document is graded via a separate LLM call, or in batch, where all documents are graded together in a single LLM call to reduce latency. This field is relevant only to the context_grader_llm ranking method. When the grading mode is not specified, the default series mode applies, in which each retrieved document is graded with a single dedicated LLM call. When the grading mode is set to batch, all retrieved documents are graded in a single LLM call, reducing overall time consumption.
   cross_encoder_model	Specifies the type of cross‑encoder model used when the cross_encoder ranking method is selected. Cross‑encoders are neural models that evaluate a query and a document together to determine relevance. Using a cross-encoder model reduces grading time by avoiding LLM calls.
   cross_encoder_min	Indicates the minimum raw score of the specifically mentioned cross_encoder_model. Each model has a different score range used to normalize cross-encoder results to the 0–5 grade threshold scale.
   cross_encoder_max	Indicates the maximum raw score of the specifically mentioned cross_encoder_model. Each model has a different score range used to normalize cross-encoder results to the 0–5 grade threshold scale.

3. Click Save.

(Optional) To view the results of the Context Grader tool

If you have used the context grader tool in your agents, by using the following API, you can analyze how it matched the user query context and graded the documents. This helps you view the context grader tool results, debug poor results, and verify document grading behavior, analyze session-level analytics, and compare grading results across queries.

(Optional) To modify the Feedback Filter Tool settings

The feedback filter tool parameters are mainly defined in the skill configurations rather than in the tool configurations. To use this tool in your agents, you must first enable feedback filtering by setting the filterContextBasedOnFeedback parameter to true at the skill level, then define the other parameters in the skill configurations as per your requirements.

1. In the HelixGPT Agent Studio, on the Skills tab, select the application and skill for which you want to enable the feedback filtering and modify its behavior.
2. Go to the Configure tab and add the following parameters in the Configuration box:

   Setting	Description
   filterContextBasedOnFeedback	Indicates whether feedback‑based context filtering is applied to filtered documents. Default value: false
   maxFeedbackQuestionsToReturn	Specifies the maximum number of feedback‑related questions considered during feedback filtering. Default value: 8
   feedbackQuestionsSimilarityThreshold	Defines the minimum similarity score required to consider a feedback question relevant to the current query. Default value: 0.75
   feedbackFilteringRelevancyThreshold	Specifies the relevance threshold a document must meet to pass feedback‑based filtering. Default value: 0.8
   feedbackPositiveWeight	Weight applied to positive user feedback in the relevancy calculation. Higher values amplify the impact of positive votes when filtering documents. Default value: 2
   feedbackNegativeWeight	Weight applied to negative user feedback (thumbs down) in the relevancy calculation. Higher values amplify the impact of negative votes when filtering documents. Default value: 1
   newContextDerivationFromFeedbackAllowed	Enables including document sections from user feedback (feedback_sources) that were not part of the original document retrieval, provided they meet the relevancy standard. When enabled, the tool can discover and include new context based on what users previously found helpful. Default value: false
   showLearningFormOnThumbsUp	Determines whether a learning feedback form is displayed when users select thumbs up on the knowledge response. Default value: true

3. Click Save.
4. Add this tool to your agent.

Related topic

Customizing the prebuilt agents that use Knowledge Modular Tools