Skip to Content

Tracing LLM calls with LangSmith

Use LangSmith to debug and monitor the LLM calls that are received in BMC HelixGPT. LangSmith provides better visibility in the LLM calls and detects errors. To use LangSmith, you must have an account with LangSmith and an API key. The API key is used to log traces and run evaluations. You must use the LangSmith API key in the HelixGPT Agent Studio to set the value of the LangSmithAPIKey parameter. 

Related topics

Skills

Before you begin

  1. Sign up for a personal or service account for LangSmith.
  2. Generate the personal or service key.

To configure the LangSmith API Key for BMC HelixGPT

  1. Log in to BMC Helix Innovation Studio and open HelixGPT Agent Studio.
  2. On the Records tab, select GlobalSetting, and click Edit data.
  3. Select the LangSmithAPIKey record, and click Edit.
  4. Specify the LangSmith API that you generated in the Value field, and click Save.

To enable call tracing for a skill in BMC HelixGPT

  1. Log in to BMC Helix Innovation Studio.
  2. From the Application launcher app_launcher_icon.png, select Failed to execute the [velocity] macro.
    .
  3. On the Skills tab, select the application and skill.
  4. Click the Configure tab.

  5. Set the traceOn parameter to true and specify the project name in the langSmithProjectName parameter.

    The following code is an example of the parameters configured for LangSmith:

    {
       "traceOn": true,
       "langSmithProjectName": "DemoDebug"
    }
  6. Click Apply.

Results

Data for each LangSmith project is visible on the LangSmith page. You can view the input query, the output sent by BMC HelixGPT, and any errors that occurred during the responses. You can analyze the errors and take appropriate actions to resolve them. The following image shows a sample of the LangSmith page:

23303_LangSmith.png

 

 

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*

BMC HelixGPT 25.3