Models in BMC HelixGPT

A model is a program or algorithm that relies on training data to recognize patterns and make predictions.

BMC HelixGPT supports different vendors and models. For more information about vendors and models, see Supported models in BMC HelixGPT.

BMC HelixGPT uses Llama3, a self-hosted solution containing the weights and configuration files required to create inferences.

Inferencing and training

Inferencing is the main task of a model, resulting in the generation of chat and summarization. Inferencing occurs in the following way:

23301_ModelInferencing.png

Data fed to the model includes example outputs, and the model is adjusted or fine-tuned regularly.

The model training is accomplished in the following methods:

  • BMC trains the model based on generic, non-sensitive data and fine-tunes the application use cases, which are not specific to any customer. For example, we provide a global prompt that establishes the tone and expectation for all responses.
  • Customers train the model by using their private data where the output is a model that is specific to the customer.
  • AISM use cases depend on ready-made models, and you can also train the model as per your requirements.

Inference service

The inference service deploys the model and exposes an API through a network endpoint. BMC Helix applications use this service at runtime during user interaction. The inference service is available in the following categories:

  • Subscribed service: A subscribed service is a generalized, large model service where you purchase access and get an API key from a third-party provider, such as Microsoft or Azure OpenAI. The vendor hosts and runs the service. BMC integrates with AI providers through REST APIs. BMC supports the Azure OpenAI provider.
  • Self-hosted service: A self-hosted service runs on one of the AI/ML platforms offered by the three major cloud vendors: Google Cloud Platform Vertex, Amazon Web Service Bedrock, and Azure ML. These platforms require the customer to be a Google Cloud Platform (GCP), Amazon Web Services (AWS), or Microsoft Azure customer. BMC supports Vertex AI on Google Cloud Platform.

Supported models

The following providers and their models are supported out of the box to create skills and prompts.

All the supported models are authenticated by using an API key. You must specify the API keys while provisioning the AI provider in BMC HelixGPT Manager. Learn more about configuring the AI provider in Provisioning-and-setting-up-the-generative-AI-provider-for-your-application.

LLM provider

LLM host

Model name

Description

Default model configuration

Products supported

BMC Helix Innovation Suite

BMC Helix Digital Workplace

BMC Helix Business Workflows

BMC Helix Virtual Agent

BMC Helix ITSM

Google
 

Google Vertex AI

Gemini 1.5

gemini-1.5-pro-002

Click here to view the Gemini 1.5 model configuration
{
"deploymentName": "",
"apiType": "google",
"location": "",
"deployedModelType":
    "gemini-1.5-pro-001",
"temperature": 0.0,
"top_p": 0.1,
"read_timeout_in_seconds": 300,
"initial_chunks_to_accumulate": 2,
"safety_settings": {}
}

✅️

 

✅️

✅️

✅️

✅️

Gemini 2.0 Flash

gemini-2.0-flash-exp    

Note:

 

Gemini 2.0 Flash details:

{{code}}
{
"deploymentName": "",
"apiType": "google",
"location": "",
"deployedModelType": 
     "gemini-2.0-flash-001",
"temperature": 0.0,
"top_p": 0.1,
"read_timeout_in_seconds": 300,
"initial_chunks_to_accumulate": 2,
"safety_settings": {}
}
{{/code}}

     

Meta

Google VertexLlama 3.2llama-3.2-90b-vision-instruct

llama-3.2.90 details:

{
"deploymentName": "",
"apiType": "llama",
"location": "",
"deployedModelType": 
     "meta/llama-3.2-90b-vision-instuct-mass",
"temperature": 0.0,
"top_p": 0.1,
"timeout": 60
}

 

  ✅️  

Oracle Cloud

Llama 3.2

llama-3.2-90b-vision-instruct

 

Click here to view the Llama 3.2 model configuration
{
"deploymentName":
    "meta.llama-3.2-90b-vision-instruct",
"apiType": "oci_genai",
"deployedModelType":
    "meta.llama-3.2-90b-vision-instruct",
"temperature": 0,
"top_p": 0.1,
"read_timeout_seconds": 300,
"supportsJsonResponse": true,
"initial_chunks_to_accumulate": 14,
"user": "ocid1.user.oc1..     aaaaaaaa6te254a4jt5j7zeyua2pympxl3d2p7dqt4l3xirt6irn44v6b7dq",
"fingerprint": "b0:51:3a:c9:83:e8:89:fe:12:15:5b:32:ad:86:01:13",
"tenancy": "ocid1.tenancy.oc1..
     aaaaaaaam3tucgu7udgrdzch76ysaj5tinoo2vcysvg7ab2mlrco46zgf4qa",
"region": "us-chicago-1",
"location": "us-chicago-1",
"compartment_id": "ocid1.compartment.oc1..
    aaaaaaaavccimb5ndn4caejsuts6pweall7in4clxsfi2z6c4zvm36rd4a5a"
}

 

✅️

✅️

✅️

✅️

❌️

Open AI

OpenAI

GPT-4o (Omni)

gpt-4o-2024-08-06

Click here to view the default model configuration for the GPT-4o (Omni) model
{
"supportsJsonResponse": true,
"temperature" : 0.0,
"apiType": "openai",
"top_p": 0.1,
"read_timeout_in_seconds":300,
"maxNumOfImages": 3,
"imageUploadSizeInMB": 2,
"captureImageDetail": "low",
 "includeImgDataInPersistHistory":true
}

❌️

✅️

❌️

✅️

❌️

GPT-4ogpt-4o-2024-11-20    

GPT 4.o details:

{{code}}
{
  "deploymentName": "gpt-4o-core-1",
  "apiType": "azure",
  "supportsJsonResponse": true,
  "temperature": 0.0,
  "top_p": 0.1,
  "read_timeout_in_seconds": 300
}
{{/code}}

 

     

Open AI

 

MS Azure

GPT-4o (Omni)

gpt-4o (2024-11-20)
Click here to view the GPT-4o (Omni) default configuration model
{
 "deploymentName": "gpt-4o-core-1",
 "apiType": "azure",
 "supportsJsonResponse": true,
 "temperature": 0.0,
 "top_p": 0.1,
 "read_timeout_in_seconds": 300
}

 

✅️

✅️

✅️

✅️

✅️

GPT-4o mini

gpt-4o-mini (2024-07-18)

Click here to view the GPT-4o mini model configuration
{"deploymentName":"gpt-4o-mini-helixgpt-qa",
"apiType":"azure","supportsJsonResponse":
true,"temperature":0,"top_p":0.1,
"initial_chunks_to_accumulate":8,
"read_timeout_in_seconds":300}

❌️

✅️

❌️

✅️

❌️

Important

  1. When creating a service request, if you observe issues with the GPT-4o (Omni) (2024-08-06) model, use the GPT-4o (Omni) (2024-11-20) model.
  2. Turkish is not supported by the models mentioned in this topic.

Details about the default configuration parameters

The following table lists the description of the parameters used in the model configuration. For more information about the parameters, see Azure OpenAI Service REST API reference and Reproducible output support.

Parameter 

Description

temperature

Controls the randomness of the text generated by the model.

A lower value generates a deterministic output and a higher value generates a randomized output.

The default value is 0.0.

apiType

Specifies the API type of the AI provider. For example, azure_ad.

deploymentName

Specifies the deployment name of the model. For example, se-gpt-4-turbo

top_p

An alternative to sampling with temperature, this parameter considers the results of the tokens with top_p probability mass. 

The default value is 0.1.

A value of 0.1 means the tokens comprising the top 10% probability mass are considered.

suppportsJsonResponse

When the value is set to true, returns a valid JSON response as an output.

read_timeout_in_seconds

Specifies the time in seconds after which the model times out.

The default value is 300 seconds.

question_max_retry

Specifies the number of times the model reattempts to respond.

The default value is 3.

seed

(Optional) Controls the reproducibility of the response, such that repeated requests with the same seed value return the same result.
For example, seed = 1

Related topics

Skills

Prompts

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*