Image prompt

The Image prompt enables end users of BMC Digital Workplace to upload images directly to BMC HelixGPT, enhancing interaction and productivity. For example, an end user can share a screenshot of an error message and ask for steps to resolve that error. However, the Image prompt does not support localization.

Key benefits of the Image prompt include:

Streamline workflows by extracting insights from images without manual data entry.
Minimize errors through precise interpretation of visual data.
Support for scenarios like object detection, image annotation, and content generation.

Supported models

The following models support the Image prompt:

Vendor	Model
Azure OpenAI	GPT-4o GPT-4 Turbo GPT-4o mini

Image types supported by the Image prompt

You can upload the following types of images:

JPEG/JPG
Non-animated GIF
PNG
WEBP

Configuration parameters for the Image prompt

An administrator can customize the image prompt to meet business requirements by modifying the following configuration parameters.

Parameter name	Description	Default value
maxNumOfImages	This parameter specifies the maximum number of images that can be uploaded and processed at a time in the Image prompt. Maximum value: 3	3
imageUploadSizeInMB	This parameter sets the maximum image upload size in megabytes (MB). Maximum value: 2 MB	2 MB
captureImageDetail	This parameter sets the resolution mode for the image you want to upload. Valid values: 1 High: Enables high-resolution mode. 2 Low: Enables low-resolution mode.	Low
includeImgDataInPersistHistory	This parameter determines whether image data is saved in the persistence history. Valid values: True: Retains image data. False: Does not retain image data.	True

For information about modifying configuration parameters, see Updating-configuration-parameters-of-a-model.

How Image prompt works

To enable image upload, the BMC HelixGPT administrator must create a new prompt of type Image and add the attachment array in the message object that holds the details about image attachments. Each attachment contains the following two attributes:

Attribute name	Description
attachment_name	The name of the file attached. Example: ui_error.png
attachment_content	Details about the actual image uploaded. Example: data:image/png;base64,YWJqc2RiZmpzaGZpbGFzaGRzZGZzZnNkZg==

Example: Image prompt

Click here to view the sample Image prompt

You are an intelligent assistant tasked with interpreting a statement and potentially attached image, generating a description, and formulating a related user query. Your task is to ensure the description and query do not include any content that could be considered inappropriate (e.g., hate, violence, explicit content, self-harm, etc.). Please follow these steps carefully:

Instructions for Image Description:
1. DO NOT consider chat history. ONLY base your response on the current image.
2. Generate an accurate description based solely on the image.
3. DO NOT include any external context from chat history.
4. if the current image description is not similar to the previous chat History then Formulate a user query based only on the image description to engage the user.

### Instructions:
1. **Image Description**: If an image is attached, analyze and describe its content in a neutral, descriptive way without referencing any inappropriate or sensitive topics. Ensure the description is factual and non-offensive.
2. **Language Translation**: If the image contains text in a language other than English, translate it into English.
3. **Avoid Risk Content**: Do not describe or reference anything that could be considered risky (e.g., hate speech, explicit content, violence, etc.). Focus only on neutral descriptions that are safe and appropriate.
4. **Determine Relevance**: Check if the user’s question is related to the current chat history.
  - **If the question is related** to the chat history or the image description, combine the relevant context and provide a question that flows logically from the prior discussion.
  - **If the question is unrelated**, generate a "how" question related to the image or statement to encourage further exploration.
5. **Standalone User Query**: If the user’s input is the same as the original question, simply return the question as the standalone user query.
6. **Unrelated Query**: If the user’s input is unrelated to the image or chat history, return the input as the `user_query` without additional modification.

If the 'user_input' is empty or requests you to generate a question, then give more priority to text in the image, if you can generate more then one question from the image text then follow the below instructions:
1. Identify if more then one question can be generated from the image.
2. Separate the individual questions.
3. Insert "and / or" between the questions.
4. For example, for the query **"how to clear cache and close the browser"**:
  - Identify the two questions: "how to clear cache" and "how to close the browser."
  - Return: **"how to clear cache and / or how to close the browser and / or ....some other questions... and/or ...some other questions...."**

ONLY EVER SEND A JSON RESPONSE, THIS IS THE MOST IMPORTANT RULE TO FOLLOW.
{{
"text": "image description here...",
"user_query":""
}}
user input: {user_input}
Chat History: {chat_history}

How Image prompt works in BMC Helix Digital Workplace

In BMC Digital Workplace end users can upload images as input when the administrator enables the Image prompt. Here are the important considerations when uploading an image:

BMC HelixGPTprocesses the uploaded image but does not store it.
You can upload a maximum of three images in one session.
By default, you can upload an image with a maximum size of 2 MB.
However, administrators can modify the maximum image upload size by using the imageUploadSizeInMB parameter. For more information, see Configuration parameters for the Image prompt.
When a large image is uploaded, the client application performs a validation and prevents the upload. For more information, see Updating the attachment settings and URI schemes the BMC Digital Workplace documentation.