Setting up your environment to leverage agentic AI capabilities in BMC Helix AIOps

Microsoft Azure AI requirements:

• You have an active Microsoft Azure subscription. 
• You must be assigned the Contributor or appropriate role to upload resources in the Microsoft Azure CLI.
• You have the Standard_NC24ads_A100_v4 quota assigned to your region:
  1. In the Azure Machine Learning studio, click Quota.
  2. Click the subscription name for the subscription where you want to host the model.
  3. Select the region.
  4. Search and select Standard NCADSA100v4 Family Cluster Dedicated vCPUs.
  5. Click Request quota.
      If the unused quota is 0 or less than 24, click Request quota and set the New cores limit to whatever the Usage is plus 24. So if Usage is 0, set Quota to 24. This quota is enough for a machine type with one A100 accelerator.  
  6. Click Submit.
     After your quota limit is approved, you must assign it to the workspace later. 

Local host requirements:

• The Microsoft Azure CLI is installed with the Azure Machine Learning (ml) extension.
• Docker Engine is installed and running.

You can deploy the model by using the Microsoft Azure Machine Learning Studio, however, this section explains how to use the Azure command line interface (CLI) for more control over the deployment.

Perform the following steps:

1. On a local host, extract the model artifacts provided by BMC Helix: tar -xzvf <helix_gpt_model_version>.tar.gz
2. Log in to the Microsoft Azure CLI: 
   az login

3.  Create a resource group:
   az group create --name <resource-group-name> --location <azure-region>

4. Create an Azure Container Registry:az acr create
   
   --resource-group <resource-group-name>
   --name <name of the registry>
   --sku Basic
   --admin-enabled true

   Parameter	Description
   resource-group	Specify the name of the resource group created in the previous step.
   name	Specify the name of the Azure Container Registry.
   sku	Specify the pricing tier: Basic, Standard, or Premium. Most users start with Basic.
   admin	Specify true to make sure that the user can upload resources to the registry.

5. Tag the docker image:
   docker tag <local-image>:<tag> helixgptreg.azurecr.io/vllm-vertex:<tag>

6. Push the docker image to the Azure Container Registry:
   docker push helixgptreg.azurecr.io/vllm-vertex:<tag>
7. Create an Azure ML workspace:
   az ml workspace create
     --name <workspace name> \
     --resource-group <resource-group-name> \
     --location <region> \
8. Set Azure CLI defaults:
   az configure --defaults workspace=<workspace name> group=<resource-group-name> location=<region>
9. Link the Azure Container Registry to the workspace:
   az ml workspace update \
   --name <workspace-name> \
   --resource-group <resource-group-name>\
   --container-registry
   /subscriptions/73ae88b6-0681-4b5f-839d-e331f68a59ae/resourceGroups/<resource-group-name>/providers/Microsoft.ContainerRegistry/registries/helixgptreg \
   --update-dependent-resources
10. Create an online endpoint:az ml online-endpoint create --file <endpoint>.yml
    endpoint.yaml:
    $schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
    name: helix-gpt-v7-25-3-endpoint
    auth_mode: key
11. Create an online deployment:
    az ml online-deployment create --file azure.yml --all-traffic
    Sample YAML file:
    Azure.yaml:
    name: helix-gpt-v7-25-3-deploy
    endpoint_name: helix-gpt-v7-25-3-endpoint
    model:
      name: helix-gpt-v7-25-3
      path: ./helix-gpt-v7-25-3
      version: 1
    environment_variables:
      AIP_HEALTH_ROUTE: "/ping"
      AIP_PREDICT_ROUTE: "/score"
      MODEL_BASE_PATH: "/var/azureml-app/azureml-models/helix-gpt-v7-25-3/1/helix-gpt-v7-25-3"
      DEPLOYMENT_SPEC: "agaqnayhu2tstm7s3z5xmnmdugrzccsa-helix-gpt-v7_2"
      AIP_STORAGE_URI: "/var/azureml-app/azureml-models/helix-gpt-v7-25-3/1/helix-gpt-v7-25-3"
    environment:
      image:attach:xwiki:IT-Operations-Management.Operations-Management.BMC-Helix-AIOps.aiops252.Setting-up-and-going-live.Setting-up-your-environment-to-leverage-agentic-AI-capabilities-in-BMC-Helix-AIOps.WebHome@filename helixgptreg.azurecr.io/vllm-vertex:dfe4802-43
      inference_config:
        liveness_route:
          port: 8080
          path: /ping
        readiness_route:
          port: 8080
          path: /ping
        scoring_route:
          port: 8080
          path: /score
    request_settings:
      request_timeout_ms: 180000
    instance_type: Standard_NC24ads_A100_v4
    instance_count: 1
12. (Optional) Get container logs:
    az ml online-deployment get-logs \
    --endpoint-name <name of the endpoint> \
    --name <name of the deployment>
13. Set traffic:
    When you set traffic to 100%, all requests sent to the endpoint are routed to that single deployment.
    az ml online-endpoint update \
    --name <name of the endpoint> \
    --resource-group <resource-group-name>\
    --workspace-name <workspace-name> \
    --traffic "<workspace-name>-deploy=100"
14. Get scoring URI:
    The scoring URI is the REST API endpoint you use to send data and get predictions from your deployed model.
    az ml online-endpoint show \
    --name <endpoint-name>-endpoint \
    --resource-group <resource-group>\
    --workspace-name <workspace-name> \
    --query "scoring_uri" \
    --output tsv
    
    Summary
    | Component | Value |
    | Workspace | <workspace-name> |
    | Endpoint Name | <endpoint-name> |
    | Deployment Name | <deployment-name> |
    | Region | <region>|
    | Scoring URL | https://<name>-endpoint.westus.inference.ml.azure.com/score
15. Test the endpoint:
    az ml online-endpoint show --name helix-gpt-v7-25_3-endpoint
    az ml online-endpoint get-credentials --name helix-gpt-v7-25_3-endpoint
    To test:
    curl -X POST <scoring-uri> \
      -H "Authorization: Bearer <key>" \
      -H "Content-Type: application/json" \
      -d '{"input": "your input here"}'
16. Continue to configure the model in BMC HelixGPT Manager in the next step. 

For more information about deploying models on Microsoft Azure AI, see Microsoft Azure Machine Learning documentation