Upgrading to the latest AI model

Make sure you have the following details:

• Docker Inference Engine
• Model artifacts

1. On a local host, extract the model artifacts provided by BMC Helix: 
   tar -xzvf <helix_gpt_model_version>.tar.gz
2. Upload the model to the Google Cloud Storage bucket:
   gsutil cp -r <helix_gpt_model_version> gs://<your-bucket>/model/
3. Prepare the Custom Inference Docker image: 
   1. If you have a Docker image tarball, load the image file:
      docker load -i /path/to/model_container.tar
   2. If using the container registry, log in and pull the image from the registry:
      # docker login containers.bmc.com        
      (Specify the credentials provided by BMC Helix)
      
      #docker pull containers.bmc.com/bmc/lpade:helix-gpt-vllm-docker-<build_number>   
      (Specify the image tag provided by BMC Helix)
      ​​​​​​
4. Push the Docker image to the Google Cloud container registry: 
   #docker tag <bmc helix image> <Google Cloud Container Registry tag>
   
   #docker push <Google Cloud Container Registry path>
   Now, the model and its artifacts are available in the Google Cloud Model Store and the Google Cloud Container Registry.
5. Navigate to Model Registry from the Vertex AI navigation menu.
6. Click Import and then click Import as new version.
7. On the Import Model page, select the previous version of the model, and add an optional description. 
8. Select the region, and click Continue.
   Select the region that matches both your bucket's region and the Vertex AI regional endpoint that you are using. 
9. Navigate to the Model settings page and select Import an existing container.
10. In the Custom container settings section, click Browse in the Container image field and then click the Container Registry tab to select the container image.
11. Click Browse in the Model artifact location and select the Cloud Storage path to the directory that contains your model artifacts.
12. In the Arguments section, specify the following parameters and click Continue:

    Field	Description	Example value
    Environment variables	Specify the file name of the Deployment spec (without the file extension) included in the model artifacts.	DEPLOYMENT_SPEC= zhp52uqvaxvacmt4u2tbezojfucjkf4f-helix-gpt-v6-instruct
    Prediction route	Specify the HTTP path to send prediction requests to.	/predictions
    Health route	Specify the HTTP path to send health checks to.	/ping
    Port	Specify the port number to expose from the container.	8080

13. On the Explainability options page, retain the default No explainability option, and click Import.
    After a few minutes, the model is displayed on the Models page. For more information about importing models in GCP Vertex AI, see the online documentation https://cloud.google.com/vertex-ai/docs/model-registry/import-model#custom-container.  

To add model to an existing endpoint

1. Select the model and then click Deploy and test.
2. Click Deploy to endpoint.
3. Select Add to existing endpoint and click Continue.
4. On the Model Settings page, click to edit the existing model version, and update the Traffic Split value to 0. 
5. Click the latest upgraded version and update the Traffic Split value to 100. 
6. Retain the access setting value to Standard and click Continue.
7. On the Model Setting Page, specify the values for the following fields and continue with default values for other fields:
   • Machine Type: a2-highgpu-1g, 12 vCPUs, 85 GiB Memory
   • Accelerator Type: NVIDIA Tesla A100
   • Accelerator Count: 1
8. Click Continue and then click Deploy.
   The latest model is deployed to an existing endpoint, and all requests are routed to the new version of the model. 
   ​​​​​​

Test the endpoint:
curl -X POST <scoring-uri> \
  -H "Authorization: Bearer <key>" \
  -H "Content-Type: application/json" \
  -d '{"input": "<add test prompt here>"}'

For example, if you provide a test prompt to evaluate the problem based on ranking and generate resolution steps, the results display the predictions, which indicates that the model is deployed and generates responses successfully.