Deploying BMC AMI AI Services on zCX or zLinux

This topic describes how to deploy BMC AMI AI Services on your zCX or zLinux instance.

Important

You cannot deploy the Large Language Model on zCX or zLinux machines owing to hardware limitations. 

This topic also describes how to deploy LLM services. You prepare your zCX or zLinux instance following your organization processes and deploy BMC AMI AI Services.

Related topics

Installing

Manual-installation

Task 1: To acquire the required infrastructure

You must acquire the following infrastructure to run BMC AMI AI Services.

To acquire a zCX/zLinux instance

Create a zCX/zLinux instance in accordance with your configuration. This instance is used to deploy BMC AMI Platform services.

Important

Make sure that 8000 and 8001 ports are accessible from the BMC AMI AI Models instance.

For optimal performance with BMC AMI AI Services use zCX/zLinux instance (CPU – 1 core, 8 GB RAM)

To acquire the infrastructure for BMC AMI AI Models

BMC AMI AI Services support the following types of infrastructure:

After acquiring the configuration, note the IP or domain name for both machines, because you will need it to perform application installation.

Task 2: To download the required scripts from EPD

  1. Make sure that you complete the steps outlined in Downloading-the-installation-files and obtain the BMC-AMI-AI-zCX.zip file.
  2. Extract the contents from the BMC-AMI-AI-zCX.zip file and save the extracted files in a folder of your choice for future use. 

Task 3: To configure BMC AMI AI Services for installation

Update the configuration in the Ansible scripts, based on the services that you need:

  • BMC AMI AI Platform Services configuration
  • BMC AMI Code Insights Explain configuration
  • BMC AMI AI Model configuration

BMC AMI AI Platform Services configuration

You can find the BMC-AMI-AI-Platform.sh shell script file in the extracted folder. This script installs BMC AMI AI Platform services.

You must configure the following fields before running the Ansible script:

Field

Description

enable_ssl 

To enable SSL for BMC AMI AI Services, set the value to true.

Important

You must create a certificate or acquire one from the certificate authority. A self-signed certificate is not accepted.

ssl_keyfile_name 

Enter the SSL key file name.

ssl_certfile_name 

Enter the SSL certificate name.

ssl_encryption_cipher

Enter the supported values for TLSv1 or TLSv1.2.

gateway_host

Enter the IP address or domain name of your x86 machine. BMC AMI AI Services communicates internally via this field.

Important

You must provide the domain name when setting the SSL.

For licensing purposes, you must configure the following fields for BMC AMI Common Enterprise Services (CES). During integration provisioning, the CES details are provided to store BMC AMI AI Services access information. The CES details are also used with BMC AMI DevX Workbench applications to access BMC AMI AI Services. If you are using Code Insights Explain, you must configure the CES fields.

Field

Description

ces_scheme

Protocol of the CES instance

ces_host

Host name or IP address of the CES instance

ces_port

Port of the CES instance

ces_username

If the CES instance requires a user name, then enter it.

ces_password

If the CES instance requires a password, then enter it.

If you have the mainframe machine's host name or IP address, enter it in the zos_host field to authenticate users. All authenticated users can access the BMC AMI AI Manager console.

If you don't have the host name or IP address, specify the administrator user name in the admin_user_id field. The default is admin. If you have set zos_host, reset the values set for the admin user in the admin_user_id field. All users are admin users.

In the admin_password field, set an admin user password encoded in base64. The default value is base64 encoding of amiaiadmin: YW1pYWlhZG1pbg== 

To configure other properties, see Configuring-the-BMC-AMI-Platform-service.

To access the BMC AMI AI Services console, user credentials are required.

BMC AMI AI Code Insights Explain configuration

Important

This step is required only if you install BMC AMI AI Code Insights Explain.

You can find the BMC-AMI-AI-Devx-Service.yml Ansible file in the extracted folder. This script installs BMC AMI AI Code Insights Explain.

You must configure the following fields before running the Ansible script:

Field

Description

enable_ssl

To enable SSL for BMC AMI AI Services, set the value to true.

Important

You must create a certificate or acquire one from the certificate authority. A self-signed certificate is not accepted.

ssl_keyfile_name 

Enter the SSL key file name.

ssl_certfile_name 

Enter the SSL certificate name.

ssl_encryption_cipher

Enter the supported values for TLSv1 or TLSv1.2.

integration_host

Update only when SSL is enabled.

Important

You must provide a domain name.

BMC AMI AI Model configuration

Use one of the following Ansible files in the extracted folder, depending on your selected configuration:

  • BMC-AMI-AI-CPU-Service.yml—CPU (entry-level configuration)
  • BMC-AMI-AI-GPU-Service.yml—GPU (mid-level and recommended configurations)

You must configure the following fields before running the Ansible script:

Field

Description

enable_ssl

To enable SSL for BMC AMI AI Services, set the value to true.

Important

You must create a certificate or acquire one from the certificate authority. A self-signed certificate is not accepted.

ssl_keyfile_name 

Enter the SSL key file name.

ssl_certfile_name 

Enter the SSL certificate name.

ssl_encryption_cipher

Enter the supported values for TLSv1 or TLSv1.2.

llm_host

Update only when SSL is enabled.

Important

You must provide a domain name.

vllm_kvcache_space

Set this value based on available CPU memory.

Example

If the available CPU memory is 64 GB, the value would be 44 (64 - 20).

Task 4: To copy the script and files to the machine

Copy the folder to the acquired instance, as follows:

  1. Locate the folder where you stored the following files:
    • BMC-AMI-AI-Platform.sh
    • BMC-AMI-AI-CPU-Service.yml
    • BMC-AMI-AI-GPU-Service.yml
    • BMC-AMI-AI-CPU.sh
    • BMC-AMI-AI-GPU.sh
    • BMC-AMI-AI-Devx.sh
  2. Move the BMC-AMI-AI-Platform.sh and BMC-AMI-AI-Devx.sh files to the zCX/zLinux and rest of the files to AWS/Azure/x86 Linux instance. You can use any scp client to copy all files.

Task 5: To deploy BMC AMI AI Services

  1. BMC AMI Platform services deployment
    1. Connect to the newly acquired zCX/zLinux instance via SSH. Make sure that the credentials used for login have admin access.

    2. Run the following commands to deploy AMI AI Platform Services: 

      chmod +x BMC-AMI-AI-Platform.sh
      sed -i -e 's/\r$//' BMC-AMI-AI-Platform.sh
      ./BMC-AMI-AI-Platform.sh
    3. When prompted, enter your EPD portal's user name and password.
  2. BMC AMI AI Code Insights Explain deployment
    1. Connect to the newly acquired zCX/zLinux instance via SSH. Make sure that the credentials used for login have admin access.

    2. Run the following commands to deploy AMI AI Platform Services: 

      chmod +x BMC-AMI-AI-Devx.sh
      sed -i -e 's/\r$//' BMC-AMI-AI-Devx.sh
      ./BMC-AMI-AI-Devx.sh
    3. When prompted, enter your EPD portal's user name and password.
  3. Model deployment 
    1. Connect to the newly acquired AWS, Azure, or x86 Linux instance via SSH. Make sure that credentials used for login have admin access.
    2. Install Python and its dependency on-premises x86 machine as follows:

      1. Verify whether Python is already installed by entering the  python3 command. If Python is not installed, then run the following commands: 

        sudo apt update
        sudo apt install python3

      2. Install pip by using the following commands: 

        sudo apt update
        sudo apt-get -y install python3-pip

    3. Run one of the following commands, based on your selected configuration. Do not run all the commands.

      Configuration type

      Command

      Recommended

      chmod +x BMC-AMI-AI-GPU.sh
      sed -i -e 's/\r$//' BMC-AMI-AI-GPU.sh
      ./BMC-AMI-AI-GPU.sh

      After running this command, enter Mixtral in the selection prompt window.

      Mid-level 

      chmod +x BMC-AMI-AI-GPU.sh
      sed -i -e 's/\r$//' BMC-AMI-AI-GPU.sh
      ./BMC-AMI-AI-GPU.sh

      After running this command, enter Llama3 or Granite in the selection prompt window.

      Entry-level

      chmod +x BMC-AMI-AI-CPU.sh
      sed -i -e 's/\r$//' BMC-AMI-AI-CPU.sh
      ./BMC-AMI-AI-CPU.sh
    4. When prompted, enter your EPD portal's user name and password.

Task 6: To verify the deployment of BMC AMI AI Services

For more information about how to verify the deployment of BMC AMI AI Services, see Verifying the installation.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*