Creating a virtual machine


This topic describes how to create a virtual network, subnets, health probes, and a virtual machine to deploy BMC AMI AI Services.

Before you begin

The following table lists the recommended Azure virtual machine configuration for better performance and efficiency.

Configuration option

LLM

Azure

Recommended

Mixtral8x7b-instruct Quantized

Standard_NC24ads_A100_v4 (1 X A100)
 

Mid-level

Meta-Llama-3-8B-instruct 4K Quantized (GPU)

Standard_NC24ads_A100_v4 (1 X A100)

Entry level

Meta-Llama-3-8B-instruct 4K Quantized (CPU)

Standard_F32s_v2

If you cannot temporarily procure GPU-enabled machines and decide to proceed with the entry-level configuration, be aware that the performance of BMC AMI AI Services will be significantly slower, and some features might be unavailable.

Important

If you already have access to a usable Azure virtual machine instance that meets these configuration criteria, you do not have to create an Azure virtual machine.

To create a virtual machine

  1. On the Azure portal, select Create a resource.
    The Create a resource window appears.
  2. Under Virtual machine, select Create.
  3. On the Basics tab, perform the following actions to specify the virtual machine settings:

    Field

    Action

    Resource group

    Click Create new and enter the following resource group name: amiai-resource-group.

    Virtual machine name

    Enter amiai-vm. (This name is an example, you can enter any name.)

    Region

    Select East US.

    Security type

    Select Standard.

    Image

    1. Select Ubuntu Server 22.04 LTS – x64 Gen2 from the menu. 
    2. Select x64 as virtual machine architecture.

    Size

    1. Click See all sizes.
    2. In the Select a VM size window, select a type of virtual machine and then click Select.

    Best practice
    We recommend that you follow the configuration list from the top of the table.

    Authentication type

    Select Password.

    Username

    Enter an administrator user name.

    Password

    Enter an administrator password.

  4. For the Public inbound port, select Allow selected ports and select SSH in Select inbound ports.
    worddav360355a48f90a4deed1dc6559fc3bcad.png
  5. Accept the other defaults and then click Next: Disks.
  6. Select the required OS disk size. The suggested size is above 200 GB.
  7. Accept other defaults on the Disks tab and then click Next: Networking.
  8. On the Networking tab, perform the following actions: 

    Field

    Action

    Virtual network

    1. Click Create new
    2. In the Create virtual network dialog box, follow these steps:
      1. In the Name field, enter amiai-virtual-network.
      2. Accept the defaults and click OK.

    Subnet

    Select the default subnet.

    Public IP

    1. Click Create new
    2. In the Create public IP address dialog box, follow these steps:
      1. In the Name field, enter amiai-vm-public-ip.
      2. Accept the defaults and click OK.
  9. Accept the other defaults and then click Next: Management.
  10. Click Next: Monitoring.
  11. On the Monitoring tab, follow these steps:
    1. In the Boot diagnostics  area, select  Disable.
    2. Accept the other defaults and then click Review + create.
  12. On the Review + create tab, review the settings, correct any validation errors, and then click Create.
  13. Click Deployment details to expand, and then click amiai-vm.
    image-2024-10-7_16-2-14-1.png
  14. In the amiai-vm window, note down the public and private IPs.
  15. Click Network Settings and expand the Rules tab.
  16. Click the existing SSH Inbound rule to edit the rule.
    1. Select Source as IP Addresses.
    2. Enter the IP address that you want to allow for SSH, HTTP, and HTTPS.

      Important

      Select Source as My IP Address if you want only your IP to have access.

    3. Select Service as Custom.
    4. Enter 22,8000, 443 as Destination port ranges.
    5. Let other values be defaults and click Save.
  17. Click Create port rule.
    1. Select Inbound port rule.
    2. From the Source window, select IP Address.
    3. In the Source IP addresses/CIDR ranges window, enter the public IP of the machine.
    4. In the Destination port ranges window, enter 8000.
    5. Click Add.
  18. Install Python and its dependency on the Azure virtual machine.
    1. Verify whether Python is already installed by entering the python3 command. If Python is not installed, then run the following commands: 

      sudo apt update
      sudo apt install python3
    2. Install pip by using the following commands: 

      sudo apt update
      sudo apt-get -y install python3-pip

To install the NVIDIA GPU driver extension

Important

You must follow this task if you run Meta-Llama-3-8B-instruct 4K Quantized (GPU) or Mixtral8x7b-instruct Quantized models. 


Follow this procedure if your Azure virtual machine has GPUs.

  1. Navigate to your Azure virtual machine, expand the Settings menu, and click Extensions + applications.
  2. Click +Add and search for NVIDIA.
    image-2024-10-7_15-58-42-1.png
  3. Select the extension, click Next, click Review + create, and then click Create.
  4. Wait for deployment to finish.
  5. Install NVIDIA Container Toolkit by using the following command: 

    sudo apt update
    sudo apt-get install -y nvidia-container-toolkit 
    sudo nvidia-ctk runtime configure --runtime=docker

    Important

    After you run the sudo apt update  command, you might get the following error message:
    image-2024-10-29_8-37-19.png

    You must run the following command to resolve this error message as follows:

    sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys <key>

Where to go from here

See Creating-an-Application-Gateway.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*