Creating a virtual machine


This topic describes how to create a virtual network, subnets, health probes, and a virtual machine to deploy BMC AMI AI Services.

Before you begin

The following table lists the recommended Azure virtual machine configuration for better performance and efficiency:

Configuration option

Supported LLM

Azure

Recommended

  • Mixtral8x7b-instruct Quantized
  • Meta-Llama-3.1-8B-instruct 4K Quantized
  • Granite-3.1-8B-Instruct

Standard_NC24ads_A100_v4 (1 X A100)
 

Important

If you already have access to a usable Azure virtual machine instance that meets these configuration criteria, you do not have to create an Azure virtual machine.

To create a virtual machine

  1. On the Azure portal, select Create a resource.
    The Create a resource window appears.
  2. Under Virtual machine, select Create.
  3. On the Basics tab, perform the following actions to specify the virtual machine settings:

    Field

    Action

    Resource group

    Click Create new and enter the following resource group name: amiai-resource-group.

    Virtual machine name

    Enter amiai-vm. (This name is an example; you can enter any name.)

    Region

    Select East US.

    Security type

    Select Standard.

    Image

    From the menu, select Ubuntu Server 22.04 LTS – x64 Gen2 from the menu. Then select x64 as virtual machine architecture.

    Size

    1. Click See all sizes.
    2. In the Select a VM size window, select a type of virtual machine and then click Select.

    Best practice
    We recommend that you follow the configuration list from the top of the table.

    Authentication type

    Select Password.

    Username

    Enter an administrator user name.

    Password

    Enter an administrator password.

  4. For the Public inbound port, select Allow selected ports and select SSH in Select inbound ports.
    worddav360355a48f90a4deed1dc6559fc3bcad.png
  5. Accept the other defaults and then click Next: Disks.
  6. Select the required OS disk size. The suggested size is above 200 GB.
  7. Accept other defaults on the Disks tab, and then click Next: Networking.
  8. On the Networking tab, perform the following actions: 

    Field

    Action

    Virtual network

    1. Click Create new
    2. In the Create virtual network dialog box, follow these steps:
      1. In the Name field, enter amiai-virtual-network.
      2. Accept the defaults and click OK.

    Subnet

    Select the default subnet.

    Public IP

    1. Click Create new
    2. In the Create public IP address dialog box, follow these steps:
      1. In the Name field, enter amiai-vm-public-ip.
      2. Accept the defaults and click OK.
  9. Accept the other defaults, and then click Next: Management.
  10. Click Next: Monitoring.
  11. On the Monitoring tab, follow these steps:
    1. In the Boot diagnostics area, select Disable.
    2. Accept the other defaults, and then click Review + create.
  12. On the Review + create tab, review the settings, correct any validation errors, and then click Create.
  13. Click Deployment details to expand, and then click amiai-vm.
    image-2024-10-7_16-2-14-1.png
  14. In the amiai-vm window, note down the public and private IPs.
  15. Click Network Settings and expand the Rules tab.
  16. Click the existing SSH Inbound rule to edit the rule as follows:
    1. Select Source as IP Addresses.
    2. Enter the IP address that you want to allow for SSH, HTTP, and HTTPS.

      Important

      If you want only your IP to have access, selectSource as My IP Address.

    3. Select Service as Custom.
    4. Enter 22,8000, 443 as Destination port ranges.
    5. Let other values be defaults and click Save.
  17. Click Create port rule.
    1. Select Inbound port rule.
    2. In the Source window, select IP Address.
    3. In the Source IP addresses/CIDR ranges windowenter the public IP of the machine.
    4. In the Destination port ranges window, enter 8000.
    5. Click Add.
  18. Install Python and its dependency on the Azure virtual machine, as follows:
    1. Verify whether Python is already installed by entering the python3 command. If Python is not installed, then run the following commands: 

      sudo apt update
      sudo apt install python3
    2. Install pip by using the following commands: 

      sudo apt update
      sudo apt-get -y install python3-pip

To install the NVIDIA GPU driver extension

Important

You must follow this task if you run Meta-Llama-3-8B-instruct 4K Quantized, Granite-3.1-8B-Instruct, or Mixtral8x7b-instruct Quantized models. 

If your Azure virtual machine has GPUs, follow these steps:

  1. Navigate to your Azure virtual machine, expand the Settings menu, and click Extensions + applications.
  2. Click +Add and search for NVIDIA.
    image-2024-10-7_15-58-42-1.png
  3. Select the extension, click Next, click Review + create, and then click Create.
  4. Wait for deployment to finish.
  5. Install NVIDIA Container Toolkit by using the following command: 

    sudo apt update
    sudo apt-get install -y nvidia-container-toolkit 
    sudo nvidia-ctk runtime configure --runtime=docker

After running the sudo apt update  command, you might get the following error message:

image-2024-10-29_8-37-19.png

To resolve the error, run the following command:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys key

Where to go from here

See Configuring-BMC-AMI-AI-Services-for-installation-on-Microsoft-Azure.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*