Creating a virtual machine
Before you begin
The following table lists the recommended Azure virtual machine configuration for better performance and efficiency.
Configuration option | LLM | Azure |
---|---|---|
Recommended | Mixtral8x7b-instruct Quantized | Standard_NC24ads_A100_v4 (1 X A100) |
Mid-level | Meta-Llama-3-8B-instruct 4K Quantized (GPU) | Standard_NC24ads_A100_v4 (1 X A100) |
Entry level | Meta-Llama-3-8B-instruct 4K Quantized (CPU) | Standard_F32s_v2 |
If you cannot temporarily procure GPU-enabled machines and decide to proceed with the entry-level configuration, be aware that the performance of BMC AMI AI Services will be significantly slower, and some features might be unavailable.
To create a virtual machine
- On the Azure portal, select Create a resource.
The Create a resource window appears. - Under Virtual machine, select Create.
On the Basics tab, perform the following actions to specify the virtual machine settings:
Field
Action
Resource group
Click Create new and enter the following resource group name: amiai-resource-group.
Virtual machine name
Enter amiai-vm. (This name is an example, you can enter any name.)
Region
Select East US.
Security type
Select Standard.
Image
- Select Ubuntu Server 22.04 LTS – x64 Gen2 from the menu.
- Select x64 as virtual machine architecture.
Size
- Click See all sizes.
- In the Select a VM size window, select a type of virtual machine and then click Select.
Authentication type
Select Password.
Username
Enter an administrator user name.
Password
Enter an administrator password.
- For the Public inbound port, select Allow selected ports and select SSH in Select inbound ports.
- Accept the other defaults and then click Next: Disks.
- Select the required OS disk size. The suggested size is above 200 GB.
- Accept other defaults on the Disks tab and then click Next: Networking.
On the Networking tab, perform the following actions:
Field
Action
Virtual network
- Click Create new.
- In the Create virtual network dialog box, follow these steps:
- In the Name field, enter amiai-virtual-network.
- Accept the defaults and click OK.
Subnet
Select the default subnet.
Public IP
- Click Create new.
- In the Create public IP address dialog box, follow these steps:
- In the Name field, enter amiai-vm-public-ip.
- Accept the defaults and click OK.
- Accept the other defaults and then click Next: Management.
- Click Next: Monitoring.
- On the Monitoring tab, follow these steps:
- In the Boot diagnostics area, select Disable.
- Accept the other defaults and then click Review + create.
- On the Review + create tab, review the settings, correct any validation errors, and then click Create.
- Click Deployment details to expand, and then click amiai-vm.
- In the amiai-vm window, note down the public and private IPs.
- Click Network Settings and expand the Rules tab.
- Click the existing SSH Inbound rule to edit the rule.
- Select Source as IP Addresses.
Enter the IP address that you want to allow for SSH, HTTP, and HTTPS.
- Select Service as Custom.
- Enter 22,8000, 443 as Destination port ranges.
- Let other values be defaults and click Save.
- Click Create port rule.
- Select Inbound port rule.
- From the Source window, select IP Address.
- In the Source IP addresses/CIDR ranges window, enter the public IP of the machine.
- In the Destination port ranges window, enter 8000.
- Click Add.
- Install Python and its dependency on the Azure virtual machine.
Verify whether Python is already installed by entering the python3 command. If Python is not installed, then run the following commands:
sudo apt update
sudo apt install python3Install pip by using the following commands:
sudo apt update
sudo apt-get -y install python3-pip
To install the NVIDIA GPU driver extension
Follow this procedure if your Azure virtual machine has GPUs.
- Navigate to your Azure virtual machine, expand the Settings menu, and click Extensions + applications.
- Click +Add and search for NVIDIA.
- Select the extension, click Next, click Review + create, and then click Create.
- Wait for deployment to finish.
Install NVIDIA Container Toolkit by using the following command:
sudo apt update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
Where to go from here