Creating an EC2 instance
Before you begin
The following infrastructure is required for optimal performance with BMC AMI AI Services.
Configuration option | Supported LLM | AWS |
---|---|---|
Recommended | Mixtral8x7b-instruct Quantized | g5.12xlarge |
Mid-level |
| g5.4xlarge |
Entry level | Meta-Llama-3-8B-instruct (CPU) | c6in.8xlarge |
If you cannot temporarily procure GPU-enabled machines and decide to proceed with the entry-level configuration, be aware that the performance of BMC AMI AI Services will be significantly slower, and some features might be unavailable.
If you don't already have them, create a Virtual Private Cloud (VPC), set up subnets, and configure an Internet Gateway as follows. If you already have them, you can reuse them later in the deployment process.
- Sign in to your AWS account.
Open the Amazon VPC console athttps://console.aws.amazon.com/vpcconsole.
- On the VPC console dashboard, click Create VPC .
- Keep all unspecified values at their default settings in the following steps.
- In the VPC settings field, click VPC and more.
- For ... name, enter a name such as BMC-AMI-AI-Services.
- Enter the IPv4 CIDR block value. For example, 10.0.0.0/16.
- For Number of private subnets, select 0 (zero).
- For VPC endpoints, select None.
- Click Create VPC.
To create an EC2 instance
- Sign in to your AWS account.
Open the Amazon EC2 console athttps://console.aws.amazon.com/ec2/ .
- On the EC2 console dashboard, in the Launch instance area, click Launch instance.
- In the Name field, in the Name and tags area, enter a name (for example, AMIAI_VM).
- In the search box in the Application and OS Images area, enter Deep Learning OSS Nvidia Driver AMI GPU PyTorch.
- From the search results, select Ubuntu/Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.2 or later (Ubuntu 22.04).
- In the Instance type area, from the menu, we recommend that you select g5.12xlarge for optimal performance.
- In the Key pair (login) area, click Create new key pair and follow these steps:
- Enter a key pair name (for example, AMIAI_KeyPair).
- Select RSA for the key pair type.
- Select .ppk for the Private Key file format.
- Click Create key pair to download the .ppk file. Store it in a safe location.
- In the Network settings area, click Edit.
- For VPC, select the VPC that you created in the previous step.
- From the Auto-assign public IP menu, select Enable.
- For Firewall, click Create security group.
- For Inbound Security Group Rules, delete all rules except for the SSH type.
- From the SSH Source type , select My IP . Only your machine has SSH access to this EC2 instance.
- In the Configure storage area, modify the storage in accordance with your needs.
- In the Summary area, enter the number of instances of EC2 required.
- Click Launch instance.
- On the confirmation page, which indicates that your instance is launching, click View all instances to close the confirmation page and return to the console.
- In the Instances area, you can view the launch status of your EC2 instance. Launching an instance takes a short time. When you launch an instance, its initial state is Pending. After the instance starts, its state changes to Running and receives a public DNS name.
- Click the Instance ID to launch your instance.
- Note down the Public and Private IPs and their DNS names.
To add the inbound rules, seehttps://docs.aws.amazon.com/finspace/latest/userguide/step5-config-inbound-rule.html.
- Add an inbound rule to the EC2 instance public IP address to allow traffic on port 8000.
- If you deployed the BMC AMI AI Services on zCX/zLinux, then add the inbound rule to the zCX/zLinux instance IP address to port 4000.
- If the SSL is enabled, then add an inbound rule on ports 8000, 8001, 8002, 8080, 8005, and 4000. You must allow these ports to the EC2 Instance public IP.
Where to go from here