> Source: https://meetrix.io/store/llama-2/
> Markdown copy of that page. Cite the URL above, not this file.

[Store](https://meetrix.io/store/)  Llama 2

Open-source LLM

# Self-hosted Llama 2 on AWS

Run Meta's Llama 2, in 7B or 70B, behind an OpenAI-compatible API on a GPU instance in your own AWS account. Existing OpenAI tools work with a new base URL, and your prompts stay private.

 [Browse Meetrix on AWS](https://aws.amazon.com/marketplace/seller-profile?id=62b53dd8-45d0-4b50-a654-bd6993167486)

-   7B and 70B models
-   OpenAI-compatible API
-   Pay per hour

![Llama 2 deployed on AWS by Meetrix](https://meetrix.io/assets/store/llama-2/llama-2.webp)

## What is Llama 2?

Llama 2 is Meta's pre-trained generative text model family. Its Llama-2-Chat models are tuned for dialogue, and the open weights let you run them on your own hardware instead of calling a hosted API.

-   ### Two sizes

    7B for low cost and speed, 70B for deeper text understanding.

-   ### Chat models

    Llama-2-Chat for assistants and support bots.

-   ### OpenAI-compatible API

    Chat, completions, embeddings and model listing endpoints.

-   ### Codex-style completions

    A /v1/engines/copilot-codex/completions endpoint for code tools.

-   ### Automatic SSL

    Certificates for Route 53-hosted domains.

-   ### Private by default

    Prompts and outputs stay in your own account.

## What's in the Meetrix Llama 2 AMIs

Each AMI ships the model and an OpenAI-compatible API server on Ubuntu 22.04, ready on a g4dn GPU instance.

-   Llama 2 7B or 70B with an OpenAI-compatible API
-   GPU-ready Ubuntu 22.04 image for g4dn instances
-   Automatic SSL when your domain is hosted on Route 53
-   A certificate script for issuing SSL manually otherwise
-   A CloudFormation template for your own AWS account
-   Email support from Meetrix at aws@meetrix.io

## How to set up a self-hosted Llama 2 API

1.  ### Pick a size

    Choose 7B or 70B and check your account has vCPU quota for the matching g4dn instance.

2.  ### Launch the stack

    Subscribe on AWS Marketplace and launch the CloudFormation stack with your domain and admin email.

3.  ### Point your domain at it

    After 5-10 minutes, create a DNS record with the PublicIp from the stack outputs.

4.  ### Call the API

    Open the DashboardUrl for the API docs, then point your OpenAI client at your server.

## Deploy on AWS

### Llama 2 on AWS

Two AMIs on AWS Marketplace, each launched on a g4dn GPU instance in your own AWS account with CloudFormation.

Deploys with

CloudFormation

7B instance

g4dn.xlarge

70B instance

g4dn.12xlarge

#### Setup guides

-   [**Developer guide** Launch, SSL, API endpoints, testing](https://meetrix.io/blogs/llama2-developer-guide/)
-   [**Install Llama 2 on AWS** Requirements and best practices](https://meetrix.io/blogs/how-to-install-llama-2/)
-   [**Video walkthrough** Watch the deployment on YouTube](https://www.youtube.com/watch?v=zvRbySK6Q6w)

 [Browse Meetrix on AWS](https://aws.amazon.com/marketplace/seller-profile?id=62b53dd8-45d0-4b50-a654-bd6993167486)

## Llama 2 models

| Model | Recommended instance | Supported instances | AWS |
| --- | --- | --- | --- |
| Llama 2 7B | g4dn.xlarge | g4dn.xlarge to g4dn.metal | [Launch on AWS](https://aws.amazon.com/marketplace/pp/prodview-f6d2k5vasskwa) |
| Llama 2 70B | g4dn.12xlarge | g4dn.12xlarge, g4dn.metal | [Launch on AWS](https://aws.amazon.com/marketplace/pp/prodview-3vkj4msxhjsq4) |

## Llama 2 API endpoints

| Endpoint | Purpose |
| --- | --- |
| /v1/chat/completions | Chat completions from a list of messages |
| /v1/completions | Completions from a prompt |
| /v1/embeddings | Embeddings for input text |
| /v1/engines/copilot-codex/completions | Codex-style code completions |
| /v1/models | List available models |

The developer guide includes a small Node.js script that checks every endpoint. [Read the Llama 2 developer guide →](https://meetrix.io/blogs/llama2-developer-guide/)

## Video: Llama AMI installation walkthrough (shown with Llama 3)

## Llama 2 FAQ

What is Llama 2?

Meta's pre-trained generative text model family, packaged here with an OpenAI-compatible API.

Which size should I choose?

7B on g4dn.xlarge for lower cost and faster responses; 70B on g4dn.12xlarge for more capable output.

Is it compatible with the OpenAI API?

Yes. It serves the OpenAI chat, completions, embeddings and models endpoints, so OpenAI SDKs work with a new base URL.

How do I test the API?

The developer guide has a Node.js script that calls /completions, /embeddings, /chat/completions and /models and prints the results.

What if I see a 502 Bad Gateway error?

The model is still loading. Wait about 5 minutes and refresh.

What if SSL does not set up automatically?

That happens when the domain is not on Route 53. SSH in and run sudo /root/certificate\_generate\_standalone.sh.

Is there a free trial?

No. The listings are billed per hour of software use plus the EC2 instance.

## Llama guides and articles

### [How to Install Llama 2 on AWS via Pre-configured AMI](https://meetrix.io/blogs/how-to-install-llama-2/)

Requirements, single-click vs manual, and AWS best practices.

### [Llama 2 Meta AI Transforming AI Across Industries](https://meetrix.io/blogs/unveiling-the-language-revolution-llama-2-meta-ai/)

Where Llama 2 fits in real workloads, 7B vs 70B.

### [Self-Host an OpenAI-Compatible LLM API on AWS](https://meetrix.io/blogs/openai-compatible-api-self-hosted/)

Swap the OpenAI base URL without rewriting app code.

## Need a hand with Llama?

We build and run self-hosted AI infrastructure every day, from model selection to GPU sizing. Tell us what you need.

[Contact us](https://meetrix.io/contact-us/)