vLLM on GCP - Developer Guide

Welcome to the Meetrix vLLM developer guide for Google Cloud Platform! vLLM is a high-throughput inference and serving engine for large language models. It uses PagedAttention and continuous batching to pack the KV cache tightly, which lets it serve many concurrent requests with low latency, and it exposes an OpenAI-compatible HTTP API so existing OpenAI SDKs and tools work against your own self-hosted model.

With the Meetrix pre-configured GCP image, you can deploy a production-ready vLLM instance on a GPU virtual machine in your own project in minutes. This guide walks you through finding the product on GCP Marketplace, configuring the deployment, choosing a model, pointing DNS and issuing SSL, and calling the API. If you run on AWS instead, the same stack is covered in our vLLM on AWS developer guide.

What You Get

The image ships a fully wired vLLM stack so you do not have to assemble the GPU toolchain yourself:

• vLLM serving an OpenAI-compatible API, running as a container on Ubuntu 26.04 LTS.
• NVIDIA drivers, CUDA, and the NVIDIA Container Toolkit pre-installed and matched to the attached GPU.
• An nginx reverse proxy for HTTP and HTTPS, tuned for streaming token responses.
• Automatic SSL via Let's Encrypt when a domain is supplied and its DNS points at the VM.
• Interactive API documentation at /docs (Swagger UI) and /redoc (ReDoc).
• A pre-hardened base image with a firewall, unattended upgrades, and the Google Cloud Ops Agent.

Launch the Product

Step 1: Find the Product

1. Log in to your Google account.
2. Go directly to the product page: vLLM Inference Server: High-Throughput Self-Hosted LLM Serving on GCP Marketplace
3. You can also browse all Meetrix products at the Meetrix Solutions Page.

Step 2: Launch the Product

1. Select your GCP project from the project selector at the top.
2. Click the Launch button.
3. Review the terms and agreements, tick the acknowledgement checkbox, and click AGREE.

Free Trial

This product includes a 5-day free trial with up to USD 50.00 in licence fee credits. To activate it, tick I accept the solution trial Terms and Conditions before proceeding.

Step 3: Configure the Deployment

You will see the deployment configuration form. Fill in the fields across the following sections.

General

• Deployment name - A unique name for this deployment (a default is pre-filled).
• Deployment Service Account - Select an existing service account that has the roles/config.agent, roles/compute.admin, and roles/iam.serviceAccountUser roles, or let GCP create a new one for you.
• Zone - Select a zone that has the GPU you want available (for example us-central1-a).

Machine Type

vLLM requires an NVIDIA GPU. In the Machine type section, choose a GPU-accelerated machine type rather than a plain CPU type:

• g2-standard-8 - one NVIDIA L4 with 24 GB of GPU memory. A solid default for 7B to 8B models such as Qwen/Qwen2.5-7B-Instruct.
• Larger L4 or a2 (A100) types - for bigger models, longer context windows, or higher concurrency.

GCP shows an estimated monthly cost, made up of the Meetrix licence fee and the underlying infrastructure fee, based on your selected machine type and disk size before you deploy. If you are still deciding which open model to serve, our roundup of the best open source LLMs to self-host covers the trade-offs between the popular families.

Step 4: Configure Networking

• Network and Subnetwork - Leave as default unless you have a custom VPC.
• External IP - Leave as Ephemeral. Select None only if you do not need public internet access.
• Allow SSH (TCP port 22) from the Internet - Enabled by default. Restrict the Source IP ranges for SSH traffic field to your own IP range if you want to limit SSH access.

Step 5: Application Settings

Scroll down to the Application Settings section and provide:

• Domain name - The domain for your vLLM API (for example vllm.yourdomain.com). Point your DNS A record to the VM IP before or shortly after deploying so SSL can issue. Leave the default if you plan to use the IP address directly.
• Admin email - The address used when requesting the Let's Encrypt SSL certificate.
• Model name - The Hugging Face repository ID of the model to serve, for example Qwen/Qwen2.5-7B-Instruct. Pick a model that fits in your machine type's GPU memory.
• Hugging Face token - An access token. Required only for gated or private models; leave it blank if your model is public.

Tick I accept the solution trial Terms and Conditions, then click Deploy and wait a few minutes for the deployment to complete.

Point DNS to Your vLLM Server

Skip this section if you kept the default domain and plan to use the external IP directly.

Step 1: Get the External IP

1. Once deployment is complete, open the VM instance from the deployment details.
2. Copy the External IP from the Network interfaces section.

Step 2: Create a DNS A Record

1. Go to your DNS provider.
2. Add an A record pointing your vLLM domain (for example vllm.yourdomain.com) to the copied external IP.
3. Wait for DNS propagation before proceeding (typically a few minutes to 1 hour).

Access the vLLM API

Once DNS has propagated and SSL has issued, open https://your-domain/docs to see the interactive Swagger UI. The server exposes the OpenAI-compatible endpoints such as /v1/models, /v1/chat/completions, and /v1/completions, along with helper routes like /health and /metrics.

Call the API

Send a chat completion request with any HTTP client. Replace the host with your domain or external IP and the model with the one you deployed:

curl https://<your-domain>/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen2.5-7B-Instruct",
    "messages": [{"role": "user", "content": "Say hello in one sentence."}]
  }'

Because the API is OpenAI-compatible, you can also point the official OpenAI SDKs at it by setting the base URL to https://<your-domain>/v1. That means tools built for the OpenAI API work unchanged: you can, for example, point a self-hosted Hermes Agent at this endpoint, or use it as the generation backend behind a retrieval pipeline built on a self-hosted Milvus vector database.

Generate an SSL Certificate Manually

vLLM tries to issue SSL automatically on first boot for the domain you passed as the Domain name. If that fails, for example because DNS had not propagated yet, you can generate it manually.

Step 1: SSH into the Server

1. Go to the VM instance page in the GCP console.
2. Click SSH to open a browser-based terminal and authorize access.

Step 2: Run the Certificate Script

The image includes a pre-configured certificate script. Run it with:

sudo bash /root/certificate_generate_standalone.sh

This script runs certbot using the domain name and admin email you provided during deployment.

Check Server Logs

Open an SSH session to the instance from the GCP console, then inspect the vLLM container. This is where you can watch the model download and the GPU worker come online:

sudo docker ps
sudo docker logs -f <container-id-prefix>

To confirm the GPU is visible to the host and see memory use, run:

nvidia-smi

Manage the Deployment

Stop the VM

To stop the VM without deleting it, go to Compute Engine → VM Instances in the GCP console, select your instance, and click Stop. Stopping the VM halts GPU billing while keeping the boot disk. You can restart it later with your data intact.

Delete the Deployment

To fully remove the deployment and stop all billing:

1. Go to Solution deployments in the GCP console.
2. Find your vLLM deployment.
3. Click Delete to remove the VM, its disk, and the networking resources the deployment created.

Note the model name and any configuration changes first so you can reapply them on a fresh deployment.

Upgrades

When a new image version is available in the GCP Marketplace, delete the previous deployment and relaunch with the new version. Record the model name and any service configuration changes beforehand so you can reapply them.

Troubleshoot

GPU Quota Errors

GCP enforces per-region GPU quotas, and new projects often have a limit of zero for accelerators like the NVIDIA L4. If deployment fails with a quota error, request a GPU quota increase for that region in IAM & Admin → Quotas, or choose a region or zone where you already have quota. Google's resource quotas documentation explains the request flow.

Insufficient Capacity Errors

If deployment fails because the GPU type is not available in the selected zone right now, try a different zone or region, or a different accelerator type.

502 Bad Gateway

If the API is temporarily inaccessible, wait 5-15 minutes and retry. On first boot vLLM downloads the model weights and loads them onto the GPU before it starts answering.

Model Fails to Load

If the container logs show an out-of-memory error, the model is too large for the GPU. Choose a smaller model, a quantized build, or a larger machine type. For gated models, confirm the Hugging Face token was set and the account has accepted the model's licence on its Hugging Face model page.

SSL Did Not Issue

Confirm your DNS A record points at the VM's external IP and has propagated, then re-run the certificate script from the "Generate an SSL Certificate Manually" section.

Disk Space

Model weights are large, so a full boot disk can stop vLLM from starting. Check usage:

df -h

If the root volume is between 90-100% full, resize the persistent disk in the GCP console, then reboot the VM and restart the service.

Conclusion

The Meetrix vLLM Deployment Guide gets a high-throughput, OpenAI-compatible inference server running on a GPU VM in your own GCP project in minutes. Once it is up, you point any OpenAI SDK or tool at a single endpoint, serve the open model of your choice on your own hardware, and keep your prompts and data inside your project. For the full source and configuration options, see the vLLM GitHub repository and the official vLLM documentation.

vLLM on AWS - Developer Guide

Deploy vLLM, the high-throughput OpenAI-compatible LLM inference server, on AWS with our step-by-step CloudFormation guide. Learn how to launch a GPU instance, pick a model, secure it with SSL, and call the API.

By Binuka Ranatunga • Meetrix.io

Technical Support

If you run into any issues, our support team is here to help. Reach out to us at support@meetrix.io and we will respond within 12 hours.