vLLM on GCP: Your Own Inference Endpoint, Billed Like Compute Instead of Tokens
Every hosted inference API works roughly the same way underneath. You send tokens, the provider runs the model on hardware you'll never see, and the bill grows in exact proportion to how much your product gets used. That's fine when usage is low. It stops being fine the month your app actually takes off and the inference line item grows faster than the revenue that's supposed to be paying for it.
Meetrix packaged vLLM as a GCP Marketplace deployment that launches to a single GPU VM in your own Google Cloud project. NVIDIA drivers and CUDA pre-installed. The container pre-pulled, so first boot doesn't stall downloading it. nginx tuned for streaming tokens. SSL handled automatically.
Want to try it? Launch the Meetrix vLLM listing on GCP Marketplace.
What Is vLLM?
vLLM is an open-source inference and serving engine for large language models. It's released under the Apache 2.0 license, originally developed at UC Berkeley's Sky Computing Lab, and now a PyTorch Foundation-hosted project.
The throughput it's known for comes from two specific mechanisms.
PagedAttention manages the KV cache the way an operating system manages memory pages. That means GPU memory doesn't get wasted on padding or fragmentation. In practice, memory utilization can jump from around 40% to 96%.
Continuous batching keeps new requests flowing into a batch that's already running, instead of waiting for one batch to finish before starting the next.
Together, those two things let one GPU serve far more concurrent requests than a naive implementation would.
It also exposes an OpenAI-compatible HTTP API at /v1. That matters more than it sounds like it should. The server currently supports Completions (/v1/completions), Chat Completions (/v1/chat/completions), Responses (/v1/responses), and Embeddings (/v1/embeddings). Code written against the OpenAI SDK, LangChain, or any tool that expects that API shape works against your self-hosted model with a base URL change and nothing else. You're not rewriting integration code just to move from a hosted provider to your own instance.
Weighing it against another serving engine first? Our vLLM vs SGLang comparison covers how they differ. vLLM is the general-purpose choice for standard chat and high-throughput batch workloads. SGLang tends to shine with structured output and shared-prefix agentic workloads.
What Pay-Per-Token Actually Costs at Real Volume
How Deployment Works
Setting this up by hand means picking a GPU machine type, installing the right NVIDIA driver and CUDA version, wiring up the container GPU runtime, configuring nginx to not buffer streamed tokens, and downloading the model weights before serving a single request. Through the Marketplace, it's four steps.
- Find and Launch From GCP Marketplace Open the Meetrix vLLM listing, select your project, and launch the deployment on a GPU-accelerated machine type. g2-standard-8 with one NVIDIA L4 is the sensible default for 7B to 8B models.
- GPU Drivers and the Container Are Already Set Up NVIDIA drivers, CUDA, and the NVIDIA Container Toolkit are pre-installed and matched to the instance's GPU. The container image is pre-pulled, so first boot doesn't spend its first minutes downloading it.
- Pick Your Model Set the Model name field to a Hugging Face repository ID, like Qwen/Qwen2.5-7B-Instruct. Add a Hugging Face token only if the model is gated or private.
- Wait for Boot, Then Call the API Give the VM a few minutes to finish starting. Then point any OpenAI-compatible client at your domain's /v1 path. Restrict access before you send it real traffic, it's open by default.
Check Your GPU Quota Before You Launch
What Meetrix Brings to This Deployment
- No GPU Toolchain to Assemble - NVIDIA drivers, CUDA, and the container GPU runtime come pre-installed and version-matched. It's not a driver compatibility puzzle you have to solve before vLLM will even start.
- A Pre-Pulled Container, Not a Cold Download - The vLLM container image is already on the instance. First boot is spent starting the service, not fetching gigabytes of image layers.
- SSL and a Streaming-Ready Proxy From the Start - Automatic Let's Encrypt certificates once DNS points at the VM. nginx is already tuned so token-by-token streaming responses don't get buffered into one slow chunk.
- Interactive API Docs Included - Swagger UI at /docs and ReDoc at /redoc ship with the instance. You can see and test every route without hunting through vLLM's own documentation first.
- Hardened by Default - A firewall, unattended upgrades, and the Google Cloud Ops Agent are active out of the box, plus IP-restricted SSH. It's not a hardening pass you do after the fact.
- People Who Actually Run This Stack - Support goes to support@meetrix.io with a response inside 12 hours. These are engineers who deploy GPU images regularly, not a queue reading from a script.
Who Is vLLM on GCP Right For?
This fits teams that want inference costs to behave like infrastructure, not like a metered utility that tracks their growth. More specifically, it's a strong match if you're:
- A team that priced out a hosted inference API against real projected token volume and didn't like watching the number scale with success
- A developer who wants to run Llama, Qwen, Mistral, Gemma, or Phi without rewriting OpenAI SDK integration code to talk to it
- An engineering org that needs model weights and inference requests to stay inside its own GCP project, not a shared third-party cluster
- A team serving high, steady request volume where a flat GPU-hour cost beats a per-token fee that never stops accumulating
- Someone who wants to swap the served model by changing a deployment field, rather than migrating to a different hosted provider entirely
- A builder who wants Swagger docs and an OpenAI-compatible endpoint on day one, not a bare inference script to wrap in an API themselves
vLLM on GCP by Meetrix vs Alternatives
| Feature | vLLM on GCP by Meetrix | Together AI | Vertex AI | Self-Managed Install |
|---|---|---|---|---|
| Hosting | Your own GCP project, fully self-hosted | Together's cloud, no self-host option for serverless inference | Google-managed infrastructure, not a VM in your own project | Your own GPU server, set up by you |
| License | Apache 2.0, fully open source | Proprietary platform | Proprietary Google service | Open source, but you own every step of running it |
| Pricing Model | Flat GPU VM rate plus licence fee, no per-token fee | Pay-per-token, roughly $0.05-$7.00 per million tokens by model | Pay-per-token or provisioned throughput, priced by model | GPU server cost, no per-token fee, but you build the stack |
| Model Choice | Any Hugging Face model that fits your GPU's memory | Together's curated model catalog | Vertex AI's Model Garden catalog | Whatever you set up yourself |
| Deployment Time | Minutes via GCP Marketplace | Instant API signup | Instant, already inside GCP | Hours to days: drivers, CUDA, container runtime, SSL |
| Data Location | Your own GCP project | Together's infrastructure | Google's shared infrastructure | Your server |
| Support | Meetrix engineers, response inside 12 hours | Support tiers, gated by plan | Google Cloud support plans, gated by tier | Community forums only |
Resources
Under the Hood
Here's what's actually running once the deployment finishes.
vLLM in a Pre-Pulled Container
Runs as a container on Ubuntu 26.04 LTS, with the image already pulled onto the instance so first boot starts the service instead of downloading it.
Drivers Matched to the Instance
NVIDIA drivers, CUDA, and the container GPU runtime come pre-installed and matched to whichever machine type you chose, not a version you have to reconcile by hand.
nginx Tuned for Streaming
Configured for token-by-token streaming responses rather than buffering, so a chat completion streams the way it's supposed to instead of arriving as one delayed block.
SSL via Let's Encrypt
Certificates issue automatically when a domain is supplied and its DNS points at the VM. No manual certbot step required.
Swagger and ReDoc Included
Interactive documentation ships at /docs and /redoc, listing every route the OpenAI-compatible API exposes.
Firewall, Unattended Upgrades, and the Ops Agent
The base image ships with a firewall active, automatic security patching enabled, and the Google Cloud Ops Agent running, plus IP-restricted SSH access.
One Thing vLLM Won't Do For You
It won't lock the door behind you. The OpenAI-compatible endpoint is open to anyone who can reach it the moment the VM boots. No API key. No auth. Nothing.
That's a reasonable default for a quick test against your own IP. It's a genuinely bad one to leave in place before pointing a public app at it.
Here's something a lot of guides gloss over: even if you turn on vLLM's --api-key option, it only protects endpoints under /v1, /v2, and /inference. Other routes on the same server, most notably /invocations, which exposes the same inference capabilities, are not authenticated by that key alone. The vLLM security documentation explicitly warns against relying on --api-key as a full server boundary and recommends deploying behind a reverse proxy for proper hardening.
So restrict the firewall source ranges. Put it behind a VPN or a reverse proxy with authentication. Treat the API key as one layer among several.
This is the one step in the whole deployment that's on you, not the image. If you outgrow a single VM, our guide to deploying open-weight LLMs on GKE covers the same server on Kubernetes, with autoscaling for larger models.
Frequently Asked Questions
What is vLLM, in plain terms?
An open-source engine for serving large language models efficiently. It uses PagedAttention and continuous batching to handle many concurrent requests on one GPU, and it speaks the same API as OpenAI, so existing code works with a URL change.
Is vLLM free?
Yes. vLLM is released under the Apache 2.0 license, so the software itself costs nothing. You pay for the GPU infrastructure it runs on, plus the Meetrix licence fee on the Marketplace listing.
What does this cost to run on GCP?
On the recommended g2-standard-8 instance with one NVIDIA L4, on-demand pricing in us-central1 runs about $0.854 per hour, roughly $623 per month. That's flat regardless of request volume.
Which machine type should I actually pick?
g2-standard-8 with one L4 GPU is the sensible default for 7B to 8B models. For larger models or higher throughput, you'll want a machine type with more GPU memory.
Which models can I actually run on this?
Any Hugging Face model that fits in your GPU's memory. The L4 has 24 GB of VRAM, so 7B to 8B models in FP16, or larger models with quantization, work well.
Is the API secured by default?
No. The endpoint is open by default. You need to restrict firewall access and add authentication before exposing it to real traffic. And vLLM's --api-key only protects /v1, /v2 and /inference routes, not every endpoint on the server.
How do I call it from my existing code?
Point your OpenAI-compatible client at your domain's /v1 path. The base URL is the only change needed for most SDKs and frameworks.
Where does the model actually run?
On a GPU VM inside your own GCP project. Model weights, inference requests and outputs never leave your Google Cloud environment.
Run Your Own Inference Server on GCP
Stop paying per token for inference you could run on your own GPU. Deploy vLLM on GCP in minutes, set up by a team that runs this stack every day.
Deploy on GCP Marketplace