vLLM vs SGLang: Which Should You Self-Host?

If you're choosing between vLLM and SGLang, start with vLLM unless your traffic looks like an agent: lots of requests that share a long prefix, such as a system prompt, tool definitions or a growing conversation. That's where SGLang's design pays off. For everything else the two are close enough that published benchmarks disagree about which one wins, and the honest advice is to test your own workload.

Both are open source under Apache 2.0, both expose an OpenAI-compatible API, and both ship a new release roughly every two weeks. As of September 28, 2026 the current versions are vLLM 0.30.0 and SGLang 0.5.20. Below is what each one does, how they handle prompt caching differently, what the benchmarks really show (and why I wouldn't trust any single one), the commands to run each, and where I'd start.

How to Choose

Five questions decide most of it, and you can answer them from your own logs before you run a single benchmark.

1. How much of each prompt is shared between requests? A fixed system prompt, tool definitions or a RAG document that repeats across calls favours SGLang's radix tree. Mostly unique prompts mean caching barely matters.
2. Is it one long conversation or many short calls? Multi-turn chats and agent loops keep growing a prefix that could be reused. One-shot completions don't.
3. What hardware do you have? Both cover NVIDIA and AMD GPUs. vLLM's list is longer, with plugins for Intel Gaudi and Apple Silicon. SGLang lists Intel Xeon CPUs, Google TPUs and Ascend NPUs.
4. Is your model supported today? vLLM documents 200+ architectures. SGLang lists the major families and says it works with most Hugging Face models. Check your exact model on both, especially a brand-new release. If you are still picking a model, our list of the best open-source LLMs to self-host is a good start.
5. How much churn can you take? Both release about every two weeks, and both remove features. If you can't afford to retest after an upgrade, pin a version and upgrade on your own schedule.

How They Cache Prompts

This is the real technical difference, and the reason the two are usually pitched at different workloads.

vLLM hashes each block of KV cache using the tokens in the block and the tokens before it. Its prefix caching design docs describe it as a hash-based system with least-recently-used eviction, and they're explicit about a limit: only full blocks are cached. SGLang's RadixAttention stores cached sequences in a radix tree instead, so a new request finds the longest matching prefix wherever it branches, without waiting for a block boundary.

In practice that matters when many requests start the same way. An agent that sends the same 2,000-token system prompt and tool definitions on every call, or a chat that appends a turn each time, reuses a lot of work. If most of your prompts are unique, there's little to reuse and the difference mostly disappears.

What the Benchmarks Say

I looked at three published head-to-head tests and SGLang's own release notes. They don't agree, and the reasons why are more useful than any single number.

Figures as published by each source. I haven't re-run any of them.

vLLM vs SGLang Side by Side

Facts from each project's own README and docs, checked on September 28, 2026.

Most of the feature rows are ticks on both sides, and that's the point. The differences that matter are the cache design, hardware and how each behaves on your traffic.

Running Each One

Both install with pip or uv and start an OpenAI-compatible server with one command. This is vLLM, from its quickstart:

uv pip install vllm --torch-backend=auto
vllm serve Qwen/Qwen2.5-1.5B-Instruct
# serves on http://localhost:8000

And this is SGLang, from its quickstart. It also publishes an official lmsysorg/sglang Docker image if you'd rather not manage CUDA wheels yourself:

uv pip install --prerelease=allow sglang
sglang serve --model-path qwen/qwen2.5-0.5b-instruct --host 0.0.0.0 --port 30000
# serves on http://localhost:30000

Because both speak the OpenAI API, a client only needs a different base URL. That makes a side-by-side test cheap:

from openai import OpenAI

# vLLM listens on 8000, SGLang on 30000
client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")

response = client.chat.completions.create(
    model="Qwen/Qwen2.5-1.5B-Instruct",
    messages=[{"role": "user", "content": "List 3 countries and their capitals."}],
    max_tokens=64,
)
print(response.choices[0].message.content)

When you compare them, replay your real prompts at your real concurrency, and make sure prefix caching is actually on for vLLM. One of the benchmarks above ran vLLM without it in its main tests and another doesn't say, so it's a real variable.

Track four numbers: median and p95 time to first token, tokens per second at your peak concurrency, the prefix cache hit rate if the engine exposes it, and GPU memory headroom. Tokens per second alone hides the thing that agent users actually feel, which is how long they wait before anything appears.

Keeping Up With Releases

Both projects release about every two weeks, and both break things between releases. Version 0.30.0 of vLLM removed GPTQ activation ordering and put its scale-out endpoints behind a flag. SGLang 0.5.20 retired the first version of prefill context parallelism, and its CUDA 12 builds ended after 0.5.19.

None of that is a criticism. It's what a fast-moving inference stack looks like. It does mean you should pin an exact version in production, read the release notes (vLLM, SGLang) before every upgrade, and keep your previous image around so you can roll back.

Self-Hosting With Meetrix

Where I'd Start

Related Articles

Ollama vs vLLM in Production

Ollama vs vLLM in production: throughput under load, latency, memory, GPUs, security and monitoring compared, with tuning tips and when to switch in 2026.

By Shalomi Umeshika • Meetrix.io

Llama vs Mistral vs DeepSeek: Best Self-Hosted LLM in 2026

The best self hosted LLM in 2026: Llama vs Mistral vs DeepSeek compared on licensing, benchmarks, and hardware footprint for real self-hosted deployments.

By Shalomi Umeshika • Meetrix.io

Self-Host a Voice AI Agent Stack

Self-host a voice AI agent stack with open-source tools: LiveKit or Pipecat, Whisper STT, a local LLM on vLLM and Kokoro TTS. Layers, costs, latency, pitfalls.

By Shalomi Umeshika • Meetrix.io