LoginSign Up
← Back to all sparks
V

vLLM

AI-ASSISTANTS
Velocity5.0

High-throughput inference and serving engine for large language models

docs.vllm.ai ↗

vLLM's release candidates are where the hardware and speculative-decoding seams get sewn.

speculative-decodinghardware-breadthtransformers-backendrelease-candidatesinference-serving
◆Current state
vLLM tags frequently and most tags carry a single commit subject as their entire changelog. The window runs from the 0.25 rc series — Transformers-backend embedding scaling and CUDA graph capture, disaggregated prefill/decode KV-load lookahead under MTP speculative decoding, a flaky ARM ShortConv test — through the 0.26.1 and 0.27.0 tags, into the current 0.27.2rc0 carrying a confidence-scheduled verification scheme for speculative decoding. Hardware breadth is constant background work: TPU, ROCm, ARM and CUDA paths all appear.
◆Where it's heading
Two things are being maintained at once. One is reach — keeping AMD, TPU and ARM honest, and keeping the Transformers modelling backend correct so new architectures run without bespoke kernels. The other is speculative decoding, which keeps producing work at its seams: first the interaction with disaggregated prefill/decode, now the verification schedule itself. The rc tags carry the interesting commits and the stable tags mostly ratify them, so reading only the stable releases understates what is moving.
◆Prediction
The confidence-scheduled verification work should surface in a 0.27.2 stable tag on the usual short rc-to-release gap. Whether it becomes a default or stays an opt-in scheduler is not answerable from a commit subject.

◆Recent moves

  1. 6d ago

    v0.27.2rc0 — DSpark confidence-scheduled spec-decode verification

    A confidence-scheduled verification scheme for speculative decoding, landing on an rc tag with nothing but sign-offs as its body. Scheduling verification by draft confidence targets the acceptance-rate trade-off directly, which is the part of speculative decoding this train keeps returning to.

    View source ↗
  2. 9d ago

    v0.27.0 — TPU compile fix for Kimi's vision tower

    The 0.27 stable tag ships carrying a TPU-specific fix that disables dynamic torch.compile for Kimi's vision tower. Small in itself, but it places multimodal-on-TPU inside the supported path rather than at the edge of it, which is consistent with the hardware-breadth work running through this whole window.

    View source ↗
  3. 22d ago

    v0.26.1rc0 — ROCm CI correctness reference fix

    A ROCm CI fix correcting the reference value in a wikitext correctness test. Test infrastructure only — it keeps the AMD path honest without changing anything about how the server behaves.

    View source ↗
  4. 1mo ago

    v0.25.0rc3 — P/D KV-load lookahead fix under MTP speculative decode

    Fixes async KV-load lookahead in disaggregated prefill/decode when MTP speculative decoding is active. Each feature is mature on its own; this is the seam between them, which is where the 0.25 train has spent most of its rcs.

    View source ↗
  5. 1mo ago

    v0.25.0rc2 — embed scaling and CUDA graph fixes in Transformers backend

    Corrects embedding scaling and CUDA graph capture in the Transformers modelling backend. That backend is how vLLM absorbs new architectures without bespoke kernels, so correctness there quietly widens the set of models that run well.

    View source ↗
  6. 1mo ago

    v0.25.0rc1 — flaky ARM ShortConv prefill test fix

    Fixes a flaky ShortConv prefill test on ARM caused by uninitialized state. Confirms CPU/ARM is still a maintained target, but nothing here changes what a user can run.

    View source ↗