LoginSign Up
← Back to all sparks
V

vLLM

AI-ASSISTANTS
Velocity6.3

High-throughput inference and serving engine for large language models

docs.vllm.ai ↗

vLLM in a six-RC sprint to stabilize v0.29.0 with Mamba and hybrid prefix caching

llm-inferenceprefix-cachingmoe-modelsmambatrt-llmspeculative-decoding
◆Current state
vLLM is in intensive release candidate territory for v0.29.0, shipping six RC builds in under a week. The work is concentrated on prefix caching for Mamba and hybrid architectures, CUTLASS MoE permutation correctness, and TRT-LLM backend synchronization. None of these are user-visible capabilities — they're pre-release bug convergence.
◆Where it's heading
Repeated prefix-cache fixes for Mamba and hybrid models signal that non-transformer architecture support is being promoted to first-class status in vLLM. The CUTLASS and TRT-LLM work shows backend coverage expanding beyond vanilla GPU inference. Once v0.29.0 stable lands, the next focus is likely speculative decoding maturity — the DSpark and DFlash2 work from earlier entries were architecturally more interesting than anything in this RC cycle.
◆Prediction
v0.29.0 stable is days away given the RC cadence. The stable release will formally include dense prefix caching as a default for Mamba models, the recurring theme across rc5 and rc6.

◆Recent moves

  1. 1mo ago

    vLLM 0.29.0-rc6: dense prefix cache defaults for hybrid architectures

    RC6 applies the dense prefix cache default to hybrid models — a followup to rc5's Mamba change, extending the same fix to attention+state-space architectures. Still a convergence fix inside the RC cycle.

    View source ↗
  2. 1mo ago

    vLLM 0.29.0-rc5: prefix cache retention defaults for Mamba models

    RC5 defaults prefix_cache_retention_interval to dense for Mamba models, fixing an inference degradation for state-space architectures that had been present without explicit configuration. One of several Mamba-specific fixes in this RC sprint.

    View source ↗
  3. 1mo ago

    v0.29.0rc4: [Bugfix] Avoid sync in TRT-LLM ragged prefill

    RC4 removes an unnecessary synchronization point in TRT-LLM's ragged prefill path, a correctness fix for deployments using the TensorRT-LLM backend. No user-visible capability change.

    View source ↗
  4. 1mo ago

    vLLM 0.29.0-rc3: CI cleanup, stale Nemotron model reference removed

    RC3 is pure CI cleanup — removing a deleted Nvidia Nemotron model reference that was causing test failures. No code change relevant to end users.

    View source ↗
  5. 1mo ago

    v0.29.0rc2

    RC2 fixes handling of prefix-covered items in the shared-memory worker cache for multimodal inputs, closing a correctness gap that could affect vision model inference during prefill.

    View source ↗
  6. 1mo ago

    v0.29.0rc1: [Bugfix] Handle padded routes in CUTLASS MoE permutations (#54747)

    RC1 fixes a correctness bug in CUTLASS MoE permutation handling for padded routes — without this fix, MoE models with route padding could produce silently wrong outputs. The fix strengthens vLLM's MoE inference reliability as these architectures become more common.

    View source ↗