LoginSign Up
← Back to all sparks
T

Transformers

AI-ASSISTANTS
Velocity6.3

Hugging Face library providing thousands of pretrained models for NLP, vision, and audio.

huggingface.co/docs/transformers ↗

Transformers is becoming a kernel-dispatch layer, and it's breaking APIs to get there

kernel-dispatchbreaking-changesvllm-backendday-0-modelsattention-backendscache-api
◆Current state
Transformers ships every two to four weeks on a split rhythm: minors carry day-0 architecture support for newly released open-weight models, patches almost exclusively unblock downstream serving runtimes. The last six releases added Meta's Muse Glimmer, Thinking Machines' Inkling, the Kimi K2.5 family and MiMo-V2-Flash, while three separate patches existed mainly to keep vLLM in sync. v5.15.0 breaks that pattern by landing four flagged breaking changes at once, including making kernel selection opt-in for linear attention models.
◆Where it's heading
The refactor visible across these releases is a consolidation onto shared attention and kernel dispatch: the T5 family moved onto ALL_ATTENTION_FUNCTIONS, every linear attention model was rewritten against one convolution standard, and Gemma 4's heterogeneous attention config was made explicit through per_layer_config. The release notes state outright that the kernels package will likely become a required dependency of transformers[torch]. Alongside that, the project is absorbing compatibility work on behalf of vLLM rather than its own direct users — weight remaps and attention-backend flags added specifically for the vLLM modelling backend.
◆Prediction
Expect kernels to move from opt-in to a hard dependency of transformers[torch], with more model families migrated onto the shared attention backend path and the eager-only route treated as a fallback. Day-0 architecture additions continue at the current pace on every minor.

◆Recent moves

  1. 8d ago

    Kernels go opt-in as T5 and linear attention move to shared backends

    ⚡ SPARK

    The clearest step yet in the consolidation onto a shared dispatch layer: kernel selection becomes opt-in for Mamba and other linear attention models, the T5 family joins ALL_ATTENTION_FUNCTIONS, and cache cropping changes to relative offsets. It also carries the usual day-0 additions — Meta's 30B Muse Glimmer, Granite SWA, A.X-K1/K2 and Cosmos3 Edge.

    View source ↗
  2. 1mo ago

    Patch fixes Inkling prefill and assisted-decoding cache bugs

    A cleanup pass on the previous release's Inkling integration: SDPA prefill with position_bias, assisted decoding for models using EncoderDecoderCache, an FP8 kernels bump and a multi-device deepgemm fix. Routine follow-up, and consistent with the pattern where day-0 model support needs a patch within days.

    View source ↗
  3. 1mo ago

    Inkling lands day-0; GPTNeoX and GPTBigCode realign for vLLM

    Day-0 support for Thinking Machines' Inkling (975B total, 41B active) plus TIPSv2, alongside a breaking GPTNeoX weight remap and a GPTBigCode attention-backend flag aimed at the vLLM modelling backend. Model additions are this project's normal cadence; the notable part is that the breaking compatibility work serves the serving runtime rather than the library's own callers.

    View source ↗
  4. 1mo ago

    Patch unblocks the latest vLLM release

    Three defensive fixes around the new linear layer type names so custom-code models keep loading, shipped with the stated purpose of enabling the current vLLM release. The second patch in this window that exists for downstream serving compatibility rather than for direct users.

    View source ↗
  5. 1mo ago

    Kimi K2.5-2.7 and MiMo-V2-Flash architectures added

    A model-addition release: the Kimi K2.5 architecture covering K2.5 through K2.7, plus Xiaomi's MoE MiMo-V2-Flash with its 256K context path. Incremental against the wider refactor, but it keeps day-0 coverage of agentic and long-context open-weight models intact.

    View source ↗
  6. 2mo ago

    Patch raises PEFT floor and fixes Mistral tokenizer resolution

    A PEFT lower-bound bump and an auto-tokenizer fix so the mistral-common backend resolves correctly. Housekeeping, cut alongside v5.10.3 to give vLLM a stable version to target.

    View source ↗