LoginSign Up
← Back to all sparks
O

ONNX Runtime

AI-ASSISTANTS
Velocity7.5

Cross-platform inference and training engine for ONNX-format machine-learning models.

onnxruntime.ai ↗

ONNX Runtime is dismantling itself into plug-ins — CUDA is now the one that ships separately.

execution-providersplugin-architecturecudawebgpudeprecationssecurity-hardening
◆Current state
ONNX Runtime is running two release tracks at once: the numbered core releases (1.25 through 1.29) and a growing set of separately versioned plug-in execution providers. WebGPU broke out first in May, and CUDA has now followed with its own 0.1.0. The core releases in between are dominated by security hardening, opset upgrades and deprecation notices rather than new capability.
◆Where it's heading
The direction is a smaller core binary with accelerators attached at runtime. The 1.26 notes stated the intent outright — CUDA moving to a dedicated execution provider rather than a package shipped from core — and 0.1.0 delivers it, with version-gated callbacks maintaining compatibility back to 1.24.4. Alongside that, the deprecation list keeps growing: CUDA 11, then CUDA 12, WebGL and JSEP, ArmNN, the duktape WGSL generator. Web inference is being consolidated onto WebGPU and native inference onto plug-ins.
◆Prediction
Expect the plug-in EPs to take over release cadence from the core, with CUDA 12 removed in 1.27 as announced and further backends following WebGPU and CUDA out of the main binary.

◆Recent moves

  1. 1d ago

    CUDA becomes a standalone plug-in execution provider

    ⚡ SPARK

    The CUDA execution provider ships as a standalone versioned plug-in and becomes the default CUDA implementation, completing the split the 1.26 notes announced. It arrives with arena allocation, CUDA Graph capture and replay, user compute streams, external allocators and IOBinding synchronization, with version-gated callbacks preserving compatibility back to ONNX Runtime 1.24.4.

    View source ↗
  2. 7d ago

    ONNX Runtime 1.29 deprecates WebGL and JSEP, adds POSIX telemetry

    ⚡ SPARK

    The 1.29 release sets the terms for browser inference by naming WebGPU the only supported path and retiring WebGL and JSEP, while extending telemetry to non-Windows platforms. It is the deprecation half of the same consolidation the plug-in split is driving.

    View source ↗
  3. 7d ago

    ONNX Runtime 1.26 adds RISC-V vector support and .ort memory mapping

    Adds RISC-V vector support to the CPU provider and optional memory mapping for .ort models, and states the plan to move CUDA out of core — the announcement the plug-in release has now delivered on.

    View source ↗
  4. 20d ago

    WebGPU plug-in: FlashAttention fusions, Qwen3 and Gemma 4 paths

    The WebGPU plug-in's first substantive update after breaking out of core: FlashAttention fusions, Qwen3 and Gemma 4 model paths, and quantized-path work. Evidence the separated providers ship faster than the core.

    View source ↗
  5. 25d ago

    ONNX 1.22 upgrade, slimmer CUDA footprint, experimental C API

    Upgrades to ONNX 1.22, makes cuDNN and cuFFT optional at runtime to shrink the CUDA redistributable footprint, and introduces an experimental C API surface. The slimming of CUDA dependencies here foreshadows the full plug-in split.

    View source ↗
  6. 1mo ago

    Patch release: QMoE batch-1 decode fast path and fixes

    A patch release of targeted bug fixes plus a QMoE batch-1 decode fast path and CI repairs. Maintenance between the structurally interesting releases.

    View source ↗