LoginSign Up
← Back to all sparks
K

KServe

AI-ASSISTANTS
Velocity5.0

Standardized model inference serving platform on Kubernetes

kserve.github.io/website ↗

KServe v0.21.0 ships as the GA release of a cycle that turned the platform into a production LLM inference layer.

llm-servingkubernetes-nativellmisvckv-cachemulti-api
◆Current state
KServe's last two major release cycles (v0.19.0, v0.20.0, now v0.21.0) delivered a comprehensive LLM serving rework: native support for OpenAI Completions, Responses API, and Anthropic Messages API; KV cache offloading for CPU tiering; traffic splitting for controlled LLM deployments; Managed DRA (Kubernetes Dynamic Resource Allocation) for GPU resource management; vLLM as a first-class runtime; LoRA adapter affinity scoring; confidential model serving; and autoscaling via KEDA and HPA. The LLMInferenceService (llmisvc) is now the platform's primary development surface, not the classic InferenceService.
◆Where it's heading
KServe is repositioning as the Kubernetes-native LLM inference platform for enterprise, not just a generic ML serving abstraction. The prefill/decode disaggregation work (llm-d integration), KV cache tiering, distributed tracing, and multi-API protocol support (OpenAI, Anthropic) all target production LLM workloads at scale. Confidential model serving and Managed DRA integration signal intent to serve regulated environments where GPU resource isolation and data protection are requirements. The llmisvc trajectory points toward multi-model routing behind a single endpoint and increasingly sophisticated scheduling.
◆Prediction
The v0.21.0 release cycle likely consolidates the llmisvc API into a stable surface. The next major release will probably ship autoscaling policies based on KV cache utilization rather than request count alone, and extend multi-model routing to cover model versioning and A/B deployments.

◆Recent moves

  1. 1mo ago

    KServe v0.21.0 released

    KServe v0.21.0 ships as the stable release, completing the release cycle that started with rc0 and rc1. The sparse commit content means the full feature scope is captured in the rc0 entries below, but the GA tag is meaningful for teams waiting on stable versions before production deployment.

    View source ↗
  2. 1mo ago

    KServe v0.21.0-rc1 release candidate

    v0.21.0-rc1 finalizes the release candidate with minor fixes before the GA tag — routine pre-release stabilization with no new features.

    View source ↗
  3. 1mo ago

    KServe v0.21.0-rc0 release candidate

    v0.21.0-rc0 opens the release candidate cycle for v0.21.0 — the entry captures the git commit, not the feature scope, which accumulated in the weeks before tagging.

    View source ↗
  4. 2mo ago

    KServe v0.20.0-rc1: TLS and KV transfer config fixes

    v0.20.0-rc1 tightens the disaggregated serving infrastructure: drops the old TLS flag for the disagg-sidecar, fixes KV transfer config quote escaping, and skips v1alpha2 InfPool reconciliation when the CRD is absent. Stability fixes preparing the v0.20.0 release.

    View source ↗
  5. 3mo ago

    KServe v0.20.0: Anthropic API, confidential serving, KV cache offloading, traffic splitting

    ⚡ SPARK

    v0.20.0-rc0 is a major capability expansion: Anthropic Messages API HTTPRoute support, confidential model serving, KV cache offloading with secondary filesystem tiering, traffic splitting API, Managed DRA for GPU resources, vLLM as a supported runtime, LoRA adapter auto-affinity scoring, and model-name-based routing. This release establishes KServe's LLMInferenceService as a serious production LLM serving platform, not just a Kubernetes abstraction.

    View source ↗
  6. 5mo ago

    KServe v0.19.0: OpenAI Responses API, dual-protocol routing, LocalModelCache for LLMISvc

    ⚡ SPARK

    v0.19.0-rc0 introduced dual-protocol routing (REST and gRPC) for Standard mode, native OpenAI Responses API support, LocalModelCache for LLMInferenceService, heterogeneous GPU load balancing, and latency predictor sidecar injection. This release moved KServe from OpenAI Completions API compatibility to full Responses API support before most Kubernetes-native serving platforms.

    View source ↗