llm-d enters CNCF governance with a production-focused 1.0 release

The llm-d distributed-inference project has reached two related milestones: version 1.0 and governance under the Cloud Native Computing Foundation. CNCF highlighted the project in a November 10 announcement about its expanding AI infrastructure portfolio, saying llm-d had joined the foundation and reached 1.0. The release itself was published on October 20. CNCF’s announcement and the project’s release record establish the dates and scope.

What reached 1.0

llm-d coordinates distributed large-language-model inference across accelerators and model servers. Its 1.0 release focuses less on a new serving engine than on the controls needed to operate shared inference fleets. The project lists four main themes: multi-tenant quality of service, zero-downtime operations, a stable configuration surface and agent-aware serving. The release notes describe priority- and fairness-aware flow control, controlled load shedding, router high availability with hot standby, and coordinated rollout management for disaggregated prefill and decode replicas.

The router configuration API is now at v1, while session-state and affinity work is intended to keep multi-turn conversations near cached context. The release also aligns most core component versions at 1.0 and adds API extensions plus preflight and topology-diagnostic utilities. llm-d is deprecating its own ROCm inference image in favor of upstream vLLM images, continuing an effort to move model-server image maintenance back to the projects that own those runtimes. Those component, image and deprecation changes are documented by the project.

Why CNCF governance matters

CNCF describes llm-d as part of a broader open stack for training, inference and autonomous agents, alongside projects including Kubeflow, KubeRay, OpenShell and Agent Substrate. Moving llm-d into that foundation gives vendors and users a neutral governance venue for work that sits between model servers, Kubernetes scheduling and traffic management. CNCF says the project advances an open framework for deploying and scaling distributed LLM inference.

The Red Hat connection is direct. Red Hat says the project was founded in 2025 by Red Hat, IBM, Google Cloud, CoreWeave and NVIDIA, and positions llm-d as a foundation for production-scale inference. Its release summary identifies intelligent routing, KV-cache offloading, prefill/decode disaggregation, wide expert parallelism, and batch and asynchronous inference across model servers including vLLM and SGLang. Red Hat’s October 20 release announcement also describes the 1.0 work on multi-tenancy, high availability and session-aware routing.

Platform teams already testing llm-d should treat 1.0 as a migration checkpoint rather than assume the major-version label makes every deployment production-ready. The project notes that not every tracked 1.0 goal was completed and publishes a separate release-testing matrix. Operators should review the image deprecations, component alignment and LeaderWorkerSet requirements before upgrading, then validate the paths for their accelerator and model server against that matrix. CNCF governance broadens the project’s stewardship; it does not replace workload-specific testing or a vendor support statement.