What Cloud-Native Still Needs to Get Right?

KubeCon + CloudNativeCon Europe 2026 in Amsterdam opened with a split personality, and no one on stage seemed to notice. In the same Day 1 keynote block, CNCF Executive Director Jonathan Bryce made a careful infrastructure-first argument: Kubernetes as the operational layer for production AI, and Solo.io used its keynote slot to argue that the future of open-source AI is fundamentally agentic. These are not the same thesis. They are not even the same conversation. That tension is the most important thing to understand about where cloud-native is headed.

Inference infrastructure at scale is an ops problem

How do you efficiently serve billions of tokens, route traffic intelligently, and manage GPU fleets economically?

Agentic AI is an application architecture problem

How do you build systems that reason, plan, and take action across tools and data?

Collapsing both into a single keynote narrative creates a split message that serves vendor positioning more than it helps practitioners decide what to do next.

The analogy: imagine if at a Windows NT launch in the mid-90s, half the keynotes were about kernel scheduling and the other half were about how the future of software is internet applications. Both were true. But they required different conversations, different audiences, and different timelines for maturity. KubeCon EU 2026 was that conference.

Understanding why requires being precise about which AI-native developers we are actually talking about, and what “operating system” actually means for them, before accepting or rejecting the AI OS claim.

The inference pivot is real, but is Misunderstood

Bryce’s infrastructure-first message was coherent and grounded. He described the moment as both a challenge and a gold rush for cloud-native inference, noting that 82% of organizations have adopted Kubernetes for AI workloads while only 7% deploy AI daily. That chasm between adoption and production is the elephant in the room that the ecosystem needs to address.

The infrastructure response was significant. The most consequential announcement was llm-d(a Kubernetes-native distributed LLM inference framework)joining the CNCF as a Sandbox project. Launched in May 2025 as a collaborative effort between Red Hat, Google Cloud, IBM Research, CoreWeave, and NVIDIA, llm-d addresses the inference problem by separating the prefill and decode phases into independently scalable pods and introducing hierarchical KV cache offloading across GPU, TPU, CPU, and storage tiers.

The compute trajectory that framed the keynote is stark: AI inference is projected to jump from 20.9 gigawatts in 2025 to 93.3 gigawatts by 2030, surpassing training to become the dominant workload in AI data centers. Against that backdrop, CNCF simultaneously tightened its Kubernetes AI Conformance Program, adding inference-specific requirements. The number of certified platforms nearly doubled following a 70% surge in certifications, and the program expanded to include agentic workload support and mandatory alignment with Kubernetes v1.35.

Projected Growth in AI Inferencing and Power Demand for AI Inferencing. Souce. KubeCon EU 2026 Keynote.

This is the right infrastructure instinct. For hyperscalers and foundation model builders running massive compute fleets, Kubernetes-native inference management is not optional: it is existential economics. For hyperscalers and foundation model builders, this is existential. Inference is now an infrastructure economics problem. And Kubernetes is positioning itself exactly where it should: at the center of that problem.

The AI Agentification Index (AI²) work we have been building at CloudDon points to the same dynamic: compute-intensive infrastructure roles face the steepest displacement curves precisely because the economics of automation are most compelling at scale. Inference is no different.

The developer audience problem

This is the question KubeCon EU raised but did not answer: for whom is Kubernetes the AI OS?

For cloud-native infrastructure teams, hyperscalers, and foundational model builders — the answer is clearly yes. These developers and operators need everything the ecosystem announced/ is working on. The tooling is maturing quickly, and the investment from the broader ecosystem is in the right direction.

But consider a different developer profile — one that the AI² Index work has made visible. The vast majority of AI-native builders today are not managing GPU clusters. They are calling inference APIs. They are building agents, workflows, and applications on top of hosted models over HTTPS. Their entire infrastructure stack is somebody else’s problem. Kubernetes, to them, is as invisible as the Linux kernel is to a developer shipping a Next.js application on the cloud.

This is not a niche edge case. This is likely the majority use case.

CNCF has to decide whether this API-calling developer is a constituency it wants, or one it consciously cedes to Vercel, Cloudflare, the hyperscaler application platforms, or the next wave of AI application developer platforms. There is no shame in the latter —Linux never tried to be the Android SDK either.

But the “AI OS” claim implies universal relevance, and that claim does not survive contact with this developer profile. The honest version of the thesis is narrower:

Kubernetes is the AI OS for teams that own their inference stack. That is a meaningful and defensible claim. It is also a smaller tent than the keynote implied.

Kubernetes as AI OS — An Assessment

Jonathan Bryce’s keynote in Amsterdam leaned into the “Kubernetes as AI OS” framing — a thesis that Janakiram first posited at KubeCon India 2025, where he argued that Kubernetes is fast becoming the foundational substrate for the GenAI revolution. The Cloud Native Generative AI Stack pyramid from Janakiram’s keynote, with Accelerated Compute at the base, then Models, then MCP, then Agents at the apex, makes a structurally similar argument: Kubernetes handles the bottom half, and the cloud-native ecosystem is climbing toward the top. It is a compelling frame. It is also, on close inspection, the gap map rather than the achievement map.

What Does ‘OS’ Actually Mean?

A modern operating system has three distinct layers sitting above the hardware substrate:

  • The kernel: process scheduling, memory management, and hardware abstraction.
  • System services: the stable interfaces between kernel and applications, including file I/O, networking, IPC, and identity.
  • The application development layer: the SDK and runtime that developers actually write against. Win32 API. Android’s Java SDK.

Anatomy of a Modern OS. Created using Claude.ai.

That last distinction matters most. It is the application development layer, not the kernel, that made Windows an OS for developers rather than just an NT kernel running services. Millions of developers built on Win32 without ever touching kernel32.dll. The Android developer has no idea what Linux kernel version is running underneath. That invisibility is the difference.

From an AI-native developer’s perspective:

The SDK is the OS. Everything below it is infrastructure.

Eight Capabilities an AI OS Needs to Own

Mapping those layers onto AI and agentic workloads yields eight specific capabilities:

Kernel layer

  1. GPU and TPU scheduling
  2. Workload orchestration

System services layer

  1. Inference routing (traffic management aware of model state and KV cache, not just HTTP paths)

  2. Model lifecycle management (versioning, storage, and serving of multi-gigabyte-weight artifacts)

  3. Agent runtime (a process model for agents)

  4. Agent identity

Application development layer

  1. Session and context management (durable state across invocations, the equivalent of what a filesystem gives a traditional application)

  2. Behavioral evaluation (the ability to assess whether an agent is doing what it is supposed to do)

AI OS Capabilities Mapped to OS Layers. Created using Claude.ai. © CloudDon.ai.

Together, these eight capabilities define what it would mean for Kubernetes to be an AI OS in the same sense that Windows is an OS for application developers.

An Honest Assessment

An honest assessment of the Kubernetes ecosystem, including related CNCF/ LF projects, is:

  • At the hardware and kernel layers, Kubernetes is genuinely strong. GPU and TPU resource management through DRA, pod scheduling, namespace isolation, and the emerging inference-specific primitives are real kernel-layer work. llm-d addresses the resource-utilization asymmetry between prompt processing and token generation by disaggregating these phases into independently scalable pods with hierarchical KV cache offloading across GPU, CPU, and storage tiers. This layer is present and maturing fast.
  • At the system services layer, it is partially built and actively being built. Inference routing via the Gateway API is real progress, but model-aware routing requires the external kubernetes-sigs Inference Extension, not a native primitive. Model lifecycle management uses OCI registries and PVCs — container artifact tooling retrofitted to handle multi-gigabyte weight files with versioning semantics that those tools were never designed for. Kagent’s own creators acknowledge that Kubernetes “lacks the contextual awareness required for GenAI and agentic applications to secure, scale, and govern agents, tools, and LLMs,” an honest admission from the project trying to fill the gap.
  • At the application development layer, this is where the honest answer gets uncomfortable. There is no native Kubernetes AI SDK. There is no developer-facing abstraction that AI-native developers naturally reach for. The closest things live entirely outside the CNCF. MCP and AGENTS.md are housed in the Agentic AI Foundation under the Linux Foundation, co-founded by Anthropic, Block, and OpenAI. Google’s A2A protocol also lives in the Agentic AI Foundation. The protocols that will define how AI agents are built are being standardized outside of CNCF.

Kubernetes + CNCF/ LF Ecosystem gap map. Created using Claude.ai. © CloudDon.ai.

Of the eight AI OS capabilities, Kubernetes covers two fully, four partially, and misses two entirely. The two absent capabilities are the most significant from the perspective of an AI-native developer. An OS without a process model for agents and without a way to evaluate whether its workloads are behaving correctly is missing foundational responsibilities that no amount of GPU scheduling can close.

The sharper version of this critique:

What Kubernetes currently is for AI is closer to what BIOS and a bootloader are for a PC than what Windows is.

It boots the hardware, allocates resources, and gets out of the way. That is not trivial. No serious AI infrastructure runs at scale without it. But it is not what people mean when they say “operating system.”

The OS claim is currently two layers premature.

Recommendations to the CNCF

If CNCF wants to own the “AI OS” framing credibly, there are three gaps it needs to commit to closing, and be explicit about which ones it owns versus which ones it partners on.

1. Formalize the CNCF-to-AAIF integration layer.

Though MCP and A2A are not CNCF projects, CNCF needs a published specification for how Kubernetes-native agent runtimes integrate with AAIF-governed protocols. Right now that integration is happening project-by-project, and the ecosystem has been here before. See what happened with the Ingress API and Ingress NGINX — vendor patches, technical debt, multi-year efforts to cleanup, etc. Do not repeat that arc for the agent protocol layer. The cost this time will be incompatible agent runtimes, proprietary identity models, and security patterns that harden before the community agrees on anything.

2. Standardize agent session and context management as a Kubernetes primitive.

No AI agent can be production-ready without durable session state, context windows that survive pod restarts, and memory that persists across invocations. Alibaba’s Sandbox Operator, presented at KubeCon EU 2026 by Mingshan Zhao and Zhen Zhang, showed a credible approach: checkpoint-and-snapshot for MCP tool containers, allowing sessions to be paused when idle and resumed without losing context. Such work needs to become a CNCF working group initiative, not remain a single vendor’s contribution.

3. Define Kubernetes-native AI agent identity.

Kubernetes provides service accounts and RBAC for process-level identity. SPIFFE/SPIRE adds cryptographic workload identity above that. Neither solves the delegation problem: an agent acting on behalf of a user, calling tools, modifying state across sessions, is a new identity principal that no Kubernetes or CNCF standard addresses today. Left unresolved, this gap will be filled by proprietary platform vendors — and then standardized around their choices rather than the community’s. This session presented by IBM Research at KubeCon EU 2026 provided a promising way to cryptographically bind agent identity along with delegated user identity.

On Agentics Day — and why MCP is the convergence bridge

I served on the review committee for the inaugural Agentics Day, a co-located event at KubeCon EU focused on AI Agents. The event was intentionally narrow and deep, focusing specifically on the practical deployment of agents and the emerging infrastructure that supports them. It was meant for platform, SRE, and infrastructure teams who are expected to operate agentic workloads at scale.

What the session mix revealed is that practitioner maturity is real but uneven. Teams are deploying agents. But they are not yet operating them with the same discipline they bring to stateless microservices — because the tooling for observability, identity, and session management is not yet at the same level of maturity.

The gap between “we have agents running” and “we can operate agents reliably” is where the most important infrastructure work of the next two years will happen.

Adobe’s AIOps session was notable precisely because it showed what that gap looks like in a real enterprise: the agents worked; the operational governance around them was still being invented in real time.

Takeaways from Adobe AIOps Session, Agentics Day: MCP + Agents, 2026, Amsterdam.

As protocols increasingly become long-lived infrastructure, the cost of fragmentation grows. The Agentics Day reinforced the cloud-native approach to scaling ecosystems through open standards and neutral governance.

MCP is where the convergence of cloud-native and AI-native becomes concrete. If MCP becomes the protocol by which AI agents interact with tools, data sources, and services, then the question of whether Kubernetes is relevant to AI-native developers becomes a question of where MCP servers run at scale.

Google’s open-source GKE MCP Server, kagent, and agentregistry are all strong signals that the ecosystem is building the plumbing for an agent-native runtime on top of Kubernetes primitives.

The Agentics Day reinforced the demand. The governance standards are the missing layer*.*

For enterprises: what to do now

Enterprises watching this from the sidelines need a practical frame, not a wait-and-see posture.

1. Separate your inference strategy from your agent strategy.

Running models at scale on your own infrastructure? Act on llm-d and the AI Conformance work now. Calling hosted APIs? That infrastructure is your providers’ problem for the time being. The AI² Index makes the dividing line concrete: regulated industries with high-volume AI workloads need to own their inference stack today. Everyone else can wait — but not indefinitely.

2. Get ahead of MCP governance now.

The security and governance patterns for MCP-based agents are being written right now, while the ecosystem is still malleable. Audit which MCP servers your teams are deploying and who governs them. This has two distinct layers. For runtime governance (which agents are approved, catalogued, and deployable) Solo.io’s agentregistry (now contributed to CNCF) and kagent are the projects to follow. For supply chain security (whether the MCP servers themselves are safe before they ever run) JFrog’s MCP Registry, which launched GA at KubeCon EU 2026, applies the same artifact governance model that enterprises already use for software packages. Both layers matter. Most teams are addressing neither.

3. Do not let application teams solve the agent context problem.

Session state, memory across invocations, and context persistence across pod restarts are platform responsibilities — the same way persistent storage is not re-invented per application. If your platform team is not already evaluating how to provide durable agent context as a service, application teams will build bespoke solutions that create exactly the fragmentation the ecosystem is trying to avoid. Get your platform team engaged in how the CNCF community standardizes this. The enterprises that contribute real-world requirements to that process will get infrastructure that fits their needs. The ones that wait will get whatever a dominant vendor decides.

The convergence is not complete

The cloud-native era was won by infrastructure.

The AI-native era will be won by developer experience and protocols.

If Kubernetes does not move up the stack fast enough:

It will remain indispensable, but increasingly invisible.**

KubeCon EU made it clear that the Kubernetes ecosystem understands the stakes. The inference pivot is genuine.

But the convergence of cloud-native and AI-native cannot be declared at a keynote — it has to be earned. The measure of success is simple: can an AI developer with no Kubernetes knowledge ship a production agent system without ever thinking about what runs underneath? That standard does not exist today. The gap map is clear —two capabilities present, two absent, and four partial. But KubeCon EU showed the community knows where it needs to go.

The real question is timing.

If the CNCF closes these gaps before the next wave of AI application frameworks makes the underlying infrastructure permanently invisible, the ecosystem wins. If it does not, the AI OS will not be Kubernetes. It will be whatever developers are already building on top of.

Disclaimer: CNCF provided me with a conference pass, and for travel & accommodation. Thank you, CNCF!