The next era of AI is sharing. Kimi K3 just made it expensive.

Industry Trends | July 24, 2026

Ahmed GuetariSenior Vice President, Product Management – Service Provider | F5

Kimi K3 may be another Sputnik moment for AI. Its most important lesson is not simply that open weights can approach the proprietary frontier. Building the model is only half the race; serving it efficiently is the other half. K3 changes both the infrastructure required to deliver frontier intelligence and the economics of who can do so profitably.

Kimi K3 is a 2.8-trillion-parameter Mixture-of-Experts model. Only 16 of its 896 experts are activated for each token, but efficiently distributing those experts becomes a first-order systems challenge at this scale. Moonshot AI recommends supernode configurations with at least 64 accelerators connected through a high-bandwidth communication domain.

The model may be open. Serving it reliably, quickly, and cost-effectively is where the real technical and commercial challenge begins.

For GPU providers, that gap between model capability and service delivery is where both risk and opportunity emerge. That pressure will only intensify as users, enterprises, and autonomous agents adopt frontier-scale open models. This shift changes the business model for GPU providers. Success depends not only on acquiring accelerators, but on combining them into services that deliver predictable latency, enforce tenant boundaries, preserve host capacity, and generate more customer value from every rack.

“NVIDIA’s scale-up architecture helps models such as Kimi K3 generate tokens efficiently. BF5 BIG-IP for Kubernetes deployed on NVIDIA BlueField DPUs help providers deliver those tokens securely, reliably, and profitably.”

The minimum unit gets bigger

Much of the GPU cloud market has been built around fragmented capacity: individual GPUs and small allocations of one, four, or eight accelerators. Those configurations remain useful for smaller models, development, fine-tuning, and many enterprise workloads. But they are not sufficient to serve K3 efficiently as a unified frontier-scale endpoint.

At four bits per parameter, the weights alone require roughly 1.4 terabytes of storage before accounting for runtime memory, KV cache, and other serving overhead. K3 therefore pushes providers toward larger guaranteed GPU allocations, high-bandwidth scale-up domains, rack-scale systems such as NVL72, topology-aware placement, disaggregated prefill and decode, and continuous KV-cache management.

This raises the minimum viable serving unit from an individual GPU to a tightly coordinated supernode. Capacity that was previously rented in small increments may need to be reorganized into larger, dedicated pools. The weights are shared, but the infrastructure becomes more concentrated and expensive.

Scale-up is only half the problem

NVIDIA has built an increasingly sophisticated architecture for this class of workload. Wide Expert Parallelism distributes experts across many GPUs, improving load distribution and leaving more GPU memory available for KV cache. Disaggregated serving separates prefill and decode so each phase can be scaled independently, while NVIDIA Dynamo coordinates request routing, GPU allocation, KV-cache placement, and high-speed cache transfer.

These technologies optimize model execution and east-west communication across the GPU scale-up fabric. But every inference request must still enter and leave the serving environment. It must be accepted from a user or agent, authenticated, secured, assigned to a tenant, routed to an available endpoint, observed, metered, and delivered against a service-level objective.

In rack-scale deployments, BlueField can provide a dedicated infrastructure domain for north-south traffic, including user requests, token responses, telemetry, storage access, and inter-cluster communication. As models such as Kimi K3 increase inference throughput and overall service traffic, the DPU becomes an increasingly important control point for scalable, secure, Kubernetes-based AI delivery.

The hidden host tax

Host processors still support networking, proxies, security, Kubernetes services, request handling, telemetry, and other functions surrounding the GPU workload. Every CPU core consumed by infrastructure is a core unavailable to model-serving software, orchestration, or services that keep expensive GPUs productive.

F5 BIG-IP for Kubernetes deployed on NVIDIA BlueField DPUs addresses this challenge by moving L4-L7 traffic processing, encryption, and security functions onto the DPU. In third-party testing, a conventional HAProxy deployment consumed approximately 12 host CPU cores, while BIG-IP for Kubernetes on BlueField used approximately two, returning roughly 10 cores per server to the AI serving environment.

BIG-IP for Kubernetes also provides accelerated traffic management, segmentation, tenant isolation, firewall, DDoS protection, and policy enforcement on BlueField, reducing the security and networking burden on host CPUs. Broader F5 services can extend this protection with WAF, API security, and AI security across the end-to-end environment.

From capacity to customer service

Wide Expert Parallelism and expert load balancing improve internal model-serving efficiency. BIG-IP for Kubernetes addresses a different but complementary challenge: turning that capacity into reliable output for customers.

In validated testing, BIG-IP for Kubernetes on NVIDIA BlueField delivered up to 47% higher output-token throughput, 60% lower time to first token, and 40% lower cost per token. These results were measured using specific models and configurations and should not be interpreted as Kimi K3 performance claims. Customer outcomes will vary by model, traffic profile, hardware, and serving architecture.

Even so, the findings demonstrate that north-south delivery architecture can materially improve output-token throughput, time to first token, and cost per token, helping operators produce more reliable, revenue-generating inference from the same GPU environment.

This distinction is particularly important for agents. Agentic systems create chains of model calls, tool invocations, and dependent decisions. Latency at the beginning of one inference step delays every subsequent step. A 20-second delay may frustrate a human; across dozens of sequential agent calls, it can make the service operationally unusable.

Frontier-scale multi-tenancy

K3’s rapid adoption highlights another challenge: model capability can grow faster than the infrastructure available to deliver it. Providers will need to share large serving systems across multiple customers to achieve acceptable economics, but sharing a frontier-scale supernode requires more than dividing GPU capacity.

Each tenant must be isolated. Traffic must be authenticated and governed. Service policies must be consistently enforced. Operators need visibility into consumption and whether one workload is degrading another.

BIG-IP for Kubernetes brings Kubernetes-native traffic management, network segmentation, tenant isolation, policy, and observability into the accelerated infrastructure path. This helps transform an expensive rack from a pool of unmanaged compute into a secure, differentiated AI service.

A service plane for inference

K3 demonstrates why AI factories need an AI service plane. The scale-up fabric optimizes GPU communication. Model-serving software executes inference. Schedulers place workloads and allocate GPUs. The AI service plane governs how applications, agents, and tenants consume the infrastructure.

BIG-IP for Kubernetes serves as a core traffic-control engine for that service plane, providing secure tenant access, inference traffic management, endpoint resilience, service-level enforcement, consumption visibility, policy, and reliable delivery.

Kimi K3 does not prove that every AI deployment requires a DPU. It does show that frontier inference is becoming a full-stack systems problem and creates exactly the conditions in which dedicated infrastructure processing becomes economically important.

NVIDIA’s scale-up architecture helps models such as Kimi K3 generate tokens efficiently. F5 BIG-IP for Kubernetes deployed on NVIDIA BlueField DPUs help providers deliver those tokens securely, reliably, and profitably.

The model may be open. Serving it is not cheap.

Whoever operationalizes the rack and turns its raw capacity into a dependable multi-tenant service captures the margin.