AI App Delivery Top 10: Unoptimized traffic steering

Lori Mac VittieDistinguished Engineer and Chief Evangelist | F5

Traffic steering determines where requests go, how they flow through the system, and which resources handle which workloads. In traditional application delivery, steering ensures efficient use of infrastructure and smooth distribution across servers, services, and regions. But AI introduces new traffic types, new dependency chains, and new compute pressures. Without modern steering strategies that account for model size, compute availability, agent behavior, and data locality, organizations face avoidable slowdowns, higher costs, and increased fragility across their delivery stack.

Traditional steering assumes predictable workloads and relatively uniform server capabilities. It expects that any backend instance can handle any valid request. That assumption collapses in AI systems, where resources differ dramatically in capability, cost, and availability. Without optimized steering, organizations end up with overloaded paths, underutilized capacity, and bottlenecks that degrade the entire system.

The AI shift

AI workloads are not interchangeable. A model running on one GPU type may not behave the same, and may not fit at all, on another. Latency varies based on batch size, token throughput, model variant, and prompt complexity. Agents chain multiple tools and services, causing traffic to traverse different layers of the stack depending on context and reasoning paths.

Steering becomes more complex because decisions must account for:

• model-specific hardware requirements
• variable inference cost
• regional data restrictions
• data gravity around embeddings and vector stores
• the unpredictable fan-out behavior of agents

In AI systems, the optimal backend is not simply “the least loaded one.” It is “the one that can correctly, legally, and efficiently perform this specific type of AI task.” Traditional steering cannot make these distinctions.

The impact on performance

Poor steering harms performance in multiple ways. Traffic may be routed to backends with insufficient GPU capacity, causing token generation slowdowns and increased latency. Requests may travel across regions or data layers unnecessarily because steering decisions ignore data locality, leading to longer response times.

When steering fails to consider workload type, agents generate traffic that lands on the wrong model tier. Think oversized models for simple tasks, undersized ones for complex tasks. This results in inefficiency and degraded throughput. As more AI services scale horizontally, inconsistent steering becomes a direct source of latency variance and processing bottlenecks.

The impact on availability

AI workloads push systems to their limits. Unoptimized steering can cause sudden overloads on specific GPU nodes or inference clusters, triggering throttling, cold starts, or outright failures. Because agents often retry or escalate when they encounter slow endpoints, poor steering indirectly multiplies traffic, worsening availability issues.

Steering that lacks awareness of regional capacity or data residency can unintentionally route requests through constrained zones, creating localized outages. If steering doesn’t incorporate health checks tuned to AI metrics such as model health, GPU saturation, or token-per-second degradation, traffic will continue to flow into unhealthy backends long after they should have been removed from rotation.

The impact on reliability

AI workloads rely on stable, predictable routing for consistent output. When steering is inconsistent or oblivious to the nature of the workload, the system behaves unpredictably. Two identical requests may be routed to different model variants with different performance characteristics, leading to erratic latency and response drift.

For agentic workflows, unreliable steering can break chains of dependent actions. If one segment routes to a slow or overloaded backend, retries and replanning ripple through the pipeline, compounding downstream risk. Unoptimized steering also complicates debugging because when traffic lands in inconsistent places, failures become harder to reproduce and diagnose.

Best practices for mitigating unoptimized traffic steering

Effective mitigations for AI-era traffic steering depend on two things: segmenting traffic so that workloads don’t collide and using programmable steering logic that can adapt in real time to unpredictable shifts in model load, agent behavior, and data locality.

Start with traffic segmentation. Separate interactive requests, batch jobs, agent-generated flows, and heavyweight inference tasks into distinct routing lanes. Different workloads demand different performance characteristics. For example, latency-sensitive tasks need fast, lightly loaded backends; bulk or exploratory agent workflows can tolerate lower priority or delayed processing. Segmentation isolates noisy or bursty AI flows so they cannot overwhelm user-facing paths or shared services.

Next, introduce programmable steering rules. Static steering strategies collapse under AI load because they assume uniform backends and predictable request shapes. Programmability allows the delivery layer to route traffic based on real-time conditions: GPU saturation, token throughput, queue depth, model type, and even prompt size. When model health drops, when a region’s embeddings store becomes overloaded, or when a particular class of request needs a specific hardware profile, programmable logic adjusts routing immediately. This keeps AI pipelines responsive and prevents cascading slowdowns.

Finally, align traffic steering with data locality. Route inference and retrieval tasks to the regions or compute pools closest to their embeddings or structured data. This reduces latency and avoids unnecessary cross-region traffic, especially in agentic workflows that chain multiple retrieval and inference steps.

These mitigations work because they combine bounded behavior (via segmentation and rate limits) with adaptive intelligence (via programmability). AI workloads refuse to behave predictably; programmable traffic steering ensures the environment doesn’t depend on predictable behavior to stay stable.

Conclusion

AI has transformed traffic from something mostly predictable into something dynamic, variable, and often self-amplifying. Traditional steering approaches built around static policies and uniform backends cannot keep pace with workloads that shift shape mid-execution, span multiple resource types, and respond to real-time model behavior. Unoptimized traffic steering doesn’t just slow systems down, it quietly erodes performance, availability, and reliability across the entire stack.

With the right steering strategy, organizations can maintain stability even as AI workloads grow more complex and more central to application behavior. Optimized traffic steering becomes the control point that preserves performance under pressure, maintains availability during spikes, and ensures reliable behavior in a world where AI no longer follows the rules of traditional traffic.

Read more about the Top 10 Application Delivery challenges faced by organizations across the globe.

About the Author

Lori Mac VittieDistinguished Engineer and Chief Evangelist | F5

Lori MacVittie is a Distinguished Engineer and Chief Evangelist in F5’s Office of the CTO with deep expertise in application delivery, automation strategy, and infrastructure. She is known for turning complexity into clarity whether she’s defining guardrails for AI agents, dissecting brittle multicloud architectures, or probing the limits of scalable systems. She brings more than thirty years of industry experience across application development, IT architecture, and network and systems operations. Before joining F5, she served as an award-winning technology editor. MacVittie holds an M.S. in Computer Science and is a prolific author whose publications span security, cloud, and enterprise architecture. She is also an avid tabletop and video gamer with unapologetically strong opinions about cheese.

More blogs by Lori Mac Vittie