AI App Delivery Top 10: Poor resource utilization

Lori Mac VittieDistinguished Engineer and Chief Evangelist | F5

The original ADC10 focuses on a problem every ops team recognizes: inefficient use of compute, storage, and network resources drives up cost and constrains scalability. Over-provision and you waste money. Under-provision and performance suffers. The fix, as outlined, is better traffic distribution, intelligent health checks, and programmable control so resources are used efficiently across the system. This remains true.

What changes with AI is that resource utilization stops being predictable.

Traditional applications consume resources in fairly consistent ways. AI systems do not. They are bursty, compute-heavy, and highly variable depending on the request. That turns what used to be a capacity planning problem into a real-time allocation problem.

Consequences of poor resource utilization

Impact on performance

In a conventional system, inefficient resource utilization shows up as uneven load distribution. Some servers are idle while others are overloaded. The result is slower response times and wasted capacity. AI introduces a different kind of imbalance.

Inference workloads are not uniform. One request might be trivial. Another might consume orders of magnitude more compute due to prompt size, context length, or reasoning depth. Even within the same model, performance can vary significantly from one request to the next.

That variability makes it much harder to predict and distribute load effectively.

If requests are routed without awareness of their computational weight, you end up with hotspots. GPUs or inference nodes saturate unexpectedly. Queues build. Latency spikes. Meanwhile, other resources sit underutilized because they weren’t selected for that workload.

So the familiar performance problem remains, uneven distribution, but the cause shifts. It is no longer just about how many requests are being sent. It is about how expensive each request is to process.

Impact on availability

The original ADC10 ties poor resource utilization to broader delivery concerns, including the ability to sustain workloads efficiently. With AI, inefficient utilization can directly impact availability in ways that feel disproportionate.

AI workloads are constrained by scarce resources, particularly specialized compute like GPUs. When those resources are exhausted, the system doesn’t degrade gracefully. It stalls. Requests queue or fail. Timeouts increase.

What makes this more challenging is that availability becomes tied to allocation decisions, not just capacity.

You may have sufficient total capacity across the system, but if it is not accessible in the right place, at the right time, or for the right type of request, the system behaves as if it is out of resources. A model endpoint may be technically online but unable to accept additional work. A region may have idle capacity that cannot be used due to latency or compliance constraints. A cheaper model may be available but not selected because routing logic is too rigid.

This creates a scenario where availability is fragmented. The system is partially available, depending on the path taken through it.

That is a harder problem than simple uptime. It requires the ability to shift workloads intelligently across models, regions, and tiers in real time.

Impact on scalability

Inefficient resource use forces organizations to scale out prematurely, adding more infrastructure to handle workloads that could have been managed more efficiently. In the past it’s lead to overprovisioning, which in cloud environments can cause costs to skyrocket. AI accelerates this pattern dramatically.

Inference is expensive. Storage for embeddings and context grows quickly. Data pipelines expand. If resources are not used efficiently, the default response is to add more capacity. That works, for a while.

But it quickly becomes unsustainable, both financially and operationally. Scaling AI systems is not just about adding more servers. It is about managing a complex mix of compute types, each with different performance and cost characteristics.

If utilization is poor, scaling amplifies inefficiency instead of solving it.

The alternative is to make smarter decisions about how resources are used. That includes selecting the right model for the task, optimizing when and how context is retrieved, and ensuring workloads are distributed based on real-time conditions rather than static rules.

In other words, scalability depends less on how much infrastructure you have and more on how intelligently you use it.

Impact on operational efficiency

Poor resource utilization has always increased operational costs and complexity. Over-provisioning wastes money. Under-provisioning creates contention and forces reactive scaling. AI compounds both sides of that problem.

Because workloads are unpredictable, teams often overcompensate. They provision more capacity than they think they need to avoid performance issues. That leads to significant idle resources, especially when demand fluctuates.

At the same time, unexpected spikes still occur. A surge in complex inference requests can overwhelm available resources despite that over-provisioning. Now teams are dealing with both wasted capacity and performance degradation.

Operationally, this creates a constant balancing act.

Teams spend time tuning thresholds, adjusting routing rules, and trying to understand why certain workloads consume disproportionate resources. Observability becomes more important, but also more difficult, because the unit of work is no longer uniform.

Efficiency, then, depends on visibility and control. Without both, resource management becomes reactive and expensive.

Best practices for optimizing resource utilization

Programmable infrastructure and dynamic allocation

For traditional applications, programmable infrastructure and dynamic traffic steering are key solutions. By routing traffic based on real-time metrics, organizations can balance load and improve utilization across the system. This approach becomes critical with AI.

Static load balancing assumes requests are roughly equal. AI breaks that assumption. Requests are no longer predictive of resource consumption. You need to route based on more than connection count or simple algorithmic distribution. You need awareness of request cost, model performance, and system state.

Programmability allows you to adapt in real time. Requests can be directed to different models based on latency, cost, or availability. Workloads can be shifted across regions or clusters as conditions change. Expensive operations can be throttled or deferred when resources are constrained.

This turns resource utilization into an active control loop rather than a passive outcome.

Efficiency through intelligent routing

At its core, ADC10 is about using resources efficiently. AI expands what “efficient” means.

It is no longer just about distributing traffic evenly. It is about matching the right workload to the right resource.

That may mean using a smaller model for simpler requests instead of defaulting to the most powerful one. It may mean limiting context size to reduce compute cost. It may mean caching or reusing results instead of recomputing them. It may mean recognizing when not to use AI at all.

All of those decisions sit above traditional load balancing. They require understanding intent, not just traffic.

This is where application delivery evolves again. Routing becomes resource orchestration. The goal is not simply to keep systems busy. It is to ensure that every unit of compute is used effectively.

Conclusion

ADC10 has always been about avoiding waste and ensuring that infrastructure is used efficiently to support performance, availability, and scalability.

What AI introduces is volatility.

Resource consumption is no longer predictable. Workloads vary widely in cost. Specialized compute introduces new constraints. And inefficiencies scale faster because the cost of each request is higher.

The response follows the same pattern as the original guidance, but with greater urgency.

You need visibility into how resources are being consumed. You need programmability to adapt in real time. And you need smarter routing decisions that account for the true cost of each request, not just the number of them.

Because in an AI-driven system, poor resource utilization is not just wasteful.

It is the fastest path to degraded performance, constrained availability, and runaway cost.

Read more about the Top 10 Application Delivery challenges faced by organizations across the globe.

About the Author

Lori Mac VittieDistinguished Engineer and Chief Evangelist | F5

Lori MacVittie is a Distinguished Engineer and Chief Evangelist in F5’s Office of the CTO with deep expertise in application delivery, automation strategy, and infrastructure. She is known for turning complexity into clarity whether she’s defining guardrails for AI agents, dissecting brittle multicloud architectures, or probing the limits of scalable systems. She brings more than thirty years of industry experience across application development, IT architecture, and network and systems operations. Before joining F5, she served as an award-winning technology editor. MacVittie holds an M.S. in Computer Science and is a prolific author whose publications span security, cloud, and enterprise architecture. She is also an avid tabletop and video gamer with unapologetically strong opinions about cheese.

More blogs by Lori Mac Vittie