Best practices for optimizing AI infrastructure at scale

Mark MengerSolutions Architect

This blog post is the fifth in a series about AI data delivery.

Most organizations don’t struggle with AI because they lack models; models are proliferating in both number and type. They struggle because their infrastructure was never designed for the way AI moves data.

Training, fine-tuning, and retrieval-augmented generation (RAG) all depend on moving massive volumes of data—reliably, securely, and repeatedly—across storage systems, networks, and compute tiers. As AI initiatives scale, even small inefficiencies in data access, security policy, or resiliency can ripple outward—slowing pipelines, breaking jobs, or driving up infrastructure costs.

What makes this especially challenging is that AI workloads stress infrastructure in new and unfamiliar ways. Data transfers run longer than traditional applications expect. Storage systems scale and rebalance more frequently. Security and regulatory requirements span on-premises, cloud, and hybrid environments. And failures that might be tolerable in a web app, such as a brief disconnect or retry, can derail hours or days of model work.

As a result, many organizations find themselves reacting to symptoms rather than solving root causes. These root causes often involve training jobs that stall during maintenance windows, fine-tuning pipelines that behave differently in each environment, RAG systems that slow down under load or during re-indexing, and security controls that are inconsistent or difficult to manage at scale.

Optimizing AI infrastructure isn’t about chasing peak performance benchmarks. It’s about designing for stability, resiliency, security, and operational clarity as everything scales at once—data, models, environments, and teams.

Eight do’s and don’ts for building scalable, resilient AI infrastructure

Whether you’re training large models, fine-tuning them for specific tasks, or powering RAG systems that retrieve and reason over enterprise data, check out the following do’s and don’ts. They highlight the infrastructure choices that most directly affect scalability and resilience.

1. Do standardize how AI workloads access storage. Don’t let every job connect directly to individual storage nodes.

• Training: Repeated reads of large datasets demand a stable, predictable storage endpoint.
• Fine-tuning: Frequent, smaller jobs benefit from consistent access without reconfiguration.
• RAG: Embedding generation and refresh pipelines rely on dependable object access.

A single, consistent access layer prevents retries, brittle scripts, and pipeline failures as storage scales or rebalances.

How F5 BIG-IP can help: F5 BIG-IP can provide a stable front door that shields AI jobs from backend change.

2. Do design for long-running, high-volume data transfers. Don’t rely on short-lived, web-style traffic assumptions.

• Training: Multi-hour or multi-day jobs must survive transient slowdowns.
• Fine-tuning: Sustained throughput matters more than burst speed.
• RAG: Large document ingests and re-indexing require reliable multipart transfers.

AI data movement behaves differently than application traffic and must be treated accordingly.

How F5 BIG-IP can help: It enables tuning for durable, high-throughput connections that align with AI data flows.

3. Do enforce consistent encryption and access policies everywhere. Don’t allow security rules to drift across environments.

• Training: Sensitive datasets remain protected regardless of where compute runs.
• Fine-tuning: Dev, test, and production behave the same way from a policy standpoint.
• RAG: Proprietary documents and embeddings stay encrypted as they move between tiers.

Centralized enforcement reduces risk and operational overhead.

How F5 BIG-IP can help: It provides a single control point for encryption and access policy across environments.

4. Do offload repetitive infrastructure work when possible. Don’t burn CPU cycles on tasks that don’t advance AI outcomes.

• Training: Host resources are freed to support preprocessing and model execution.
• Fine-tuning: Turnaround time improves when many jobs run in parallel.
• RAG: Ingestion and retrieval remain responsive under encrypted traffic.

Infrastructure work is necessary, but it shouldn’t compete with AI workloads.

How F5 BIG-IP can help: It supports architectures where traffic handling and encryption are managed efficiently outside the AI hosts.

5. Do isolate stable client endpoints from changing backends. Don’t expose infrastructure churn to AI pipelines.

• Training: Jobs continue uninterrupted as storage nodes scale or are replaced.
• Fine-tuning: Iteration isn’t disrupted by backend maintenance.
• RAG: Retrieval endpoints remain stable during failover events.

This separation is critical to resilience.

How F5 BIG-IP can help: It routes around unhealthy or changing backends without breaking client connections.

6. Do treat resiliency as an operational baseline. Don’t wait for outages to discover weaknesses.

• Training: Long-running jobs survive planned maintenance.
• Fine-tuning: Updates can roll out without pausing experimentation.
• RAG: Always-on availability supports real-time queries.

Resilience should be routine, not exceptional.

How F5 BIG-IP can help: It enables graceful maintenance and controlled failover for data paths.

7. Do prioritize critical AI data flows under load. Don’t let background traffic starve active workloads.

• Training: Active runs stay performant during replication or sync jobs.
• Fine-tuning: Iterative pipelines complete on schedule at peak times.
• RAG: Query and retrieval traffic remains responsive.

Not all traffic matters equally.

How F5 BIG-IP can help: It acts as a policy point to shape and prioritize traffic by intent.

8. Do optimize for clarity and operability, not just performance. Don’t build an AI stack only specialists can operate.

• Training: Troubleshooting is faster when jobs slow or fail.
• Fine-tuning: Iteration quickens because teams understand the data path.
• RAG: Separation of concerns is cleaner as systems scale.

The most scalable systems are the ones teams can understand, operate, and evolve.

How F5 BIG-IP can help: It provides a clear, consistent control layer for AI data movement.

When scaling AI infrastructure works best

AI infrastructure scales best when data access is stable, security is consistent, and infrastructure change is absorbed by the delivery layer—not exposed to models and pipelines.

For more information about our AI data delivery solutions, visit our AI Data Delivery and Infrastructure Solutionswebpage. Also, stay tuned for the next blog post in our AI data delivery series focusing on delivering AI applications at scale and the role of application delivery controllers.

F5’s focus on AI doesn’t stop with data delivery. Explore how F5 secures and delivers AI apps everywhere.

Be sure to check out our previous blog posts in the series:

Fueling the AI data pipeline with F5 and S3-compatible storage

Optimizing AI by breaking bottlenecks in modern workloads

Tracking AI data pipelines from ingestion to delivery

Why AI storage demands a new approach to load balancing

About the Author

Mark MengerSolutions Architect

Mark Menger is a Solutions Architect at F5, specializing in AI and security technology partnerships. He leads the development of F5’s AI Reference Architecture, advancing secure, scalable AI solutions. With experience as a Global Solutions Architect and Solutions Engineer, Mark contributed to F5’s Secure Cloud Architecture and co-developed its Distributed Four-Tiered Architecture. Co-author of Solving IT Complexity, he brings expertise in addressing IT challenges. Previously, he held roles as an application developer and enterprise architect, focusing on modern applications, automation, and accelerating value from AI investments.

More blogs by Mark Menger