A programmable storage front door for AI and enterprise deployments

Most security architectures require compromise

Object storage works in the cloud because capacity, throughput, and endpoints grow behind a single address without the application noticing. On premises that is harder, because a cluster is a finite set of access nodes behind a fixed endpoint. Storage that used to serve backups and archives now feeds model training, fine-tuning, and RAG index builds, so scale and resiliency are no longer optional.

F5 BIG-IP closes that gap. lg F5 BIG-IP sits in the S3 data path as a full proxy and gives the storage team control of the endpoint clients resolve, how requests are distributed across access nodes, and how failures are handled. BIG-IP dynamically optimizes, secures, and routes object storage traffic by:

• Ensuring continuous availability with active, real-S3 health checks on every access node.
• Terminating SSL/TLS on dedicated, purpose-built hardware so storage CPU is spent serving objects rather than handshakes.
• Maximizing throughput with custom TCP profiles tuned specifically for high-bandwidth S3 workloads.
• Steering traffic intelligently with native, bucket-level, key-prefix, and operation-based iRules.

Key benefits

Load balance and route S3 traffic intelligently

BIG-IP presents one virtual S3 endpoint and maps it to whatever the storage team decides should sit behind it, whether that’s additional nodes, a second cluster, a different vendor, or a dataset coming back from public cloud. Weighted pools let a workload move incrementally rather than in a single high-stakes cutover. Steering by bucket or key prefix means a migration can land one dataset at a time. Anyone who has sat through an S3 interoperability discussion knows that implementations differ in small but consequential ways, and iRules give you a place to absorb those differences, rewriting addressing styles and headers at the edge so applications never have to care which implementation is answering.

Hold throughput steady with per-tenant QoS

Hundreds of concurrent training workers resolving the same endpoint tend to pile onto whichever access nodes they reached first. The resulting imbalance saturates a few access nodes while others sit idle, and no amount of aggregate cluster bandwidth hides it. BIG-IP spreads that concurrency across every node, reuses back-end connections, and terminates TLS in hardware so storage CPU cycles can serve data rather than handshakes. TCP profiles tuned for sustained large-object reads and for high-concurrency, small-object access keep throughput predictable across mixed pipelines. Additionally, per-tenant rate controls stop one aggressive job or checkpoint burst from dragging every other workload below the utilization threshold against which the platform is measured.

Ride out failures and planned work

BIG-IP application-level monitors issue real S3 operations against each node and pull unhealthy members out of rotation before any client notices a difference. That makes it routine to drain a node for a firmware update, replace failed hardware, or expand a cluster while reads and writes keep flowing. If an entire site degrades, BIG-IP DNS steers traffic to a healthy location. Planned storage work stops requiring a change advisory board meeting and a room full of application owners, because from the application side nothing appears to have happened at all.

See and govern every object request

Direct client-to-bucket connections leave the storage team with capacity graphs and little else. As a proxy, BIG-IP logs who requested what, which bucket, which operation, and how long it took. This turns chargeback, audit response, and capacity planning into reporting rather than forensics. The same point enforces policy, protecting S3 endpoints and backup repositories from API abuse, layer 3 through 7 DDoS, credential attacks, and unauthorized egress of data that took months to curate, and it offloads TLS to hardware so no cluster CPU is spent on encryption. And because curated datasets live for years, BIG-IP negotiates hybrid post-quantum key exchange on the wire today, closing the harvest now, decrypt later window before it opens.

Features

BIG-IP deploys in front of any S3-compatible target as hardware, a virtual edition, or a container, with no changes required to the storage nodes.

TCP profiles tuned for S3 traffic

• Window scaling and buffer sizing matched to sustained large-object reads over high bandwidth-delay paths
• Profile options for the high-concurrency small-read patterns typical of language and RAG datasets
• Congestion and idle-timeout behavior suited to long-lived multipart upload sessions
• Full-proxy architecture; client-side and storage-side TCP stacks are tuned independently

S3-native programmability with F5 iRules

• Request steering by bucket, key prefix, HTTP method, or S3 operation
• Translation between virtual-hosted and path-style addressing at the edge
• Canary routing that validates a new backend against a single dataset before wider cutover
• Custom per-request telemetry and enforcement of bucket naming or tagging conventions

Endpoint abstraction

• One virtual server and DNS name in front of many backends
• Weighted pools for phased migration, repatriation, and vendor evaluation
• Read and write split across clusters or tiers
• SigV4 preserved on pass-through, or re-signed by iRules when translating endpoints

Health monitoring and failover

• Monitors issue live S3 requests, not just TCP or ICMP checks
• Graceful node drain, priority groups, and fallback pools
• Multi-site and multi-cloud steering with F5 BIG-IP DNS
• High availability pairs with connection mirroring

Performance offload and S3 QoS

• Hardware TLS offload and certificate management on F5 rSeries and F5 VELOS
• Back-end connection reuse with OneConnect
• Bandwidth controllers with guaranteed floors and hard ceilings per workload class
• Rate and connection limits per client, tenant, or bucket, so training reads outrank background copies and restores

Multi-network and dual-stack access

• Dual-stack IPv4 and IPv6, with translation between client and storage networks
• One S3 endpoint published across data, backup, and partner networks using VLANs and route domains
• Per-network isolation of tenants and policies on shared storage clusters
• SNAT and self-IP control for multi-homed cluster topologies

Security at the storage edge

• F5 WAF for S3 API protection, layer 7 DoS mitigation, and a hardened path to backup repositories
• F5 BIG-IP Advanced Firewall Manager for network DDoS defense and endpoint access control
• F5 BIG-IP Zero Trust Access for mutual TLS and machine identity on ingestion clients
• Post-quantum readiness, hybrid key exchange in TLS 1.3 with ML-KEM per FIPS 203
• Per-request logging by tenant, bucket, and operation, streamed to SIEM and observability platform

Level 3 view for engineer & architect audiences of an AI architecture

Figure: F5 delivers and secures every stage of the AI reference architecture. In the ingestion paths, F5 presents a single S3 endpoint and tunes, steers, and protects the training, fine-tuning, and RAG traffic that reaches the object store.

Specifications

Resources

AI data delivery resiliency validation: F5 BIG-IP and Dell ObjectScale

Optimize AI data processing costs and enhance security with F5 rSeries, VELOS, and BIG-IP LTM

Enabling AI data ingestion

High-performance AI data delivery for S3-compatible storage