Setting the standard: Why AI guardrail efficacy testing can't wait

Ian LauthDirector, Product Marketing, AI | F5

AI is moving fast. Faster, in many cases, than the security controls designed to govern it.

Enterprises are deploying generative AI, agentic systems, and AI-assisted workflows at a pace that consistently outstrips formal security review. Business units are moving ahead, and security leaders are left trying to close gaps that are widening in real time.

In the middle of all this urgency, there's a problem that doesn't get enough attention: We don't yet have broadly accepted, rigorous standards for evaluating whether AI security controls actually work. Vendors make claims. Demonstrations are narrowly scoped. Assurances are grounded in architecture diagrams rather than real-world evidence. Buyers, CISOs, GRC leaders, and enterprise architects are left trying to make consequential decisions without a reliable way to distinguish meaningful controls from good marketing.

That's the gap we set out to close.

Today, the independent cyber security testing firm NSS Labs is publishing a two-part research series on how to evaluate enterprise AI security, developed in collaboration with F5, AWS, and Microsoft.

• Part 1, AI Security Beyond the Model: What Enterprises Need to Care About and Why, makes the case for why enterprise AI security is a system and governance challenge, not a model-only problem. It walks through the full risk surface: input integrity and instruction control, output risk and data exfiltration, system resilience under stress, agentic AI and delegated authority, observability and forensics, and the governance requirements that tie it all together.
• Part 2, Evaluating Enterprise AI Security: Questions Every Buyer Should Be Able to Answer, translates that framework into action. It gives CISOs, enterprise buyers, and GRC teams the specific questions to ask vendors, the red flags to watch for, and the quantitative evaluation criteria needed to make confident, defensible decisions when selecting AI security controls.

AI guardrails are the core of AI security

If you want to understand what AI guardrails are and why they matter, I'd encourage you to read our earlier post on the topic. The short version: AI guardrails are the runtime security controls that operate outside the model, enforcing policy, protecting data, constraining agentic behavior, and producing the audit trail that governance requires.

They are not the model itself. They are the enterprise security layer that makes

The industry needed a reference point

One of the hardest things to do in any fast-moving technology space is to pause long enough to establish what good looks like. The pressure to ship, to compete, and to respond to customer demand works against the kind of deliberate, vendor-neutral thinking that produces durable frameworks.
The companies that collaborated on this whitepaper—F5, AWS, and Microsoft—work with a wide range of the largest and most sophisticated enterprises in the world who are adopting AI at scale. We see the threat patterns, the deployment failures, the governance gaps, and the edge cases that most vendors never encounter.

Pairing that depth of real-world exposure with NSS Labs' rigor for independent testing methodology gave us the foundation to develop something we hope is genuinely useful. Not a vendor pitch, but a practitioner framework built by people who work on these problems every day.

I'm grateful to everyone who contributed their time and expertise to make it happen.

The work ahead

Standards are only useful if they're adopted. Our hope is that this framework becomes a reference point that enterprises use to evaluate the AI security solutions they're considering.

Buyers who understand what rigorous testing looks like will make more informed purchases, and we hope that expectation helps push the broader security community forward.

AI is not slowing down. The attack surface is not shrinking. And the gap between AI deployment and AI accountability is not going to close on its own.

You can read the two-part NSS Labs whitepaper here. Also, please check out our press release announcing this collaboration.

About the Author

Ian LauthDirector, Product Marketing, AI | F5

More blogs by Ian Lauth