Why model routing may become security infrastructure

Lori Mac VittieDistinguished Engineer and Chief Evangelist | F5

For the last two decades, enterprise security has centered on identity. Authenticate the user. Authorize the action. Audit the request. Agentic AI is exposing the limits of that model.

As organizations increasingly rely on AI systems to retrieve information, make decisions, and execute tasks, the question is no longer simply who is making the request. The question is which model should be trusted to perform the work in the first place.

At the same time, a second trend is emerging. AI capabilities are moving onto the endpoint. PCs are shipping with neural processing units (NPUs) and embedded models capable of running inference locally. Small language models are becoming practical components of everyday applications rather than cloud-only services.

These two trends intersect in an interesting way.

As AI moves onto the endpoint, model routing may evolve from a performance and cost optimization mechanism into a fundamental element of security architecture.

The current assumption: One model, many permissions

Most agent architectures today assume a general-purpose model operating with access to an expanding collection of tools, APIs, data sources, and enterprise systems.

Security is typically layered around the model:

• Identity systems determine who initiated a request.
• Access controls determine which resources are available.
• Guardrails attempt to constrain behavior.
• Auditing tracks actions after execution.

This approach mirrors traditional application architecture, where a single application receives permissions and security controls determine what it can do. The challenge is that large language models are increasingly being asked to perform tasks with dramatically different risk profiles.

Summarizing a document, approving an expense report, modifying a customer record, and deleting production infrastructure are not equivalent operations. Yet many architectures treat them as variations of the same interaction.

As agent capabilities expand, this creates pressure to continuously add more policy, more controls, and more oversight around increasingly capable systems.

The result is growing complexity in both governance and enforcement.

An alternative approach: Authority through specialization

What if the solution is not stronger controls around a general-purpose model? What if the solution is reducing the authority available to any single model?

Historically, distributed systems evolved toward specialization because specialization reduces complexity and limits failure domains. Databases perform database functions. Proxies perform proxy functions. Load balancers perform load balancing functions.

AI architectures may follow a similar path.

Instead of one highly privileged model, organizations may deploy portfolios of specialized models:

• Local models for summarization and classification
• Retrieval models for enterprise knowledge access
• Workflow models for orchestration
• Action models for specific business processes
• Highly constrained models for privileged operations

Each model would possess only the capabilities required for its intended function. Rather than relying solely on policy enforcement after model selection, architects could increasingly bind authority directly to model selection itself.

The model chosen becomes part of the security decision.

Model routing as policy routing

Today, model routing is primarily discussed in terms of cost and performance.

Organizations route requests to different models based on latency requirements, token costs, or reasoning capabilities.

Over time, routing criteria may expand significantly.

A future routing decision might evaluate:

• Data sensitivity
• Regulatory requirements
• Required authority level
• Tool access requirements
• Data residency constraints
• User trust level
• Endpoint capabilities

In this model, routing becomes less about finding the cheapest model and more about selecting the safest execution environment.

• A request involving sensitive financial data may be restricted to local processing.
• A request requiring access to customer systems may be routed to an enterprise-hosted model with approved tooling.
• A request involving destructive actions may require escalation into a tightly controlled execution environment.

The routing layer effectively becomes a policy decision point.

Why endpoint AI changes the equation

The emergence of endpoint AI may accelerate this shift.

Historically, cloud-hosted models forced organizations into relatively centralized architectures. Data traveled to the model because the model was located in the cloud. As capable models become available directly on user devices, new architectural patterns emerge.

Sensitive information may never need to leave the endpoint. Certain workflows may be completed entirely within local trust boundaries.

Organizations gain additional options for balancing privacy, performance, and governance. More importantly, they gain an additional security primitive.

The ability to decide whether a task should be executed locally or remotely becomes a meaningful architectural control. In some scenarios, the safest cloud model may still represent greater exposure than a sufficiently capable local model with no network access at all.

The case for a local AI gateway

This creates an opportunity for a new infrastructure layer.

Rather than allowing applications to directly invoke models, organizations may increasingly deploy local AI gateways responsible for:

• Model selection
• Policy enforcement
• Tool authorization
• Data movement controls
• Audit generation
• Context filtering

One possible implementation path is a lightweight WebAssembly-based runtime operating as a local inference and policy proxy. Such a layer would allow organizations to apply consistent governance regardless of which model ultimately performs the work. More importantly, it separates security decisions from model behavior.

Instead of hoping a model obeys a policy, the gateway controls whether the model receives the capability in the first place.

That distinction becomes increasingly important as models grow more autonomous.

Strategic implications

Several broader implications emerge from this shift.

First, model routing may become a core architectural function rather than a platform optimization feature.

Second, smaller specialized models may become more valuable than larger general-purpose models in enterprise environments because they offer clearer authority boundaries.

Third, endpoint AI may become a governance strategy as much as a performance strategy.

Finally, organizations may begin treating model selection as part of access control itself.

In that future, security decisions are no longer limited to questions about users and resources. They also include questions about which model should be trusted with a given task.

Looking ahead

For years, identity has served as the primary control plane for enterprise computing. Agentic systems are challenging that assumption.

As AI becomes more deeply integrated into business processes, security architectures will need new mechanisms for constraining authority, limiting exposure, and reducing operational complexity.

Model specialization offers one possible path. Rather than building ever-larger security frameworks around increasingly capable models, organizations may choose to distribute authority across specialized models and use routing infrastructure to enforce those boundaries.

If that happens, model routing will evolve beyond performance optimization to policy routing. And policy routing has always been security infrastructure.

About the Author

Lori Mac VittieDistinguished Engineer and Chief Evangelist | F5

Lori MacVittie is a Distinguished Engineer and Chief Evangelist in F5’s Office of the CTO with deep expertise in application delivery, automation strategy, and infrastructure. She is known for turning complexity into clarity whether she’s defining guardrails for AI agents, dissecting brittle multicloud architectures, or probing the limits of scalable systems. She brings more than thirty years of industry experience across application development, IT architecture, and network and systems operations. Before joining F5, she served as an award-winning technology editor. MacVittie holds an M.S. in Computer Science and is a prolific author whose publications span security, cloud, and enterprise architecture. She is also an avid tabletop and video gamer with unapologetically strong opinions about cheese.

More blogs by Lori Mac Vittie