NetOps Need Advocates not Adversaries

Lori Mac VittieDistinguished Engineer and Chief Evangelist

You're ready to dig into a meal you've looked forward to all day when you notice that it's undercooked. Frustrated, you speak harshly to the server and perhaps even reduce their tip. They take it with a smile even though it's not their fault. After all, they didn't prepare your meal. But they are your interface to the kitchen, and ultimately it is they who pay the price for failures that happen out of sight.

Whether it's the server in a restaurant or a customer service representative for <insert service here>, it is generally the person who interfaces with you that winds up bearing the brunt of your angst/frustration/anger when something goes wrong.

That's true in the data center, as well.

As IT is digitally transforming itself with a goal of attaining higher optimization and speed, it is the NetOps teams who are most likely to interact with internal "customers" and thus bear the brunt of disgruntled users when processes don't move as quickly as desired.

It is important to recognize that it's not always "NetOps" that's getting in the way of deploying the latest thing/app/service. Impediments to speed are often due to a failure to adopt all the premises of DevOps as organizations seek to transform IT operations.

Are you doing DevOps or just automating?

CAMS is the most commonly used means to disseminate the core tenets of DevOps. CAMS stands for: culture, automation, measurement, and sharing.

Of these four, automation is most likely to be embraced with enthusiasm. It is the other three that tend to be left behind or outright ignored in the quest to improve service velocity within IT.

Of particular note is that the three commonly ignored concepts are intertwined. It's difficult to change the culture when teams are still isolated by function and focused on metrics that don't matter to other teams. We work toward that which we are measured on. If we're measured on network uptime, that's what we're going to focus on - even if our counterparts in operations are trying to improve application uptime.

To wit, you may recall the State of Network Automation research in which we joined forces with Red Hat to dive into the murky world of DevOps and NetOps and automation. In it, we found broad disparity between the metrics (measurement) sought after by NetOps and those involved in development and operations.

Nearly three-quarters (73%) of NetOps cited "network uptime" as their primary metric. On the other side of the fence, 59% of developers and operations told us "application uptime" was their primary metric. Conversely, nearly twice as many developers and operations (30%) are measured on frequency of deploys than are NetOps (16%) and Security (17%).

Why is this disparity important? If my primary goal is to keep the network available, I'm going to spend my time focusing on the network. Instrumentation and monitoring - the latter which is critical to the sharing component of DevOps - will focus first on the network, and then perhaps on the application. If there's time. No one has time for security and no one is measured on it anyway. In fact, the number one measurement cited for security was <drum roll please> "network uptime" at 59% of security pros measured on this metric.

People, who are still at the heart of IT and comprise the teams that must implement automation and orchestration, are not necessarily aligned on the same goals. A factor in this misalignment is the continued isolation of operational domains. NetOps and Security groups are more likely to work in the construct of a "single function" team. NetOps worries about the network. Security? Security. Operations? System ops.

But it goes even deeper than that. Because inside the BIG siloes are even smaller siloes. Like the matryoshka - a Russian nesting doll - there are many different teams within NetOps upon which that seemingly simple "new site" request must flow before it is complete. A lot of infrastructure and application services must be provisioned and onboarded before such a request can be accomplished. A new site means not only the compute and network resources to host it, but a bevy of other requirements. A web server and its storage. Access control. Certificates and key management. Load balancing. Firewall rules. The list of intra-IT siloes through which this "simple" request must traverse goes on and on.

And if one of those siloes within the NetOps silo is not automated, the process comes to a screeching halt. Days if not weeks can be added to the time it takes to deliver on such a request.

To the outside - to the requestor - it looks like NetOps is just falling down on the job. It is the "counterpart", the "liaison", the "IT partner" that bears the angst of those demanding to know why it's taking so long to fulfil what appears to be a simple request. We blame NetOps the way technical neophytes blame the local provider for Internet hiccups when the problem is really a router deep in the backbone managed by some other provider.

Be an advocate, not an adversary

The shift toward a more collaborative, transparent IT is still just a blip on the horizon for many organizations. While some groups in IT see the need and embrace the changes necessary, not all do. In the five years we've conducted research on application services we did not see "DevOps" reach the level of strategic importance required to really initiate the cultural and organizational changes necessary to achieve the kind of speed the business wants - and needs. Instead, organizations have embraced automation - and forgotten about the other three components critical to DevOps.

This failure to recognize the DevOps movement into IT as being about more than just automation is troubling. It is a failure to recognize that if you're going to gain speed you have to automate the pipeline, and that pipeline traverses just about every operational domain and silo within IT. That means everyone affected has to move toward automation, or you're going to fail at realizing the speed and frequency of deployments you're after. You cannot just embrace automation and expect to succeed. When automation has to cross lines between siloed groups, you will fail without significant cultural change.

The change needed has to come from the top. Particularly organizational change. We can't very well re-organize ourselves, can we? We can't reprioritize our goals and use an entirely different set of metrics, can we?

We can't, and unless we educate and convince those who can make the necessary changes, we will find ourselves stuck with manual processes in the midst of an otherwise automated pipeline.

So let's stop using NetOps as a scapegoat and recognize they may be frustrated, too. Instead, remind decision makers of the need to reevaluate organizational structures and reprioritize measurements to realize better align with the business and the rest the continuous pipeline.

Yelling at the NetOps folks on the frontlines may feel satisfying, but it doesn't actually change the situation that gave rise to the anger in the first place. And without change, the pipeline isn't going to get any faster.

Be your NetOps advocate instead of their adversary. They need all the help they can get.

About the Author

Lori Mac VittieDistinguished Engineer and Chief Evangelist

More blogs by Lori Mac Vittie