Recently, I was reminded of the importance of calibrating measurements when I re-entered the realm of reef keeping. Like rapid changes in the application landscape, reef keeping has dramatically changed in the past five years.
Today, I'm enabled by connected monitors and probes that automatically track and alert me if something is awry. Web and mobile dashboards enable monitoring, programming, and an at-a-glance view of the health of my tank and status of the myriad devices that keep it running.
Astute readers will note that PH appears to be "below minimum." No need to worry, all is well. You see the probe (sensor) is not properly calibrated. It's a common problem; calibrating probes is a process fraught with failure. At the moment I know that the probe is calibrated low, and automatically adjust the measurement based on that knowledge.
Like the health of a reef tank, the health of applications requires careful attention to key metrics. Deviations, especially wild fluctuations, can indicate a problem. Manual adjustment of metrics is not a process you want to mimic when it comes to applications. Manual adjustments may work for one application or even two, but the average organization has between 100 and 200 applications in its digital portfolio. You need accurate measurements calibrated against typical health patterns.
As with most network and application metrics, this means taking samples for a period of time and learning the "highs" and "lows." Thresholds can then be used to determine anomalous behavior.
The issue is not the principle, but rather execution.
First, we're generally focused on only one measurement point: the application. Interestingly, the health of a reef tank requires measuring salinity and PH along with temperature because both values are impacted by temperature. Measuring application health is much the same; it is impacted by other measures, such as network performance and load. Unfortunately, most organizations aren't necessarily taking a holistic view of application health. The application itself may be fine, but the customer experience might be abysmal thanks to a poorly performing connected device or network.
We need to broaden our view of application health by expanding what we measure. Moreover, we need to calibrate additional measurements to make sure we can identify what's typical and what is not. Because "what is not typical" can be indicative of a problem or, worse, an attack.
Second, the challenge of scale arises from the need to calibrate across multiple points for every application we need to monitor (spoiler: that's all of them). We can't expect operators to manually calibrate that many data points. It's not humanly possible.
That's where machines come in.
Advanced Analytics
Advanced analytics and machine learning are one of the answers to the issue of scale. Machines can, and do, process vast volumes of telemetry at significant rates. They can ingest, normalize, and analyze patterns and relationships in quantities of data that we, as human beings, simply cannot manage. In this way, machine learning provides the ability to calibrate "normal" across a range of related data points and immediately detect deviant patterns that indicate a problem.
It's easy enough to correlate performance problems with an application to a Monday morning surge of logins. What isn't easy is to recognize that Bob usually doesn't log in until Monday afternoon. And yet today, he is. That's an anomaly that isn't readily recognizable by human operators because we don't have that level of visibility. With enough telemetry emitted by the application, client, and application services that comprise the code-to-customer experience, advanced analytics can detect that anomaly. It can also flag it or push a new business flow that verifies Bob is actually trying to log in.
That capability is like what many applications do today on a device level. Many digital processes push verification codes and ask us to prove we're human by identifying all the cars in a blurry image. But it's the device details that trigger the new business flow, not the behavior of logging in at an unusual time of day. In the future, we'll need to be able to trigger flows based on both, especially if we continue to support a distributed workforce.
That makes calibration a critical part of the process. And calibration is achieved by taking (a lot of) measurements and coming up with "normal." That, too, is a process that challenges human scale and requires machines to ingest and analyze significant amounts of telemetry.
Advanced analytics will ultimately enable observability and give rise to new services able to uncover hidden application insights (DEM), enable smarter app service orchestration (AI Ops), and produce previously undiscoverable business value (AI-Enhanced Services).
To do that we need to generate copious amounts of telemetry so we can calibrate "normal" behavior for applications, users, and everything in the data path in between.
About the Author
Related Blog Posts
F5 accelerates and secures AI inference at scale with NVIDIA Cloud Partner reference architecture
F5’s inclusion within the NVIDIA Cloud Partner (NCP) reference architecture enables secure, high-performance AI infrastructure that scales efficiently to support advanced AI workloads.
F5 Silverline Mitigates Record-Breaking DDoS Attacks
Malicious attacks are increasing in scale and complexity, threatening to overwhelm and breach the internal resources of businesses globally. Often, these attacks combine high-volume traffic with stealthy, low-and-slow, application-targeted attack techniques, powered by either automated botnets or human-driven tools.
F5 Silverline: Our Data Centers are your Data Centers
Customers count on F5 Silverline Managed Security Services to secure their digital assets, and in order for us to deliver a highly dependable service at global scale we host our infrastructure in the most reliable and well-connected locations in the world. And when F5 needs reliable and well-connected locations, we turn to Equinix, a leading provider of digital infrastructure.
Volterra and the Power of the Distributed Cloud (Video)
How can organizations fully harness the power of multi-cloud and edge computing? VPs Mark Weiner and James Feger join the DevCentral team for a video discussion on how F5 and Volterra can help.
Phishing Attacks Soar 220% During COVID-19 Peak as Cybercriminal Opportunism Intensifies
David Warburton, author of the F5 Labs 2020 Phishing and Fraud Report, describes how fraudsters are adapting to the pandemic and maps out the trends ahead in this video, with summary comments.
The Internet of (Increasingly Scary) Things
There is a lot of FUD (Fear, Uncertainty, and Doubt) that gets attached to any emerging technology trend, particularly when it involves vast legions of consumers eager to participate. And while it’s easy enough to shrug off the paranoia that bots...