F5’s OpenTelemetry Contribution Advances Our Adaptive Apps Vision

Kara Sprague

At F5, we envision a better digital world in which customers can focus on delighting their customers and growing their business without getting overwhelmed by security and application performance risks. Telemetry is key to making that happen.

Today, more than ever, applications and APIs are the most important assets of the digital enterprise. To optimize the security, performance, and value of those apps, organizations must instrument them, as well as their supporting services and infrastructure, to produce meaningful telemetry to inform actions. Many organizations today already use telemetry to detect issues in their environments. A growing number are making investments in observability and automation solutions to also predict likely failures. The ultimate goal of these efforts is to hone in on the right telemetry, insights, and automation to prevent most classes of failures.

Indeed, strong telemetry is the foundation for application health and safety. And to use telemetry effectively, we need a common way to collect and transport high volumes of data across multiple systems and applications so it can be turned into actionable insights. A consistent telemetry standard gets past the problem of combining disparate data that’s been formatted in incompatible ways from different vendors and products.

Today, we’re thrilled to announce a new telemetry protocol that improves the compression ratio by roughly 100% over the existing OpenTelemetry Protocol (OTLP)—unlocking significant cost savings and performance benefits for organizations adopting (or considering) advanced telemetry systems. The new OTel Arrow Protocol is part of our ongoing support for the open source project OpenTelemetry from the Cloud Native Computing Foundation, which provides a standardized format for how telemetry data is collected and exported. We look forward to seeing this new protocol pave a way toward even more significant optimizations that will accelerate data processing itself.

Led by F5 Distinguished Engineer Laurent Quérel, and in partnership with leading digital workflow company ServiceNow, the new OTel Arrow Protocol, on average, doubles the compressibility of OTLP, depending on the data workload. This, in turn, reduces network bandwidth and associated network costs by about half. In some situations, the observed compression can be even greater when the metrics are multivariate in nature, as with F5 products (see the diagram below).

The new OTel Arrow Protocol, on average, doubles the compressibility of OTLP, with the observed compression even greater when the metrics are multivariate in nature

This newly designed protocol comes at a time when the quantity of telemetry organizations generate is surging due to the proliferation of devices and sensors, the increasing reliance on data-driven and AI-driven technologies, and the shift from monolithic application deployments to more granular forms such as containers and serverless functions. At the same time, telemetry data is becoming increasingly distributed—spanning data centers, multiple clouds, and the edge. Together, these developments amplify the urgency to optimize the way telemetry is transported across the Internet.

Telemetry Is Key to Realizing Our Adaptive Apps Vision

The OTel Arrow Protocol sets the stage for future enhancements in telemetry data processing, and it’s also an integral part of our adaptive apps vision. At F5, we are working toward a future in which customers can rapidly detect and neutralize security threats, improve application performance and resilience, speed deployment of new apps, and easily unify policies across their on-premises, public cloud, and edge environments. The ultimate goal is to reduce the cost and complexity of operating apps across disparate application environments—using automation and machine learning techniques that efficiently turn vast amounts of real-time telemetry data into actionable insights for our customers.

To stay ahead of security and app performance risks, customers need visibility into their entire application portfolio. They also need automated predictive analytics to help them rapidly identify risks before they turn into potential problems. For predictive analytics to become automated, large quantities of data are required to train and refine machine learning models so that telemetry can be analyzed with increased precision over time. Open source contributions that improve the way telemetry is transported, processed, stored, and queried are key to making predictive analytics simpler and less costly.

Performing Predictive Analytics at Scale

As we work to make adaptive apps a reality for customers, we’ve been implementing the OpenTelemetry observability framework with F5 BIG-IP, F5 NGINX, and F5 Distributed Cloud Services. These investments are positioning us to better perform data analytics at scale, while leveraging AI capabilities and machine learning models that improve over time.

With a solid data infrastructure in place, we’re working toward giving customers a holistic view into the health and performance of their apps and APIs, not just within specific F5 products but across their entire application portfolios. This will help us proactively give organizations the rapid security and app performance insights they need to quickly identify and neutralize new risks.

Real-time telemetry is key, and what puts F5 at the forefront is that our technology lies in the data path of thousands of customers. F5 technology powers 85% of the Fortune 500 and nearly half of the world’s applications. The massive amount of data we have insight into provides instant visibility into risks, helping us to build automated solutions that rapidly protect organizations from new threats as they emerge. As we continue to turn telemetry into actionable insights, organizations will be able to spend less time managing risks and more time developing the digital innovations that improve business efficiency and enhance the customer experience.

At its core, our adaptive apps vision hinges on the ability to efficiently analyze massive amounts of telemetry from disparate sources—and our ongoing OpenTelemetry contributions are helping to optimize the data aggregation and analysis that’s required to bring this vision to life.

To learn more about F5’s recent telemetry protocol contribution, read our news release. Also, see the Part 1 and Part 2 articles on the Apache Arrow blog.

About the Author

Kara Sprague

More blogs by Kara Sprague