The Network team at MarketAxess was slogging through a lengthy and painful migration from a legacy product to an F5 solution — while also performing multiple manual IT tasks each day. Much of this manual work could only be done within a brief period of time after hours, between the close of financial markets in one hemisphere and their opening in another. MarketAxess leveraged F5’s Professional Services teams and their partnership with Ansible to implement an automation solution that resulted in greater productivity, faster migration and a solid foundation for the future.
MarketAxess is the leading fixed income online trading platform for institutional investors and broker dealers and a change accelerator in the credit industry. With an average trade flow of approximately $7B a day, the company’s IT team is tasked with maintaining and managing the organization’s single most valuable asset — its network — so its customers can easily access the online trading platform during market hours.
Because MarketAxess is a financial institution, the team can’t work on production systems while customers are using the platform.
"MarketAxess is a trading platform, so if you break a client connection on a production device during the day, it could mean a loss of revenue that's flowing through the system at that moment,” explained Amrisha Sinha, Network DevOps Engineer at MarketAxess.
“We have a window of approximately one - two hours in the evening on weekdays to do maintenance and migrations, between the time the markets close here in the U.S. and when they open in Asia. Our end goal is for clients to have a seamless experience after any type of work is performed. That is our team’s definition of ‘success,’” said Sinha.
Because these tasks had to be done manually, an IT engineer always had to be present during that narrow window outside of business hours, to ensure that any patches, upgrades, migrations, and maintenance done at those times went smoothly.
The IT team knew that with automation, they could schedule tasks after hours without having an engineer present. More importantly, it would ensure consistency: however diligent they may be, when multiple people work on code, small changes can creep in—which can lead to spending time and energy on fixes later. Automation would also make existing processes go faster and free up staff time to work on other tasks that can’t be scripted.
"My background is in both programming and networking,” said Sinha. “I have a dual title of Network DevOps engineer, but the common language for both of those is automation. Everyone wants more consistency and standardization—and automation is how you achieve it. The more consistency I can build in, and the more methods and processes I can automate with F5 and Ansible, the more the DevOps and Network teams are able to accomplish.”
Over the last two years, the MarketAxess Network team worked on migrating away from the company’s legacy load balancing product to a solution built on the BIG-IP platform. MarketAxess engaged F5 Professional Services at the beginning of their migration, with the intention of automating early in the migration process. When Sinha attended the F5 Agility conference in 2017, she consulted with F5 and Ansible representatives she had been in touch with via Professional Services. With access to these resources, MarketAxess was able to setup an automation strategy to leverage in their environment.
The cooperation between Ansible and F5 enabled MarketAxess to develop a single platform for all of its automation efforts, including firewalls and routers. Ansible playbooks are also used to build, enable and disable Virtual Servers.
“F5 was able to help us create a brand new environment that was purpose-built for automation. In fact, 95% of our interaction with BIG-IP are through scripts and automated tasks, and the best part is that we don’t have to learn and integrate a new tool as we continue to automate the network,” said Sinha. “Automating with F5 and Ansible has positioned us to begin automating further – it’s a gateway for us to automate the other pieces of our network.”
By automating with the F5 BIG-IP platform and Ansible, MarketAxess made time-consuming manual processes fast and easy, increasing the IT team’s productivity. MarketAxess can deploy a more streamlined infrastructure, and engineers no longer have to provide coverage outside of business hours to perform routine work on production systems.
Thanks to automation with F5 and Ansible, the Network team at MarketAxess can do their work faster and more efficiently.
For example, every day the team receives client requests to whitelist IP addresses within its trading platform’s firewall. Previously, a member of the team had to manually process anywhere from 2 to 20 new tickets each day—spending two or three hours validating and approving each work ticket and preparing the script. Then, during the brief time frame between close of business and the opening of the markets in Asia, that engineer would log into the production device and enter those new IPs into the whitelist, checking and double-checking to make sure there were no errors.
With automation in place, the team can now input that same information into the system at any time of the day, knowing that the appropriate script will execute the process that evening. “Because that one daily task got automated, nobody has to stay late just to make sure it’s done consistently,” said Sinha. “That’s just one example of many ways working with Ansible has sped things up and reduced our workload.”
Automation has enabled MarketAxess to consistently migrate 50 virtual IPs per week within a 1-hour maintenance window—close to 4,000 virtual IP addresses (VIPs) in the space of only two years. Without automation, and without the collaboration of the F5 Professional Services team, MarketAxess estimates this project would have required twice the time and effort.
“Automation is the only way I’m able to work this quickly,” Sinha said. “Before, I could have migrated one application per maintenance window with confidence. Now I’m able to migrate five applications.”
“My maintenance window starts at 6:00 p.m.,” she continued. “I can start the 12-minute automated process to build and enable my VIPs, then go do something else that I don’t have a script for. In the space of that 12 minutes while my script is running, I get 24 minutes of work done.”
The collaboration between MarketAxess and F5 Professional Services was key to the success of the project. “My experience with F5 has always been one of collaboration,” said Sinha. “I got to work directly with the F5 dev who wrote literally every Ansible module I use. Having access to these resources helped me write robust scripts, and get quick responses on questions I had on the functionality of these modules.”
Having F5 experts available helped the MarketAxess team ensure that they were setting up their F5 automation environment according to best practices—and that it could be used to its fullest.
“Professional Services helped us as we automated, providing BIG-IP expertise, and access to automation resources within F5.” said Sinha. “They got us started on the right track, and helped me fully understand what I was building. Having that foundation in place cut down the amount of work I have to do now, because I don’t have to write a script for every use case. As our network footprint evolves and grows, we are now able to apply this concept to the next challenge, and that is very exciting.”
Because of the successful automation project, the MarketAxess team is partnering with F5 specialists on other upcoming projects, including a shift to the public cloud. The IT team is now working with F5 cloud specialists to deploy new applications in the cloud and leveraging the existing automation framework to configure BIG-IPs as the organizations explores other F5 modules such as BIG-IP AFM, ASM and APM.