App Tiers Affected:
The F5 Security Incident Response Team (F5 SIRT) helps customers tackle security incidents in real time. F5 Labs, F5’s threat intelligence team, took the data collected by the F5 SIRT and examined the top security incidents over the past three years. Specifically, we looked at incidents from 2017, 2018, and 2019 reported by telecom service providers, both landline and mobile, across the globe. The data shows that the majority of reported security incidents were either distributed denial-of-service attacks (DDoS) or brute force attacks. However, we also noted some interesting trends in attacks using service provider-specific management protocols.
(Note that we do not mention specific organizations or divulge numbers in order to protect customer confidentiality.)
DDoS Attacks Trending Up for Service Providers
The most common attack for service providers, as an average for the three-year period we reviewed, was DDoS, at 49% of reported incidents. While the number of attacks was fairly constant in 2017 and 2018, DDoS saw a huge jump in 2019, accounting for 77% of all incidents handled by the F5 SIRT, up from just a third of all incidents in 2017 (see Figure 1).
What Does a Typical DDoS Attack Look Like at a Service Provider?
A denial-of-service attack against service providers usually targets either the core services that their customers use (such as DNS) or the applications that allow their users to view their bills, see usage, and the like. Such attacks can overwhelm the bandwidth of a service provider network, although this is rare. Attacks are often seen to be sourced from within the service providers’ subscription base and may not (in the case of DNS attacks) directly target the service provider but, instead, use their resources to attack others.
The F5 SIRT primarily had reports of DNS DDoS attacks such as water torture and reflection attacks. Reflection attacks use service provider-hosted resources (such as DNS or NTP) to reflect spoofed traffic so that the responses from the leveraged service end up going to the target, not to the initiator. DNS water torture attacks, also known as pseudo-random subdomain attacks, are a form of reflection attack. These attacks use intentionally incorrect DNS queries to generate response traffic directed to the DNS servers that are authoritative for the queried domains. However, these requests still go through the service provider’s local DNS servers, generating increased load on these servers that occasionally rises to the level of denial of service.
Typically, the first indication that a service provider’s operations team has is an increase in network traffic. The team is naturally focused on the performance of their network as a whole and, given the diversity of traffic types seen, it can take some effort to determine what exact services are being used as part of the attack, and thus, allow the attack to be mitigated.
The next most common indication of these attacks tended to be customer complaints, such as slow network service or complaints that DNS servers weren’t responding, impacting the customer’s ability to get their work done.
From a defensive point of view, as mentioned above, these attacks can appear to simply be either a general outage of a service, such as DNS, or a surge of network traffic. The ability to quickly compare the characteristics of normal, expected network traffic against samples of traffic while the attack condition exists is critically important. Additionally, the ability to quickly enable in-depth logging for network services like DNS in order to identify unusual queries is key to detecting and mitigating these attacks.
Authentication Attacks: Significant but Slowly Waning
Some attacks against service providers target various authentication systems with the goal of either breaching individual customers or trying to find administrative accounts to penetrate deeper into the target network. Brute force attacks were the second most commonly reported incident, accounting for 39% in 2018.
Brute force attacks involve a bad actor trying massive numbers of usernames and passwords against an authentication endpoint. Sometimes these are credentials that have been obtained from other breaches, which are then used to target the service in an attack known as “credential stuffing.” Other forms of brute force attacks simply use common lists of default credential pairs (for example, admin/admin), commonly used passwords, or even randomly generated password strings.
We saw this attack trend significantly downward over the three-year period we reviewed, from 72% of total F5 SIRT incidents in 2017 to less than 20% in 2019 (see Figure 2). As a reference point, it’s worth noting that we’ve seen an upward trend in this attack when looking specifically at the financial vertical. We’ll explore what’s happening in the financial sector in a future article.
What Does a Typical Brute Force Attack Look Like at a Service Provider?
Brute force attack methods vary. It’s not uncommon, for example, for mobile phone service providers to encounter unauthorized online account activity in which an attacker accesses accounts by trying a phone number in combination with a PIN code. These types of attacks are very easy to implement because phone numbers are easily available, and 4-digit PINs only allow for 10,000 combinations, making exhaustive, automated attacks quick.
Service providers also see brute force attacks against web login pages, such as those used for webmail or account access. Because many users continue to use the same credentials for multiple web-based accounts, credential stuffing is an effective attacker technique.
Further, attackers use “password spraying,” a form of brute force attack, to avoid detection. This technique involves the attacker rapidly trying a password across a large number of accounts that individually may only allow a few login attempts before the account is locked. In this way, over time, an attacker can hit many accounts again and again with new password attempts and stay below the threshold for simple detection techniques. The lack of widespread adoption of multi-factor authentication also enables attackers to conduct brute force activities with greater ease.
For a user, the result of a successful brute force attack can be account takeover in the worst case, or being locked out of their account if countermeasures are triggered. This leads to frustration for the customer and increased support desk calls for the provider.
As with DDoS attacks, the first indications of such an attack are customer complaints of account lockout rather than any sort of automated detection. This, in itself, can constitute a denial of service if a large number of accounts are locked out, and certainly can cause increased, at times overwhelming, stresses on support desk capabilities.
Defensively, early detection is key. If an increase in failed login attempts over a short period of time can be identified, this give defenders a window of time to take effective actions to mitigate the attack before customers are affected.
Other Attacks against Service Providers
DDoS and authentication attacks are the most common attacks the F5 SIRT helped customers mitigate, but they are by no means the only attack types observed. Compromised devices within service provider infrastructure accounted for 8% of the cases seen in 2018. These were usually detected because of increased outbound traffic because the compromised devices were used to launch denial-of-service attacks.
“General” web attacks also accounted for 8% in 2019. These can take many forms, but injections were by far the most commonly observed vector. These attacks attempt to leverage bugs in web application code to cause command execution or, in the case of SQL injection, attempt to execute SQL commands on backend database servers, often leading to data exfiltration, if successful. These attacks are usually caught by WAF technologies, or via alerts triggered from web server logs.
Service Provider Targeting in IoT Bot Building
An IoT bot named Annie, a fast-following variant of Mirai, was discovered in November of 2016 after causing at outage at Deutche Telekom.2 Annie was implicated in DDoS attacks against Liberian ISPs as well as attempts to take over routers of ISPs in the UK.3 The bot targeted custom protocols TR-069 and TR-0644 used by ISPs to remotely manage large fleets of routers over port 7547. Whereas the threat actor who created Annie admitted to not using that bot in December of 2016, targeting of port 7547 is still prevalent and increasing in 2019.
In F5 Labs’ Hunt for IoT Report series, we have been following the targeting of port 7547 by botnets, as well as other ports commonly used to remotely administer SOHO routers. Attacker interest in port 7547 dipped off in 2018, however, interest in that port, as well as the Mikrotik remote management port 8291, has increased exponentially over the past 6 months.
We look at the threat landscape region to region in our Regional Threat Perspectives research series. Interestingly, ports 7547 and 8291 were top targeted ports only in the Middle East and Latin America in the fourth quarter of 2019. This is also indicative of the variable use of these ports from region to region. ISPs in Europe surely learned from the news of Annie years ago, and attackers focus their efforts where there are gains to be had. ISPs in Middle East and Latin America likely still have some work to do protecting these ports.
The above data indicates that while service providers have made some strides to defend their networks from attacks against authentication systems, they are still very much subject to being used as part of denial-of-service attacks. This is quite predictable; attackers use ISPs as connectivity to the web, just like everyone else. The attacks they launch—whether directly targeted towards the service provider or not, affect the service provider network.
Our data also indicates there is significant room for improvement in terms of early detection of such attacks. With the large and complex networks that service providers operate, they face special challenges, both in terms of scale and complexity. How they can best determine what is attack traffic versus the normal variability in their traffic is a challenging problem, and unfortunately, it is difficult to do before the customer is impacted. Additionally, finding effective ways to remotely manage their customers’ routers, and choosing the technologies to do so securely, continues to be a challenge from both a scale and difficulty of procurement perspective.
To mitigate these types of attacks, consider implementing the following security controls based on your specific circumstances: