This content has been updated to add more information on the state of EPSS and how to integrate it into your environment.
Summary
- It’s infeasible to patch all the open and critical-rated vulnerabilities in our systems.
- There is a provably more effective way to predict which vulnerabilities will be exploited that is superior to dead reckoning.
- Cyentia and Kenna teamed up to analyze several years of vulnerability data using machine learning to create a predictive model.
- Using something as simple a few free web sources and a spreadsheet, security teams can use this model to predict which vulnerabilities should be prioritized.
- This model can even be used with qualitative metrics for those who don’t like math.
Too Many Vulnerabilities, Not Enough Time
In 2018, 14,238 vulnerabilities were released1. Think about it: That’s nearly 275 vulnerabilities a week that need to be examined, vetted, prioritized, and possibly patched. It’s no wonder that vulnerability teams are extremely busy. We need an effective vulnerability prioritization process to sort out which of these vulnerabilities actually represent a danger to our organizations.
Many operational teams simply prioritize based on the criticality rating of the vulnerability itself. However, the same analysis of the vulnerabilities found that 55% of them were rated critical or high. That means, on average, every 8 to 9 hours, a significant vulnerability is released. That is still an overwhelming continuous flood to keep up with. But do you need to patch them all? Research by the Cyentia Institute found that only somewhere between “2 to 5% of CVEs are (detected as) exploited in the wild.”2
A Data-Driven Approach to Vulnerability Management
Ideally, we patch only the vulnerabilities that would actually be exploited to cause a security incident. Patching anything else is unnecessary work. But analysis is needed to tell which vulnerabilities should be patched and which will be ignored by attackers. Many IT and security teams prioritize patching using the Common Vulnerability Scoring System (CVSS) rating of 1 (lowest) to 10 (most severe). But in 2013, Mike Roytman of Kenna Security and Dan Geer of In-Q-Tel published an article showing how CVSS is a flawed system for prioritizing patching. Specifically, their analysis showed that "a high CVSS score does not imply impending risk in need of immediate mitigation."3 Instead, published exploit code was a better indicator of likely attack than vulnerability severity scoring.
The latest research in this area was released at Blackhat 2019 by Mike Roytman and Jay Jacobs of the Cyentia Institute in a presentation called “Predictive Vulnerability Scoring System,” which unveiled their “Exploit Prediction Scoring System (EPSS)” model.4
To build this model, they looked at 25,159 vulnerabilities published by MITRE’s Common Vulnerability Enumeration (CVE) over a two-year window between June 1, 2016 and June 1, 2018. This was compared against 921 observations of exploitations in the wild during the same period. The data was crunched and analyzed using machine learning to find which aspects were tell-tale signs of exploitation. Like most machine learning derived results, most of the key measures made sense but there were a few interesting nuances as well.
Finding the Likelihood of Exploitation of a Vulnerability
Based on the machine learning analysis, there were 16 significant factors usable to determine predictability.5 Each of those factors carries either a positive (more likely) or negative (less likely) weight towards predictability. Table 1 list those factors.
Factor | Weight |
Is this a vulnerability in a Microsoft product? | 2.44 |
Is this a vulnerability in an IBM product? | 2.07 |
Has vulnerability been weaponized as an attack tool exploit? | 2.00 |
Is this a vulnerability in an Adobe product? | 1.91 |
Is this a vulnerability in an HP product? | 1.62 |
Is there a proof-of-concept exploit for this vulnerability? | 1.50 |
Is this a vulnerability in an Apache project? | 1.10 |
Count of vendor references in the vulnerability | Log(count +1 ) * 1.01 |
Does this vulnerability allow arbitrary code execution? | 0.57 |
Can this vulnerability be exploited remotely? | 0.23 |
Does this exploit cause denial of service? | 0.22 |
Does this vulnerability impact web service? | 0.06 |
Does this exploit cause memory corruption? | -0.20 |
Can this vulnerability only be exploited locally? | -0.63 |
Is this a vulnerability in a Google product? | -0.89 |
Is this a vulnerability in an Apple product? | -1.92 |
There are four major categories of factors: the affected software origination, the type of impact, how the vulnerability can be exploited, and the count of references published with the vulnerability. In short, popular software that is easily remotely exploited to inject arbitrary code was more likely to be attacked. Note that none of these factors include the CVSS score, although some of the same factors that drive the CVSS score (such as exploit code, remote access) can also be found within the EPSS model.
The model works by tallying up all these weights and adding them to a base factor of -6.18 to get the LogOdds variable. The probability can be calculated with LogOdds with the formula 1 / (1 + e-LogOdds).
Collecting the Factors Related to the Exploitation of a Vulnerability
Let’s work through an example here with an eye towards how to measure each of these factors and perform the calculation. From 2019, we can look at CVE-2019-11043, a vulnerability in the web programming language PHP. Pulling up the National Vulnerability Database (NVD) entry for this at https://nvd.nist.gov/vuln/detail/CVE-2019-11043, we can get some basic information on the vulnerability as shown in Figure 1.
Right off the bat, PHP is a web service, so we can see that we can tick off one of the factors: "Does this vulnerability impact web service.” So that’s the first weight of 0.06.
Further down the page, there is a list of vendor references, some of which are shown in Figure 2.
We can simply count the number of references and see there are 25. In Excel, we can calculate the weight of these with “1.01*(LN(25+1))”, yielding 3.29.
Within those references, there are links to published exploits, as shown in Figure 3.