We analyzed 3 million cybersecurity records and here is what we came up with: Vulners AI Score v2

When it comes to good cybersecurity hygiene, continuous and timely patching is a basic principle. But basic doesn't mean it's always easy to do in practice.

According to the Cyentia Institute, one of the main trusted cybersecurity research teams, a typical organization remediates about 1 out of 10 vulnerabilities in their environment within a given period. This seems to hold true for companies large, small, and anywhere in between. Why so?

Patching can hurt

Patching takes time, costs money, often introduces new risks, and can be very resource consuming. For example, you cannot patch a production system with a single click of a button because even a short-term downtime has consequences: customer churn, revenue loss, business reputation damage, etc.

The larger an organization is, the more resources it can throw at patching problem, but they also have much more stuff to fix. Regardless the scale of an organization, patching is an overwhelming and often thankless job, as according to our stats, hundreds of new vulnerabilities are published daily:

To tackle this problem, you need a proper approach to vulnerability prioritization and remediation. Once a vulnerability is patched, the threat is eliminated, and the target system is back up and running to provide business continuity. But how to make sure you are doing it efficiently?

Prioritization is the key, and Vulners can make a difference

Basic remediation strategy is to focus on the riskiest vulnerabilities with known exploits and put all the non-exploited flaws on the backburner. But how can you tell in real-time which vulnerabilities that can affect your system are being actively exploited in the wild?

Here at Vulners.com, we aggregated and correlated the most complete cybersecurity intelligence available online to help security specialists make timely, smart, and well-informed decisions.

Vulners in numbers

We collated 200+ information sources (i.e., national and corporate vulnerability databases, newsfeeds, vulnerabilities severity metrics, public and private exploits collections and exploitation evidence sources, bug bounties, and more) that cover 222,000+ CVEs with over 3 million security advisories. With this experience we are certain that such torrent of information is impossible to parse manually, especially when we are dealing with hyped up vulnerabilities and fluctuating severity estimation that can promptly change over time.

The rising trend of vulnerabilities AI scoring

When a vulnerability is disclosed, a swift assessment is imperative. In such scenarios, every indicator holds significance in achieving high-quality results. Even if a vulnerability is deemed critical but the potential impact is assessed as low, hasty patching is not the recommended approach. Historical instances illustrate that attackers can exploit new vulnerabilities within a matter of days or even hours following their publication. Nevertheless, each of these cases necessitates a comprehensive evaluation within the context of the secured infrastructure.

For example, CVE-2023-36884:

On July 11, 2023, Microsoft released advisory: "very critical, high exploitability, no exploit and no patch yet."
A whole week after publication, on July 18, the flaw was fixed with an official patch.

Technically, Microsoft has offered a fix without an official patch, but this is a rather dangerous approach. According to Qualys, typical vulnerability scanners start detecting the vulnerability on average one week after the official patch. This means that it could take about two weeks from the release of the advisory to manually patch the vulnerability until it is detected by a scanner. If you add in the time it takes to prioritize the patch, the timeframe gets even longer.

In time gaps like these, early-warning indicators such as Exploit Prediction Scoring System (EPSS) (predicts the likelihood that the flaw will be exploited within the next 30 days) are becoming very popular. But is EPSS alone enough?

For this CVE, the EPSS score was 0.00051 (basically saying there is nothing to worry about) up until July 17, when it sharply jumped to 0.499
Then up to 0.8787 by August 1.

We believe that various predictive systems should be considered to identify truly critical vulnerabilities before they strike, and it seems like the community is thinking the same.

Vendors are beginning to introduce their own scoring systems more and more often, which we believe is a good trend as it provides more information for decision-making. For example, Qualys TruRisk offers a proprietary score that provides an additional metric for assessing the criticality of a vulnerability.

With the exponential growth of CVEs, the emergence of AI scores is undeniably gaining momentum, and the trend is set to reshape the vulnerability assessment landscape. As vendors and organizations adopt this approach, AI scores are becoming a powerful addition to the cybersecurity professional's arsenal. Here at Vulners, we recognize the significance of this trend and are at the forefront of it.

Vulners AI score v1 vs. v2

The Vulners team is comprised of cybersecurity enthusiasts with 15+ years of experience in building defenses for fintech, IT, IoT, telecom, and mobile. They are authors of several CVEs, members of the "hall of fame" in many Bug Bounty programs. As part of our commitment to staying ahead of emerging threats, we've developed our own approach to vulnerability criticality prediction — the Vulners AI score.

In 2020, we launched the first iteration of the Vulners AI score, which uses a news graph methodology to analyze vulnerabilities. This first feature set the cornerstone for our dedication to innovation and ongoing enhancement. Over time, the approach used to predict the severity of new vulnerabilities outgrew its usefulness. This was evident in the widening gap between the AI score and actual circumstances. Of course, like any metric, AI scores cannot achieve flawless 100% accuracy. However, maintaining an acceptable margin of error remains critical.

Drawing from invaluable feedback shared by our customers and harnessing the latest strides in artificial intelligence and machine learning, we are thrilled to unveil the arrival of Vulners AI Score v2. At Vulners, we believe in continuous improvement. Our AI score is not static but evolves with the changing threat landscape and vulnerability content. It adapts to emerging risks, keeping you ahead of the curve.

This new scoring system leverages our powerful vulnerability intelligence database and ensures timely severity prediction for efficient prioritization and patching of vulnerabilities, which are crucial for proper assets and data protection.

Vulners AI score is not static but evolves with the changing threat landscape and vulnerability content. It adapts to emerging risks, keeping you ahead of the curve. The algorithm is retrained once a week to ensure up-to-date performance.

Under the hood of Vulners AI

The core idea behind Vulners AI was to predict the Common Vulnerability Scoring System (CVSS) of any security record using machine learning algorithms and vulnerability intelligence content which emerges around the flaws. Our team proposed the hypothesis of incorporating graph features into themodel to improve the prediction accuracy, since records are interconnected and can be represented as a graph.

Initial data analysis revealed that about one-third of records are related to other records, and a significant number of nodes form connected components in the graph.

To meet the requirements of real-time computation and uncertainty estimation while being CPU friendly, we used a basic AI model built on CatBoost, a tree-boosting algorithm with open-source library. This regression model with gradient boosting over decision trees analyzed vulnerability descriptions and available record fields to achieve a Mean Absolute Error (MAE) in the range of 1.

What is CatBoost?
Imagine you have a big collection of data, and you want the computer to learn from it. CatBoost uses a method called "gradient boosting" to do this. It's like having a team of decision-makers (trees) who are not very smart on their own but together can make better decisions. Also, your team is large (lots of trees) and one team members refine the response of other team members.
What makes CatBoost special is that it's great at dealing with things like "red" or "blue" without you having to change them into numbers. It's like if you told your team to make decisions based on colours, and they understood exactly what you meant.
Here are some key points:
Handling Categorical Data: One of the biggest challenges in machine learning is handling categorical data. Typically, this requires manual preprocessing steps like one-hot encoding. CatBoost handles categorical data automatically, eliminating the need for extensive preprocessing.
Gradient Boosting: At its core, CatBoost is a gradient boosting algorithm. This means it builds an ensemble of decision trees by iteratively correcting the errors of the previous trees.
Speed and Efficiency: CatBoost is optimized for speed. This makes it particularly useful for large datasets.
Interpretable: Like other tree-based algorithms, CatBoost can be quite interpretable. You can analyze individual trees, see feature importance, and even get SHAP values for understanding model predictions at a granular level.

To improve the performance of the model, graph features were introduced, and lightweight algorithms were used for fast recalculation when adding new nodes or updating links. These model features include:

ancestors_count — the total number of ancestors in the graph.
ancestors_true_count — the number of ancestors with given CVSS scores.
ancestors_min — the minimum CVSS score of the ancestors.
ancestors_max — the maximum CVSS score of the ancestors.
ancestors_median — the median CVSS score of the ancestors.

In addition, categorical features such as bulletinFamily and text descriptions of bulletin were also used in the model to improve predictions. Incorporating these graph features into the base model improved its quality of MAE to 0.63, an improvement of approximately 30%.

The most influential feature on performance was the number of ancestors. It was observed that nodes with a higher number of ancestors are likely to have lower CVSS scores because they represent combinations of multiple vulnerabilities and do not provide significant new information to the community. Using this feature allowed the model to avoid overestimating such vulnerabilities, even if their descriptions mention well-known tools or technologies (which often inflate CVSS scores in text-based models).

Uncertainty estimation using the RMSEWithUncertainty loss function was used to identify explicit model errors, considering both overall and data uncertainty. The algorithm is retrained once a week to ensure up-to-date performance.

Why Vulners AI score

Like the widely used CVSS and EPSS, our AI score provides an additional, data-driven metric for vulnerability prioritization. But why augment existing metrics with Vulners AI score?

Early warning. In a fast-paced cyber environment, time is of the essence. The Vulners AI score ranks any piece of content appearing in the database — an advisory, exploit, newsfeed, or CVE — and predicts the CVSS score when one is not yet assigned. It empowers you to quickly prioritize vulnerabilities and distil the truly critical ones.
Second opinion. Vulners AI score enables you to make decisions based on the eminent data, which is updated non-stop and augments the CVSS, EPSS, or any other indicator. You gain unprecedented visibility by harnessing the power of AI and massive data from 200+ sources to make strategic security decisions aligned with your objectives.

Let's search the Vulners database for some non-trivial examples with AI score.

Example 1: OpenSSL Vulnerabilities (CVE-2022-3602 , CVE-2022-3786)

Metrics:

100+ mentions on Twitter: Indicates a high level of public attention and discussion around the vulnerability.
High CVSSv3: Suggests a vulnerability with potentially severe consequences if exploited.
Low EPSS: Indicates a low probability of successful exploitation based on historical data or exploit patterns.
Low AI score: The AI scoring system rates this vulnerability as not being actively exploited.

Analysis:

This OpenSSL vulnerability gained significant attention on social media, possibly due to the reputation of OpenSSL and its historical impact. The high CVSS score indicates severe potential consequences. However, the low EPSS and AI scores suggest that, despite its critical nature, there might be factors that make it challenging for attackers to exploit. It's essential not to rush to patch this vulnerability but instead perform a careful assessment of its impact in your specific environment.

Example 2: CVE-2023-20867 (VMware ESXi)

Metrics:

High AI: The AI scoring system rates this vulnerability as actively exploited.
Low CVSSv3: The Common Vulnerability Scoring System gives it a lower severity rating than expected.
Real-world APT group exploitation: Indicates that threat actors are actively using this vulnerability.

Analysis:

CVE-2023-20867 in VMware ESXi is a case where the CVSSv3 rating might not accurately reflect the true risk. Despite having a low CVSS rating, the high AI rating and evidence of exploitation in the wild by APT groups highlights its importance. This discrepancy could be due to the complexity of the attack or the presence of effective mitigations. It's crucial to prioritize patching or mitigating this vulnerability promptly to defend against active threats.

In both examples, the key takeaway is that vulnerability assessment should consider various factors beyond just the CVSS score. Real-world context, exploitability, and the specific environment in which the software or system is deployed play critical roles in determining the urgency of patching or mitigation.

Additional examples can be found through the following queries:

Key takeaways and a gift

Continuous and timely patching remains the single most important thing you can do to secure your technology. It is pure basics, but basic does not mean easy. Sticking to these best practices will save you time, money, and in the end of the day, leave your adversaries wasting their efforts in vain:

#1 Take well-informed decisions based on data

Content is king. Vulners.com is the most complete an_d the only fully correlated security intelligence with forever-free access for web users. It is a one-stop shop for cybersecurity professionals, which goes through constant updates and links 200+ data sources. It contains 3 million+ security entries, including CVEs, all major scores like CVSS and EPSS, advisories, exploits, and newsfeeds — everything you need to stay abreast of the latest security threats._

#2 Assign risk levels right and on time

The Vulners AI score, crafted by security experts for security experts and provided gratis, allows you to make instant and clear-cut CVSS score predictions and take quick, well-informed decisions, even when CVSS or EPSS are not assigned yet._ It also complements the CVSS and EPSS scoring systems when a second opinion is highly appreciated. No single indicator is universal and sufficient. Building your risk assessment model based on various scoring systems is a smart choice.

#3 Keep up with vendor advisories and patch announcements

According to our stats, 1,000+ security updates are issued daily. By signing up for custom Vulners notifications, you make sure to catch the latest updates on vulnerabilities and exploits as soon as they pop up. It will skyrocket your awareness and boost vulnerability management efficiency. Here are some free _preset subscriptions that we have prepared for you:

Since day zero, supporting the cybersecurity community has been a core part of Vulners strategy. Over the years, we have relentlessly added new features to our vulnerability intelligence platform to best serve the various scenarios, use cases, and needs of our comrades. Vulners AI is a great example of how hard we push ourselves to improve the quality of service we deliver.

However, we understand that no technology is flawless. Our team spent months training the Vulners AI model and checking thousands of output results, but what are thousands against millions? To reach maximum performance and precision, we encourage you to go test-drive vulners.com for free with all the scrutiny and give us fair feedback. Your opinion and input are highly appreciated.

And remember, we are just one click away at [email protected].

Stay safe and secure!