Methodology

The Risky Tech Ranking (RTR) is derived using a three step process:

1. Collect all vulnerabilities in the National Institute of Standards and Technologies (NIST) National Vulnerability Database (NVD) and all vendors in NIST’s Common Platform Enumeration (CPE) dictionary.
2. Map vulnerabilities to vendors using NIST enrichment when available, defaulting to an AI model when it is not available.
3. Score vendors by adding up all their vulnerabilities weighted by their exploitability.

This page provides information on how each step is carried out.

Step 1: Identify Vendors and Vulnerabilities

Vendors

We primarily score vendors that appear in NIST's Official CPE Dictionary. In order to maintain fidelity with the National Vulnerability Database, we score some additional vendors. We do not replace names appearing in the CPE dictionary with the name of the legal entity that “owns” the vendor. This means the following:

Collect all vulnerabilities in the National Institute of Standards and Technologies (NIST) National Vulnerability Database (NVD) and all vendors in NIST’s Common Platform Enumeration (CPE) dictionary.

1. “Facebook” and “Whatsapp” are distinct vendors in the RTR, even though both are products owned by Meta.
2. For an example where this is not true, “Google” is a vendor responsible for multiple individual products, like Chrome, Android, and so on.

NIST offers a channel (cpe_dictionary@nist.gov) whereby vendors can request changes to the CPE dictionary. After all calculations are complete, we map CPE strings to names that are better formatted for presentation on the website.

Vulnerabilities

We source vulnerabilities in NIST’s National Vulnerability Database for various reasons:

1. The database was created to track vulnerabilities in a standardized way.
2. It is widely used in industry and academia.
3. Vendors can easily verify and challenge the accuracy of the information as most mainstream vendors are CVE Numbering Authorities that are solely responsible for issuing vulnerabilities in their products.

These factors are not true of proprietary vulnerability databases.

Step 2: Map Vulnerabilities to Vendors

The NVD contains partial mapping of CVE IDs to CPE vendors, which is known as enrichment and maintained by NIST. However, this is incomplete, in part due to the enrichment crisis that started in February 2024.

To map CVE IDs to vendors, we use the following hierarchy:

1. We use NIST’s enrichment where available.
2. If there is no NIST enrichment, we use third-party enrichment where available.
3. If there is no NIST or third-party enrichment, we use an AI system to map the remaining CVE IDs to vendors.

Step 3: Score Vendors

Our final step is to assign a Vendor Score to vendors that takes into account all the vulnerabilities that we identified in Step 2.

To account for risk, each vulnerability is weighted by its Coalition Exploit Scoring System (Coalition ESS) score, which is based on the probability of exploitation. This means high-risk vulnerabilities contribute more to the aggregate score than low-risk vulnerabilities.

To calculate Vendor Score, we sum the Coalition ESS scores of all vulnerabilities affecting that vendor in a time period. The formula is mathematically equivalent to:

Interpretation

The vendor score aggregates the Coalition ESS score of vulnerabilities impacting all products associated with a particular vendor. This formula has a number of implications:

• Vendors with more products have a higher score, all else being equal.
• A high score can be the result of a large number of vulnerabilities with low individual scores or a small number of vulnerabilities with high scores. 
• An increasing vendor score over time can be the result of either more vulnerabilities being disclosed than in the past or changes in the threat landscape.

Reproduction

Coalition ESS scores in the Risky Tech Ranking are based on data pulled from NIST's NVD on April 1, 2025. In order to reproduce our ranking, we recommend using the following resources:

• The model file for Coalition ESS, used to assign a weighting to each vulnerability.
• CISA’s vulnrichment project, used to map a CVE to a vendor if NIST is not available. Use our AI enrichment results/models to map a CVE to a vendor if Step 2 is not available.

Please note, this file contains one row per vulnerability record from the NVD. It contains the following columns:

• cve_id: The unique identifier for the vulnerability (e.g., "CVE-2024-1234").
• coalition_ess_score: A numeric value (e.g., 0.2, 0.5) representing the exploitability score.
• vendor: the final vendor assignment used in the calculations. It is the combination of the NIST vendor, third-party, LLM evaluation, and post-processing rules. Each value contains a list of vendors per CVE, with comma-delimited vendor names.