EvaluationmediumIntermediate

Severity Matrix

Also known as: Severity Rating System / Prioritization Matrix

Analytical tool to classify, prioritize and assign criticality to usability problems found.

What is it

Strategic value

Focuses attention on problems that really matter. Avoids the frustration of delivering lists of problems without prioritization that paralyze developers and analysts.

Category: Communication and reportingEstimated time: 2-3 hours (with problems already identified)

The Severity Matrix is an analytical tool used after user research to classify, prioritize and assign a criticality level to usability problems found. It transforms an overwhelming list of errors into a structured action plan, evaluating problem impact against its frequency of occurrence.

What it is for

Classify and prioritize usability problems by impact and frequency
Transform extensive error lists into structured action plans
Justify to executives or developers why certain changes should be prioritized
Create a common language between UX and development to discuss problem severity

Research methods that feed it

Usability tests (Think Aloud)Heuristic evaluationsModerated remote testsError analysis in analytics

When to make it

When to use it

After usability tests or heuristic evaluations with high volume of problems
When there are time or resource constraints preventing fixing everything before launch
To justify to executives or developers why certain changes should be prioritized

When NOT to use it

In very fast agile environments where documenting each problem with scales delays iteration
If technological cost to fix is minimal, better to fix immediately
When there is debate about the validity of severity scales among specialists

How it is made

Required components

Issues list: description of identified usability problems
Impact/severity scale: problem magnitude for user (Low, Medium, High, Catastrophic)
Frequency scale (occurrence probability): percentage of users experiencing the problem
Criticality/priority score: result of combining impact and frequency (Level 1, 2, 3)

Optional components

○Business objectives impact: how much it affects commercial goals
○Implementation/technological costs: effort or time to fix
○Persistence: if it will repeatedly annoy or be overcome once learned
○Core task importance: how central is the affected task
○Workaround: if there's an alternative way to accomplish the task

How to create it step by step

1Identification and listing: Document all problems found during usability testing.
2Scale definition: Choose a consistent rating system (1-4, 1-5, or categorical).
3Independent evaluation and consensus: More than one specialist assigns ratings independently, then discusses differences.
4Criticality calculation: Use chosen method: Sum (Rubin), Multiplication, or Decision tree.
5Tier classification: Order from highest to lowest priority and assign action plan.

Tips for small teams

Use a simple 3-level scale (Critical, Important, Minor) instead of complex scales
If you're the only evaluator, document your reasoning for each rating
Include screenshots alongside each problem
Share the matrix with development before starting fixes to align priorities

Contextualized example

Context: Post-heuristic evaluation of e-learning platform with 47 identified problems.

Application: Using Criticality = Severity + Frequency (Rubin method): 8 problems classified Tier 1 (critical), 15 problems Tier 2 (important), 24 problems Tier 3 (minor). Tier 1 included: registration flow without email confirmation (Severity 4 + Frequency 4 = 8). Result: development prioritized all 8 Tier 1 problems for the current sprint.

How to tell it is done well

Quality criteria

Consistency and transparency: same scale across all studies to allow historical comparisons
Common language: definitions of each level understandable by non-UX audience (developers, analysts, managers)
Actionability: transforms hundreds of data points into manageable, prioritized task list

Common mistakes

Subjectivity and lack of standardization: using own judgment without standard process creates conflicts
Evaluator Effect: tendency to rate problems you discovered as more severe
Reluctance to use low ratings: avoiding 'Low' category for fear development will ignore those problems
Treating it as exact science: believing ratings are irrefutable

Authority quotes

“Figuring out the criticality of a problem is a simple matter of adding the severity rating and the frequency rating.”
— Rubin & Chisnell

“Severity ratings help focus attention on the problems that really matter.”
— Handbook of Usability Testing

“By asking only three questions about any usability problem, we can classify its severity.”
— Usability Testing Essentials

Keep going

Templates

○Findings and Severity MatrixplanillaDownload (XLSX)

Phase

Evaluation

Audience

product-teamdevelopment

Complexity

medium

Experience level

Intermediate