How to Choose the Right UX Research Method: A Step-by-Step Guide for UX Researchers

How to Choose the Right UX Research Method: A Step-by-Step Guide for UX Researchers

By Paulina ContrerasUpdated on 1 min read
UX ResearchUX Methodology
View contents

Summary: Choosing the right UX research methodology can be overwhelming with so many options available. This guide shows you the 6 key criteria that UX Research literature recommends for making this decision, in what order to prioritize them, and how to apply them in real projects.

This article was born as a complement to the UX Research Methodology Selector tool I published a couple of months ago. Now I wanted to share the "behind the scenes" of how to choose a methodology, because although the tool helps filter options, there's a thought process worth understanding.

Choosing the right UX Research method is one of those things that seems simple until you have to explain it to others. It's happened to me more than once to be facing a new project, with stakeholders waiting for results, and wondering: should I do user interviews? A usability test? A survey? All of them together? It gets easier each time, but explaining to others why this method and not another that might seem more obvious isn't that simple.

After reading quite a bit of literature, making several mistakes, and learning from those errors, I came to understand that there's a logical order for making this decision. It's not magic, (...drumroll) it's method.

Why is choosing a user research method so difficult?

When you search "UX Research methods" on Google, you find lists of 10, 20, with 35 techniques or more, all different. Interviews, usability tests, card sorting, A/B testing, ethnography, surveys, eye tracking... The number of options can be paralyzing.

The problem isn't the lack of information about each method. The problem is that no one explains when to use which one and in what order to make the decision.

The travel analogy for choosing UX methodologies

I like to think of choosing a methodology as planning an important trip. It sounds simple, but this analogy has helped me a lot to explain it to teams that don't come from the UX world:

  1. First you define the destination (where do you want to go?)
  2. Then you choose the route (is it an exploration or validation trip?)
  3. You decide what you're going to record (photos, times, opinions?)
  4. You review your budget and time
  5. You choose the right travel companions

If you don't know where you're going, it doesn't matter which bus you take. And if you choose the bus just because it's cheap, you probably won't get where you need to go.

This map sums up the whole route, with the six criteria in order:

Decision map for choosing a UX research method: a route from the business question to the method through six criteria in order (problem and objectives, explore or validate, type of data, practical constraints, participants, context and format), plus product stage, how to combine methods and four questions to ask first

Download the map as a PDF (A3 size) to read it in full or print it.

The 6 criteria for choosing a UX research method (in order of priority)

After reviewing Rohrer from Nielsen Norman Group, Travis & Hodgson, Tullis & Albert, and other authors who have written about this, I found there's quite a consensus on how to prioritize decision criteria. Here I order them according to the importance that literature gives them:

1. Define the problem and research objectives (Maximum Priority)

This sounds obvious but it's where we fail the most. The research question defines which methods you'll use, who you recruit, how you analyze, and what you report. Everything revolves around it.

Before even thinking about methodologies, you need to answer:

  • What is the business objective?
  • What specifically do we want to learn?
  • What are the team's doubts?
  • What decision will be made with the results?

As Travis & Hodgson say in "Think Like a UX Researcher": focus first on the data you need to collect, and the method will follow. If your initial question is too vague ("do users like our app?"), the method you choose probably won't give you useful answers.

My learning: In my first projects, I would arrive at kickoff meetings already thinking about doing interviews or testing, because it was what I knew. Now I arrive with questions: What do you need to know? What decision will you make with this information? What happens if we discover X versus Y?

The two-spaces model

A simple way to think about it is to ask yourself: Is my question in the problem space or in the solution space?

  • Problem space: We don't yet know well what to build. We need to understand users, context, needs, pain points. This is where generative research lives.
  • Solution space: We already have something (an idea, a prototype, a product) and we want to know if it works. This is where evaluative research lives.

If your question is "what do our users need?" → you're in the problem space.
If your question is "does this design solve the problem?" → you're in the solution space.

Double Diamond design model: two diamonds side by side on an axis that runs from uncertainty to certainty; the first diamond is where the problem is defined, the second is where you experiment with the solution

What development stage is the product in?

Another way to see it is through the product lifecycle:

StageType of researchTypical methods
Concept/IdeationGenerativeInterviews, ethnography, desk research
Design/PrototypeFormative evaluativeUsability tests, card sorting
Development/Pre-launchFormative evaluativeTests with working prototypes
Post-launchSummative evaluativeAnalytics, surveys, A/B testing

This is useful when you talk to product teams, because it connects your research to their roadmap.

2. Identify the project stage: Generative or evaluative research?

Once you're clear on the objective, you need to know at what moment of the design process you are. This defines whether your research should be generative or evaluative.

Generative Research (Exploratory or Discovery)

It's used when you're at the beginning of the project, trying to understand the user, their needs, problems, and pain points. The objective is to generate new knowledge about who your users are and what they need.

Typical methods: in-depth interviews, ethnographic research, field studies, diary studies, focus groups.

Evaluative Research (Validation)

It's used when you already have something concrete (a prototype, a product, a hypothesis) and you want to know if it works. The objective is to validate or refute what you've designed.

Typical methods: usability tests, A/B testing, tree testing, heuristic evaluation.

The formative vs. summative distinction by Tullis & Albert

Tullis and Albert (2008, pp. 45-46) describe a useful distinction:

Formative Research: You collect data to improve the design before launching. It's iterative.

Summative Research: You measure how well objectives were achieved after launching. It's about measurement and benchmarking.

3. Determine the type of data you need: qualitative, quantitative, behavioral, or attitudinal

This is where it gets interesting. The choice of method depends on what type of data you're looking for. There are two dimensions to consider:

Qualitative vs. Quantitative

  • Qualitative data: Helps you understand the why and how. Depth over breadth.
  • Quantitative data: Gives you the how many and how often. Breadth over depth.

Attitudinal vs. Behavioral

  • Attitudinal data: What users say, their opinions and perceptions.
  • Behavioral data: What users do, their actual behavior.

Here's an important point I learned from Travis & Hodgson that changed my way of thinking:

"Behavioral data is strong. Opinion data is weak."

This doesn't mean that surveys or interviews aren't useful, but that if you can observe the user "in the act" doing something, that data is more reliable than asking them later what they did or what they prefer. People don't always know how to explain their behavior, or they rationalize it afterward.

My learning: I used to prioritize interviews a lot because I felt comfortable with them. Now I try to combine: if I do interviews, I also include observation (Google analytics, heatmaps, etc.) or a test where I can see what they actually do, not just what they say they do.

4. Evaluate practical constraints: time, budget, and resources

We reach the uncomfortable point. You're clear on your objective, you know what phase of the project it is, you know what type of data you need... but you have 2 weeks and zero budget.

Feasibility is a critical factor. You need to consider:

  • Available time: How many days or weeks do you have?
  • Budget: Is there money for incentives, tools, recruitment?
  • Trained team: Who can execute the research?
  • Available tools: Do you have access to testing software, recording, analysis?

Here there's an interesting discrepancy in the literature. Savarit says something that stuck with me:

"Even if some people tell you that you choose your research methods depending on your budget, I disagree: you need to figure out first what you want to find out based on your research brief; then the researcher will be able to decide which research method will be the more appropriate to answer your questions. Only then can you get a quote or estimate. Not the other way around." (Savarit, 2020, p. 92)

This seems important to me. Objectives must be defined first, and then constraints limit the options. But never the other way around: don't choose a destination just because the bus is cheap.

My learning: This was hard for me to accept. In my first investigations, I was very strict with methodological rigor (sufficient number of participants, complete transcriptions, always the best methodology). Now I understand that the best methodology is doing what's possible with available resources. Quick guerrilla testing is infinitely better than perfect research that never gets done.

5. Consider access to representative participants

The right profile is vital for the validity of findings. You need to make sure that the people who participate are representative of your target users.

How many people you need depends on what you want to know:

To find usability problems: 5 participants per round, as long as they are comparable users. Jakob Nielsen recommends several small rounds over one large study, and with very different user groups, 3 to 4 people per group when there are two and 3 per group when there are three or more (Nielsen, 2000). Watch out for rare problems: one that affects 10% of users has only a 41% chance of showing up in five sessions. The problem discovery calculator runs that calculation for your case.

For interviews and other qualitative studies: there is no number you can fix in advance. Sample size is justified by the study aim, sample specificity, established theory, quality of dialogue and analysis strategy (Malterud et al., 2016), and it is reviewed during fieldwork against saturation. The qualitative sample size tool covers both moments: before fieldwork, to justify your sample, and during fieldwork, to assess saturation.

To measure (quantitative research): Nielsen Norman Group recommends 40 participants for most quantitative usability studies (Budiu & Moran, 2021). That number comes from precision, not statistical significance: with 40 people, a binary metric such as success rate carries a ±15% margin of error at 95% confidence. If you accept more risk, it goes down: 28 participants at 90% confidence, or 15 if you also accept a ±20% margin. The quantitative usability calculator runs the numbers with your data.

If you don't have access to real users, consider alternatives like proto-personas or guerrilla testing, but be aware of the limitations.

6. Define the context of use and evaluation format

Finally, you decide the logistical details that affect data quality:

  • In lab or in the field? If the user's real environment is critical, it's worth going to their context.
  • In-person or remote? Remote is more accessible but loses some nuances.
  • Moderated or unmoderated? Moderated allows you to go deeper, unmoderated scales better.

Step-by-step process: how I choose methodologies in my projects

I'll share what this process looks like in my day-to-day as a UX Researcher:

Step 1 - Kickoff meeting: I arrive with questions, not answers. I ask what they need to know, what decisions they'll make, what happens if we discover different scenarios. Sometimes a research project doesn't continue because at this stage we realize it wasn't necessary to do it, which is much less costly than having executed it and at the end realizing it wasn't necessary, or that it couldn't be implemented, or that we didn't have clarity on the target group, or... the list is very long.

Step 2 - I define the research question: I write it and refine it until it's specific and actionable. This isn't something stakeholders need but you as a UX Researcher, it's your map for the journey. One thing is what the business needs to know, another is what the UX Researcher needs to go find (in clinical psychology we would distinguish it as: reason for consultation versus therapeutic objective, both are different, but we relate them in the therapeutic process).

Step 3 - I identify the stage: Are we in discovery (generative) or validation (evaluative)? There are data collection methods that are more suitable for one stage than another.

Step 4 - I choose the type of data: I try to prioritize behavioral data when possible, complemented with attitudinal. But the priority is the research objective.

Step 5 - I review constraints: Time, budget, access to users. (This would be our reality check criterion)

Step 6 - I select the methodology: Only here do I open the methodology selector or my mental list of options.

Step 7 - I plan the logistics: Remote/in-person, moderated/unmoderated, number of participants.

Pro tip: combine methodologies

You don't have to choose just one. In fact, combining methods usually gives better results:

  • Benchmark + Interviews: First you map what the competition is doing, then you dive deep with users to understand what really matters to them.
  • Survey + Usability Test: The survey gives you the "what" at scale, the test gives you the "why" in depth.
  • Analytics + Interviews: Data shows you where they drop off, interviews explain why.

The typical combination is: one quantitative/behavioral method + one qualitative/attitudinal method. This way you get both breadth and depth.

The most common mistake when choosing UX research methods (and how to avoid it)

The mistake I see most (and that I myself made many times) is jumping directly to step 6: "Let's do interviews" or "Let's do a usability test" without going through the previous steps.

The result is usually research that delivers interesting but not actionable information, or that answers questions nobody was asking.

The solution: Before proposing any method (or someone else requesting one), make sure you can answer:

  1. What specific question are we trying to answer?
  2. What decision will be made with this information?
  3. Are we exploring or validating?
  4. Do we need to know what users do or what they think?

Free tool: UX Research Methodology Selector

The Methodology Selector I built tries to help with steps 3-6 of this process. It allows you to filter by phase and goal, type of data, effort, experience level, access to participants and format to find methodologies that could work for your case.

But the tool doesn't replace steps 1 and 2. Those you have to do yourself, with your team, with your stakeholders. And they're the most important.

Conclusions: 4 principles for choosing UX Research methods

  • Define the destination first, then choose the vehicle. Research objectives always come before methodology.
  • Behavioral data is stronger than opinion data. Observe what they do, not just what they say.
  • The best methodology is the one you can do. Methodological rigor + real constraints = smart pragmatism.
  • There's NO ONE best way. Each project is unique, and your background also brings a different perspective.

I hope this article helps you in your own process of choosing methodologies. If you want to continue the conversation about this, write to me on LinkedIn.

Frequently asked questions about UX research methods

What is the best UX Research method for beginners?

For those starting out, I recommend beginning with in-depth interviews (for generative research) and think-aloud usability tests (for evaluative research). They're versatile methods, relatively simple to execute, and give you a lot of learning about how to interact with users.

How many users do I need for UX research?

It depends on what you want to know. To find usability problems, 5 participants per round with comparable users, and several small rounds beat one large study (Nielsen, 2000). For interviews, there is no fixed number: sample size is justified before fieldwork and reviewed against saturation during it, and the qualitative sample size tool walks you through both steps. To measure usability, the usual recommendation is 40 participants, or fewer if you accept more risk (Budiu & Moran, 2021). Surveys and A/B tests have their own calculation, which depends on population size or on the difference you want to detect: the sample calculator handles them.

What's the difference between generative and evaluative research?

Generative research is done at the beginning of the project to discover needs and opportunities (what should we build?). Evaluative research is done when you already have something designed to validate if it works (does what we built work?).

Is qualitative or quantitative research better?

There's no "better" one. Ideally, you combine them. Qualitative gives you depth and the "why" behind behaviors. Quantitative gives you scale and allows you to validate patterns. A good strategy is to use qualitative to discover hypotheses and quantitative to validate them.

Bibliographic references

Related articles:

Related Articles