Three types of user research

I’m a lead user researcher at GDS. My job is to support our user researchers in helping their teams make better products. This means I think a lot about the relationship our product teams have with user research.

Over the last two years I’ve stumbled into a useful model for talking about this relationship with researchers and their teams. The model helps them understand what to expect from each other, recognise and support each others’ strengths, and work together to make better products.

The model? There are three types of user research product teams should care about:

Testing things the team have built
Working out what the team should build next
Understanding potential users and their lives

Not revolutionary. Not innovative. Just helpful in doing the trickier parts of my job. Like adding a new user researcher to a team, making the role of our user researchers clear, and helping our teams get value from research.

1. Testing things the team have built

For many people, doing usability testing on existing software is all there is to user research. It is the easiest type of user research to grasp. So this is where we start.

The benefits are clear. It helps find and fix problems before you release software to the world. It is easy to get started for researchers (easy to learn) and for teams (easy to prepare to do). It fits neatly with agile because you can slot it into every sprint.

The problem is harder to see. Building software costs lots of money. Even in agile. This sunk cost makes it hard to tackle the big problems hard-coded in architectural decisions. This limits the scope of designers to make necessary changes. At its worst, teams make tiny, insignificant changes and ignore the bigger, more important stuff.

User researchers and their teams must be alert to this. If big problems are showing up in testing, teams should be spending more of their research effort on working out what the team should build next...

2. Working out what the team should build next

Research can help you figure out what to build before you start building. This mostly involves interviewing users about specific tasks and watching them use prototypes.

The benefits are strong. You understand the big architectural choices before sinking money into building them. You empower designers by giving them space to explore multiple options and do rapid iterations. This leads to huge progress, quickly.

There are two problems that block teams from doing this well:

At one extreme, teams avoid this work. Maybe because it doesn’t fit neatly with agile. Or because they won't invest their developers' time in prototyping.
At the other extreme, teams get stuck doing too much prototyping. They want all the answers before they build. Prototypes can never give you all the answers.

This is the hardest research to get right. You must balance doing too little (too risky) against doing too much (too slow). You have to work closely with designers, technologists and product managers. There is an art to this.

Although it’s the hardest research to do right, it’s not the most harmful to get wrong. That dubious honour falls to not understanding potential users and their lives...

3. Understanding potential users and their lives

Understanding people that might use your product (not just current users) is about listening to their stories and observing their daily lives. Away from your product.

The benefit is ensuring you have a workable proposition. This research can trigger innovative ideas, establish whether a market exists, and even help you pivot to new users when you need to. It’s worth saying that the findings from this last a long time too. People’s lives don’t change as fast as your product. This is worth doing well.

The problem is that many teams don’t value this work. Early on, they’re sure their assumptions are right. Later, they’d rather avoid knowing their assumptions were wrong. They say this work is a waste of time because it doesn't focus on the product or lead to immediate action. This work doesn't fit neatly into sprints, either, and things that are hard to manage often get ignored.

Let's be honest though. User researchers can get too fascinated by people and their lives. We sometimes get lost in this work for its own sake. Maybe some complaints about this work being a waste of time are not baseless. We should think on that.

Whatever, the most frustrating thing about my career has been watching teams of talented people working on things with flawed propositions. User research doesn’t have all the answers, no, but remaining unaware of hard truths about people and their lives is a pretty reliable way to undermine your product.

OK. That's the model. As Simon Wardley would say, all models are wrong, but some are useful. Here are three ways this model is useful to me.

Adding a new user researcher to a team

Talking about these three types of user research is useful when a new user researcher joins a team. This happens a lot in my job.

If the team is starting a new thing, focus the user research on understanding potential users and their lives. No discovery should be building protoytypes, let alone working software, so there's no choice but to start here.
If the team is underway but doesn't appreciate user-centred design, focus the user research on testing things the team has built. All teams grasp this and it’s a great way to build the trust you will need later.
If the team is up and running smoothly with user-centred design, focus the user research on working out what to build next. In mature teams with good propositions, the biggest value that research can bring is to find the riskiest thing on the roadmap and work on that with the designer and product manager.

It’s helpful to have this clear focus when you join a team. Over time the researcher’s job is to expand to cover all three types of user research.

Making the role of our user researchers clear

I tell our user researchers that their job is to make sure that all three types of research are being done by their teams (except in discovery of course). Framing the role like this gives our researchers plenty of freedom to come up with their own approaches to their work. This has worked out well for us.

Beyond that I don’t much care what methods they use or how often they do research. I want to hear how they are balancing the three types of research on their team and whether their teams are blocking any of the three types. This is a good conversation.

Researchers tend to prefer one of the three types. Some are most comfortable testing, some love getting in there with designers and prototypes, and others just want to explore the wide open spaces of people and their lives. It's fine to have a preference, but I expect researchers in product teams to do all three well. So the three types are a useful tool for conversations about professional development too.

Helping our teams get value from research

Finally, talking about these three types of user research lets me have better conversations with product teams about how user research can solve their problems.

Some teams get obsessed with building things above all else. This can be costly as they’re not using design to de-risk development properly. I’ll ask them to slow down and go back to working out what to build next.
Other teams get paralysed working out what to build. This can be costly as they're wasting too much time on certainty. We are gamblers, not scientists. I’ll push them to take a punt so they can start testing things the team has built.
We’ve all been on teams with a flawed proposition. This is not only costly but also emotionally difficult. I think of James Baldwin, who said “not everything that is faced can be changed, but nothing can be changed until it is faced”. I think of Rebecca Solnit, who said “we prefer ugly scenarios to the pure unknown”. Then I’ll try and help teams face the unknown by understanding users and their lives.

These are difficult conversations to have. Honestly, I've not always been successful in having them. But it helps me, in these conversations, to have a model of the value that user research brings to our product teams.

Sharing this model with you

I have thought long and hard about sharing this model. Leisa Reichelt, who I admire more than any other user researcher in the world, flatly told me that we didn’t need yet another way to talk about user research when I shared it with her six months ago. So I went away and wrote about discoveries instead. Thank you Leisa.

But I haven’t been able to let this go. On my good days I think this is because it’s a model with universal applicability. On my bad days I worry that it’s because - like all designers - I’ve fallen in love with my own idea.

All I know now is that it’s been useful to me. Maybe it will be useful to you too.

Let me know what you think on @myddelton. One final thing. You may be wondering what is wrong with discovery, alpha, beta, live as a model. The problem is people end up equating each stage with a type of research. Given that teams spend most of their time in beta and live, it's harmful to think they should only be doing usability testing...