User research is a team sport
‘User research is a team sport’ is the most powerful thing I’ve learned in government. Forget user needs. Forget discovery, alpha, beta, live. Forget agile. Forget the service standard and service assessments. This is the one thing I would save in a fire.
I love ‘user research is a team sport’ for two reasons. First, it increases the impact of our user research on the services we make. Secondly, and less obviously, it rapidly improves our research practice as individuals. These are precious outcomes.
Yet recently I've heard some worrying interpretations of what ‘user research is a team sport’ means. That it means anyone in a team can and should do user research. Not true. Or that it means user research is not a specialist skill. Also not true. I’ve even heard stories of ‘user research is a team sport’ being used to shame user researchers for not sharing every aspect of their work. Unacceptable.
These interpretations are starting to spook user researchers into disavowing ‘user research is a team sport’. Given the precious outcomes, this is a tragedy.
So I’m going to talk about my own experiences of ‘user research is a team sport’. I’ll show why it’s powerful. I’ll show that it goes hand in hand with being a specialist user researcher. But most of all I’ll give you a detailed guide of how to make it work in practice so you can do more of it yourself.
Big-reveal research doesn’t work
I used to be that researcher that swans in and thinks their job is to save the client with user research. Elaborate methods, psychological concepts, scientific analyses, fancy graphics, flashy presentations. This made me feel heroic, knowing and enlightened.
The problem was that it didn’t work. Little changed after I left the room. No matter how ‘right’ or compelling or evidence-based my findings felt to me.
I slowly realised that change in organisations comes from change within other people’s minds. Not my own. Most people only change their mind when they experience things themselves. People don’t respond well to being told what to do.
So I changed my research practice to be more collaborative. This worked well. And then, inspired by ‘user research is a team sport’, I decided to go in-house with GDS.
Involving the team in research does work
I spent the first six months at GDS working out how to apply ‘user research is a team sport’ in practice. On GOV.UK Notify, through gradual trial and error, I found ways to involve the team in five critical areas of the research process:
Generating research questions
Observing sessions
Taking notes
Attending playbacks
Prioritising findings.
As I did this, I watched my research make changes to a real product for the first time in my career. The team responded to the research without being told what to do. It finally clicked. If you involve the team in the research, in a way they understand, they will make better decisions whenever they make decisions. Not just when you’re there.
Something else happened too. The team began to share their thoughts about ways to improve the research. This group of non-researchers helped me learn some of the deepest lessons I’ve learned about research. Stop researching when you’ve learned the big things. Pick the riskiest things from the roadmap first. Know that some things are OK to ignore completely because they’re easy to fix later if they blow up.
This is the power of ‘user research is a team sport’. It increases the impact of our research and it improves our research practice at the same time. Precious outcomes.
A field guide to ‘user research is a team sport’
I want to save you from the trial and error I went through in working out how to apply ‘user research is a team sport’. To do this I’m going to step through the four stages of the user research process and show how it worked for me:
At each stage I’ll talk through the most productive ways I found to involve my team in the research. I’ll also show that specialist researchers are central to ‘user research is a team sport’ by covering the things that I did on my own, away from the team.
1. Planning research as a team sport
Planning research is about coming up with research questions and turning them into a research plan made up of different activities. It’s also about arranging participants, a location, a discussion guide and your research materials.
These were the best ways I found to involve my team in planning research:
Generating and prioritising research questions. Every three months I would drag my whole team into a room for an hour and ask them what they wished they knew about their product. They wrote questions on post-its, clarified what these meant, and then prioritised them. These sessions took a little practice to get right but ensured that my research was relevant to my team’s needs.
Discussing the riskiest items on the roadmap. Once I’d won the trust of the team I found ways to meet regularly with the product manager, technical architect and designer to understand what was coming up. Research has to work a little ahead (all that planning!) and I wanted enough time to research the riskiest assumptions. Looking ahead helped me avoid blocking the team’s progress.
And these were the things that I did on my own:
Finalising the research questions. My team came up with loads of research questions in a million different formats. It was my job, alone, to synthesise my team’s questions into a coherent research plan.
Choosing research methods. My team were not experts in research methods. I knew what methods we could use to answer the research questions in our research plan. I tended to use simpler methods to bring my team along with me.
Recruiting participants and arranging the space. Any time there was detailed, logistical, grunt work this was my job alone. That was OK with me. My team had better things to do than help with with research admin.
Preparing the discussion guide and research materials. I’m the one that knew how to write a discussion guide that answered our research questions, that moved from broad to specific, and that left enough room for improvisation.
That’s what it means to me to plan research as a team sport. You invite your team to help you choose the research focus and then you - as a specialist user researcher who owns the process and is accountable for the results - do the detailed work to prepare for success on the day.
2. Doing research as a team sport
Doing research is about running sessions with participants. You might be in a lab or in someone’s house. You might be interviewing or testing or observing. There is always someone moderating the session and some other people observing and taking notes.
These were the best ways I found to involve my team in doing research:
Observing research sessions. Everybody on the team observed some research. For lab sessions I invited the team into the observation room. When we went out in the field I invited a non-researcher to come with me. I even combined the two, putting the team in the lab’s observation room and streaming back sessions live from the field. It’s important to note that I never insisted the whole team spend the whole day observing. One or two sessions each because people are busy.
Taking notes in research sessions. Observing was great but people got bored and started checking emails unless they had something to do. So I made them take notes. I set up a different person as the primary note taker for each session. I asked other observers to note down the five most important things they saw at the end of the session. This made sure everyone paid attention and had a voice in the notes. It also helped me jump-start analysis many times.
And these were the things that I did on my own:
Moderating the research sessions. It has taken me years to be a passable moderator. Leading questions. Body language. Neutral confirmations. Knowing when to probe. Getting consent. Dealing with ethical situations. Navigating dodgy prototypes. Slipping into role playing. I am still learning all of this.
Ensuring notes were high quality. My team wrote terrible notes at first because I hadn’t found a way to explain that I wanted observations (what individuals do) and quotes (what individuals say) but not findings (what clusters of people do) or solutions (what changes we should make). Over time I coached them into writing better notes. Until then I re-did the notes myself where it mattered.
Storing research materials safely and securely. My team were not trained to worry about the ethical implications of the personal information stored in our recordings. Nor were they accountable. It was my job to store things safely.
That’s what it means to me to do research sessions as a team sport. You invite your team to observe the sessions but you - as a specialist user researcher who owns the process and is accountable for the results - do the moderation and ensure that the notes and recordings are dealt with appropriately.
3. Analysing research as a team sport
Analysing research is about taking the observations from your research sessions and turning them into compelling findings that prompt your team to make better decisions.
The analysis stage has the trickiest relationship to ‘user research is a team sport’. Bluntly, I tell less experienced user researchers to do analysis on their own until they’re 100% confident in their own approach. But even for experienced researchers there are two reasons that analysis is tricky as a team sport:
Analysis is time-consuming. It easily takes as long as the research sessions themselves but you can’t divide it up between different team members like with research sessions. Having the whole team spending a day is usually unjustifiable.
Analysis is a subtle, personal, intuitive process. The best analyses I have done were rambling journeys. I used my intuition to discount or emphasise things. I inferred things beyond the visible data. I was influenced by outputs that I imagined. I know this makes analysis sound like weird magical alchemy. I’m sorry. Analysis has always resisted my attempts at clear, rational explanation.
That said, these were the best ways I found to involve my team in analysing research:
Quick and dirty usability analysis. In sprints I often needed findings the next day and wasn’t bothered about being comprehensive because I knew we’d be back in two weeks. So I’d invite multiple team members to each session and ask them to write down the five most important things they saw for each participant. The next day we would cluster these, name the clusters, rewrite as findings and prioritise as a team. This flushed out 80% of the big problems. The team ended up so bought-in that they often didn’t even need a playback.
Transcript-based exploratory analysis. Occasionally with depth interviews I’d invite the team to a multi-hour analysis session. Each person went through a transcript, highlighted quotes, and passed the transcript on until each transcript had multiple eyes on it. Then each person wrote up the 10 (ish) most compelling passages onto post-its for the transcript they were holding. Again, we would cluster, name, write findings and prioritise. This was a great way to immerse the team in discovery research but it wasn’t something I felt able to do often.
Remote tagging analysis. Sometimes I used the team to tag a large volume of research data. This took a bit of preparation on my side because I needed to make sure the tags worked and explain how to apply them. When it worked it exposed the team to research they would not have otherwise paid attention to.
And these were the things that I did on my own:
Preparing materials for analysis. Whether it was sorting through the post-its generated by my team, cutting up my own notes onto index cards, or transcribing my recordings, there was a lot of solo work preparing for analysis. This work remained mine whether I was doing analysis on my own or with the team.
Hunting down that last 20%. Finding only the biggest problems was OK some of the time but most of the time I needed to go deeper. In these situations I would do the hard, deep work of analysis myself. Or I’d take what the team had done and then go back over it to fill in the gaps.
Writing clear research findings. Throughout analysis I am turning noun-based clusters of observations (e.g. “password problems”) into clear finding statements (e.g. “the password rules are so complex people can’t remember what they entered”). This last step - writing a bold, clear research finding - is the critical moment in my research. It’s something that only I will ever take responsibility for.
That’s what it means to me to do analysis as a team sport. Most of the time you do analysis on your own, but sometimes you involve your team. Either way you - as a specialist user researcher who owns the process and is accountable for the results - prepare the materials and take responsibility for writing clear research findings.
4. Communicating research as a team sport
Communicating research is about taking the research findings and inserting them into the minds of the people making decisions. This might be standing in front of a wall telling stories, it might be presenting slides, or it might even be a big old report.
These were the best ways I found to involve my team in communicating research:
Whole-team dedicated research playback. Every two weeks I would gather the whole team for 30 minutes and share findings from the research. This always felt like it came around too often but it forced me to get better at sharing findings earlier. It also helped me see whether the research findings had impact. There’s nothing like watching someone’s eyes close as you are talking to them.
Prioritising research findings together. When my team bought into the findings prioritisation they responded to the research. So I ended every playback with an invitation to correct my prioritisation. These conversations forced the team to reveal what mattered most to them. It also meant that I could walk away at this point and give the team creative freedom to solve the most important problems.
Critiquing the user research process. I used to run a research retro every four or five playbacks. I asked each team member to post up what worked well, what didn’t work well, and what they would change about the research. This was exposing at first - it’s hard listening to non-researchers critique your practice - but it ended up as my favourite part of the whole research process.
And these were the things that I did on my own:
Doing the first-pass prioritisation of findings. I would often have upwards of 50 findings from analysis. But I knew my team would rarely pay attention to more than 10. It was my job to work out which 10 findings to share. This is an art.
Preparing a compelling playback. My team were often grumpy about being collared to listen to research for half an hour. So I spent a lot of time thinking about better ways to communicate things to them.
Documenting findings for the future. I involved the team to make sure the research findings got into their heads. But I also had to document the most important findings so that they were available for other teams or future members.
That’s what it means to me to communicate research as a team sport. You stand up and share your findings but only after you - as a specialist user researcher who owns the process and is accountable for the results - have prepared a compelling playback.
‘User research is a team sport’ is a big ask
That’s my field guide to doing ‘user research is a team sport’. I have to be honest with you though. Doing research in this way is hard work. It means we need these things:
Confidence in our own abilities. Once we open up our practice to our teams they ask difficult questions. We need to feel secure about our core practice.
Clarity in our communication. Whether coaching our team to take better notes or preparing for our playbacks, we need to be able to express ourselves clearly.
Ease with facilitating groups. For many researchers this is the hardest part. Our teams are not expecting to take part in research. We have to find ways to make it engaging, useful and fun. This means being able to facilitate reluctant groups.
Humility to accept we’re often wrong. The smart people we work with will show us many ways where our approach isn’t right. We need to be able to hear this.
These things are hard for us. Lots of us are introverted, feel anxious, or have imposter syndrome. Opening ourselves up to critique, or dragging grumpy people away from their screens, is challenging. This is a tough journey for all of us.
It’s the right journey though. At a certain point, being a better user researcher is not about learning new research methods, it’s about finding ways to increase the impact of the work you are doing. That’s what ‘user research is a team sport’ is about.
There’s one more thing that is hard. It can feel like we are giving up control of the quality of the research. This is not even close to being true. The researcher finalises the research questions, creates the research plan, writes the discussion guide, moderates the sessions, has the final say on the notes, determines analysis approach, and is the one who writes research findings. Our control remains intact.
There are many ways to play our team sport
The stories I’ve shared here are my personal take and I’m often wrong. Yes, I’ve seen this work for me and the researchers I’ve supported over the years. But time and again I’ve learned from other researchers who have different ways of doing things and who disagree - often vehemently - with the things I say.
That’s OK. There is no One True Way. All models are wrong, but some are useful.
I wrote this partly to make it clear that ‘user research is a team sport’ is not at odds with being a specialist user researcher. These things go hand in hand.
But mostly I wrote this to pass on a framework for thinking about where to involve our teams and where to work alone. ‘User research is a team sport’ is the most powerful thing I’ve learned in government and I’d love to see more of us doing it...
Let me know what you think on @myddelton. And thank you to the researchers, developers, product managers, designers and other team members I’ve worked with. You taught me these things, one halting step at a time, and I’m forever grateful.