Scientists, designers and The Truth

This feels heretical for a user experience designer to say, but it’s been bugging the hell out of me for a couple of years, so here goes…

Design research is not about uncovering The Truth.

We are not getting at The Truth when we do user interviews, usability tests and contextual enquiries. What we’re doing is trying to learn enough to Make Something Better. This is a slightly more ambitious version of making something happen, something I've written about before, a couple of times.

This doesn’t mean I don’t value design research. It’s hands-down the most important part of my job and will continue to be. It’s just that it’s not scientific research, or hard evidence, or proof, or reflective of some kind of universal truth.

And I’m tired of hearing people talk about it as if it is.

The difference between science and business data

Jaron Lanier has a great passage in his otherwise-flawed book Who Owns The Future? where he talks about the differences between the use of data in the scientific/academic world and in business.

I am going to grossly oversimplify here, but his point is that scientific research and business research seem superficially similar because they have attributes that you can easily map from one to the other:

1. They are both done by ‘researchers’
2. They both involve a methodology to eliminate biases
3. They both involve collecting data, statistical and/or qualitative
4. They end with a statement of truth.

But these superficial similarities mask two vastly different realities.

Scientists search for The Truth

Scientific research is concerned with uncovering The Truth.

Scientists, the people searching for The Truth, spend decades working on incredibly specific problems. It is not unusual for them to throw away years of research because of a tiny error in their methodology.

Their methodologies are designed to carefully isolate single variables (if they’re lucky enough to be physicists) or to compare small groups of variables (if they’re lucky enough to be biologists) or to attempt comparisons between huge groups of variables (if they have the misfortune to be economists). Go and read how Daniel Kahneman set up the experiments to find out the truths we all misrepresent fromThinking Fast and Slow. Meticulous.

Their collection of evidence is even more painstaking. I don’t know what you know about statistics, but statistical significance is only the tip of the iceberg. Scientists deal in measures of statistical validity that we have never heard of. Every measurement is understood in terms of error bars. When did you last see an error bar?

Finally, it comes to stating The Truth. This cannot happen without publishing everything - the hypotheses, the meticulous details of the methodologies, the raw data collected during all the experiments, even the calculations used to process the data.

Only then can this research claim to be The Truth.

And still it will be attacked and probed and questioned by experts from all over world. In fact, the very premise of being The Truth requires you to accept it always remains a hypothesis waiting to be 'falsified’ by another, better-designed experiment.

That’s how scientific research finds The Truth. It literally takes centuries of painstaking specific detail being added to painstaking specific detail. Standing on the shoulders of giants and all that.

Now, let’s compare that to design research…

Designers fight to make something better

Design research is about fighting to Make Something Better. Because if we don’t fight, nothing will happen, and it’s all a waste of time.

Designers, the people fighting to Make Something Better, do not just work on one incredibly specific thing. We definitely do not have years and decades to Make Something Better. If you’re lucky you’ll have months, but usually it’s weeks or days.

Our methodologies are designed to fit inside a few months (if you’ve got an enormous budget), a few weeks (if you work at a user-centred design consultancy) or wherever the hell they can among the chaos that is working in a business (most of the rest of the world). They are not designed to isolate variables beyond cursory attempts to eliminate the biggest biases and leading questions. When was the last time you threw away your research because of a methodological error?

Our collection of evidence is haphazard. Where statistics are used, the sample sizes are incredibly small, there is almost never any use of error bars or confidence intervals. Where we go qualitative, interesting quotes are cherry picked to make specific points. Patterns are detected and amplified into stories because that’s what you need to Make Something Happen. In all honesty, our research is rarely subjected to even socialscience levels of rigour.

Finally it comes time to Make Something Better. We might summarise our goals (changed many times throughout the project), show a high level outline of methodologies (usually in bullet points), publish a tiny fraction of the raw data to a tiny audience (some quotes and video clips) and make some massive generalisations (like design principles).

Only then can this research start to Make Something Better.

And still the likelihood is it will be glanced over, forgotten and nothing will happen. Because it’s a messy world and there are people.

That’s how design research gets to Make Something Better. It’s a desperate search for answers to too many questions followed by a headlong dash to assemble them into something as convincing as you can possibly muster.

They’re different, and that’s OK

So, scientific research and design research sound similar when you talk about the nouns (researchers, methodology, evidence, data, analysis, conclusion) but they are actually very different when you fill in all the verbs (share/summarise, publish/present, prove/convince).

None of this is to say that one method is better or worse than the other. They are different. They have different goals.

If you want to find The Truth, don’t use design research because it will be falsified by a high school student in minutes. She’ll tear apart your methodology ruthlessly.

If you want to Make Something Better, don’t use the scientific method because you’ll still be working on the first premise long after your company goes bust.

The correct response to this is NOT to try and make design research 'more scientific’. Yes, we should attempt to eliminate biases but we must never lost sight of the fact that we are here to Make Something Better. Finding The Truth just takes more time than we have available, and is mostly unnecessary in any case.

If you want a stark reminder of the differences between our two different worlds, remember that it’s OK for scientists to use Comic Sans. When physicists at CERN found the Higgs’ Boson their choice of typeface was mocked in the design world, but it didn’t invalidate their work. Because they had found The Truth and history doesn’t care about the typeface that the The Truth is written in.

Try doing that as a designer.

Making things better is hard enough

I’m not saying we shouldn’t do research. I’m definitely not saying we shouldn’t try to avoid biases. I’m just saying we need to be aware of the differences between these two superficially-similar methods. They are both types of 'research’, yes, but you shouldn’t be using language from one method if you’re actually deploying the other.

For example, I get sick of hearing designers saying they have 'proved’ something with user research. For designers, research is about action, not proof. Tell me how your design research led to something better happening in the world, not what you think you 'proved’. All our proofs are subject to change. Times change. Contexts change.

This kind of lazy talk leads to something else that I am growing weary of, which is hearing clients demand that we find out The Truth. We should come clean: we don’t have access to The Truth and we should stop talking as if we do.

Because our truth is we don’t need The Truth to do great design work.

What we need is a bunch of design research tools to Make Something Better. Because design is usually a wicked problem, not a simple equation, and 'better’ is the best you can do with wicked problems. There is never a 'right’ answer. The Truth is an expensive mirage.

And actually, for most businesses (and charities, and governments, and in fact nearly everyone) even just being able to Make Something Happen is the most precious thing imaginable, because it’s in the doing of new things that people learn how to do newer and better things themselves in the future. Teach a woman to fish and all that…