Are Study Participants Brain Dead?
We recently moved our office and during packing I stumbled across a collection of articles dating back 15+ years related to where market research failed to predict the ultimate success of a new-to-world innovation. I was under some pressure to declutter, but after thumbing through the folder I decided these articles would make the trek across town.
It struck me that there was a common theme among all of these authors. For many innovations, it is very difficult for end-users to even recognize that they even have a need for something.
Early concept testing for the microwave oven and the mobile phone was not positive. At that point in time, consumers could not imagine the need for a large metal box that cooks with radiation. Similarly, one of the articles described how consumers struggled with understanding the usefulness of a briefcase-sized phone used in the car. A more recent example is Google Glass. With the ubiquitous smartphone, what exactly is the relative advantage of Internet access via a pair of glasses?
As researchers, a common postmortem conclusion is that we didn’t ask the right questions, or that we didn’t “nail the insight.” I’ve conducted and observed hundreds of focus groups over the past twenty years and for the most we do a good job of asking questions. More often than not, I find that study participants are terrible at giving answers.
The more we learn about how the brain works, it reveals several shortcomings with traditional qualitative research in the evaluation of new-to-world product concepts.
We Rely on the Wrong People
Deeply articulating the benefits of a never-seen-before innovation requires a highly-imaginative person—someone who can see beyond the hear-and-now. According to the Myers Briggs Type Indicator®, we know that three-quarters of U.S. adults are not optimal for this task. It is possible to screen for the right characteristics, and focus on the quality of the participant vs. the quantity.
We also need to rethink the research process. The traditional approach to qualitative research is not optimal. Whether you are highly-imaginative or not, adults need think time. Hitting someone cold with a conceptual topic and expecting sparkling insights in 30 minutes is tremendously difficult.
Make it Tangible
The cognitive demands required to give feedback on something intangible are enormous. Most people find it difficult to visualize how a two-dimensional rendering can benefit them or incorporate into their life. Reaction to something tangible is much preferred—even if the early idea is made from construction paper and rubber bands.
A favorite sport among study observers is rating participants on their effectiveness. Yes, some participants are better than others. But, we need to put ourselves in the participants’ shoes and realize that what we ask them to do is not easy. No, participants are not brain dead. The problem is ours: we often fail to pick the right brains for the assignment and often do not set up the right environment so their brains can flourish. The science certainly continues to bear this out.