You have been tasked with mapping out a territory. From all of the initial input from your client, views of the territory using existing maps, and a couple of fly-over excursions let us say that you have so far determined that this territory is big and densely and impenetrably green and that most people would call it a jungle. Now, how do you begin to understand what is happening on the ground?
Perhaps you value an experiential approach – you don’t want to project your bias or your client’s bias into the findings. To get these data, you hire a group of individuals, equip them with basic supplies, drop them off in various locations, tell them to radio in what they have seen each day, to take good notes for later discussions, and to meet-up in the middle of the jungle at the end of the week.
Your team members each venture into the jungle without knowing exactly what the task is at hand while knowing that they only have a week to do it. Some members, timid or methodical, stay near the edge, while others, anxious for adventure or restless, strike out for the heart of the wilderness. Each member knows that they are there to be observers and that their observations will inform some important scientific discovery, so, they do their best to perform the task as best as they think they understand it. One individual is in awe of the flowers and ferns he sees; a couple of them are struck by how vast the jungle seems to be; most of them are concerned that they do not know where exactly the pick-up site is; all of them are deeply worried by the growls coming from the jungle at night; and, none of them really understand the point of the trip or the dangers of the jungle.
After a couple of weeks have passed you notice that your team no longer radios in, and no one has arrived at the pick-up site.
The summary report to your client is thus: Several features of the territory located and described. The team and equipment assumed to be lost. Further research required . . .
In all estimations this project is a failure. Your report cannot properly discriminate or identify common objects across reports; it cannot contextualize the impact of each report; and because you have almost assuredly lost your team, as well as all of the equipment, you have no idea how to effectively repeat, and thereby expand or corroborate the study because there is no one left to report what worked and what did not.
This apocryphal tale seems absurd, but it details the method most often (re)produced in market research. In many market research studies the researcher will gather a panel of subjects to inform his/her study; the researcher will then provides each subject with a starting point — “Today, we are here to discuss frozen yogurt.” — and then the researcher will probably offer a set of discussion topics — “What flavor do you like?”; “Where do you shop?”; “What price would you pay?”; “What advertisements are appealing?” etc… The subject does not know how to navigate this territory; s/he is not aware of the pitfalls that exist within it; s/he does not know the purpose for entering into it in the first place; and, s/he has no idea what the endpoint looks like. The researcher has just signed over his or her subject to the mercy of the jungle: “Here you go kid, this is the edge of the wilderness and this is your compass: good luck, and try not to get eaten by a grue.”
The summary report on frozen yogurt to your client might read like this: The participants liked vanilla most, many seem concerned with the number of calories, and nearly all shop for frozen yogurt between 4-5 miles from home. Further research may be required . . . and, luckily, casualties with this study were not so high and everyone was able to leave eating their favorite flavor of frozen yogurt.
Many market research reports, like the one above, fail in the same ways that the report in the jungle story failed. The report is fragmentary, improperly contextualized, and comes with such a poor treatment of methods that there will be no telling why the study was successful or unsuccessful — which also happens to mean that the research cannot be reproduced reliably. The reader is not equipped to evaluate the report, just as the research participants are not equipped to add maximum value to the data collected. As just an initial battery of issues with the above report, consider the following outstanding questions: did the participants prefer vanilla because it is truly their favorite flavor, or because vanilla is a social default, or because they are allergic to chocolate? Are participants concerned with calories because frozen yogurt is a dessert item, or do frozen yogurt eaters, as a group tend to be more health conscious? Is distance to home the proper distance anchor, or is distance from parks, work, movie theaters, or other social venues more appropriate? While the data above are not incorrect or useless, they fail to provide any reasonable assistance in navigating the territory in question; moreover, no actual case of yogurt buying was related in the report – whatever that story may have been, it did not make it out of the jungle.
Improving the fate of the subjects in the jungle story is straightforward enough: you need to hire a guide. Therefore, let us reimagine the story in the following way:
With the same task in mind, and still valuing an experiential approach, you hire a group of individuals, but this time you pair each individual with a fully equipped guide. The guide, who may not have ever been in this jungle, or these particular 200 square miles of this jungle, is still an expert scout. S/he will be able to keep the subject moving, avoid being eaten, read the jungle in a way that allows the subjects’ observations to be tracked, and deliver the subject to the pick-up site by the end of the week safely (that is, to avoid being eaten)… so, effectively, the logistics of the excursion are accounted for, and the subjects can focus on their role, as they perceive it.
The team members in this story still venture into the jungle without knowing exactly what the task is at hand. However, this time the subjects move through the jungle with some regulated pace – the guide keeps each person moving each day, within his or her own limits. While still uncertain, each observer is reassured by the scout that in fact they have merely crossed the same river twice, or is encouraged by the scout that in fact the observer has noticed some truly novel species. Additionally, even though the subjects are unaware of it, the scout methodically marks the distance covered each day and the location of each observation. The uncertainty of each observer is eased by the presence of the scout and the observers are able to proceed through the jungle calmly unaware of all of the danger that surrounds them. At the end of the week the observers are collected at the pick-up site and each one is chatting in an animated manner about his or her experience in the jungle.
Market research can and should be conducted in the same way. Proper exploration of new territory requires a seasoned guide. In market research this takes the form of a researcher who has studied research methodology extensively, and especially individuals who have studied cognitive psychology. Experiential knowledge is useful to an extent, but, going back into the jungle for a moment, consider which guide you would prefer to trust yourself to:
A: “Oh, I have seen a root like that before. It could be edible, but the last time my group members ate something like that five people died. We should probably avoid eating it.”
B: “This plant seems to be related to the well known Manihot esculenta, that is to say “cassava”. To avoid possible poisoning by cyanogenic glycoside,we need to boil this thing, if we are going to eat it.”
Everyone wants guide B, because with guide A you are always worried that you are going to be his or her next learning experience. The level of detail that guide B goes into is possible in market research if the researcher has a strong research background. Many of the pitfalls that sidetrack (and often consume!) research subjects are cognitive biases that subjects have acquired over time in order to navigate their world with minimal cognitive stress. Most of these heuristics are invisible to the subject. A basic search for information on cognitive biases will yield a long (very long) list; for illustrative purposes, we will describe only one below.
Confirmation bias is a cognitive bias that pervades our process of searching for information and accessing stored memory. When recalling information in particular, people have a tendency to prefer information that confirms previously established beliefs – this is especially true for self-image reports. When subjects are asked to report information about frozen yogurt buying habits, they will preferentially report the information that confirms their beliefs about themselves: I am a strong willed, health-conscious parent; my children are well behaved, and I am budget-oriented. The weight of these beliefs will lead the subject to report all of the times that his or her children finished their homework early and bought (and shared) a small frozen yogurt with their own saved-up allowance money, and to completely ignore the times that he or she drove an extra ten minutes out of his or her way to get frozen yogurt two hours before dinner-time, and ended up buying two cups instead of one because his or her kids were fighting and would not share. The latter is not reported because it is an event which seems to disprove the absolute nature of the subject’s strongly held beliefs. In the latter case, the parent was inconvenienced, bought frozen yogurt near mealtime, and spent twice as much as normal. This process is not a conscious or active manipulation, the subject simply remembers the confirmatory set of events more often and more strongly over the dissenting set. Have you ever wondered why so many people seem to report having adorable, perfectly behaved wunderkinder?
There are many biases out there that affect how subjects respond to a researcher. Researchers with a background in cognitive psychology and research methodology, who expect these reportative distortions, are able to develop research plans that do not inflate and thereby confirm these biases. Additionally, for those biases that cannot be directly avoided in the process of data collection, these properly trained researchers are also much better equipped to identify biases that surface in the analysis of data. While an experienced researcher without this background may be able to approximate these skills and this level of sophistication, there is one final advantage to having an established background in cognition and research methods that cannot be matched. In the above example comparing the jungle guides and the ‘cassava’ root, guide B does something that guide A cannot. Guide A summarily avoids the root in fear, while guide B engages a potentially poisonous plant and renders it extremely useful (cassava is a basic diet staple for millions). Understanding cognitive biases well enough to identify them in action allows the properly trained researcher to interact with them, in real time, to yield incredibly rich data – especially concerning norms and expectations. Like boiling the cassava root, you have to have the know-how to turn the otherwise toxic material into something bountiful.
The Paradigm2 Research Team has a knowledge base that is rooted in cognitive psychology and market research. From this knowledge base the Team has pioneered the use of qualitative research for small businesses, making it possible to access hard-to-reach populations and generate deep insights through innovative methods. Some of these methods were developed and published by the research staff, as we continue to change the way high-potential organizations think about their marketing research, competitive intelligence, and strategy development. If you would like to contribute to this discussion or talk to the Paradigm2 Research Team about your research needs, please contact us here.