“If I can do a survey with 200 respondents, why would I want to conduct 10 or 12 in-depth interviews? Where’s the value? What can I expect to learn from such a small population?” Researchers who do qualitative studies hear these questions all the time. This paper is intended to address them, shed some light on the theoretical differences between quantitative and qualitative research, and explain how qualitative studies are often misunderstood.
Qualitative research is often criticized because its findings rest on data from a small sample of the population, sometimes a very small sample. This criticism is based on trying to understand qualitative research results through expectations that are based in quantitative research. This is a common misunderstanding associated with qualitative research. This misunderstanding has led to both an undervaluation of the strengths of qualitative research and the inability to determine the quality and usefulness of qualitative research findings. In this paper we will explain some of the fundamental motivational differences between quantitative and qualitative research methods, the kinds of questions for which qualitative research is better suited, and crucially, how to assess the quality of qualitative research findings. We also include a summary table following the discussion.
Quantitative research is all about variables and systematic relationships. In quantitative research, populations are simplified and translated into a table of properties or variables – which could be any observable unit of data. While many variables relevant to a study can be directly obtained through a simple survey question (e.g. age, education, job title, tenure, etc.), many others have to be operationalized – which is the process of anchoring an abstract concept to some measurable piece of data (e.g. employee productivity as defined by individual sales data). We will return to this process in our discussion of qualitative research. In quantitative research we often attempt to simulate the laboratory process where one can establish control cases and build test cases, and then employ the scientific method to discover the independent variables at play. Concerning social phenomena, including business, where the research is very often conducted in the absence of a clean control case, the process of isolating those independent variables changes. Studying social phenomena differs from the laboratory setting in that the relationship claimed between variables is often, at best, just that the variables are interrelated. The discovery of these statistical relationships is a stand-alone finding that often requires further analysis. The valuation of or contextualization of these statistical interrelations is beyond the survey itself – the survey cannot inform the researcher whether there is direct causality, or whether there are unmeasured moderator variables at work. Recognizing the separate need for establishing the meaning within, or for, a statistical relationship is crucial to understanding the motivation behind both quantitative and qualitative research. Therefore, the purpose of using quantitative research is to discover relationships between different variables within a sample, and then generalize these findings to an entire population. Because we are concerned with finding average relationships between observable units, the sample has to be sufficiently large to represent the broader population.
Qualitative research is focused on examining incidents or meaning-structures that can be used to describe the observation units or objects in question. In qualitative research, individual cases are examined as a complete whole and the differences among them are attributed to the individual case and not the population as a whole. That is to say that there is no sense in describing differences among variables, as one would in quantitative research, for two reasons. One, each case is considered “true” and therefore has the ability to describe the research object in question on its own without respect to the other cases. Two, no case is necessarily privileged and therefore there is no sense in which one can elect any particular case as a control case. While the differences between individuals are relevant to describing the research objectives, their relevance cannot be ranked without reverting to bias, or facile aesthetics arguments.
While the methods used for extracting meaning from a qualitative study are numerous and varied, all claims made to explain the data are ideographic. Alasuutari (1995)§ describes this understanding of qualitative analysis by relating it to historical research: “[t]here are no historical situations that differ in terms of certain variables, nor is it possible to reconstruct any so that in some of them [the Second World War] would break out and in others it would not.” All we have to work with are the cases as they are presented, and our task is to develop rich descriptive and explanatory models. However, it is worth noting that this does not mean that the claims made might not hold for, and be useful in describing, other cases. (Those who fail to study history are doomed to repeat its mistakes, after all.) We must only be certain to keep in mind that our claims, in and of themselves, are about describing instances and not general laws. However, this does not deregulate the claims that can be made; it, in fact, has quite the opposite effect. By treating all instances as equally true and valid examples, it entails that any claim made under qualitative analysis must be true for all test cases. If there are data that fail a claim within a qualitative study, the claim has to be amended such that it is inclusive of the data. This constraint also makes it quite easy for a reader of a study to test the validity of the claims he is presented. If a claim fails to hold true of the data within a study, the author has either made an error in his analysis or the data have been misrepresented – in either case the reader has cause to be sceptical. This constraint on analysis makes qualitative research claims often much more powerful, if less general, than quantitative claims.
UNDERSTANDING WHEN TO DO QUALITATIVE RESEARCH
A statistician was at a cookout, he stood up to his ankles in the ice chest and had his face laying on the grill; on average, he felt fine.
The humor of this joke derives from the fact that, sometimes, the average case is clearly an absurd notion. In such studies where the average case is an absurd (or at least irrelevant) notion, qualitative research is preferable to quantitative research. However, do keep in mind that qualitative and quantitative approaches are not mutually exclusive and that in the life-cycle of a research schedule their findings often correspond in complementary ways. In this section we will focus on some of the contexts in which qualitative research is preferable to quantitative research.
First, keeping in mind the fact that qualitative and quantitative research fundamentally try to answer different kinds of questions, qualitative research provides an equally rich, if not richer data landscape than quantitative research does. For a very crude example, let us assume that the process of conducting qualitative interviews – because of the ability to react to real-time events, the contact time with an individual case, the ability to observe and record non-verbal reactions, and the general capacity for subjects to offer spontaneous responses – allows the researcher to extract fifty percent more data (or extract data that is fifty percent more valuable) than s/he would in a quantitative survey. Consider the following typical study sizes: a quantitative survey has 200 individuals provide 10 minutes worth of data – for a total of 2,000 data-minutes; a qualitative interview series has 12 individuals provide 120 minutes worth of data – for a total of 1,440 data-minutes. With our assumption that the time spent in the qualitative interviews is fifty percent more productive, the data-minute count for the qualitative research is 2160 (the original 1,440*1.5) – which for concession we will consider essentially equal to the data-minutes in the quantitative survey. It is clear, though, from this rough comparison that if we were to more realistically weight the data gathered during a schedule of qualitative interviews that the data-minute count for a qualitative study would quickly run away from the quantitative study. This is not to argue that qualitative is better than quantitative research; again, the two approaches are extremely useful for different research aims. This instead is meant to demonstrate that qualitative studies are data-rich and capable of standing on their own.
Second, some research objectives are most interested in atypical cases. The kinds of research that benefit from examining extreme cases include exploratory projects and pilot studies, and various sorts of decision-making studies. For these situations qualitative methods are preferred, since quantitative research can only describe average relationships between variables. In these kinds of inquiry, the research objectives are best explored through understanding what is possible – not what is typical. Therefore, in these cases a non-representative, non-average sample is desired.
In the case of exploratory or pilot research projects, one is most interested in mapping out a territory. Like with a jigsaw puzzle, we often seek to put together the edges of the puzzle first in order to “fit in” the remaining pieces. In research, in early stages we are often not concerned with knowing what the whole picture looks like. Rather, we are concerned with trying to find limits or boundaries that can be used to determine and assess future research questions, and by which we can measure future analysis judgments. Specifically, by piloting a research schedule with a qualitative inquiry one can identify and describe the most relevant or interesting variables to pursue in subsequent quantitative studies. Once quantitative research is informed and directed in this way, it becomes an even more powerful tool.
In the specific case of researching decision making, the average decision maker does not have robust enough heuristics to provide the researcher reliable access to the research object. Instead, the individual cases that tend to have the most to bear on decision making are either experts within the decision making domain, or they are paragons of a process within the decision making domain; both of these are atypical cases. Generally speaking, when the research object concerns progressive idea development, or understanding peculiarities, qualitative research is preferable.
Qualitative research is not an “other” category of misfit research methods that did not make the cut to be included in quantitative research; in some ways qualitative research is much more rigorously constrained than quantitative research. Qualitative research has its own unique position in the research process, and instead of trying to understand it through its inability to do what quantitative research does, we should explore its strengths and the ways that it can complement quantitative research as part of a mixed-method research program. Qualitative research is good for exploring new ideas, describing and contextualizing variables (especially when feeding quantitative studies), and understanding unique cases. It is also much better-suited to addressing questions of context – understanding not just how someone responds to a question we asked, but also learning the cues and factors that influence them in ways we never thought to ask about.There are sophisticated ways of auditing a qualitative analysis but counting the respondents is not one of them.
Summary explanation of the differences between qualitative and quantitative research:
|Question domains||How, Why||Who, What, When, Where|
|Common sample sizes and contact time||10-15 respondents
45-240 minutes each
10-20 minutes each
|Validity||Must be true of each case Ideographic (case oriented)||Must be true of most of the data Nomothetic (law oriented)|
|Question domains||How, Why||Who, What, When, Where|
|Question domains||How, Why||Who, What, When, Where|
|Variables||Describe them and establish meaning-structures and contexts||Define their relationships and establish a general case|
|Outliers||Valuable descriptive cases with unique ‘access’ to average cases||Unique positions lost to the weight of the average or ignored outright|
|Examples||Interviews, literature content review, real world observations, case study, narratives, ethnography||Surveys, numerical counts and statistical analyses, mathematical modeling|
Paradigm2 Research has pioneered the use of qualitative research for small businesses, making it possible to access hard-to-reach populations and generate deep insights through innovative methods. Some of these methods were developed and published by the research staff, as we continue to change the way high-potential organizations think about their marketing research, competitive intelligence, and strategy development. If you would like to contribute to this discussion or talk to the Paradigm2 Research Team about your research needs, please contact us here.
§Alasuutari, Pertti. “What Is Qualitative Research.” Researching Culture: Qualitative Method and Cultural Studies. London: SAGE, 1995. 6-22. Print.