07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 214 CHAPTER 7 Designing a Qualitative Study Joseph A. Maxwell T raditionally, works on research design (most of which focus on quantitative research) have understood \"design\" in one of two ways. Some take designs to be fixed, standard arrangements of research conditions and methods that have their own coherence and logic, as possible answers to the question, \"What research design are you using?\" (e.g., Campbell & Stanley, 1967). For example, a randomized, double-blind experiment is one research design; an interrupted timeseries design is another. Beyond such broad categories as ethnographies, qualitative interview studies, and case studies (which often overlap), qualitative research lacks any such elaborate typology into which studies can be pigeonholed. In addition, typologies are usually based on a limited number of features of the study, and by themselves do little to clarify the actual functioning and interrelationship of the component parts of a design. Other models present design as a logical progression of stages or tasks, from problem formulation to the generation of conclusions or theory, that are necessary in planning or carrying out a study (e.g., Creswell, 1997; Marshall & Rossman, 1999). Such models usually resemble a flowchart with a clear starting point and goal and a specified order for doing the intermediate tasks. Although some versions of this approach are circular or iterative (see, e.g., Bickman & Rog, Chapter 1, this volume), so that later steps connect back to earlier ones, all such models are linear in the sense that they are made up of one-directional sequences of steps that represent what is seen as the optimal order for conceptualizing or conducting the different components or activities of a study. Neither of these models adequately represents the logic and process of qualitative research. In a qualitative study, \"research design should be a reflexive process operating through every stage of a project\" (Hammersley & Atkinson, 1995, p. 24); 214 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 215 Designing a Qualitative Study the activities of collecting and analyzing data, developing and modifying theory, elaborating or refocusing the research questions, and identifying and dealing with validity threats are usually going on more or less simultaneously, each influencing all of the others. In addition, the researcher may need to reconsider or modify any design decision during the study in response to new developments or to changes in some other aspect of the design. Grady and Wallston (1988) argue that applied research in general requires a flexible, nonsequential approach and \"an entirely different model of the research process than the traditional one offered in most textbooks\" (p. 10). This does not mean that qualitative research lacks design; as Yin (1994) says, \"Every type of empirical research has an implicit, if not explicit, research design\" (p. 19). Qualitative research simply requires a broader and less restrictive concept of \"design\" than the traditional ones described above. Thus, Becker, Geer, Hughes, and Strauss (1961), authors of a classic qualitative study of medical students, begin their chapter titled \"Design of the Study\" by stating, In one sense, our study had no design. That is, we had no well-worked-out set of hypotheses to be tested, no data-gathering instruments purposely designed to secure information relevant to these hypotheses, no set of analytic procedures specified in advance. Insofar as the term \"design\" implies these features of elaborate prior planning, our study had none. If we take the idea of design in a larger and looser sense, using it to identify those elements of order, system, and consistency our procedures did exhibit, our study had a design. We can say what this was by describing our original view of the problem, our theoretical and methodological commitments, and the way these affected our research and were affected by it as we proceeded. (p. 17) For these reasons, the model of design that I present here, which I call an interactive model, consists of the components of a research study and the ways in which these components may affect and be affected by one another. It does not presuppose any particular order for these components, or any necessary directionality of influence. The model thus resembles the more general definition of design employed outside research: \"An underlying scheme that governs functioning, developing, or unfolding\" and \"the arrangement of elements or details in a product or work of art\" (Frederick et al., 1993). A good design, one in which the components work harmoniously together, promotes efficient and successful functioning; a flawed design leads to poor operation or failure. Traditional (typological or linear) approaches to design provide a model for conducting the researcha prescriptive guide that arranges the components or tasks involved in planning or conducting a study in what is seen as an optimal order. In contrast, the model presented in this chapter is a model of as well as for research. It is intended to help you understand the actual structure of your study as well as to plan this study and carry it out. An essential feature of this model is that it treats research design as a real entity, not simply an abstraction or plan. Borrowing 215 07-Bickman-45636:07-Bickman-45636 216 7/28/2008 6:13 PM Page 216 APPLIED RESEARCH DESIGNS Kaplan's (1964, p. 8) distinction between the \"logic-in-use\" and \"reconstructed logic\" of research, this model can be used to represent the \"design-in-use\" of a study, the actual relationships among the components of the research, as well as the intended (or reconstructed) design (Maxwell & Loomis, 2002). This model of research design has five components, each of which addresses a different set of issues that are essential to the coherence of a study: 1. Goals: Why is your study worth doing? What issues do you want it to clarify, and what practices and policies do you want it to influence? Why do you want to conduct this study, and why should we care about the results? 2. Conceptual framework: What do you think is going on with the issues, settings, or people you plan to study? What theories, beliefs, and prior research findings will guide or inform your research, and what literature, preliminary studies, and personal experiences will you draw on for understanding the people or issues you are studying? 3. Research questions: What, specifically, do you want to learn or understand by doing this study? What do you not know about the things you are studying that you want to learn? What questions will your research attempt to answer, and how are these questions related to one another? 4. Methods: What will you actually do in conducting this study? What approaches and techniques will you use to collect and analyze your data, and how do these constitute an integrated strategy? 5. Validity: How might your results and conclusions be wrong? What are the plausible alternative interpretations and validity threats to these, and how will you deal with these? How can the data that you have, or that you could potentially collect, support or challenge your ideas about what's going on? Why should we believe your results? I have not identified ethics as a separate component of research design. This isn't because I don't think ethics is important for qualitative design; on the contrary, attention to ethical issues in qualitative research is being increasingly recognized as essential (Christians, 2000; Denzin & Lincoln, 2000; Fine, Weis, Weseen, & Wong, 2000). Instead, it is because I believe that ethical concerns should be involved in every aspect of design. I have particularly tried to address these concerns in relation to methods, but they are also relevant to your goals, the selection of your research questions, validity concerns, and the critical assessment of your conceptual framework. These components are not substantially different from the ones presented in many other discussions of qualitative or applied research design (e.g., LeCompte & Preissle, 1993; Lincoln & Guba, 1985; Miles & Huberman, 1994; Robson, 2002). What is innovative is the way the relationships among the components are conceptualized. In this model, the different parts of a design form an integrated and interacting whole, with each component closely tied to several others, rather than being linked in a linear or cyclic sequence. The most important relationships among these five components are displayed in Figure 7.1. 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 217 Designing a Qualitative Study Conceptual framework Goals Research questions Methods Figure 7.1 Validity An Interactive Model of Research Design SOURCE: From Qualitative Research Design: An Interactive Approach, by J. A. Maxwell, 2005. Copyright by SAGE. There are also connections other than those emphasized here, some of which I have indicated by dashed lines. For example, if a goal of your study is to empower participants to conduct their own research on issues that matter to them, this will shape the methods you use, and conversely the methods that are feasible in your study will constrain your goals. Similarly, the theories and intellectual traditions you are drawing on in your research will have implications for what validity threats you see as most important and vice versa. The upper triangle of this model should be a closely integrated unit. Your research questions should have a clear relationship to the goals of your study and should be informed by what is already known about the phenomena you are studying and the theoretical concepts and models that can be applied to these phenomena. In addition, the goals of your study should be informed by current theory and knowledge, while your decisions about what theory and knowledge are relevant depend on your goals and questions. Similarly, the bottom triangle of the model should also be closely integrated. The methods you use must enable you to answer your research questions, and also to deal with plausible validity threats to these answers. The questions, in turn, need to be framed so as to take the feasibility of the methods and the seriousness of particular validity threats into account, while the plausibility and relevance of particular validity threats, and the ways these can be dealt with, depend on the questions and methods chosen. The research questions are the heart, or hub, of the model; they connect all the other components of the design, and should inform, and be sensitive to, these components. There are many other factors besides these five components that should influence the design of your study; these include your research skills, the available resources, perceived problems, ethical standards, the research setting, and the data and 217 07-Bickman-45636:07-Bickman-45636 218 7/28/2008 6:13 PM Page 218 APPLIED RESEARCH DESIGNS Perceived problems Personal experience Goals Conceptual framework Personal goals Participant concerns Funding and funder goals Ethical standards Research setting Figure 7.2 Researcher skills and preferred style of research Exploratory and pilot research Thought experiments Research questions Methods Existing theory and prior research Validity Preliminary data and conclusions Research paradigm Contextual Factors Influencing a Research Design preliminary conclusions of the study. In my view, these are not part of the design of a study; rather, they either belong to the environment within which the research and its design exist or are products of the research. Figure 7.2 presents some of the environmental factors that can influence the design and conduct of a study. I do not believe that there is one right model for qualitative or applied research design. However, I think that the model I present here is a useful one, for three main reasons: 1. It explicitly identifies as components of design the key issues about which decisions need to be made. These issues are therefore less likely to be ignored, and can be dealt with in a systematic manner. 2. It emphasizes the interactive nature of design decisions in qualitative and applied research, and the multiple connections among the design components. 3. It provides a model for the structure of a proposal for a qualitative study, one that clearly communicates and justifies the major design decisions and the connections among these (see Maxwell, 2005). SOURCE: From Qualitative Research Design: An Interactive Approach, by J. A. Maxwell, 2005. Copyright by SAGE. 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 219 Designing a Qualitative Study 219 Because a design for your study always exists, explicitly or implicitly, it is important to make this design explicit, to get it out in the open, where its strengths, limitations, and implications can be clearly understood. In the remainder of this chapter, I present the main design issues involved in each of the five components of my model, and the implications of each component for the others. I do not discuss in detail how to actually do qualitative research, or deal in depth with the theoretical and philosophical views that have informed this approach. For additional guidance on these topics, see the contributions of Fetterman (Chapter 17, this volume) and Stewart, Shamdasani, and Rook (Chapter 18, this volume) to this Handbook; the more extensive treatments by Patton (2000), Eisner and Peshkin (1990), LeCompte and Preissle (1993), Glesne (2005), Weiss (1994), Miles and Huberman (1994), and Wolcott (1995); and the encyclopedic handbooks edited by Denzin and Lincoln (2005) and Given (in press). My focus here is on how to design a qualitative study that arrives at valid conclusions and successfully and efficiently achieve its goals. Goals: Why Are You Doing This Study? Anyone can find an unanswered, empirically answerable question to which the answer isn't worth knowing; as Thoreau said, it is not worthwhile to go around the world to count the cats in Zanzibar. Without a clear sense of the goals of your research, you are apt to lose your focus and spend your time and effort doing things that won't contribute to these goals. (I use goals here in a broad sense, to include motives, desires, and purposesanything that leads you to do the study or that you hope to accomplish by doing it.) These goals serve two main functions for your research. First, they help guide your other design decisions to ensure that your study is worth doing, that you get out of it what you want. Second, they are essential to justifying your study, a key task of a funding or dissertation proposal. In addition, your goals inevitably shape the descriptions, interpretations, and theories you create in your research. They therefore constitute not only important resources that you can draw on in planning, conducting, and justifying the research, but also potential validity threats, or sources of bias, that you will need to deal with. It is useful to distinguish among three kinds of goals for doing a study: personal goals, practical goals, and intellectual goals. Personal goals are those that motivate you to do this study; they can include a desire to change some existing situation, a curiosity about a specific phenomenon or event, or simply the need to advance your career. These personal goals often overlap with your practical or research goals, but they may also include deeply rooted individual desires and needs that bear little relationship to your \"official\" reasons for doing the study. It is important that you recognize and take account of the personal goals that drive and inform your research. Eradicating or submerging your personal goals and concerns is impossible, and attempting to do so is unnecessary. What is necessary, in qualitative design, is that you be aware of these concerns and how they may be shaping your research, and that you think about how best to deal with their consequences. 07-Bickman-45636:07-Bickman-45636 220 7/28/2008 6:13 PM Page 220 APPLIED RESEARCH DESIGNS To the extent that you have not made a careful assessment of ways in which your design decisions and data analyses are based on personal desires, you are in danger of arriving at invalid conclusions. However, your personal reasons for wanting to conduct a study, and the experiences and perspectives in which these are grounded, are not simply a source of \"bias\" (see the later discussion of this issue in the section on validity); they can also provide you with a valuable source of insight, theory, and data about the phenomena you are studying (Marshall & Rossman, 1999, pp. 25-30; Strauss & Corbin, 1990, pp. 42-43). This source is discussed in the next section, in the subsection on experiential knowledge. Two major decisions are often profoundly influenced by the researcher's personal goals. One is the topic, issue, or question selected for study. Traditionally, students have been told to base this decision on either faculty advice or the literature on their topic. However, personal goals and experiences play an important role in many research studies. Strauss and Corbin (1990) argue that choosing a research problem through the professional or personal experience route may seem more hazardous than through the suggested [by faculty] or literature routes. This is not necessarily true. The touchstone of your own experience may be more valuable an indicator for you of a potentially successful research endeavor. (pp. 35-36) A second decision that is often influenced by personal goals and experiences is the choice of a qualitative approach. Locke, Spirduso, and Silverman (1993) argue that \"every graduate student who is tempted to employ a qualitative design should confront one question, 'Why do I want to do a qualitative study?' and then answer it honestly\" (p. 107). They emphasize that qualitative research is not easier than quantitative and that seeking to avoid statistics bears little relationship to having the personal interests and skills that qualitative inquiry requires (pp. 107-110). The key issue is the compatibility of your reasons for \"going qualitative\" with your other goals, your research questions, and the actual activities involved in doing a qualitative study. Besides your personal goals, there are two other kinds of goals that I want to distinguish and discuss, ones that are important for other people, not just yourself: practical goals (including administrative or policy goals) and intellectual goals. Practical goals are focused on accomplishing somethingmeeting some need, changing some situation, or achieving some goal. Intellectual goals, on the other hand, are focused on understanding something, gaining some insight into what is going on and why this is happening. Although applied research design places much more emphasis on practical goals than does basic research, you still need to address the issues of what you want to understand by doing the study and how this understanding will contribute to your accomplishing your practical goals. (The issue of what you want to understand is discussed in more detail below, in the section on research questions.) There are five particular intellectual goals for which qualitative studies are especially useful: 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 221 Designing a Qualitative Study 1. Understanding the meaning, for participants in the study, of the events, situations, and actions they are involved with, and of the accounts that they give of their lives and experiences. In a qualitative study, you are interested not only in the physical events and behavior taking place, but also in how the participants in your study make sense of these and how their understandings influence their behavior. The perspectives on events and actions held by the people involved in them are not simply their accounts of these events and actions, to be assessed in terms of truth or falsity; they are part of the reality that you are trying to understand, and a major influence on their behavior (Maxwell, 1992, 2004a). This focus on meaning is central to what is known as the \"interpretive\" approach to social science (Bredo & Feinberg, 1982; Geertz, 1973; Rabinow & Sullivan, 1979). 2. Understanding the particular context within which the participants act and the influence this context has on their actions. Qualitative researchers typically study a relatively small number of individuals or situations and preserve the individuality of each of these in their analyses, rather than collecting data from large samples and aggregating the data across individuals or situations. Thus, they are able to understand how events, actions, and meanings are shaped by the unique circumstances in which these occur. 3. Identifying unanticipated phenomena and influences and generating new, \"grounded\" theories about the latter. Qualitative research has long been used for this goal by survey and experimental researchers, who often conduct \"exploratory\" qualitative studies to help them design their questionnaires and identify variables for experimental investigation. Although qualitative research is not restricted to this exploratory role, it is still an important strength of qualitative methods. 4. Understanding the processes by which events and actions take place. Although qualitative research is not unconcerned with outcomes, a major strength of qualitative studies is their ability to get at the processes that lead to these outcomes, processes that experimental and survey research are often poor at identifying (Maxwell, 2004a). 5. Developing causal explanations. The traditional view that qualitative research cannot identify causal relationships is based on a restrictive and philosophically outdated concept of causality (Maxwell, 2004b), and both qualitative and quantitative researchers are increasingly accepting the legitimacy of using qualitative methods for causal inference (e.g., Shadish, Cook, & Campbell, 2002). Such an approach requires thinking of causality in terms of processes and mechanisms, rather than simply demonstrating regularities in the relationships between variables (Maxwell, 2004a); I discuss this in more detail in the section on research questions. Deriving causal explanations from a qualitative study is not an easy or straightforward task, but qualitative research is not different from quantitative research in this respect. Both approaches need to identify and deal with the plausible validity threats to any proposed causal explanation, as discussed below. These intellectual goals, and the inductive, open-ended strategy that they require, give qualitative research an advantage in addressing numerous practical goals, including the following. 221 07-Bickman-45636:07-Bickman-45636 222 7/28/2008 6:13 PM Page 222 APPLIED RESEARCH DESIGNS Generating results and theories that are understandable and experientially credible, both to the people being studied and to others (Bolster, 1983). Although quantitative data may have greater credibility for some goals and audiences, the specific detail and personal immediacy of qualitative data can lead to the greater influence of the latter in other situations. For example, I was involved in one evaluation, of how teaching rounds in one hospital department could be improved, that relied primarily on participant observation of rounds and open-ended interviews with staff physicians and residents (Maxwell, Cohen, & Reinhard, 1983). The evaluation led to decisive department action, in part because department members felt that the report, which contained detailed descriptions of activities during rounds and numerous quotes from interviews to support the analysis of the problems with rounds, \"told it like it really was\" rather than simply presenting numbers and generalizations to back up its recommendations. Conducting formative studies, ones that are intended to help improve existing practice rather than simply to determine the outcomes of the program or practice being studied (Scriven, 1991). In such studies, which are particularly useful for applied research, it is more important to understand the process by which things happen in a particular situation than to measure outcomes rigorously or to compare a given situation with others. Engaging in collaborative, action, or \"empowerment\" research with practitioners or research participants (e.g., Cousins & Earl, 1995; Fetterman, Kaftarian, & Wandersman, 1996; Tolman & Brydon-Miller, 2001; Whyte, 1991). The focus of qualitative research on particular contexts and their meaning for the participants in these contexts, and on the processes occurring in these contexts, makes it especially suitable for collaborations with practitioners or with members of the community being studied (Patton, 1990, pp. 129-130; Reason, 1994). A useful way of sorting out and formulating the goals of your study is to write memos in which you reflect on your goals and motives, as well as the implications of these for your design decisions (for more information on such memos, see Maxwell, 2005, pp. 11-13; Mills, 1959, pp. 197-198; Strauss & Corbin, 1990, chap. 12). See Exercise 1. Conceptual Framework: What Do You Think Is Going On? The conceptual framework of your study is the system of concepts, assumptions, expectations, beliefs, and theories that supports and informs your research. Miles and Huberman (1994) state that a conceptual framework \"explains, either graphically or in narrative form, the main things to be studiedthe key factors, concepts, or variablesand the presumed relationships among them\" (p. 18). Here, I use the term in a broader sense that also includes the actual ideas and beliefs that you hold about the phenomena studied, whether these are written down or not. Thus, your conceptual framework is a formulation of what you think is going on with the phenomena you are studyinga tentative theory of what is happening and 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 223 Designing a Qualitative Study why. Theory provides a model or map of why the world is the way it is (Strauss, 1995). It is a simplification of the world, but a simplification aimed at clarifying and explaining some aspect of how it works. It is not simply a \"framework,\" although it can provide that, but a story about what you think is happening and why. A useful theory is one that tells an enlightening story about some phenomenon, one that gives you new insights and broadens your understanding of that phenomenon. The function of theory in your design is to inform the rest of the designto help you assess your goals, develop and select realistic and relevant research questions and methods, and identify potential validity threats to your conclusions. What is often called the \"research problem\" is a part of your conceptual framework, and formulating the research problem is often seen as a key task in designing your study. It is part of your conceptual framework (although it is often treated as a separate component of a research design) because it identifies something that is going on in the world, something that is itself problematic or that has consequences that are problematic. The conceptual framework of a study is often labeled the \"literature review.\" This can be a dangerously misleading term, for three reasons. First, it can lead you to focus narrowly on \"literature,\" ignoring other conceptual resources that may be of equal or greater importance for your study, including unpublished work, communication with other researchers, and your own experience and pilot studies. Second, it tends to generate a strategy of \"covering the field\" rather than focusing specifically on those studies and theories that are particularly relevant to your research (Maxwell, 2006). Third, it can make you think that your task is simply descriptive to tell what previous researchers have found or what theories have been proposed. In developing a conceptual framework, your purpose is not only descriptive, but also critical; you need to treat \"the literature\" not as an authority to be deferred to, but as a useful but fallible source of ideas about what's going on, and to attempt to see alternative ways of framing the issues (Locke, Silverman, & Spirduso, 2004). Another way of putting this is that the conceptual framework for your research study is something that is constructed, not found. It incorporates pieces that are borrowed from elsewhere, but the structure, the overall coherence, is something that you build, not something that exists ready-made. Becker (1986, 141ff.) systematically develops the idea that prior work provides modules that you can use in building your conceptual framework, modules that you need to examine critically to make sure they work effectively with the rest of your design. There are four main sources for these modules: your own experiential knowledge, existing theory and research, pilot and exploratory studies, and thought experiments. Before addressing the sources of these modules, however, I want to discuss a particularly important part of your conceptual frameworkthe research paradigm(s) within which you situate your work. Connecting With a Research Paradigm One of the critical decisions that you will need to make in designing your study is the paradigm (or paradigms) within which you will situate your work. This use 223 07-Bickman-45636:07-Bickman-45636 224 7/28/2008 6:13 PM Page 224 APPLIED RESEARCH DESIGNS of the term paradigm, which derives from the work of the historian of science Thomas Kuhn, refers to a set of very general philosophical assumptions about the nature of the world (ontology) and how we can understand it (epistemology), assumptions that tend to be shared by researchers working in a specific field or tradition. Paradigms also typically include specific methodological strategies linked to these assumptions, and identify particular studies that are seen as exemplifying these assumptions and methods. At the most abstract and general level, examples of such paradigms are philosophical positions such as positivism, constructivism, realism, and pragmatism, each embodying very different ideas about reality and how we can gain knowledge of it. At a somewhat more specific level, paradigms that are relevant to qualitative research include interpretivism, critical theory, feminism, postmodernism, and phenomenology, and there are even more specific traditions within these (for more detailed guidance, see Creswell, 1997; Schram, 2005). I want to make several points about using paradigms in your research design: 1. Although some people refer to \"the qualitative paradigm,\" there are many different paradigms within qualitative research, some of which differ radically in their assumptions and implications (see also Denzin & Lincoln, 2000; Pitman & Maxwell, 1992). You need to make explicit which paradigm(s) your work will draw on, since a clear paradigmatic stance helps guide your design decisions and to justify these decisions. Using an established paradigm (such as grounded theory, critical realism, phenomenology, or narrative research) allows you to build on a coherent and welldeveloped approach to research, rather than having to construct all of this yourself. 2. You don't have to adopt in total a single paradigm or tradition. It is possible to combine aspects of different paradigms and traditions, although if you do this you will need to carefully assess the compatibility of the modules that you borrow from each. Schram (2005) gives a valuable account of how he combined the ethnographic and life history traditions in his dissertation research on an experienced teacher's adjustment to a new school and community. 3. Your selection of a paradigm (or paradigms) is not a matter of free choice. You have already made many assumptions about the world, your topic, and how we can understand these, even if you have never consciously examined these. Choosing a paradigm or tradition primarily involves assessing which paradigms best fit with your own assumptions and methodological preferences; Becker (1986, pp. 16-17) makes the same point about using theory in general. Trying to work within a paradigm (or theory) that doesn't fit your assumptions is like trying to do a physically demanding job in clothes that don't fitat best you'll be uncomfortable, at worst it will keep you from doing the job well. Such a lack of fit may not be obvious at the outset; it may only emerge as you develop your conceptual framework, research questions, and methods, since these should also be compatible with your paradigmatic stance. Experiential Knowledge Traditionally, what you bring to the research from your background and identity has been treated as \"bias,\" something whose influence needs to be eliminated 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 225 Designing a Qualitative Study from the design, rather than a valuable component of it. However, the explicit incorporation of your identity and experience (what Strauss, 1987, calls \"experiential data\") in your research has recently gained much wider theoretical and philosophical support (e.g., Berg & Smith, 1988; Denzin & Lincoln, 2000; Jansen & Peshkin, 1992; Strauss, 1987). Using this experience in your research can provide you with a major source of insights, hypotheses, and validity checks. For example, Grady and Wallston (1988, p. 41) describe how one health care researcher used insights from her own experience to design a study of why many women don't do breast self-examination. This is not a license to impose your assumptions and values uncritically on the research. Reason (1988) uses the term critical subjectivity to refer to a quality of awareness in which we do not suppress our primary experience; nor do we allow ourselves to be swept away and overwhelmed by it; rather we raise it to consciousness and use it as part of the inquiry process. (p. 12) However, there are few well-developed and explicit strategies for doing this. The \"researcher identity memo\" is one technique; this involves reflecting on, and writing down, the different aspects of your experience that are potentially relevant to your study. Example 7.1 is part of one of my own researcher identity memos, written when I was working on a paper of diversity and community; Exercise 1 involves writing your own researcher identity memo. (For more on this technique, see Maxwell, 2005.) Doing this can generate unexpected insights and connections, as well as create a valuable record of these. Ex ampl e 7. 1 Identity Memo on Diversity I can't recall when I first became interested in diversity; it's been a major concern for at least the past 20 years . . . I do remember the moment that I consciously realized that my mission in life was \"to make the world safe for diversity\"; I was in Regenstein Library at the University of Chicago one night in the mid-1970s talking to another student about why we had gone into anthropology, and the phrase suddenly popped into my head. However, I never gave much thought to tracing this position any further back. I remember, as an undergraduate, attending a talk on some political topic, and being struck by two students' bringing up issues of the rights of particular groups to retain their cultural heritages; it was an issue that had never consciously occurred to me. And I'm sure that my misspent youth reading science fiction rather than studying had a powerful influence on my sense of the importance of tolerance and understanding of diversity; I wrote my essay for my application to college on tolerance in high school society. But I didn't think much about where all this came from. (Continued) 225 07-Bickman-45636:07-Bickman-45636 226 7/28/2008 6:13 PM Page 226 APPLIED RESEARCH DESIGNS (Continued) It was talking to the philosopher Amelie Rorty in the summer of 1991 that really triggered my awareness of these roots. She had given a talk on the concept of moral diversity in Plato, and I gave her a copy of my draft paper on diversity and solidarity. We met for lunch several weeks later to discuss these issues, and at one point she asked me how my concern with diversity connected with my background and experiences. I was surprised by the question, and found I really couldn't answer it. She, on the other hand, had thought about this a lot, and talked about her parents emigrating from Belgium to the United States, deciding they were going to be farmers like \"real Americans,\" and with no background in farming, buying land in rural West Virginia and learning how to survive and fit into a community composed of people very different from themselves. This made me start thinking, and I realized that as far back as I can remember I've felt different from other people, and had a lot of difficulties as a result of this difference and my inability to \"fit in\" with peers, relatives, or other people generally. This was all compounded by my own shyness and tendency to isolate myself, and by the frequent moves that my family made while I was growing up. The way in which this connects with my work on diversity is that my main strategy for dealing with my difference from others, as far back as I can remember, was not to try to be more like them (similarity-based), but to try to be helpful to them (contiguity-based). This is a bit oversimplified, because I also saw myself as somewhat of a \"social chameleon,\" adapting to whatever situation I was in, but this adaptation was much more an interactional adaptation than one of becoming fundamentally similar to other people. It now seems incomprehensible to me that I never saw the connections between this background and my academic work. [The remainder of the memo discusses the specific connections between my experience and the theory of diversity and community that I had been developing, which sees both similarity (shared characteristics) and contiguity (interaction) as possible sources of solidarity and community.] SOURCE: From Qualitative Research Design: An Interactive Approach, by J. A. Maxwell, 2005. Copyright by SAGE. Existing Theory and Research The second major source of modules for your conceptual framework is existing theory and researchnot simply published work, but also unpublished papers and dissertations, conference presentations, and what is in the heads of active researchers in your field (Locke, Spirduso, & Silverman, 2000). I will begin with theory, because 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 227 Designing a Qualitative Study it is for most people the more problematic and confusing of the two, and then deal with using prior research for other purposes than as a source of theory. Using existing theory in qualitative research has both advantages and dangers. A useful theory helps you organize your data. Particular pieces of information that otherwise might seem unconnected or irrelevant to one another or to your research questions can be related if you can fit them into the theory. A useful theory also illuminates what you are seeing in your research. It draws your attention to particular events or phenomena and sheds light on relationships that might otherwise go unnoticed or misunderstood. However, Becker (1986) warns that the existing literature, and the assumptions embedded in it, can deform the way you frame your research, causing you to overlook important ways of conceptualizing your study or key implications of your results. The literature has the advantage of what he calls \"ideological hegemony,\" making it difficult for you to see any phenomenon in ways that are different from those that are prevalent in the literature. Trying to fit your insights into this established framework can deform your argument, weakening its logic and making it harder for you to see what this new way of framing the phenomenon might contribute. Becker describes how existing theory and perspectives deformed his early research on marijuana use, leading him to focus on the dominant question in the literature and to ignore the most interesting implications and possibilities of his study. Becker (1986) argues that there is no way to be sure when the established approach is wrong or misleading or when your alternative is superior. All you can do is try to identify the ideological component of the established approach, and see what happens when you abandon these assumptions. He asserts that \"a serious scholar ought routinely to inspect competing ways of taking about the same subject matter,\" and warns, \"Use the literature, don't let it use you\" (p. 149; see also Mills, 1959). A review of relevant prior research can serve several other purposes in your design besides providing you with existing theory (see Locke et al., 2004; Strauss, 1987, pp. 48-56). First, you can use it to develop a justification for your studyto show how your work will address an important need or unanswered question. Second, it can inform your decisions about methods, suggesting alternative approaches or revealing potential problems with your plans. Third, it can be a source of data that you can use to test or modify your theories. You can see if existing theory, the results of your pilot research, or your experiential understanding is supported or challenged by previous studies. Finally, you can use ideas in the literature to help you generate theory, rather than simply borrowing such theory from the literature. Pilot and Exploratory Studies Pilot studies serve some of the same functions as prior research, but they can be focused more precisely on your own concerns and theories. You can design pilot studies specifically to test your ideas or methods and explore their implications, or to inductively develop grounded theory. One particular use that pilot studies have in qualitative research is to generate an understanding of the concepts and theories held by the people you are studyingwhat I have called \"interpretation\" (Maxwell, 1992). 227 07-Bickman-45636:07-Bickman-45636 228 7/28/2008 6:13 PM Page 228 APPLIED RESEARCH DESIGNS This is not simply a source of additional concepts for your theory; instead, it provides you with an understanding of the meaning that these phenomena and events have for the actors who are involved in them, and the perspectives that inform their actions. In a qualitative study, these meanings and perspectives should constitute an important focus of your theory; as discussed earlier, they are one of the things your theory is about, not simply a source of theoretical insights and building blocks for the latter. Thought Experiments Thought experiments have a long and respected tradition in the physical sciences (much of Einstein's work was based on thought experiments) but have received little attention in discussions of research design, particularly qualitative research design. Thought experiments draw on both theory and experience to answer \"what if \" questions, to seek out the logical implications of various properties of the phenomena you want to study. They can be used both to test your current theory for logical problems and to generate new theoretical insights. They encourage creativity and a sense of exploration and can help you make explicit the experiential knowledge that you already possess. Finally, they are easy to do, once you develop the skill. Valuable discussions of thought experiments in the social sciences are presented by Mills (1959) and Lave and March (1975). Experience, prior theory and research, pilot studies, and thought experiments are the four major sources of the conceptual framework for your study. The ways in which you can put together a useful and valid conceptual framework from these sources are particular to each study, and not something for which any cookbook exists. The main thing to keep in mind is the need for integration of these components with one another and with your goals and research questions. Concept Mapping A particularly valuable tool for generating and understanding these connections in your research is a technique known as concept mapping (Miles & Huberman, 1994; Novak & Gowin, 1984). Kane and Trochim (Chapter 14, this volume) provide an overview of concept mapping but focus on using concept mapping with groups of stakeholders for organizational improvement or evaluation, employing mainly quantitative techniques. However, concept mapping has many other uses, including clarification and development of your own ideas about what's going on with the phenomena you want to study. Exercise 2 is designed to help you develop an initial concept map for your study (for additional guidance, see the sources above and Maxwell, 2005). Research Questions: What Do You Want to Understand? Your research questionswhat you specifically want to learn or understand by doing your studyare at the heart of your research design. They are the one 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 229 Designing a Qualitative Study component that directly connects to all the other components of the design. More than any other aspect of your design, your research questions will have an influence on, and should be responsive to, every other part of your study. This is different from seeing research questions as the starting point or primary determinant of the design. Models of design that place the formulation of research questions at the beginning of the design process, and that see these questions as determining the other aspects of the design, don't do justice to the interactive and inductive nature of qualitative research. The research questions in a qualitative study should not be formulated in detail until the goals and conceptual framework (and sometimes general aspects of the sampling and data collection) of the design are clarified, and should remain sensitive and adaptable to the implications of other parts of the design. Often, you will need to do a significant part of the research before it is clear to you what specific research questions it makes sense to try to answer. This does not mean that qualitative researchers should, or usually do, begin studies with no questions, simply going into the field with \"open minds\" and seeing what is there to be investigated. Every researcher begins with a substantial base of experience and theoretical knowledge, and these inevitably generate certain questions about the phenomena studied. These initial questions frame the study in important ways, influence decisions about methods, and are one basis for further focusing and development of more specific questions. However, these specific questions are generally the result of an interactive design process, rather than the starting point for that process. For example, Suman Bhattacharjea (1994; see also Maxwell, 2005, p. 66) spent a year doing field research on women's roles in a Pakistani educational district office before she was able to focus on two specific research questions and submit her dissertation proposal; at that point, she had also developed several hypotheses as tentative answers to these questions. The Functions of Research Questions In your research design, the research questions serve two main functions: to help you focus the study (the questions' relationship to your goals and conceptual framework) and to give you guidance for how to conduct it (their relationship to methods and validity). A design in which the research questions are too general or too diffuse creates difficulties both for conducting the studyin knowing what site or informants to choose, what data to collect, and how to analyze these dataand for clearly connecting what you learn to your goals and existing knowledge (Miles & Huberman, 1994, pp. 22-25). Research questions that are precisely framed too early in the study, on the other hand, may lead you to overlook areas of theory or prior experience that are relevant to your understanding of what is going on, or cause you to pay too little attention to a wide range of data early in the study, data that can reveal important and unanticipated phenomena and relationships. A third problem is that you may be smuggling unexamined assumptions into the research questions themselves, imposing a conceptual framework that doesn't fit the reality you are studying. A research question such as \"How do elementary school teachers deal with the experience of isolation from their colleagues in their 229 07-Bickman-45636:07-Bickman-45636 230 7/28/2008 6:13 PM Page 230 APPLIED RESEARCH DESIGNS classrooms?\" assumes that teachers do experience such isolation. Such an assumption needs to be carefully examined and justified, and without this justification it might be better to frame such a question as a tentative subquestion to broader questions about the nature of classroom teachers' experience of their work and their relations with colleagues. For all these reasons, there is real danger to your study if you do not carefully formulate your research questions in connection with the other components of your design. Your research questions need to take account of what you want to accomplish by doing the study (your goals), and of what is already known about the things you want to study and your tentative theories about these phenomena (your conceptual framework). There is no reason to pose research questions for which the answers are already available, that don't clearly connect to what you think is actually going on, or that would have no direct relevance to your goals in doing the research. Likewise, your research questions need to be ones that are answerable by the kind of study you can actually conduct. There is no value to posing questions that no feasible study could answer, either because the data that could answer them could not be obtained, or because any conclusions you might draw from these data would be subject to serious validity threats. A common problem in the development of research questions is confusion between research issues (what you want to understand by doing the study) and practical issues (what you want to accomplish). Your research questions need to connect clearly to your practical concerns, but in general an empirical study cannot directly answer practical questions such as, \"How can I improve this program?\" or \"What is the best way to increase students' knowledge of science?\" To address such practical questions, you need to focus on what you don't understand about the phenomena you are studying, and investigate what is really going on with these phenomena. For example, the practical goal of Martha Regan-Smith's (1992) dissertation research was to improve the teaching of the basic sciences in medical school (see Maxwell, 2005, 117ff.). However, her research questions focused not on this goal but on what exceptional teachers in her school did that helped students learn science something she had realized that she didn't know and that she believed would have important implications for how to improve such teaching overall. A second confusion, one that can create problems for interview studies, is that between research questions and interview questions. Your research questions identify the things that you want to understand; your interview questions generate the data that you need to understand these things. This distinction is discussed in more detail below, in the section on methods. There are three issues that you should keep in mind in formulating research questions for applied social research. First, research questions may legitimately be framed in particular as well as general terms. There is a strong tendency in basic research to state research questions in general terms, such as, \"How do students deal with racial and ethnic difference in multiracial schools?\" and then to \"operationalize\" these questions by selecting a particular sample or site. This tendency can be counterproductive when the goal of your study is to understand and improve some particular program, situation, or practice. In applied research, 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 231 Designing a Qualitative Study it is often more appropriate to formulate research questions in particular terms, such as, \"How do students at North High School deal with racial and ethnic difference?\" Second, some researchers believe that questions should be stated in terms of what the respondents report or what can be directly observed, rather than in terms of inferred behavior, beliefs, or causal influences. This is what I call an instrumentalist or positivist, rather than a realist, approach to research questions (Maxwell, 1992; Norris, 1983). Instrumentalists formulate their questions in terms of observable or measurable data and are suspicious of inferences to things that cannot be defined in terms of such data. For example, instrumentalists would reject a question such as, \"How do exemplary teachers help medical students learn science?\" and replace it with questions such as, \"How do medical students report that exemplary teachers help them learn science?\" or \"How are exemplary teachers observed to teach basic science?\" Realists, in contrast, don't assume that research questions about feelings, beliefs, intentions, prior behavior, effects, and so on need to be reduced to, or reframed as, questions about the actual data that one uses. Instead, they treat their data as fallible evidence about these phenomena, to be used critically to develop and test ideas about what is going on (Campbell, 1988; Maxwell, 1992). The main risk of using instrumentalist questions is that you will lose sight of what you are really interested in, and define your study in ways that obscure the actual phenomena you want to investigate, ending up with a rigorous but uninteresting conclusion. As in the joke about the man who was looking for his keys under the streetlight (rather than where he dropped them) because the light was better there, you may never find what you started out to look for. An instrumentalist approach to your research questions may also make it more difficult for your study to address important goals of your study directly, and it can inhibit your theorizing about phenomena that are not directly observable. My own preference is to use realist questions and to address, as systematically and rigorously as possible, the validity threats that this approach involves. The seriousness of these validity threats (such as self-report bias) needs to be assessed in the context of a particular study; these threats are often not as serious as instrumentalists imply. There are also effective ways to address these threats in a qualitative design, which I discuss below in the section on validity. The risk of trivializing your study by restricting your questions to what can be directly observed is usually more serious than the risk of drawing invalid conclusions. As the statistician John Tukey (1962) put it, \"Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise\" (p. 13). One issue that is not entirely a matter of realism versus instrumentalism is whether research questions in interview studies should be framed in terms of the respondents' perceptions or beliefs rather than the actual state of affairs. You should base this decision not simply on the seriousness of the validity threats, but also on what you actually want to understand. In many qualitative studies, the real interest is in how participants make sense of what has happened, and how this perspective informs their actions, rather than determining precisely what took place. 231 07-Bickman-45636:07-Bickman-45636 232 7/28/2008 6:13 PM Page 232 APPLIED RESEARCH DESIGNS Finally, many researchers (consciously or unconsciously) focus their questions on variance rather than process (Maxwell, 2004a; Mohr, 1982, 1995, 1996). Variance questions deal with difference and correlation; they often begin with \"Is there,\" \"Does,\" \"How much,\" or \"To what extent.\" For example, a variance approach to Martha Regan-Smith's (1992) study would ask questions such as, \"Do exemplary medical school teachers differ from others in their teaching of basic science?\" or \"Is there a relationship between teachers' behavior and students' learning?\" and attempt to measure these differences and relationships. Process questions, in contrast, focus on how and why things happen, rather than whether there is a particular difference or relationship or how much it is explained by other variables. Regan-Smith's actual questions focused on how these teachers helped students learnthat is, the process by which their teaching helped the students learn. In a qualitative study, it can be dangerous for you to frame your research questions in a way that focuses on differences and their explanation. This may lead you to begin thinking in variance terms, to try to identify the variables that will account for observed or hypothesized differences, and to overlook the real strength of a qualitative approach, which is in understanding the process by which phenomena take place. Variance questions are often best answered by quantitative approaches, which are powerful ways of determining whether a particular result is causally related to one or another variable, and to what extent these are related. However, qualitative research is often better at showing how this occurred. Variance questions are legitimate in qualitative research, but they are often best grounded in the answers to prior process questions (Maxwell 2004a). Qualitative researchers therefore tend to generate two kinds of questions that are much better suited to process theory than to variance theory: (1) questions about the meaning of events and activities to the people involved in them and (2) questions about the influence of the physical and social context on these events and activities. (See the earlier discussion of meaning and context as research goals.) Because both of these types of questions involve situationspecific phenomena, they do not lend themselves to the kinds of comparison and control that variance theory requires. Instead, they generally involve an open-ended, inductive approach to discover what these meanings and influences are and how they are involved in these events and activitiesan inherently processual orientation. Developing relevant, focused, answerable research questions takes time; such questions cannot be thrown together quickly, nor in most studies can they be definitively formulated before data collection and analysis begin. Generating good questions requires that you pay attention not just to the questions themselves but to their connections with all the other design components: the goals that answering the questions might serve, the implications for your questions of your conceptual framework, the methods you could use to answer the questions, and the validity threats you will need to address. As is true with the other components of your design, writing memos about these issues is an extremely useful tool for developing your questions (Maxwell, 2005, pp. 76-78). 07-Bickman-45636:07-Bickman-45636 7/28/2008 6:13 PM Page 233 Designing a Qualitative Study 233 Methods: What Will You Actually Do? There is no \"cookbook\" for doing qualitative research. The appropriate answer to almost any question about the use of qualitative methods is, \"It depends.\" The value and feasibility of your research methods cannot be guaranteed by your adhering to methodological rules; rather, they depend on the specific setting and phenomena you are studying and the actual consequences of your strategy for studying it. Prestructuring a Qualitative Study One of the most important issues in designing a qualitative study is how much you should attempt to prestructure your methods. Structured approaches can help ensure the comparability of data across sources and researchers and are therefore particularly useful in answering variance questions, questions that deal with differences between things and the explanation for these differences. Unstructured approaches, in contrast, allow the researcher to focus on the particular phenomena studied; they trade generalizability and comparability for internal validity and contextual understanding and are particularly useful for understanding the processes that led to specific outcomes, what Huberman and Miles (1988) call \"local causality.\" Sayer (1992, 241ff.) refers to these two approaches as \"extensive\" and \"intensive\" research designs, respectively. However, Miles and Huberman (1994) warn that highly inductive, loosely designed studies make good sense when experienced researchers have plenty of time and are exploring exotic cultures, understudied phenomena, or very complex social phenomena. But if you're new to qualitative studies and are looking at a better understood phenomenon within a familiar culture or subculture, a loose, inductive design is a waste of time. Months of fieldwork and voluminous case studies may yield only a few banalities. (p. 17) They also point out that prestructuring reduces the amount of data that you have to deal with, functioning as a form of preanalysis that simplifies the analytic work required. Unfortunately, most discussions of this issue treat prestructuring as a single dimension, and view it in terms of metaphors such as hard versus soft and tight versus loose. Such metaphors have powerful connotations (although they are different for different people) that can lead you to overlook or ignore the numerous ways in which studies can vary, not just in the amount of prestructuring, but in how prestructuring is used. For example, you could employ an extremely open approach to data collection, but use these data for a confirmatory test of explicit hypotheses based on a prior theory (e.g., Festinger, Riecker, & Schachter, 1956). In contrast, the approach often known as ethnoscience or cognitive anthropology (Werner & Schoepfle, 1987a, 1987b) employs highly structured data collection techniques, but interprets these data in a largely inductive manner with very few preestablished 07-Bickman-45636:07-Bickman-45636 234 7/28/2008 6:13 PM Page 234 APPLIED RESEARCH DESIGNS categories. Thus, the decision you face is not primarily whether or to what extent you prestructure your study, but in what ways you do this, and why. Finally, it is worth keeping in mind that you can lay out a tentative plan for some aspects of your study in considerable detail, but leave open the possibility of substantially revising this if necessary. Emergent insights may require new sampling plans, different kinds of data, and different analytic strategies. I distinguish four main components of qualitative methods: 1. The research relationship that you establish with those you study 2. Sampling: what times, settings, or individuals you select to observe or interview, and what other sources of information you decide to use 3. Data collection: how you gather the information you will use 4. Data analysis: what you do with this information to make sense of it It is useful to think of all these components as involving design decisionskey issues that you should consider in planning your study and that you should rethink as you are engaged in it. Negotiating a Research Relationship Your relationships with the people in your study can be complex and changeable, and these relationships will necessarily affect you as the \"research instrument,\" as well as have implications for other components of your research design. My changing relationships with the people in the Inuit community in which I conducted my dissertation research (Maxwell, 1986) had a profound effect not only on my own state of mind, but also on who I was able to interview, my opportunities for observation of social life, the quality of the data I collected, the research questions I was able to answer, and my ability to test my conclusions. The term reflexivity (Hammersley & Atkinson, 1995) is often used for this unavoidable mutual influence of the research participants and the researcher on each other. There are also philosophical, ethical, and political issues that should inform the kind of relationship that you want to establish. In recent years, there has been a growing interest in alternatives to the traditional style of research, including participatory action research, collaborative research, feminist research, critical ethnography, and empowerment research (see Denzin & Lincoln, 2005; Fetterman et al., 1996; Oja & Smulyan, 1989; Whyte, 1991). Each of these modes of research involves different sorts of relationships between the researcher and the participants in the research and has different implications for the rest of the research design. Thus, it is important that you think about the kinds of relationships you want to have with the people whom you study, and what you need to do to establish such relationships. I see these as design decisions, not simply as external factors that may affect your design. Although they are not completely under your control and cannot be defined precisely in advance, they are still matters that require systematic planning and reflection if your design is to be as coherent as possi