TESOL Matters. Vol. 8 No. 6 (December 1998/January 1999) Approaches to Empirical Research in TESOL Kathleen C.

Bailey, President of TESOL, 1998-1999 Research is one of the five components of TESOL's Forward Plan, yet there is much disagreement about the sorts of research we need in TESOL and which approach to research is best. Empirical research is based on the collection and analysis of primary data, which may be quantified (such as test scores) or not quantified (as in journal entries or observational field notes). Non quantified data are often referred to as qualitative data because they deal with the qualities of the people or processes being studied. The three broad approaches to empirical research are experimental research, naturalistic inquiry, and action research. The experimental tradition is the most familiar and has been the dominant approach. But it is not without problems, especially in addressing language-related phenomena. As a result, the alternative paradigms of naturalistic inquiry and action research are quickly gaining ground. The experimental research tradition consists of a family of research designs (standardized plans for conducting research) and analytic procedures based onprobability theory. The so-called "true experimental designs" include the randomization of subjects, control groups, and experimental groups (which get the "treatment") as well as a high degree of control over the variables that might influence the study's out come. These designs are considered the strongest because they eliminate many threats to validity. In the less powerful designs, the researcher exerts less control over variables and therefore cannot make the causal claims possible with true experimental designs. This control/causality relationship is tied to the three main goals of experimentalresearch: to build theory by testing hypotheses and/or answering research questions; to discover relationships, especially causal relationships; and to generalize results beyond the subjects in the study to the wider population. Advocates feel the experimental approach has several advantages. First, its highlycodified procedures provide an elabo rate culture that is shared by experimental researchers worldwide. It uses agreed-upon criteria for determining statistical significance and attempts to replicate the objectivity of the physical sciences. Because this tradition has prestige in universities, preservice teachers can usually get training in the experimental approach. Because of its prescribed systematicity, comparisons across studies are often possible.

There are, however, several disadvantages to the experimental approach, particularly in research on language teaching and learning, because one can seldom control all the variables influencing the outcome of a treatment designed to "cause" learning. Language learning is too complex to be reduced to simple causal claims. The learner's aptitude, motivation, previous language learning experience, and intelligence are just a few of the variables involved. Thus, powerful research designs that require a high level of control over variables are not always possible in language research with human subjects, for ethical and practical reasons. Other disadvantages of experimental research are data-related. Most statistical procedures require quantifiable data, but many linguistic phenomena are not easily or validly measured or counted. In the pursuit of "hard data" and objectivity, we may therefore end up trivializing key variables because they are not easily quantified. This results in studies in which individual learners are represented only by test scores. Their individuality is further obscured when the scores are averaged. Furthermore, many common statistics require that individual items of the data be independent (i.e., not related to each other). This condition is questionable when we are analyzing discourse-length data, such as conversations or classroom transcripts. Most statistical procedures work more reliably with large numbers of subjects, which may not be available to language researchers. Because of these concerns, many researchers have turned to other approaches toconduct research on language teaching and learning. In my opinion, the two most important are naturalistic inquiry and action research. Whereas experimental research uses control and experimental groups randomly selected for the study, naturalistic inquiry works with naturally occurring settings and groups. In this tradition, preselection, or control, of variables is purposefully avoided. The main goal of naturalistic inquiry is to discover patterns in behavior by describing phenomena, rather than to find causes of observed behavior. This approach o research is interpretive and exploratory: Its goal is to understand the phenomena beinginvestigated. Naturalistic inquiry includes many different research methods, such as ethnographies and case studies.

Ethnography is "the study of people's behavior in naturally occurring, ongoing settings, with a focus on the cultural interpretation Of behavior" (WatsonGegeo, 1998, p. 576). A key tenet of ethnography is the emic principle the idea that ethnographers must try to understand the participants' perspective in any given situation. A distinction is made between the emic perspective, held by the members of a group, and the etic perspective, based on the researcher's interpretive framework. Another important method of naturalistic inquiry is the case study. The case may be oneperson or a few people, single or multiple classrooms, or even schools. In the experimental research tradition, case studies are held in low regard because their lack of control over variables precludes making strong causal claims. In that tradition, theprimary value of casestudies is in g enerating hypotheses. However, in naturalistic inquiry, welldocumented, longitudinal case studies are highly valued for their illuminating insights and vivid exemplars. In our field, case studies have been especially important in second language acquisition research. The naturalistic inquiry tradition offers several advantages. It permits in-depth study of individuals, settings, or interactions. In emphasizing both the emic and the eticperspectives, no point of view is disregarded. Naturalistic inquiry promotes discussion of language issues that are often lost in statistical analyses associated with experimental studies. In addition, its analytic procedures are not restricted to quantifiable data.Naturalistic inquiry typically generates readerfriendly reports that do not require statistical training to be understood. It also yields rich interpretive accounts of linguistic and sociolinguistic phenomena. There are, however, several disadvantages to naturalistic inquiry. Data collection, reduction, and analysis are extremely labor-intensive, particularly since ethnographies and case studies are longitudinal by nature. There is also a lack of agreed-upon criteria for determining the significance of the outcomes. Furthermore, the analytic procedures of naturalistic inquiry are not as well understood or codified in our field as are those of experimental research, and training in this approach is somewhat less accessible. The third major approach to empirical research in language teaching and learning is called action research. Until recently, this approach was less well known than the other two, but it is rapidly gaining ground and is very promising for teachers who wish to conduct research in language classrooms. The term action research refers to a reiterated cycle of procedures. After planning, anaction (or "smallscale intervention") is implemented to improve a situation. The apparent results are systematically observed through a variety of data collectionprocedures (e.g., audio or video recordi ngs, observers' notes, etc.). The action researcher then reflects on the outcome and plans a subsequent action, after which the cycle begins again. Action research, like naturalistic inquiry, is conducted in naturally occurring settingsrather than with artificially composed experimental groups. Action research is sometimes described as "participatory research" because the researchers are usually members of the community under study -- a clear contrast to the desired distance and objectivity of experimental research. The broad goals of action research are to seek local understanding and to bring aboutimprovement. "Action research is a form of self-reflective enquiry undertaken byparticipants in social situations in order to improve the rationality and justice of their own social or educational practices, as well as their understanding of these practices and the situations in which these practices are carried out" (Kemmis and McTaggart, 1989, p. 2). Action research has several advantages. It does not require quantifiable data, largenumbers of subjects, or artificial control over variables. By definition, it is intended to lead directly to applicable results, and it involves participants in investigating andimproving their own settings. Thus, conducting action research can be an empowering process. There are also disadvantages to action research. Although this approach began in the1940s, for various reasons it has not enjoyed much prestige in some parts of the world. As a result, relatively few published examples are available in the literature on linguistics or language education, and there is still limited professional status associated withconducting action research. One of the reasons for this current lack of popularity is because there are no agreed-upon criteria for determining the significance of the results in action research. Critics claim that, because there is little or no control over variables, no strong causal statements can be made. Because the subjects are not andomly selected from the population, the findings may not be generalizable beyond the particular setting and people involved. Also, conducting

action research creates additional work for teachers. One possible solution is to use a collaborative model of action research in which teams of teachers and outside researchers pair up to conduct action research in the teachers' settings. Since the goals of action research are to develop local understanding and bring aboutmprovement in one's own context, action researchers typically do not concern themselves with issues of generalizability. But if action research is conducted exclusivelyby participants in a given setting, the results may be limited to an entirely emic perspective. And if there is no concern for generalizability, then action researchers may not be motivated to disseminate their findings, and much useful information could be hidden in staff-room discussions. I have been discussing experimental research, naturalistic inquiry, and action research as if they were completely separate approaches -- and indeed, their underlying philosophies and goals are quite different. But we should not overlook commonalities or affinities among the three. For example, some experimental research uses both quantitative and qualitative data to provide a richer interpretation in the data analysis. Some qualitative data collected in the naturalistic inquiry or action research traditions can be codified and quantified in a variety of ways. When the experimental tradition was dominant, and alternative research paradigms were scorned for yielding "soft" data, researchers seldom combined procedures drawn from the different traditions. Nowadays, language education researchers use diverseprocedures to address questions of interest. So, while there are clear differences in goals and philosophies, these various approaches to research may not be as mutually exclusive as they may appear at first glance. These three approaches to empirical research in language education have very different purposes as well as differing advantages and disadvantages, but none can be said, a priori and out of context, to be superior or inferior to the others. It is simply an issue of appropriateness. A researcher must choose the approach that is right for the hypothesesor research questions under investigation. Arguments over which approach is bestoverlook the multiple purp oses of empirical research and also ignore questions of personal preference. Some people are more comfortable with the control aspect of the experimental research tradition; others, finding such control dehumanizing, prefer naturalistic inquiry or action research. Regardless of our choice of approach, it is important for us as ESL and EFL educators to be well informed about research alternatives. It is my hope that in the future, the potential of each approach will be more fully realized and that teaching and learning will be improved, in part, through more effective research.

References Kemmis, S., and McTaggart, R. (1989). Action research. IATEFL Newsletter , 102, 2-3.Watson-Gegeo, K. A. (1988). Ethnography in ESL: Defining the essentials. TESOLQuarterly , 22, 575-592.Kathleen C. Bailey, President of TESOL, 1998-1999