Some Cautions in Synthesizing Research Studies

Abstract

Meta-analysis is quickly gaining momentum in the educational research community as a useful approach to synthesizing research studies, a way of systematically taking stock of what we know about a particular area of study. Investigators using this approach amass all obtainable studies on a topic with the goal of making a general statement about the presence and magnitude of a particular phenomenon. (For a general presentation of metaanalysis, see Cook & Leviton, 1980; Cooper & Rosenthal, 1980; Glass, 1976, 1977; Glass, McGaw, & Smith, 1981.) Reviewers have used meta-analysis to investigate broad questions about a wide range of domains, such as: Is psychotherapy effective (Smith & Glass, 1977; Smith, Glass, & Miller, 1980)? Are interpersonal expectancy effects present in research studies (Rosenthal & Rubin, 1978)? Is class size related to student achievement (Glass & Smith, 1979)? Is Keller's system of personalized instruction effective (Kulik, Kulik, & Cohen, 1979)? Is sex bias present in counseling and psychotherapy (Smith, 1980b)? Is there a relationship between the types of questions teachers ask and the achievement of students (Redfield & Rousseau, 1981)? Although meta-analysis may be an ap-