How Much Do Students Learn in College?

Abstract

Cross-Sectional Estimates Using the College BASE Introduction What students learn during the undergraduate years in postsecondary education has been the focus of a large body of inquiry for the past 30 years (e.g., Bowen, 1977; Gardner, 1994; Kuh, Douglas, Lund, & Ramin-Gyurnek, 1994; Osterlind, 1997; Pascarella & Terenzini, 1991; Stage, Muller, Kinzie, & Simmons, 1998). The vast majority of these studies document the change, or growth, that occurs between the first and subsequent years of college. Overall, this research has taken three different forms. The first form is longitudinal (or panel) studies that trace changes in the same students over time on some standardized, objective measure of cognitive development or knowledge (e.g., critical thinking, mathematical reasoning, literacy, history, science reasoning). The average change between the freshman and senior year of college, for example, might then be an estimate of the impact of college. Recent examples of studies of this type are by Baxter Magolda (1990, 1991), Giancarlo and Facione (1997), Hart, Rickards, and Mentkowski (1995), King and Kitchner (1994), Kube and Thorndike (1991), May (1990), Saucier (1995), Thorndike, Andrieu-Parker, and Kube (1990), Thorndike and Andrieu-Parker (1992), and Zhang and RiCharde (1999). The second form is cross-sectional (or cohort) studies that administer some standardized objective measure of cognitive development or knowledge to students with different levels of exposure to postsecondary education. For example, one might administer the same instrument at the same time to first-year students and seniors and assume that the difference between the average scores of the two groups is an estimate of the impact of college. Recent examples of this type of study are found in ACT (1993), Durham, Hays, and Martinez (1994), Evans (1989), Hill (1995), Jehng, Johnson, and Anderson (1993), King and Kitchener (1994), McDonough (1997), Osterlind (1997), Pearson and Rogers (1998), and Wood (1997). The third form of study employs students' retrospective self-reports of how much they change or gain along various dimensions of knowledge or cognitive development during college. The assumption, not without some empirical support (e.g., Anaya, 1999a, 1999b; Bradburn & Sudman, 1988; Trusheim, 1994), is that students will be both truthful and accurate in responding to such instruments. Recent examples of this type of study are by Bauer (1996, 1998), Dollar (1991), Feldman (1994), Lincoln (1991), Tan (1995), and Williams (1996). Each of these three popular forms of research provides legitimate approaches to understanding the change and growth in knowledge and cognitive development that occur during college, and in the clear majority of cases they report statistically significant and sometimes sizable indications of growth or change. Yet, if one is interested in estimating the net impact of college (i.e., how much of what happens is attributable to the experience of college and not to coincident or competing influences, or measurement error), then each of these forms of research is also limited. For example, longitudinal panel studies of change during college have the problem of not knowing whether similar students who do not attend college might also change to the same or to a similar extent. Additionally, there is the problem of the practice effect (part of the change from the first to subsequent testings may be due simply to students taking the test more than once and not to the impact of college). Cross-sectional or cohort studies have their own methodological problems too. Probably the most important as an estimate of the impact of college is concerned with sample mortality. Because of sample mortality, seniors might represent a more academically select and motivated group than first-year students. Thus, higher average scores by seniors versus first-year students, for example, might reflect comparison group differences in academic aptitude and motivation, not simply the impact of college. …