The harms of inequity are rarely overestimated: Comment on Marks and O'Connell (2021)

Abstract

AbstractMarks and O'Connell's (Review of Education, 2021, e3293) claim that an even‐handed review of the literature shows that social and economic background does not matter to educational outcomes in advanced economies, relies on questionable scholarship and statistical errors. Much of the evidence against the relevance of social and economic background relies on research that estimates gain scores, where the effects of student background on educational outcomes are much smaller because they are residual effects. They also claim that parental education and occupation are poor measures of student background but do not recognise that this leads to attenuated effects and thus argues against their thesis. Finally, there is a lack of serious engagement with the literature: cherry‐picked numbers, summaries of studies that are at odds with the conclusions of the original authors and even two cases where thought experiments are presented as empirical evidence. Research into inequity does at times lack rigour in measurement and analysis, and not accounting for heritable abilities can confound and exaggerate the association between student background or school composition and educational outcomes, but simple calculations suggest that the resulting bias is offset by attenuation due to measurement error.Context and implicationsRationale for this studyThis comment was written to address shortcomings in a 2021 review article, ‘Inadequacies in the SES–Achievement model: Evidence from PISA and other studies’, by Gary Marks and Michael O'Connell.Why the new findings matterBecause review articles are an essential building block for the growth of a science, it is important that they are an even‐handed evaluation of the available evidence and do not rationalise or misinterpret the primary research they claim to summarise, a bar that the original review did not meet. When readers are aware of areas in which the original review fell short, they can form a more nuanced opinion of the merits and shortcomings of current research that incorporates a measure of social and economic background.Implications for researchers and policy makersThis comment explains ways in which the estimated effect of noisy measures of socioeconomic status may be disattenuated, usually leading to bigger estimates with less bias. It explains in what situations researchers should be careful of the opposite inflationary effect of genetic confounding but also explain that for many kinds of research it is unlikely to matter. Policy makers should know that the effects of social and economic background are hard to quantify and interpret, but that existing research into inequity and disadvantage is not generally misleading or flawed beyond repair, as Marks and O'Connell would have it, and instead feel confident that such measures will continue to be useful in setting educational policy.