Missing in Measurement: Why Identifying Learning in Integrated Domains Is So Hard

Abstract

Integrating computational thinking (CT) and science education is complex, and assessing the resulting learning gains even more so. Arguments that assessment should match the learning (Biggs, Assessment & Evaluation in Higher Education, 21(1), 5–16. 1996; Airasian and Miranda, Theory into Practice, 41(4), 249–254. 2002; Hickey and Zuiker, Journal of the Learning Sciences, 21(4), 522–582. 2012; Pellegrino, Journal of Research in Science Teaching, 49(6), 831–841. 2012; Wiggins, Practical Assessment, Research and Evaluation, 2(2). 1990) lead to a performance-oriented approach to assessment, using tasks that mirror the integrated instruction. This approach reaps benefits but also poses challenges. Integrated CT is a new approach to learning. Movement is being made toward understanding what it means to operate successfully in this context, but consensus is neither general nor time tested (Kaput and Schorr, Research on technology and the teaching and learning of mathematics: Case and perspectives (Vol. 2, pp. 211–253) 2008). Movement is also being made toward developing methods for assessing CT. Despite the benefits of matching assessment with pedagogy, there may be intrinsic losses. One problem is that interactions between the two domains may invalidate the results, either because the gains in one may be easier to measure at certain times than the gains in the other, or because interactions between the two domains may cause measurement interference. Our examination draws upon both theoretical basis and also existing practices, particularly from our own work integrating CT and secondary science. We present a mixed-methods analysis of student assessment results and consider potential issues with moving too quickly toward relying on a rubric-based approach to evaluating this student learning. Centrally, we emphasize the importance of assessment approaches that reflect one of the most important affordances of computational environments, that is, the expression of multiple ways of knowing and doing (Turkle and Papert, Journal of Mathematical Behavior, 11(1), 3–33. 1992).