Reflections on the PIAAC Literacy and Numeracy Frameworks

Abstract

The Program for International Assessment of Adult Competencies (PIAAC) results, released in 2013, have spurred researchers to engage in rich analyses. In addition to making it possible to compare skill levels among the 23 participating nations, the PIAAC data have enabled researchers to analyze relationships among cognitive literacy, numeracy, and problem solving in technology-rich environments skills; age, gender, race/ethnicity, educational level; and a host of personal and social outcomes including health, economic well-being, and civic engagement. In 2014 and 2015, invited researchers reported their findings at PIAAC research conferences organized by the American Institutes for Research (AIR). The comments offered below comprise a revised version of discussant remarks I made in response to AIR-commissioned papers by Donna Curry and Amy Trawick presented at the 2015 conference. The authors described how the definitional frameworks for literacy and numeracy that guided the development of the PIAAC assessment tasks could also guide practice. The underlying assumption of these commissions is that in order for instruction to improve literacy levels, it should reflect current understandings of what literacy and numeracy are. I am a qualitative researcher and, admittedly, have neither the know-how to conduct highly sophisticated analyses to fruitfully mine PIAAC data, nor even the best understanding of what they mean. However, from where I sit looking at overviews and findings summarized in research conference papers, it seems PIAAC has given us a finer grained, more sophisticated description of a problem we all already know a great deal about--far too many people do not have the literacy skills they need--and we are making little, if any, progress in ameliorating the problem. In fact, Murray, Binkley, and Shillington (2015) demonstrated we are making negative progress. It is not a pretty picture. When I think about the PIAAC results, it is difficult not to feel some despair and to wonder why we cannot do better. One way to explain our lack of progress in improving adult literacy and numeracy skills comes from a guiding principle at the Institute for Healthcare Improvement. According to the founder, Don Berwick (as cited in Donahue, 2015), the principle is so foundational to improvement work, it is practically tattooed on the body of everyone who works there: Every system is perfectly designed to achieve exactly the results it gets (para. 3). This suggests the PIAAC findings are simply a reflection of the current U.S. adult basic education (ABE) system. If we want to get better outcomes, we have to change the system that produces them. This is obvious and simple to understand, yet so difficult to act on. PIAAC data probably cannot be used to point out effective program and policy interventions, although they can spur us to action by highlighting the failings of the system. However, as described by Curry (2015) and Trawick (2015), the literacy and numeracy frameworks that undergird PIAAC can be the basis of applied implementation and evaluation research with potential to improve results. The frameworks define, in elaborated and complex ways, what literacy and numeracy are. If we think of PIAAC as a test, the frameworks provide helpful guidance about how to teach to the test (i.e., raise literacy levels). Usually, teaching to the test is considered a gross neglect of good practice--something to avoid not only because it games the test but also because it equates to poor learning opportunities and narrow educational outcomes. But if the test measures things we really care about, then teaching to the test is ideal. This is exactly what the frameworks give us--the possibility of teaching to a test, for which the results would be meaningful. Curry (2015) and Trawick's (2015) papers tell us, in better and more sophisticated ways, what we already know to be good practice. They add descriptions of what it would look like to utilize these frameworks if we backward mapped to instructional design and classroom practice for outcomes as assessed by PIAAC. …