Are large surveys of adult literacy skills as comparable over time as we think?

Samuel Vézina,Alain Bélanger

doi:10.1186/s40536-020-00080-3

Abstract

Recent literature shows that younger cohorts have lower levels of literacy ceteris paribus in Canada, the United States, Norway and other developed countries. Very few explanations are provided to justify the existence of this negative cohort effect. Yet this decline has serious implications for the economy, education system and society. In this paper, we focus on Canada and replicate the results published in the literature using the same methodology (synthetic cohorts). We use the same data from surveys of adult skills, namely the International Adult Literacy Survey (IALS), the Adult Literacy and Life Skills Survey (ALL) and the Survey of Adult Skills, a product of the OECD Programme for the International Assessment of Adult Competencies (PIAAC). Second, we conduct further analyses to better understand the effect of time on adults’ literacy skills. We then show that age has a negative effect on the literacy score, but also that a significant “period” effect (or rather a “survey” effect) can only be explained by a change in the instrument used to measure literacy skills from one survey to another. This article reveals that the negative cohort effect mentioned in the literature may be fallacious as it is exacerbated when the control for measuring instrument distortions is omitted. This paper contributes to advancing knowledge about the effect that age and cohort has on adult literacy levels in Canada. The results weaken the idea of a clear negative cohort effect; much of this effect would in fact be caused by a non-comparable methodology that systematically assigned individuals lower scores in the most recent survey cycles. These findings are important because they highlight the limitations of analyses that can be done with the synthetic cohort method using cross-sectional surveys on adult literacy skills not only in Canada, but also in all other countries where these surveys have been conducted.

Highlights

The concept of literacy, as defined in this article, is broader than just being able to read or write
Literacy skills here refers to the “the ability to understand, evaluate, use and engage with written texts to participate in society, to achieve one’s goals, and to develop one’s knowledge and potential” (OECD 2013, p. 20)
Law enforcement and even civic duties, such as tax returns, require high-level literacy skills. These skills are highly valorized on the job market, because they are formally required for any skilled employment, and because they are crucial in the development of more complex communication and reasoning skills (Carey 2014; Organisation for Economic Co-operation and Development (OECD) 2012)

Summary

Introduction

The concept of literacy, as defined in this article, is broader than just being able to read or write. In developed countries such as Canada, the challenge around literacy is to measure the number of people who can or cannot read and write, and to assess the degree of an individual’s understanding when reading and processing different types of day-to-day information. These basic literacy skills are becoming increasingly essential to enable individuals to thrive in contemporary societies. These skills are highly valorized on the job market, because they are formally required for any skilled employment, and because they are crucial in the development of more complex communication and reasoning skills (Carey 2014; OECD 2012)

Objectives

Results

Conclusion