PISA Data Research Articles

Positive Geographical Spillovers of Human Capital on Student Learning Outcomes

Human capital spillovers arise when the presence of individuals with high levels of human capital makes others more productive. If the students’ higher achievement scores are associated with human capital spillovers, a social return to education is generated. In this context, this paper developed the rationale and statistic methods for examining the association between regional human capital stocks and student learning outcomes. We used PISA data for Spain from the 2015 wave. On average, 15-year-old students scored 486 points in mathematics, but there were statistically significant differences across the 17 regions of Spain (comunidades autonomas). Three-level regression modeling showed that an extra year of region-level average schooling is associated with PISA math scores 14.5 points higher, after controlling for student and school level variables. We used mediation analysis to test whether educational expectations mediated this effect of regional human capital on student achievement. The mediation analysis showed that students who live in regions with a high prevalence of well-educated residents expect more years of education and perform better on the PISA test. This paper also found that the current expenditure per student in the different comunidades autonomas does not explain the regional differences in math performance.

Is the math gender gap associated with gender equality? Only in low-income countries

In their seminal article, Guiso et al. (2008) uncover a positive relationship between several measures of gender equality and the math gender gap (which tends to favor boys) by exploiting cross-sectional variation in PISA test scores from 39 countries - the majority of which belong to the OECD - at a given year (2003). Using five waves of PISA data spanning the period 2003-2015 and exploiting variation both across- and within-countries, we find that the positive association between the female-male gender gap in math test scores and several measures of gender equality vanishes in OECD countries once we account for country fixed effects. Interestingly, our analysis also uncovers a positive and statistically significant association between the math gender gap and several gender equality indicators for countries in the bottom quartile of per capita GDP. This association is robust to controlling for country-level time-invariant unobserved heterogeneity.

Contextual effect of school SES on reading performance: a comparison between countries in the European Union

ABSTRACT A central objective of schooling processes at the international level is reading skills development. Unfortunately, many students in the European Union underperform at this, and these low performances can be more pronounced in countries with lower Human Development Index (HDI) values. This study analysed the contextual effect of school socioeconomic status (SES) on reading performance using PISA data from 27 countries. We found that school SES had a positive contextual effect on student reading performance, especially countries with lower HDI values, via reading self-competence. However, in middle and higher HDI countries, the contextual effect was smaller, and we did not observe an indirect effect via reading self-competence. We conclude that school SES, rather than student SES, matters the most. Therefore, the modification of school composition and resources might act as a buffer for low student SES, particularly in lower HDI countries.

Gender Equality and the Math Gender Gap

In their seminal article, Guiso et al. (2008) uncover a positive relationship between several measures of gender equality and the math gender gap (which tends to favor boys) by exploiting cross-sectional variation in PISA test scores from 39 countries – the majority of which belong to the OECD – at a given year (2003). Using five waves of PISA data spanning the period 2003-2015 and exploiting variation both across – and within – countries, we find that the positive association between the female-male gender gap in math test scores and several measures of gender equality vanishes in OECD countries once we account for country fixed effects. Interestingly, our analysis also uncovers a positive and statistically significant association between the math gender gap and several gender equality indicators for countries in the bottom quartile of per capita GDP. This association is robust to controlling for country-level time-invariant unobserved heterogeneity.

A scoping review of the relationship between students’ ICT and performance in mathematics and science in the PISA data

This scoping literature review examines the relationship between ICT and performance scores in mathematics and science for students around the world included in the PISA assessment. In this review we examined 25 publications and showed that the relationship between ICT and academic achievement is not consistent. The different types of ICT revealed different relationships with performance, depending on the subject and country of the students being examined. Although there is a lack of overall consensus, it seems that moderate use, rather than high or no use of ICT, can positively predict academic scores. Although autonomy, interest, and use of ICT as a topic in conversations have been less scrutinized by researchers, they seem to positively predict both mathematics and science scores in 15-year-old students. Implications, limitations, and recommendations are discussed.

Longitudinal Study of Socioeconomic Segregation Between Schools in the UK

The aim of this study is to understand the behavior of school segregation by socioeconomic level in the UK. To do this, all data from the United Kingdom are analyzed in the PISA Assessment from 2000 to 2015 and the Gorard index, Dissimilarity index, and the Isolation index are estimated. The analysis has shown that socio-economic segregation between schools has declined somewhat in the UK from 2000 to 2015, although the clustering of the 25% poorest of students remained relatively static since 2006. England remains more highly segregated by poverty than Wales, Scotland and Northern Ireland. The segregation levels of the 10% poorest student has declined in state-maintained schools but shown a sharp increase in private schools. The level of isolation of disadvantaged students is less in state-maintained schools than private schools. The findings show that poverty segregation trends using PISA data match with segregation trends previously analysed using the national datasets using only state-maintained schools for England. This finding leads to research implications for a detailed analysis of national school segregation trends, including student data from private school.

Gibbs-Slice Sampling Algorithm for Estimating the Four-Parameter Logistic Model

The four-parameter logistic (4PL) model has recently attracted much interest in educational testing and psychological measurement. This paper develops a new Gibbs-slice sampling algorithm for estimating the 4PL model parameters in a fully Bayesian framework. Here, the Gibbs algorithm is employed to improve the sampling efficiency by using the conjugate prior distributions in updating asymptote parameters. A slice sampling algorithm is used to update the 2PL model parameters, which overcomes the dependence of the Metropolis–Hastings algorithm on the proposal distribution (tuning parameters). In fact, the Gibbs-slice sampling algorithm not only improves the accuracy of parameter estimation, but also enhances sampling efficiency. Simulation studies are conducted to show the good performance of the proposed Gibbs-slice sampling algorithm and to investigate the impact of different choices of prior distribution on the accuracy of parameter estimation. Based on Markov chain Monte Carlo samples from the posterior distributions, the deviance information criterion and the logarithm of the pseudomarginal likelihood are considered to assess the model fittings. Moreover, a detailed analysis of PISA data is carried out to illustrate the proposed methodology.

ICT Use at home for school-related tasks: what is the effect on a student’s achievement? Empirical evidence from OECD PISA data

ABSTRACT In this paper, we have employed data from the OECD’s Programme for International Student Assessment (PISA, 2012 edition) on the EU-15 countries in order to investigate the effect of using ICT at home on achievement. By employing Propensity Score Matching, we provide robust evidence that in most countries there is a negative association between using computers intensely for homework and achieving lower test scores across all subjects. Such negative effect affects the achievement of both low- and high-performing students and is robust to a specification that consider unobservable self-sorting of students across schools. Our findings suggest that a more cautious approach should be taken with regards to the wide-spread use of digital innovation as a means to support students’ out-of-school work.

Is PISA more important to school reforms than educational research? The selective use of authoritative references in media and in parliamentary debates

ABSTRACT Two separate data searches underlie this analysis of how references to educational research and to PISA are used in the Swedish education debate. The data consist of 380 newspaper articles from the eight largest print media outlets in Sweden and 200 protocols from parliamentary debates (2000 to 2016) that made explicit reference to ‘PISA’ and/or to ‘educational research’. Based on a content analysis of this material, in which notions of policy borrowing and de-/legitimization are central, we describe the result as a selective use of PISA data and of educational research in the education debate. PISA is used to legitimize selective (party political) solutions that are oriented towards problems of teaching. The analysis also shows that politics and the media debate concerning education seems disinterested in educational research in a broader sense and that PISA seems to offer sufficient and ‘neutral’ authoritative knowledge and support for policy and reforms. We argue that PISA is the first way to obtain legitimate support for educational reforms. In so doing, the kinds of problems that these reforms aim to solve have been narrowed down to teaching- or practice-oriented problems.

Uma proposta interdisciplinar entre Literatura, Linguística e Ensino

Esta resenha apresenta discussões desenvolvidas no evento Abralin ao Vivo, durante a conferência Literatura, Linguística e Ensino, proferida pela professora Marisa Lajolo. Ela articulou dados recentes do PISA sobre os níveis de leitura no país com textos midiáticos que mostram valores sociais do século XIX para mostrar que os problemas nesse âmbito não são recentes. Além disso, a professora fez reflexões importantes sobre as práticas atuais de leitura, o que se ensina nos cursos de Letras e o que se aprende nas aulas de linguagens do ensino básico para, por fim, apresentar sua proposta interdisciplinar. Desse modo, a integração entre as pesquisas em estudos linguísticos e literários e a prática docente pode contribuir para um saber efetivo e crítico.

Robust Haebara Linking for Many Groups: Performance in the Case of Uniform DIF

The comparison of group means in item response models constitutes an important issue in empirical research. The present article discusses a slight extension of the robust Haebara linking approach of He and Cui by proposing a flexible class of robust Haebara linking functions for comparisons of many groups. These robust linking functions are robust against violations of invariance. In this article, we investigate the performance of robust Haebara linking in the presence of uniform DIF effects. In an analytical derivation, it is shown that the robust Haebara linking approach provides unbiased estimates of group means in the limiting case p=0. In a simulation study, it is demonstrated that the proposed variant of the Haebara linking approach outperforms existing implementations of Haebara linking to some extent. In an empirical application using PISA data, it is illustrated that country means can be sensitive to the choice of linking functions.

Bayesian covariance structure modelling for measurement invariance testing

In a Bayesian Covariance Structure Model (BCSM) the dependence structure implied by random item parameters is modelled directly through the covariance structure. The corresponding measurement invariance assumption for an item is represented by an additional correlation in the item responses in a group. The BCSM for measurement invariance testing is defined for mixed response types, where the additional correlation is tested with the Bayes factor. It is shown that measurement invariance can be tested simultaneously across items and thresholds for multiple groups. This avoids the risk of capitalization on chance that occurs in multiple-step procedures and avoids cumbersome procedures where items are examined sequentially. The proposed measurement invariance procedure is applied to PISA data, where the advantages of the method are illustrated.

Introduction

Introduction

Performance of missing data approaches under nonignorable missing data conditions

Approaches for dealing with item omission include incorrect scoring, ignoring missing values, and approaches for nonignorable missing values and have only been evaluated for certain forms of nonignorability. In this paper we investigate the performance of these approaches for various conditions of nonignorability, that is, when the missing response depends on i) the item response, ii) a latent missing propensity, or iii) both. No approach results in unbiased parameter estimates of the Rasch model under all missing data mechanisms. Incorrect scoring only results in unbiased estimates under very specific data constellations of missing mechanisms i) and iii). The approach for nonignorable missing values only results in unbiased estimates under condition ii). Ignoring results in slightly more biased estimates than the approach for nonignorable missing values, while the latter also indicates the presence of nonignorablity under all simulated conditions. We illustrate the results in an empirical example on PISA data.

Comparing the evolution of productivity and performance gaps in education systems through DEA: an application to Latin American countries

The main objective of this paper is to propose a tool for measuring the productivity and performance gaps across a set of Decision Making Units for monitoring their evolution and analyzing their components over time. To do this, we use the approach proposed by Aparicio and Santín (Eur J Oper Res 267(1):227–235, 2018) which is grounded on a base-group base-period productivity index and Data Envelopment Analysis. Additionally, we propose a new index for measuring the performance gap between two or more groups of production units and its decomposition in effectiveness gap and outcome possibility set gap. As an empirical illustration of the approach, we focus our attention on the educational sector. In particular, we analyze six Latin American countries over time. For this purpose, we rely on OECD–PISA data aggregated at school level. Over the period 2006–2018, performance and productivity followed very different paths in each country showing that the correlation between school performance and productivity is very low. Therefore, we suggest that the simultaneous analysis of performance and productivity gaps together with their evolution over time is a must in order to benchmark countries and monitor improvements and weaknesses in education systems.

What PISA and ASPIRES studies tell us about the nuanced influence of cultural capital on student learning: Construct complexity, student outcomes and contexts

This article juxtaposes the evolving scholarship investigating the influence of cultural capital on student learning and the developing trajectory of sociologists employing quantitative research methods. It critically reviews results from 34 studies analysing PISA or ASPIRES data to generate three key insights demonstrating complexities in relationships between cultural capital and student outcomes. First, cultural capital is a complex construct with many variables (different forms of cultural capital, highbrow/non‐highbrow cultural capital) varying in their association with student literacy and parents’ cultural capital affecting student literacy both directly and indirectly. Second, the influence of cultural capital may vary with types of student outcomes measured. Specifically, there are cultural variables associated with student literacy in different subjects, some cultural capital variables are more important for student literacy for a subject, and cultural capital influences student aspirations in addition to their literacy. Third, the influence of cultural capital on student literacy may vary with educational and social contexts. Cultural capital is more important in decentralised education systems, in low‐achieving schools within differentiated education systems, in higher‐quality schools and in more unequal, developed societies. In contrast, it matters less in high‐achieving and standardised education systems, and in societies valuing education. Given the complexity in cultural capital, future research can ascertain how each cultural capital variable contributes to student learning and compare their effectiveness. It is also important to identify which cultural capital variables matter for specific student outcomes. Lastly, these insights highlight the need to recognise contextual differences in studies of social inequality.

Supporting and thwarting interpersonal dynamics and student achievement: a multi-level examination of PISA 2015

ABSTRACT Stage-environment fit theory posits that the interpersonal dynamics and structural (i.e. school-level) resources afforded by a student’s academic environment can either support or thwart student thriving (e.g. achievement). Thus, it is important to examine how these factors impact student achievement at the student- and school-level. This investigation utilized the 2015 PISA data set (N = 210,381 students in 7964 schools from 32 OECD countries) to examine the extent to which school belonging (supporting) and teacher unfairness (thwarting) predict students’ achievement (in reading, mathematics, and science) at both the student- and school-level. Findings indicate that at the student-level (but not school-level), teacher unfairness significantly thwarts achievement; in contrast, at the school-level (but not the student-level), school-belonging significantly supports achievement. These findings suggest that distinct interpersonal dynamics differentially impact achievement more saliently either at the student- or school-level – underscoring the importance of conducting multilevel modelling when investigating these social phenomena.

Exposure to crime and academic achievement: A case study for Costa Rica using PISA data

We analyse the effect of violent crime on students' results in Costa Rica. The combination of PISA data and highly disaggregated information on crime at the district level is a main contribution of this article. Our methodological approach provides also some novelties in the field, the most important being the use of a simultaneous equations model solved with instrumental variables. This allows us to obtain unbiased and efficient estimators in the presence of simultaneity between education and crime. We conclude that exposure to crime, measured by the homicide rate, has a negative and significant impact on academic achievement. This effect is particularly important in the case of students attending schools situated in districts with a lower social development.

Material resources, school climate, and achievement variations in the Philippines: Insights from PISA 2018

Educational reform programs usually institute systemwide changes. However, this assumption of similar school problems mask variations between schools and students. The 2018 Philippine PISA data (n = 7233) reveal salient variations in academic achievement that happen between schools and among students. Between schools, material resource and school climate problems have different origins and influence depending on school type and location, while among students, gender and socioeconomic status act as predictors but fail to fully explain the variation. This has implications for school-specific investments for material resource and school climate improvement, and student-specific interventions that detect and instruct those who are lagging behind.

Substitution or Complementation: The Relationship between School Education and Shadow Education

In order to explore the relationship between school education and shadow education, this study applied two-level Bernoulli Model to estimate the school-level determinants of private tutoring by analyzing the data of PISA 2015 of four regions of China (Beijing, Shanghai, Jiangsu, and Guangdong). The results indicated that: (1) the four regions have large scale of private tutoring ranging from 58.7% to 74.0% for science, mathematics, and reading; (2) the participation rate of private tutoring in villages is much higher than that in cities; (3) the quality of school teachers significantly negatively affects the likelihood for receiving tutoring. Shadow education plays both substitutional and complementary roles for school education, and the growth of shadow education is in accompany with the low quality of school education in rural areas but with high quality of school education in urban areas. Instead of increasing Education resources and improving infrastructure, increasing teacher quality will reduce the participation probability of private tutoring.