Same but different? Measurement invariance of the PIAAC motivation-to-learn scale across key socio-demographic groups

Julia Gorges,Tobias Koch,Judith Offerhaus,Débora B Maehler

doi:10.1186/s40536-017-0047-5

Abstract

BackgroundData from the Programme for the International Assessment of Adult Competencies (PIAAC) revealed that countries systematically differ in their respondents’ literacy, numeracy, and problem solving in technology-rich environments skills; skill levels also vary by gender, age, level of education or migration background. Similarly, systematic differences have been documented with respect to adults’ participation in education, which can be considered as a means to develop and maintain skills. From a psychological perspective, motivation to learn is considered a key factor associated with both skill development and participation in (further) education. In order to account for motivation when analyzing PIAAC data, four items from the PIAAC background questionnaire were recently compiled into a motivation-to-learn scale. This scale has been found to be invariant (i.e., showing full weak and partial strong measurement invariance) across 21 countries.MethodsThis paper presents further analyses using multiple-group graded response models to scrutinize the validity of the motivation-to-learn scale for group comparisons.ResultsResults indicate at least partial strong measurement invariance across gender, age groups, level of education, and migration background in most countries under study (all CFI > .95, all RMSEA < .08). Thus, the scale is suitable for comparing both means and associations across these groups.ConclusionsResults are discussed in light of country characteristics, challenges of measurement invariance testing, and potential future research using PIAAC data.

Highlights

Background: Data from the Programme for the International Assessment of Adult Competencies (PIAAC) revealed that countries systematically differ in their respondents’ literacy, numeracy, and problem solving in technology-rich environments skills; skill levels vary by gender, age, level of education or migration background
We tested the three levels of MI across gender, age groups, level of education, and migration background within each of the 21 countries provided that the information from the datasets fulfilled the prerequisites
We summarize results for each grouping variable; in the Appendix, we provide more details concerning the specified multiple-group graded response models

Summary

Introduction

Data from the Programme for the International Assessment of Adult Competencies (PIAAC) revealed that countries systematically differ in their respondents’ literacy, numeracy, and problem solving in technology-rich environments skills; skill levels vary by gender, age, level of education or migration background. In order to account for motivation when analyzing PIAAC data, four items from the PIAAC background questionnaire were recently compiled into a motivation-to-learn scale This scale has been found to be invariant (i.e., showing full weak and partial strong measurement invariance) across 21 countries. Going beyond cross-national comparisons, the general OECD report (OECD 2013a) and country-specific publications (e.g., Maehler et al 2013; Statistics Canada 2013) provide in-depth analyses of specific population subgroup competencies that are relevant for researchers, educators, and policy-makers alike These analyses show systematic skill differences across gender, age groups, level of education, and migration. In order to account for motivation when analyzing PIAAC data, four items from the PIAAC background questionnaire have recently been compiled into a motivation-to-learn scale (Gorges et al 2016). The scale so far has been found to measure motivation to learn in an equivalent way across 21 countries and allows comparative analyses

Objectives

Methods

Results

Conclusion