Large-scale International Surveys Research Articles

Juxtaposing Acquired and Required Skills: Latent Class Analysis of Self-Assessment Scales in an International Survey

Ensuring comparability of Likert-style items across different countries is a widespread challenge for authors of large-scale international surveys. Using data from the EUROGRADUATE Pilot Survey, this study employs a series of latent class analyses to explore which response patterns emerge from self-assessment of acquired and required skills of higher education graduates and how these patterns vary between eight participating countries. Results show that countries differ in the number of classes which most accurately fit the data structure. The effort to overcome national specifics by combining the levels of acquired and required skills into a single measurement of skill surplus/deficit reduces heterogeneity of patterns across countries and slightly increases comparability, yet the notion that respondents understand the scale in the same way across countries (measurement invariance) is not supported.

Cross-country variation in patience, persistent current account imbalances and the external wealth of nations

This paper is the first to utilize large-scale international surveys on economic preferences to examine the long-run relationships between patience, current accounts and external wealth. We find robust empirical evidence that countries with more patient individuals tend to run persistent current account surpluses, which in turn result in the accumulation of foreign assets. This theoretically plausible but empirically unexplored relationship holds true for euro area current account imbalances, global current account imbalances and net foreign asset positions worldwide. While the existing current account literature concentrates on proximate macroeconomic determinants, this paper’s extension of inferences from deep determinants (i.e., time preferences) to the distribution of external wealth of nations (i.e., net foreign asset positions) makes a unique contribution to the literature. Our finding showing that deep heterogeneities contribute to external imbalances suggests that the pattern of current account imbalances as well as the distribution of external wealth of nations might be very persistent.

Abilities in the blind spot of testing regimes: Eliciting the benefits and the limitations of participatory research approaches for numeracy in adult basic education

In the past decade, the numeracy component in adult basic education has gained scholarly attention. The issue has been addressed by large-scale assessments of adults’ skills and intergovernmental policy agendas, but also by qualitative research into numeracy from the perspective of social practice theory. However, some aspects of numeracy are still under-researched. This article argues that adults with learning difficulties (also referred to by some as intellectual disabilities) and their numeracy-related abilities are still hidden in the blind spot of large-scale testing regimes. To address this underrepresentation, the authors present an overview of the key paradigms of a global testing culture, outline their two areas of interest, disability and numeracy, and identify the blind spots of large-scale international surveys. They consider ways of extending the range of methodological approaches to investigating (literacy and) numeracy, and opt for a participatory research approach, for which they identify five guiding principles. They showcase and reflect on these guiding principles by presenting selected data from a small-scale qualitative study on numeracy practices of adults with learning difficulties. Their aim is to demonstrate the benefits and limitations of using participatory research approaches in the context of investigating numeracy among all members of a country’s adult population.

Qualitative Assessment of Adult Patients' Perception of Atopic Dermatitis Using Natural Language Processing Analysis in a Cross-Sectional Study.

IntroductionAtopic dermatitis (AD) is an incurable, inflammatory skin disease characterized by skin barrier disruption and immune dysregulation. Although AD is considered a childhood disease, adult onset is possible, presenting with daily sleep disturbance and functional impairment associated with itch, neuropsychiatric issues (anxiety and depression), and reduced health-related quality of life. Although such aspects of adult AD disease burden have been measured through standardized assessments and based on population-level data, the understanding of the disease experienced at the patient level remains poor. This text-mining study assessed the impact of AD on the lives of adult patients as described from an experiential perspective.MethodsNatural language processing (NLP) was applied to qualitative patient response data from two large-scale international cross-sectional surveys conducted in the USA and countries outside of the USA (non-USA; Canada, France, Germany, Italy, Spain, and the UK). Descriptive analysis was conducted on patient responses to an open-ended question on how they felt about their AD and how the disease affected their life. Character length, word count, and stop word (common words) count were evaluated; centrality analysis identified concepts that were most strongly interlinked.ResultsPatients with AD in all countries were most frequently impacted by itch, pain, and embarrassment across all levels of disease severity. Patients with moderate-to-severe AD were more likely than patients with mild AD to describe sleep disturbances, fatigue, and feelings of depression, anxiety, and a lack of hope that were directly associated with AD. Centrality analysis revealed sleep disturbance was strongly linked with itch. Collectively, these concepts revealed that patients with AD are impacted by both physical and emotional burdens that are intricately connected.ConclusionsQualitative data from NLP, being more patient-centric than data from clinical standardized measures, provide a more comprehensive view of the burden of AD to inform disease management.

Contributing to the common good? Media coverage of the international largescale assessment of adult skills (PIAAC) in four national contexts

This paper focuses on the rapidly expanding field of largescale international assessment surveys and their impact on the field of adult education and learning. I take the case of OECD’s survey assessing adult skills (PIAAC) and situate it within the wider context of the datification of educational policy and practice. The claims made for the policy effects of surveys like PIAAC are far-reaching and include the promotion of economic growth and more inclusive and equitable societies. I examine how these claims are translated into national contexts by examining documentary data collected from the OECDs publicity materials and media coverage of the second round PIAAC survey findings in 2016 in four of the nine countries that took part: Singapore, Greece, New Zealand and Slovenia. Using a socio-material approach, I discuss how these discourses were managed by the OECD and national actors and show how the survey findings are framed and interpreted through existing public debates. The paper concludes that international assessments do not serve the goals of growth and equity in any straightforward way, since many interests and contextual factors may intervene to create a mismatch between the testers’ intentions, media discourse and policy outcomes.

Wages, mental abilities and assessments in large scale international surveys: Still not much more than g

We examine the role of General Mental Ability (GMA or g), versus specific abilities, in predicting wages among 69,901 participants from 19 countries in the Programme for International Assessment of Adult Competencies (PIAAC). We define GMA as the first principal component in a battery of three ability tests, and specific abilities as the low order components. Our initial results – a difference of 52%, between a g only model and a g + specific abilities model (R2s of 0.061 and 0.093, respectively) – is considerably different from earlier results suggesting that "there is not much more than g" in predicting performance. However, further analyses show that this difference is reduced to 0.5% when crucial non-cognitive individual differences (age and sex) are controlled for (R2s of 0.0763 and 0.0767, respectively). Path models of the relationships between individual differences, specific abilities, GMA and wage shed light on these results. Implications for the understanding of the relationship between mental abilities and wage, and to the understanding of cognitive test scores as representing various skills versus general ability, are discussed.

Ett demokratilyft för Sverige? Om relationen mellan literacy, skönlitteratur och demokrati i Läslyftet

A lift for democracy? About the relationship between literacy, fiction and democracy.The declining results in reading skills revealed in large-scale international surveys such as PISA are seen as a ...

Worsening work conditions and rising levels of job satisfaction? Measuring the happiness of academics in Japan

Working conditions in academia are generally considered to be deteriorating. Data from surveys which look at the job satisfaction of academics, however, do not clearly support this notion. This appears to be especially true for the case of Japan. Much of the recent literature on academics’ job satisfaction globally relies on the comparison of two large-scale international surveys from 1992 and 2007, despite significant shortcomings of such an approach. In the case of Japan, where a third survey - also from 2007 - is available, these shortcomings become particularly apparent. After presenting an overview of how the working conditions of Japanese academics have changed in the twenty-five years between these surveys, the present paper identifies some of the inconsistencies and problems involved in such surveys and then explores the ‘paradox’ of rising levels of job satisfaction in Japan despite the objective worsening work conditions over the same period.

Confucius institutes in the US: Platform of promoting China’s soft power

Abstract This study examines the driving force behind China’s “soft power” strategy and how Confucius Institute (CI) fits in the strategy. Drawing on data from the large-scale international surveys from PewResearchCenter and the ten Hanban/CI annual reports (2006–2015), the study finds that China’s outreach through CI has increased the understanding between China and other countries and built a favored image of “a responsible big nation” in the international community. The study shows that, by December 2015, there were 500 CIs and 1,000 Confucius Classrooms (CCs) in 134 countries/regions in the world. Collectively, they offered a total of 211,500 classes and registered over 5 million students from 2006 to 2015. They also attracted 94.27 million participants for cultural exchanges and 23.73 million participants for Chinese language tests in the same time period. Data from the researcher’s 3-month-long field study on two CCs shows that the Chinese language and culture are successfully promoted in American schools and exerting tremendous attraction to students, parents, and communities.

Reliability, factor structure, and measurement invariance of the Dominic Interactive across European countries: Cross-country utility of a child mental health self-report.

Large-scale international surveys are important to globally evaluate, monitor, and promote children's mental health. However, use of young children's self-reports in these studies is still controversial. The Dominic Interactive, a computerized DSM-IV-based child mental health self-report questionnaire, has unique characteristics that may make it preeminently appropriate for usage in cross-country comparisons. This study aimed to determine scale score reliabilities (omega) of the Dominic Interactive in a sample of 8,135 primary school children, ages 6-11 years old, in 7 European countries, to confirm the proposed 7-scale factor structure, and to test for measurement invariance of scale and item scores across countries. Omega reliability values for scale scores were good to high in every country, and the factor structure was confirmed for all countries. A thorough examination of measurement invariance provided evidence for cross-country test score comparability of 5 of the 7 scales and partial scale score invariance of 2 anxiety scales. Possible explanations for this partial invariance include cross-country differences in conceptualizing items and defining what is socially and culturally acceptable anxiety. The convincing evidence for validity of score interpretation makes the Dominic Interactive an indispensable tool for cross-country screening purposes. (PsycINFO Database Record

Обзор международных образовательных проектов в области послешкольного образования

<p>В данной статье авторы рассматривают международные образовательные исследования. Целью статьи является обзор наиболее масштабных исследовательских международных образовательных проектов в области послешкольного образования. Рассматриваются такие проекты, как «Оценка результатов обучения в высшем образовании» (AHELO), «Программа международной оценки компетенций взрослых» (PIAAC), а также «Международное исследование преподавания и обучения» (TALIS) и «Международное исследование навыков по отношению к занятости и производительности труда» (STEP).</p><p>Целью перечисленных исследований является оценка компетенций, т. е. способности применять знания и навыки в нестандартных или новых ситуациях профессионального или повседневного характера, а также факторов, влияющих на уровень компетенций. В статье описываются методология проектов, структура измерительных материалов, объем и состав выборки, реализовывался ли проект в Российской Федерации.</p><p>Результаты обзора позволяют сделать вывод о необходимости участия Российской Федерации в масштабных исследовательских международных проектах с целью включения в международные базы данных, получения возможности реализации сравнительных исследований и последующего анализа, которые позволят понять сильные и слабые стороны российской образовательной системы, оценить риски и возможности, а также использовать наилучшие международные практики и опыт при модернизации образовательного сектора. Основываясь на результатах вышеперечисленных проектов, Россия сможет более эффективно формировать и использовать необходимые компетенции, поддерживая и внедряя идею обучения взрослых людей в течение жизни, преодолевая существующие несоответствия спроса-предложения на специалистов различной квалификации.</p>

Creating and using the evidence base: the case of the Active Ageing Index

The value of evidence-based policy-making depends on the quality and robustness of the available data. Many conceptual and operational difficulties restrict the comparability of the data collected in large-scale international surveys and their usefulness as an evidence base. Taking the Active Ageing Index (AAI) as a case study, this article argues that a greater understanding of the context of data sourcing and application is vital to the usefulness and transferability of the evidence generated. The difficulties of determining the effectiveness of evidence-based policy must also be understood in the context of the economic and political volatility of particular countries and, possibly, different traditions across nations in applied and academic research. Drawing on recent AAI findings, the article suggests how the AAI could be used most effectively as a toolkit by policy-makers seeking to devise evidence-informed active ageing strategies.

Exploring Young People's Civic Identities through Gamification: A Case Study of Finnish, Swedish and Norwegian Adolescents Playing a Social Simulation Game

This article is based on a case study where groups of Finnish, Swedish and Norwegian young people played a simulation game that stimulated collective deliberation on social issues. The game has been designed to provoke students to deliberate and to reflect on social problems relating to issues of citizenship and democracy. The analysis of the discussions, based on playing the simulation game, suggests that Nordic young people have a strong civic identity: they were ready to voice their opinion and they had the knowledge and skills to feel comfortable when expressing their ideas. The message of the study differs considerably from the more pessimistic view of Nordic young people's civic skills in large-scale international surveys. A qualitative approach seems to provide a deeper view on young people's skills in deliberative argumentation and the issues they want to contribute to and influence, painting a more positive picture of these future decision makers' level of engagement.

Assessing the Hypothesis of Measurement Invariance in the Context of Large-Scale International Surveys

In the field of international educational surveys, equivalence of achievement scale scores across countries has received substantial attention in the academic literature; however, only a relatively recent emphasis on scale score equivalence in nonachievement education surveys has emerged. Given the current state of research in multiple-group models, findings regarding these recent measurement invariance investigations were supported with research that was limited in scope to few groups and relatively small sample sizes. To that end, this study uses data from one large-scale survey as a basis for examining the extent to which typical fit measures used in multiple-group confirmatory factor analysis are suitable for detecting measurement invariance in a large-scale survey context. Using measures validated in a smaller scale context and an empirically grounded simulation study, our findings indicate that many typical measures and associated criteria are either unsuitable in a large group and varied sample-size context or should be adjusted, particularly when the number of groups is large. We provide specific recommendations and discuss further areas for research.

Test-retest reliability of selected items of Health Behaviour in School-aged Children (HBSC) survey questionnaire in Beijing, China

BackgroundChildren's health and health behaviour are essential for their development and it is important to obtain abundant and accurate information to understand young people's health and health behaviour. The Health Behaviour in School-aged Children (HBSC) study is among the first large-scale international surveys on adolescent health through self-report questionnaires. So far, more than 40 countries in Europe and North America have been involved in the HBSC study. The purpose of this study is to assess the test-retest reliability of selected items in the Chinese version of the HBSC survey questionnaire in a sample of adolescents in Beijing, China.MethodsA sample of 95 male and female students aged 11 or 15 years old participated in a test and retest with a three weeks interval. Student Identity numbers of respondents were utilized to permit matching of test-retest questionnaires. 23 items concerning physical activity, sedentary behaviour, sleep and substance use were evaluated by using the percentage of response shifts and the single measure Intraclass Correlation Coefficients (ICC) with 95% confidence interval (CI) for all respondents and stratified by gender and age. Items on substance use were only evaluated for school children aged 15 years old.ResultsThe percentage of no response shift between test and retest varied from 32% for the item on computer use at weekends to 92% for the three items on smoking. Of all the 23 items evaluated, 6 items (26%) showed a moderate reliability, 12 items (52%) displayed a substantial reliability and 4 items (17%) indicated almost perfect reliability. No gender and age group difference of the test-retest reliability was found except for a few items on sedentary behaviour.ConclusionsThe overall findings of this study suggest that most selected indicators in the HBSC survey questionnaire have satisfactory test-retest reliability for the students in Beijing. Further test-retest studies in a large and diverse sample, as well as validity studies, should be considered for the future Chinese HBSC study.

Assessment of cocaine and other drug dependence in the general population: “Gated” versus “ungated” approaches

Background There is a need for large-scale epidemiological surveys to be (a) faithful to diagnostic specifications and (b) constrain time and participant burden associated with each section of a possibly lengthy interview. Objective To examine whether one “gating” approach devised for recent large-scale international psychiatric surveys results in a reduced number of identified cases of drug dependence and/or biases in estimated associations with background characteristics. Design and setting Data from a recently released cross-sectional, nationally representative household survey, the United States National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) were analysed. Participants Forty-three thousand ninety-three English speaking adults aged 18 years and over. Main outcome measures Dependence upon cocaine and other illegal drug dependence, defined in two ways: “ungated” and “gated”. “Ungated” dependence included all persons meeting criteria for DSM-IV dependence, without regard for DSM-IV drug abuse clinical features. “Gated” dependence required at least one feature of DSM-IV drug abuse. Results There was no statistically robust decrement in the estimated prevalence of cocaine or other drug dependence using a “gated” assessment. Patterns of association of cocaine dependence with background characteristics were not appreciably different when the gated and ungated approaches were applied. Conclusions In panoramic mental health surveys, the inefficiency of an ungated approach must be balanced against the anticipated number of cases of dependence without associated social role impairments or harm. In this study, the reduction in the number of identified cocaine dependence cases appeared to be so small that even in a sample of over 40,000 participants, attenuation in population prevalence would prove difficult to detect.