Multilevel Item Response Theory Research Articles

Healthy aging trajectories and their predictors among Chinese older adults: Evidence from a 7-year nationwide prospective cohort study

ObjectivesThis study aimed to identify healthy aging trajectories of Chinese older adults, and explore the factors contributing to these trajectories. MethodsWe used data from four waves (2011–2018) of China Health and Retirement Longitudinal Study. We developed a healthy aging metric based on the healthy aging framework of World Health Organization (WHO) and Bayesian multilevel item response theory (IRT) method. The healthy aging trajectories were identified using the latent class growth analysis. The predictors of trajectories were explored using multinomial logistic regression analysis. Additionally, we developed two alternative metrics for healthy aging based on the Chinese Healthy Ageing Index (CHAI) and Rowe and Kahn's model of successful aging, respectively. We compared these metrics to the one developed based on the WHO's healthy aging framework. ResultsWe identified three distinct healthy aging trajectories with varying scores and decline rates. Individuals who were female, had lower educational levels, resided in rural areas, experienced depression, had more chronic diseases, participated in fewer social activities, had fewer childhood friends, experienced more adverse childhood events, and had worse family financial status in childhood were more likely to experience a worse healthy aging trajectory compared to their counterparts. Supplementary analysis showed that healthy aging metric based on WHO definition and IRT method had the strongest association with health outcomes compared to the metrics based on CHAI, as well as Rowe and Kahn model. ConclusionsOur findings provide a foundation for the development of tailored interventions to enhance and sustain healthy aging among Chinese older adults.

Approaches to Item-Level Data with Cross-Classified Structure: An Illustration with Student Evaluation of Teaching

Student evaluation of teaching (SET) questionnaires are ubiquitously applied in higher education institutions in North America for both formative and summative purposes. Data collected from SET questionnaires are usually item-level data with cross-classified structure, which are characterized by multivariate categorical outcomes (i.e., multiple Likert-type items in the questionnaires) and cross-classified structure (i.e., non-nested students and instructors). Recently, a new approach, namely the cross-classified IRT model, was proposed for appropriately handling SET data. To inform researchers in higher education, in this article, the cross-classified IRT model, along with three existing approaches applied in SET studies, including the cross-classified random effects model (CCREM), the multilevel item response theory (MLIRT) model, and a two-step integrated strategy, was reviewed. The strengths and weaknesses of each of the four approaches were also discussed. Additionally, the new and existing approaches were compared through an empirical data analysis and a preliminary simulation study. This article concluded by providing general suggestions to researchers for analyzing SET data and discussing limitations and future research directions.

Semiparametric multivariate joint model for skewed-longitudinal and survival data: A Bayesian approach.

Joint models and statistical inference for longitudinal and survival data have been an active area of statistical research and have mostly coupled a longitudinal biomarker-based mixed-effects model with normal distribution and an event time-based survival model. In practice, however, the following issues may standout: (i) Normality of model error in longitudinal models is a routine assumption, but it may be unrealistically violating data features of subject variations. (ii) Data collected are often featured by the mixed types of multiple longitudinal outcomes which are significantly correlated, ignoring their correlation may lead to biased estimation. Additionally, a parametric model specification may be inflexible to capture the complicated patterns of longitudinal data. (iii) Missing observations in the longitudinal data are often encountered; the missing measures are likely to be informative (nonignorable) and ignoring this phenomenon may result in inaccurate inference. Multilevel item response theory (MLIRT) models have been increasingly used to analyze the multiple longitudinal data of mixed types (ie, continuous and categorical) in clinical studies. In this article, we develop an MLIRT-based semiparametric joint model with skew-t distribution that consists of an extended MLIRT model for the mixed types of multiple longitudinal data and a Cox proportional hazards model, linked through random-effects. A Bayesian approach is employed for joint modeling. Simulation studies are conducted to assess performance of the proposed models and method. A real example from primary biliary cirrhosis clinical study is analyzed to estimate parameters in the joint model and also evaluate sensitivity of parameter estimates for various plausible nonignorable missing data mechanisms.

Socio-economic status and trajectories of a novel multidimensional metric of Active and Healthy Ageing: the English Longitudinal Study of Ageing

Healthy ageing research largely has a unidimensional focus on physical health, negating the importance of psychosocial factors in the maintenance of a good quality-of-life. In this cohort study, we aimed to identify trajectories of a new multidimensional metric of Active and Healthy Ageing (AHA), including their associations with socio-economic variables. A latent AHA metric was created for 14,755 participants across eight waves of data (collected between 2004 and 2019) from the English Longitudinal Study of Ageing (ELSA), using Bayesian Multilevel Item Response Theory (MLIRT). Then, Growth Mixture Modelling (GMM) was employed to identify sub-groups of individuals with similar trajectories of AHA, and multinomial logistic regression examined associations of these trajectories with socio-economic variables: education, occupational class, and wealth. Three latent classes of AHA trajectories were suggested. Participants in higher quintiles of the wealth distribution had decreased odds of being in the groups with consistently moderate AHA scores (i.e., ‘moderate-stable’), or the steepest deterioration (i.e., ‘decliners’), compared to the ‘high-stable’ group. Education and occupational class were not consistently associated with AHA trajectories. Our findings reiterate the need for more holistic measures of AHA and prevention strategies targeted at limiting socio-economic disparities in older adults’ quality-of-life.

The development of a Brief Working Alliance Inventory for clients and therapists using multilevel factor analysis and item response theory in the United States and China.

This study aimed to investigate the multilevel factor structure of the therapist and client versions of the 12-item Working Alliance Inventory-Short Revised (WAI; Hatcher & Gillaspy, 2006) in the United States and China, and to create a three-item brief version (WAI-B3) using multilevel factor analysis (M-FA) and multilevel item response theory (M-IRT). We gathered eight data sets from two samples each in United States and China with a total of 21,623 sessions from 376 therapists and 2,455 clients. M-FA results with the first four data sets (two American and two Chinese) suggested that the 12-item WAI across therapist and client versions, and in both United States and China showed a dominant general WA factor with three specific subgroup factors corresponding to the Goal, Task, and Bond items. We then constructed a three-item WAI-B3 by selecting items 11, 10, and 9 through M-IRT, as the best representative of Goal, Task, and Bond subscales, respectively, for both the therapist and client versions in United States and China. With the other four data sets (two American, two Chinese) to test the WAI-B3, we found adequate multilevel reliability, structural validity, and convergent validity with the original 12-item WAI scores. Multilevel measurement invariance tests provided tentative and mixed support for the equivalence of WAI-B3 between the American and Chinese data sets and between therapist and client versions. We recommend that the WAI-B3 be used in routine clinical practice to track therapy process, and that the WAI-B3 be tested with other client and therapist populations and cultures. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Development of a brief therapist presence inventory in China using multilevel factor analysis and item response theory

Objectives The aims of this study were to investigate the multilevel factor structure of the Therapist Presence Inventory-Therapist version (TPI-T) in a sample of Chinese beginning trainees, and to develop a 6-item brief version (TPI-T-Brief) using multilevel item response theory (M-IRT). Methods: Participants included 131 therapists from a master’s level counseling training program in China. After every session, therapists were sent the TPI-T and measures assessing their ratings of working alliance and session quality. Results: Multilevel factor analysis indicated that though conceptually TPI-T was hypothesized as having a unidimensional structure, the positively and negative worded items emerged as two statistical artifact factors. Using M-IRT, we shortened the 21-item TPI-T into the 6-item TPI-T-Brief, with three positive and three negative items. Conclusions: Evidence was found for the multilevel reliability, convergent validity, and concurrent validity of TPI-T-Brief given its significant associations with therapist- and client-rated working alliance and session quality. Further, TPI-T-Brief had significant predictive effect of session quality above and beyond working alliance at the session level. We discussed limitations (including using only one dataset) and recommended that the TPI-T-Brief be used in routine clinical practice to track therapy process.

Multilevel IRT models for the analysis of satisfaction for distance learning during the Covid-19 pandemic

The Covid-19 pandemic played a relevant role in the diffusion of distance learning alternatives to “traditional” learning based on classroom activities, to allow university students to continue attending lessons during the most severe phases of the pandemic. In such a context, investigating the students' perspective on distance learning provides useful information to stakeholders to improve effective educational strategies, which could be useful also after the end of the emergency to favor the digital transformation in the higher educational setting.Here we focus on the satisfaction in distance learning for Italian university students. We rely on data comprising students enrolled in various Italian universities, which were inquired about several aspects related to learning distance.We explicitly take into account the hierarchical nature of data (i.e., students nested in universities) and the latent nature of the variable of interest (i.e., students' learning satisfaction) through a multilevel Item Response Theory model with students' and universities' covariates.As the main results of our study, we find out that distance learning satisfaction of students: (i) depends on the University where they study; (ii) is affected by some students' socio-demographic characteristics, among which psychological factors related to Covid-19; (iii) is affected by some observable university characteristics.

Identifying and Supporting Academically Low-Performing Schools in a Developing Country: An Application of a Specialized Multilevel IRT Model to PISA-D Assessment Data.

Performance-targeted interventions are an important tool in improving educational outcomes and are often applied at the school level, where low-performing schools are selected for participation. In this paper, we aim to identify low-performing schools in Cambodia that are in need of support on improving students' abilities in formulating math problems. Using data from the PISA for Development project, we present an application of a structured multilevel mixture item response theory (IRT) model that utilizes strategic constraints in order to achieve our research aims. The approach utilized in this application draws on psychometric traditions in multilevel IRT modeling, mixture IRT modeling, and constrained mixture IRT modeling. Results support classifications of Cambodian schools participating in PISA-D as low- and non-low-performing schools, as well as provide insight into these schools various contexts. Implications for future school interventions in Cambodia as well as future extensions to this modeling approach are discussed.

The Role of Self-Control in Predicting Intimate Partner Violence Specialization in an Australian Community Sample

PurposeThis study builds on knowledge of intimate partner violence (IPV) perpetration by examining theoretically important aspects of this crime within an Australian community sample. More specifically, the current study examines the relationship between IPV offenders’ levels of self-control and offense specialization and determines whether this relationship differs by gender.MethodAn online survey advertised on social media yielded an analysis sample of 155 females and 170 males. Self-control was measured using the Brief Self-Control scale and specialization was measured with the Offense Specialization Coefficient (OSC) and a multilevel item response theory measurement approach.ResultsFindings indicate that IPV offenders with low levels of self-control are more versatile in their offending and that this relationship was more evident and pronounced for females.ConclusionsFindings are mostly supportive of Gottfredson and Hirschi’s General Theory of Crime and highlight important differences within IPV offender populations that should be considered when designing targeted prevention and intervention methods.

RETRACTED ARTICLE: Meaning in life measure: a five-Item short form and its measurement invariance across gender, age, and cultural contexts

We, the Editors and Publisher of the journal Psychotherapy Research, have retracted the following article: Xu Li, Wonjin Sim & Clara E. Hill (2023) Meaning in life measure: a five-Item short form and its measurement invariance across gender, age, and cultural contexts, Psychotherapy Research, 33:3, 387-400, DOI: 10.1080/10503307.2022.2098077 Since publication, concerns had been raised about informed consent. The University of Maryland College Park Institutional Review Board investigated and they found that some data included in the publication were obtained from therapy clients who had either not been asked to provide consent or who had withdrawn consent for their data to be included in the study research. The investigation concluded that this article should be retracted. The authors agree with the retraction of this article. We have been informed in our decision-making by our editorial policies and COPE guidelines. The retracted article will remain online to maintain the scholarly record, but it will be digitally watermarked on each page as ‘Retracted’.

Does education protect families' well-being in times of crisis? Measurement issues and empirical findings from IT-SILC data

This study analyses the relationship between education and material well-being from a longitudinal perspective using the European Survey on Income and Living Conditions (EU-SILC) data collected in Italy in four waves (2009–2012). It has two main aims: (i) to measure household material well-being on the basis of householders’ responses to multiple survey items (addressed to gather information on the household availability of material resources) by advancing indexes, which can account for global and relative divergences in households’ material well-being across survey waves; (ii) to assess how education and other sociodemographic characteristics affect absolute well-being and its variation (i.e. relative well-being) in the time span considered. Both aims are pursued, combining measuring and explanatory modelling approaches. That is, the use of the Multilevel Item Response Theory model allows to measure the global household material well-being and its yearly variation (i.e. relative material well-being) in the four waves. Meanwhile, the use of a multivariate (and multivariate multilevel) regression model allows to assess the effects of education and other sociodemographic characteristics on both components (absolute and relative well-being), controlling for the relevant sources of heterogeneity in the data. The value added to using the proposed methodologies with the main findings and economic implications are discussed.

Social correlates of specialized versus versatile offending patterns in intimate partner violence: A register-based study in Finland

PurposeThe purpose of the current study was to examine the tendencies toward specialization and generalist offending among intimate partner violence (IPV) offenders and to assess whether some well-known correlates of criminal offending are differentially associated with various offending patterns. MethodWe use large-scale register-based data from Finland including all offenders in police-recorded cases of IPV between 2015 and 2019 (N = 19,030). Two different analytic approaches suggested for research on offense specialization are used: the multilevel item response theory (IRT) approach and latent class analysis (LCA). ResultsSignificant tendencies toward both specialization and generalist offending were found in the data using both analysis methods. In addition, the correlates were differentially associated with specialized versus versatile offending patterns. Specialization in IPV was associated with, for example, female gender, older age, higher socioeconomic status, and having an immigrant background. The findings also show IPV specialization and generalist offending to be differentially associated with different victimization types. ConclusionsThe findings suggest that the idea of IPV offenders as specialists who do not engage in violence and crime in other contexts is not empirically fully accurate. Implications for future research, theory, and prevention policies are discussed.

Assessing the Victim-Offender Overlap Among Adolescents in Rural China

Grounded in criminological theories (routine activity, social bonding, social disorganization, control balance, differential association, and general strain), this study extended the victim-offender overlap research by considering the specificity of rurality. We collected data from 2,839 adolescents in rural China and applied multilevel item response theory modeling. Both victim-offender overlap and differentiation were evident among rural adolescents. The victimization-offending overlap was associated with unstructured socializing, family and neighborhood control, moral beliefs, peer delinquency, and certain forms of social strain (loss of positively valued stimuli and exposure to negative stimuli). The differential tendency toward victimization over offending was a function of non-deviant solitary routines, failure to achieve positively valued goals, and strain-specific depression. The insignificance of control imbalance may need clarification of its conceptual relevance to rurality.

Evidence on the Bidirectional Relationship Between Health and Life Satisfaction in Older Adults

Previous research on health and life satisfaction in older adults has suggested a bidirectional relationship. However, most evidence either is based on cross-sectional data, being unsuitable for inferring any directionality on the results, or disregards the within-person stability of both variables over time, thus providing potentially biased results. We analysed data from 11,667 older adults interviewed between 2008 and 2016 within the English Longitudinal Study of Ageing. A health measure including self-reported and measured tests on cognitive and physical performance was computed using a Bayesian multilevel item-response theory approach. Life satisfaction was assessed with the Satisfaction with Life Scale. Cross-lagged models using a Structural Equation Modelling approach were used to analyse the longitudinal interrelation between health and life satisfaction. Models accounting and not accounting for the within-person stability in the measures were computed. Additional sensitivity models were ran using an exclusively self-reported measure of health. Health and life satisfaction were significant longitudinal predictors of one another only when the within-person stability was not considered. The effect of life satisfaction on health became negligible once that stability was modelled. The impact of not accounting for within-person stability was larger when health was measured exclusively with self-reported information. Our study suggests that health and life satisfaction are not bidirectionally related in older adults, but only health predicts a better life satisfaction over time. Thus, policies aimed at fostering older adults’ life satisfaction by focusing on health enhancement and maintenance may be fruitful, whereas the opposite may not.

An Application of Multilevel Mixture Item Response Theory Model

Although the mixture item response theory (IRT) models are useful for heterogeneous samples, they are not capable of handling a multilevel structure that is very common in education and causes dependency between hierarchies. Ignoring the hierarchical structure may yield less accurate results because of violation of the local independence assumption. This interdependency can be modeled straightforwardly in a multi-level framework. In this study, a large-scale data set, TEOG exam, was analyzed with a multilevel mixture IRT model to account for dependency and heterogeneity in the data set. Sixteen different multilevel models (different class solutions) were estimated using the eighth-grade mathematics data set. Model fit statistics for these 16 models suggested the CB1C4 model (one school-level and four student-level latent classes) was the best fit model. Based on CB1C4 model, the students were classified into four latent student groups and one latent school group. Parameter estimates obtained with maximum likelihood estimation were presented and interpreted. Several suggestions were made based on the results.

A Multilevel Mixture IRT Framework for Modeling Response Times as Predictors or Indicators of Response Engagement in IRT Models.

Disengaged item responses pose a threat to the validity of the results provided by large-scale assessments. Several procedures for identifying disengaged responses on the basis of observed response times have been suggested, and item response theory (IRT) models for response engagement have been proposed. We outline that response time-based procedures for classifying response engagement and IRT models for response engagement are based on common ideas, and we propose the distinction between independent and dependent latent class IRT models. In all IRT models considered, response engagement is represented by an item-level latent class variable, but the models assume that response times either reflect or predict engagement. We summarize existing IRT models that belong to each group and extend them to increase their flexibility. Furthermore, we propose a flexible multilevel mixture IRT framework in which all IRT models can be estimated by means of marginal maximum likelihood. The framework is based on the widespread Mplus software, thereby making the procedure accessible to a broad audience. The procedures are illustrated on the basis of publicly available large-scale data. Our results show that the different IRT models for response engagement provided slightly different adjustments of item parameters of individuals’ proficiency estimates relative to a conventional IRT model.

Longitudinal Analysis of Patient-Reported Outcomes in Clinical Trials: Applications of Multilevel and Multidimensional Item Response Theory

With decades of advance research and recent developments in the drug and medical device regulatory approval process, patient-reported outcomes (PROs) are becoming increasingly important in clinical trials. While clinical trial analyses typically treat scores from PROs as observed variables, the potential to use latent variable models when analyzing patient responses in clinical trial data presents novel opportunities for both psychometrics and regulatory science. An accessible overview of analyses commonly used to analyze longitudinal trial data and statistical models familiar in both psychometrics and biometrics, such as growth models, multilevel models, and latent variable models, is provided to call attention to connections and common themes among these models that have found use across many research areas. Additionally, examples using empirical data from a randomized clinical trial provide concrete demonstrations of the implementation of these models. The increasing availability of high-quality, psychometrically rigorous assessment instruments in clinical trials, of which the Patient-Reported Outcomes Measurement Information System (PROMIS®) is a prominent example, provides rare possibilities for psychometrics to help improve the statistical tools used in regulatory science.

Bayesian Analysis of a Quantile Multilevel Item Response Theory Model.

Multilevel item response theory (MLIRT) models are used widely in educational and psychological research. This type of modeling has two or more levels, including an item response theory model as the measurement part and a linear-regression model as the structural part, the aim being to investigate the relation between explanatory variables and latent variables. However, the linear-regression structural model focuses on the relation between explanatory variables and latent variables, which is only from the perspective of the average tendency. When we need to explore the relationship between variables at various locations along the response distribution, quantile regression is more appropriate. To this end, a quantile-regression-type structural model named as the quantile MLIRT (Q-MLIRT) model is introduced under the MLIRT framework. The parameters of the proposed model are estimated using the Gibbs sampling algorithm, and comparison with the original (i.e., linear-regression-type) MLIRT model is conducted via a simulation study. The results show that the parameters of the Q-MLIRT model could be recovered well under different quantiles. Finally, a subset of data from PISA 2018 is analyzed to illustrate the application of the proposed model.

The Impact of Test and Sample Characteristics on Model Selection and Classification Accuracy in the Multilevel Mixture IRT Model.

The standard item response theory (IRT) model assumption of a single homogenous population may be violated in real data. Mixture extensions of IRT models have been proposed to account for latent heterogeneous populations, but these models are not designed to handle multilevel data structures. Ignoring the multilevel structure is problematic as it results in lower-level units aggregated with higher-level units and yields less accurate results, because of dependencies in the data. Multilevel data structures cause such dependencies between levels but can be modeled in a straightforward way in multilevel mixture IRT models. An important step in the use of multilevel mixture IRT models is the fit of the model to the data. This fit is often determined based on relative fit indices. Previous research on mixture IRT models has shown that performances of these indices and classification accuracy of these models can be affected by several factors including percentage of class-variant items, number of items, magnitude and size of clusters, and mixing proportions of latent classes. As yet, no studies appear to have been reported examining these issues for multilevel extensions of mixture IRT models. The current study aims to investigate the effects of several features of the data on the accuracy of model selection and parameter recovery. Results are reported on a simulation study designed to examine the following features of the data: percentages of class-variant items (30, 60, and 90%), numbers of latent classes in the data (with from 1 to 3 latent classes at level 1 and 1 and 2 latent classes at level 2), numbers of items (10, 30, and 50), numbers of clusters (50 and 100), cluster size (10 and 50), and mixing proportions [equal (0.5 and 0.5) vs. non-equal (0.25 and 0.75)]. Simulation results indicated that multilevel mixture IRT models resulted in less accurate estimates when the number of clusters and the cluster size were small. In addition, mean Root mean square error (RMSE) values increased as the percentage of class-variant items increased and parameters were recovered more accurately under the 30% class-variant item conditions. Mixing proportion type (i.e., equal vs. unequal latent class sizes) and numbers of items (10, 30, and 50), however, did not show any clear pattern. Sample size dependent fit indices BIC, CAIC, and SABIC performed poorly for the smaller level-1 sample size. For the remaining conditions, the SABIC index performed better than other fit indices.

CREATION OF A COMMON METRIC OF HEALTH STATUS IN THE HARMONIZED DATASET OF THE ATHLOS PROJECT

Although life longevity has increased across the world, evidence suggests some heterogeneity of the ageing process across individuals. To investigate different ageing patterns, the ATHLOS project harmonised data from 411,000 individuals across 17 existing cohort studies. The harmonised dataset provides comparable information on functioning measures, cognition, mental health, sociodemographic and lifestyle behaviours. To measure the process of healthy ageing across time and cohorts, we employed a Bayesian Multilevel Item Response Theory(IRT) and created a common metric of health status by using items of functioning. The IRT measurement model includes parameters describing the difficulty and discriminatory power of each item. We adopted the Bayesian Multilevel framework as it allows item parameters to vary among studies and the simultaneous estimation of all parameters under a Markov Chain Monte Carlo(MCMC) method. Finally, we assessed the predictive validity of the metric against mortality by performing a Receiver Operating Characteristic(ROC) curve analysis.