Item Response Theory Models Research Articles

Measurement of bullying in the Progress in International Reading Literacy Study 2021 cohort: Evidence for model misspecification

We evaluated the use of the Rasch model in the scoring and scaling of the context questionnaire scales from the Progress in International Reading Literacy Study 2021 cohort. Focusing on the bullying scale of the Progress in International Reading Literacy Study, we contrasted polytomous item response theory models using free (two-parameter logistic modeling) and fixed (Rasch modeling) discrimination parameters. Our analyses included data from 354,074 students from 63 countries or entities. Across all countries and entities, the Rasch-based model significantly distorted levels of bullying. The rank ordering of countries was consistent between models in 32 of the cases (50.8%), while in other cases there were changes in rank of up to five places. This study provides a robust basis for evidence-based decisions on whether a country is lagging or exceeding the global norm in aberrant behaviors such as bullying.

The Use of a Spanish-Translated PrEP Stigma Scale among the Latino Sample of the UNITE Cohort Study.

Pre-Exposure Prophylaxis (PrEP) related stigma is linked to inadequate PrEP uptake, yet there are no validated scales to test this association among Spanish-speaking LSMM. The current study examined if the Spanish-translated PrEP Stigma Scale (PSS) was psychometrically appropriate for implementing in Spanish language dominant Latino/e/x Sexual Minority Men (SMM). Recruitment was conducted using geosocial networking applications, social media sites, and e-mail blasts (N=3,049). First, we utilized Item Response Theory (IRT) modeling to evaluate the reliability of the PSS items and the latent construct across both language groups (nEnglish = 2844 and nSpanish = 205). Subsequently, we applied the PSS scale in a theoretical application by examining its association with key steps in the PrEP uptake cascade (i.e., perceived PrEP candidacy, PrEP willingness, PrEP intentions, and having spoken to provider about PrEP) stratified by language. Results of the IRT analyses provided evidence that the translated version of the PSS was appropriate for use among this sample. Further, among English respondents, PrEP stigma was negatively associated with perceived PrEP candidacy (B=-0.30, p=<.001), PrEP willingness (B=-0.46, p=<.001), and PrEP intentions (B=-0.23, p=.003). PrEP stigma, among Spanish respondents, was not significantly associated with any of the PrEP cascade steps. This study demonstrated that the PSS scale performs adequately for both English and Spanish-speaking Latino SMM. However, researchers and health professionals alike should pay close attention to the nuanced effects in U.S. based English and Spanish language samples as PrEP stigma may impact the PrEP cascade for one language sample and not the other.

Multidimensionality of children’s foundational skills: an IRT-based validation of the NEPSY factorial structure

ABSTRACT Objective Measurement tools for children’s foundational skills provide important insights towards the developmental mechanisms of these domains. This study examines the structure of sensorimotor, visuospatial processing (VP), executive function (EF) skills and their interrelationship. The NEPSY is a widely-used neuropsychological instrument, but while validated at the domain-level through classical test theory approaches, no studies have conducted in-depth item-level analyses on its structural and psychometric properties. Method This study examined whether the designed structure of NEPSY is empirically supported by fitting unidimensional and multidimensional Item Response Theory models to item response data of early elementary students in the United States. A confirmatory factor analysis was also conducted to examine the factorial structure of the data. Results Results provide empirical support to the factorial structure of NEPSY, both at the domain level and within the VP sub-domain. The results show that VP and EF are separate constructs and subtests within the VP domain can further be modelled as interrelated but distinct factors – non-motor and motor-based processing. Conclusion Our specific results support the theoretical stance that VP consists of multiple component skills that are distinct because they require different neurological processes. This multidimensional NEPSY model shares some structural commonalities with the Cattell–Horn–Carroll model.

Psychometric analysis and the implications for the use of the scoliosis research society questionnaire (SRS-22r English) for individuals with adolescent idiopathic scoliosis

Despite widespread usage of the SRS-22r questionnaire (Scoliosis Research Society Questionnaire-22r), the English version has only sparingly been subjected to analysis using modern psychometric techniques for patients with adolescent idiopathic scoliosis (AIS). The study purpose was to improve interpretation and clinical utility of the SRS-22r for adolescents with AIS by generating additional robust evidence, using modern statistical techniques. Questions about (1) Structure and (2) Item and Scale Functioning are addressed and interpreted for clinicians and researchers. This retrospective case review analyzed SRS-22r data collected from 1823 patients (mean age 14.9±2.2years) with a primary diagnosis of AIS who clinically completed an SRS-22r questionnaire.Individual SRS-22r questions and domain scores were retrieved through data queries. Patient information collected through chart review included diagnosis, age at assessment, sex, race and radiographic parameters. From 6044 SRS-22r assessments, 1 assessment per patient was randomly selected. Exploratory structural equation modeling (ESEM) and item response theory (IRT) techniques were used for data modeling, item calibration, and reliability assessment. ESEM demonstrated acceptable fit to the data: χ2 (130)=343.73, p<.001; RMSEA=0.035; CFI=0.98; TLI=0.96; SRMR=0.02. Several items failed to adequately load onto their assigned factor. Item fit was adequate for all items except SRSq10 (Self-Image), SRSq16 (Mental Health), and SRSq20 (Mental Health). IRT models found item discriminations are within normal levels for items in psychological measures, except items SRSq1 (pain), SRSq2 (pain), and SRSq16 (mental health). Estimated reliability of the Function domain (ρ=0.69) was low, however, Pain, Self-Image and Mental Health domains exhibited high (ρ>0.80) reliability. Modern psychometric assessment of the SRS-22r, in adolescent patients with AIS, are presented and interpreted to assist clinicians and researchers in understanding its strengths and limitations. Overall, the SRS-22r demonstrated good psychometric properties in all domains except function. Cautious interpretation of the total score is suggested, as it does not reflect a single HRQoL construct.

Using ECHO program data to develop a brief measure of caregiver support and cognitive stimulation using the home observation for measurement of the environment-infant/toddler (HOME-IT).

Data from three NIH Environmental influences on Child Health Outcomes (ECHO) Program cohorts that collected the HOME-Infant-Toddler (HOME-IT age 0-3 years) version were used to examine the reliability of a brief scale of caregiver support and cognitive stimulation. Participants with HOME-IT data (N = 2518) were included in this analysis. Mean child age at HOME-IT assessment was 1.51 years, 48% of children were female, and 43% of children identified as Black. A four-stage analysis plan was used to evaluate item response theory assumptions, item response theory model fit, monotonicity, scalability, item fit, and differential item functioning. Results indicate the feasibility of developing a reliable 10-item scale (reliability >0.7) with particularly high precision for children with lower levels of cognitive stimulation.

Development and validation of the Health Literacy Index for the Community (HLIC) for the Korean National Health and Nutrition and Examination Survey.

We developed and validated the Health Literacy Index for the Community (HLIC) to assess the health literacy of the Korean population within the framework of the Korean National Health and Nutrition and Examination Survey. The HLIC was developed through 1) defining the conceptual framework and generating the item pool and 2) finalizing the items and identifying the cutoff value. Interviews were conducted to examine items' face validity, and a cross-sectional survey was performed to analyze the item-response theory and Rasch models to investigate the instrument's psychometric properties. In this study of 1,041 participants, most had no difficulty understanding health information; however, 67.9% struggled to assess the reliability of health information from the internet or media. A 4-factor structure was identified through factor analysis, leading to the exclusion of some items. This resulted in 10 items across 4 domains: 1) disease prevention, 2) health promotion, 3) health care, and 4) technology and resources. The HLIC demonstrated good internal consistency, with a Cronbach's ɑ of 0.87. It also showed high test-retest reliability and correlations with other health literacy instruments. A sociodemographic analysis of the HLIC revealed disparities in health literacy across various age groups, education levels, and income brackets. The HLIC was developed to systematically measure health literacy in Korea's general population. Its simplicity and conciseness ensure reliability and validity and improve its accessibility, making it particularly suitable for the broader Korean population, including those with lower literacy levels.

Personality Measurement of Students Using Item Response Theory Models: Stability Responses from Nigerian Institutions

Item Response Theory (IRT) is utilised to detect bias in assessment tools and address issues such as faked or manipulated responses, enhancing the reliability and stability of conclusions in personality assessment. This article examines the item parameter estimates of a scale and the effectiveness of one-, two-, and three-parameter logistic models in analysing response stability in personality measurement from repeated administration. Three hundred undergraduate students at three tertiary institutions in Nigeria were sampled using a multi-stage sampling procedure. Data was collected using an adapted version of the Big Five Inventory (BFI) with a reliability coefficient of 0.85. The results showed that the item parameter estimates (mean threshold) are within the recommended benchmarks. A comparison of the three IRT models based on the Likelihood ratio (InL), Akaike Information Criterion (AIC), and Bayesian Information Criterion (BIC) values revealed that the two-parameter logistic model best fit the personality data among undergraduates from repeated administration. It is recommended that, rather than relying solely on a statistical decision-making process, IRT fit and model comparison should be applied to gain insight into the functioning of items and tests.

Impact of Violating Unidimensionality on Rasch Calibration for Mixed-Format Tests

ABSTRACT Mixed-format tests that include both multiple-choice (MC) and constructed-response (CR) items have become widely used in many large-scale assessments. When an item response theory (IRT) model is used to score a mixed-format test, the unidimensionality assumption may be violated if the CR items measure a different construct from that measured by MC items. In this simulation study, we explore the impact of different degrees of violation on the unidimensionality assumption on Rasch item parameter recovery and model-data fit statistics. Our results suggest that, when the unidimensionality assumption is violated for mixed-format tests, both estimated Rasch item parameters and model-data fit statistics are distorted, with greater impact corresponding to higher degrees of violations. The findings from this study may provide guidance for test practitioners for working with mixed-format tests within the Rasch context.

Item and Test Characteristic Curves of Rank-2PL Models for Multidimensional Forced-Choice Questionnaires

ABSTRACT A process is proposed to create the one-dimensional expected item characteristic curve (ICC) and test characteristic curve (TCC) for each trait in multidimensional forced-choice questionnaires based on the Rank-2PL (two-parameter logistic) item response theory models for forced-choice items with two or three statements. Some examples of ICC and TCC plots from a real pair form and triplet form are provided, and they appear to help identify misfit trait scores at the item and test levels. TCC plots with negative statements changed to positive ones are also proposed.

Development and validation of the MultiScent-20 digital odour identification test using item response theory

Although validated and reliable psychophysical tests of olfactory function are available, an easy-to-use and feasible test has yet to be developed. This study aimed to design a digital odour identification test, evaluate its validity, assess its reliability, establish a normative curve, and explore the impact of demographic factors. The odour identification test was presented with the Multiscent-20, a hand-held, tablet-like digital scent device that features an integrated odour digital delivery system. The identification performance on the 20 odours was assessed using item response theory (IRT). The normative curve was established by administering the test to a large sample of participants (n = 1299). The mean identification score was 17.5 (SD = 2.1). The two-parameter logistic IRT model provided the best fit, revealing variation in item discrimination and difficulty parameters. Educational attainment influenced performance, with primary education associated with lower scores. Additionally, sex was not found to be associated with performance. This study provides initial evidence supporting the validity and reliability of use of the Multiscent-20 as a digital odour identification test. The test’s automation and portability enable the standardized delivery of olfactory stimuli and efficient automatic recording and scoring of responses.

Using a nonparametric item response theory model to identify patterns of cognitive decline: The Mokken scale analysis.

Cognitive decline, particularly in dementia, presents complex challenges in early detection and diagnosis. While Item Response Theory (IRT) has been instrumental in identifying patterns of cognitive impairment through psychometric tests, its parametric models often require large sample sizes and strict assumptions. This creates a need for more adaptable, less demanding analytical methods. This study aimed to evaluate the effectiveness of Mokken scale analysis (MSA), a nonparametric IRT model, in identifying hierarchical patterns of cognitive impairment from psychometric tests. Using data from 1164 adults over 60 years old, we applied MSA to the orientation subscale of ACE-III. Our analysis involved calculating scalability, monotone homogeneity, invariant item ordering (IIO) and response functions. The MSA effectively retrieved the hierarchical order of cognitive impairment patterns. Most items showed strong scalability and consistent patterns of cognitive performance. However, challenges with IIO were observed, particularly with items having adjacent difficulty parameters. The findings highlight MSA's potential as a practical alternative to parametric IRT models in cognitive impairment research. Its ability to provide valuable insights into patterns of cognitive deterioration, coupled with less stringent requirements, makes it a useful tool for clinicians and researchers.

Exploring circular consumption: Circular attitudes and their influence on consumer behavior across the product lifecycle

AbstractThis study explores the dynamic and multidimensional domain of circular consumption, emphasizing the relevance of a lifecycle perspective in understanding consumer behaviors within the circular economy. Moving beyond prior research that predominantly focused on specific actions such as recycling or purchasing second‐hand products, we advocate for a more comprehensive understanding of consumer choices throughout a product's lifecycle. Drawing on attitudes theory, we employ item response theory (IRT) models to unveil latent attitudes reflecting diverse circular consumer behaviors. These attitudes, inferred from manifested actions across different product lifecycle stages, include environmental‐centric, resource‐centric, and societal‐centric circular attitudes. The study involves a representative sample of 5,124 respondents across five European countries. Our findings underscore the complexity of consumer motivations in the circular economy, revealing distinct links between circular attitudes and behaviors. This research contributes to a nuanced understanding of circular consumption, emphasizing the importance of a lifecycle approach, and driving the development of robust measurement scales for circular consumer actions.

A Comparison of Limited Information Estimation Methods for the Two-Parameter Normal-Ogive Model with Locally Dependent Items

The two-parameter normal-ogive (2PNO) model is one of the most popular item response theory (IRT) models for analyzing dichotomous items. Consistent parameter estimation of the 2PNO model using marginal maximum likelihood estimation relies on the local independence assumption. However, the assumption of local independence might be violated in practice. Likelihood-based estimation of the local dependence structure is often computationally demanding. Moreover, many IRT models that model local dependence do not have a marginal interpretation of item parameters. In this article, limited information estimation methods are reviewed that allow the convenient and straightforward handling of local dependence in estimating the 2PNO model. In detail, pairwise likelihood, weighted least squares, and normal-ogive harmonic analysis robust method (NOHARM) estimation are compared with marginal maximum likelihood estimation that ignores local dependence. A simulation study revealed that item parameters can be consistently estimated with limited information methods. At the same time, marginal maximum likelihood estimation resulted in biased item parameter estimates in the presence of local dependence. From a practical perspective, there were only minor differences regarding the statistical quality of item parameter estimates of the different estimation methods. Differences between the estimation methods are also compared for two empirical datasets.

Estimation of Standard Error, Linking Error, and Total Error for Robust and Nonrobust Linking Methods in the Two-Parameter Logistic Model

The two-parameter logistic (2PL) item response theory model is a statistical model for analyzing multivariate binary data. In this article, two groups are brought onto a common metric using the 2PL model using linking methods. The linking methods of mean–mean linking, mean–geometric–mean linking, and Haebara linking are investigated in nonrobust and robust specifications in the presence of differential item functioning (DIF). M-estimation theory is applied to derive linking errors for the studied linking methods. However, estimated linking errors are prone to sampling error in estimated item parameters, thus resulting in artificially increased the linking error estimates in finite samples. For this reason, a bias-corrected linking error estimate is proposed. The usefulness of the modified linking error estimate is demonstrated in a simulation study. It is shown that a simultaneous assessment of the standard error and linking error in a total error must be conducted to obtain valid statistical inference. In the computation of the total error, using the bias-corrected linking error estimate instead of the usually employed linking error provides more accurate coverage rates.

The Rank-2PL IRT Models for Forced-Choice Questionnaires: Maximum Marginal Likelihood Estimation with an EM Algorithm

The rank two-parameter logistic (Rank-2PL) item response theory models refer to a set of models applying the 2PL model in a sequential ranking process that occurs in forced-choice questionnaires. The multi-unidimensional pairwise preference with 2PL model (MUPP-2PL) is a Rank-2PL model for items with two statements. Focusing on items with three statements, we develop a maximum marginal likelihood estimation with an expectation-maximization algorithm to estimate item parameters and their standard errors. A simulation study is conducted to check parameter recovery, and then the model is applied to a real dataset. Finally, the findings are summarized and discussed, and future research is suggested.

The Beck Hopelessness Scale's psychometric features: A new short form based on item response theory

BackgroundThe assessment of hopelessness plays a significant role in preventing various psychological disorders and major life events within the general population. However, the psychometric properties of the Beck Hopelessness Scale (BHS) have been a subject of controversy, primarily studied in clinical groups. The aim of the present study was to gain new psychometric insights and propose a new short version of the BHS for the general population using the Item Response Theory (IRT) approach. MethodsA total of 2164 Argentinean individuals completed the BHS alongside the Inventory of Suicide Orientation-30. We compared IRT models with two and three parameters for the original BHS version, exploring the removal of redundant and less informative items. Convergent and discriminant validity was also examined. ResultsOur results support the 2PL model for the BHS-19. In addition, the BHS-10 short version adequately depicted the same range of the measured trait as the original version, showing reasonable measurement accuracy in the middle-high levels of the trait (marginal reliability = 0.70, Cronbach's α = 0.86). Notably, a positive correlation was found between the factorial score of BHS-10, BHS-19, and suicide orientation. ConclusionsIn conclusion, our findings support the use of a simplified version as a practical and valuable tool for both research and clinical practice in the future.

The General Psychopathology 'p' Factor in Adolescence: Multi-Informant Assessment and Computerized Adaptive Testing.

Accumulating evidence supports the presence of a general psychopathology dimension, the p factor ('p'). Despite growing interest in the p factor, questions remain about how p is assessed. Although multi-informant assessment of psychopathology is commonplace in clinical research and practice with children and adolescents, almost no research has taken a multi-informant approach to studying youth p or has examined the degree of concordance between parent and youth reports. Further, estimating p requires assessment of a large number of symptoms, resulting in high reporter burden that may not be feasible in many clinical and research settings. In the present study, we used bifactor multidimensional item response theory models to estimate parent- and adolescent-reported p in a large community sample of youth (11-17 years) and parents (N = 5,060 dyads). We examined agreement between parent and youth p scores and associations with assessor-rated youth global functioning. We also applied computerized adaptive testing (CAT) simulations to parent and youth reports to determine whether adaptive testing substantially alters agreement on p or associations with youth global functioning. Parent-youth agreement on p was moderate (r =.44) and both reports were negatively associated with youth global functioning. Notably, 7 out of 10 of the highest loading items were common across reporters. CAT reduced the average number of items administered by 57%. Agreement between CAT-derived p scores was similar to the full form (r =.40) and CAT scores were negatively correlated with youth functioning. These novel results highlight the promise and potential clinical utility of a multi-informant p factor approach.

Racial-Ethnic Disparities in Quality of Care Among Medicaid Beneficiaries With Schizophrenia.

The authors sought to update and expand the evidence on the quality of health care and disparities in care among Medicaid beneficiaries with schizophrenia. Adult beneficiaries of New York State Medicaid with schizophrenia receiving care during 2016-2019 were identified. Composite quality scores were derived from item response theory models by using evidence-based indicators of the quality of mental and general medical health care. Risk-adjusted racial-ethnic differences in quality were estimated and summarized as percentiles relative to White beneficiaries' mean quality scores. The study included 71,013 beneficiaries; 42.8% were Black, 22.9% Latinx, 27.4% White, and 6.9% other race-ethnicity. Overall, 68.8% had a mental health follow-up within 30 days of discharge, and 90.2% had no preventable hospitalizations for chronic obstructive pulmonary disease or asthma. Among beneficiaries receiving antipsychotic medications, medication adherence was adequate for 43.7%. Fourteen indicators for mental and general medical health care quality yielded three composites: two for mental health care (pharmacological and ambulatory) and one for acute mental and general medical health care. Mean quality of pharmacological mental health care for Black and Latinx beneficiaries was lower than for White beneficiaries (39th and 44th percentile, respectively). For Black beneficiaries, mean quality of ambulatory mental health care was also lower (46th percentile). In New York City, Black beneficiaries received lower-quality care in all domains. The only meaningful group difference in the quality of acute mental and general medical health care indicated higher-quality care for individuals with other race-ethnicity. Disparities in the quality of Medicaid-financed health care persist, particularly for Black beneficiaries. Regional differences merit further attention.

Sustainability and high-level corruption in healthcare procurement: Profiles of Italian contracting authorities

Addressing corruption is crucial for building a sustainable healthcare system that ensures access, quality, and equity in healthcare delivery. Despite that, many current strategies to combat corruption in the healthcare sector do not evaluate high-level corruption, such as corruption risks occurring at sub-national levels. This work bridges this gap by providing corruption risk profiles of Italian contracting authorities responsible for procuring goods and services for healthcare facilities in the public procurement process. Using an array of 14 red flags of corruption risk and an extended Item Response Theory model applied to a big data source made available by the Italian Anti-corruption Authority, our main findings show that: i. the risk of corruption is a multidimensional occurrence, which can be represented as a four-dimensional latent variable; ii. there are eight clusters of contracting authorities, having distinct and well-defined risk profiles over the four ascertained dimensions of corruption risk; iii. the distribution of risk profiles at sub-national level showcases relevant geographic variations and emphasises the need for tailored anti-corruption strategies to effectively address region-specific challenges and risk factors.

Item Response Theory Models for Polytomous Multidimensional Forced‐Choice Items to Measure Construct Differentiation

AbstractMultidimensional forced‐choice (MFC) items have been found to be useful to reduce response biases in personality assessments. However, conventional scoring methods for the MFC items result in ipsative data, hindering the wider applications of the MFC format. In the last decade, a number of item response theory (IRT) models have been developed, majority of which are for MFC items with binary responses. However, MFC items with polytomous responses are more informative and have many applications. This paper develops a polytomous Rasch ipsative model (pRIM) that can deal with ipsative data and yield estimates that measure construct differentiation—a latent trait that describes the degree to which the personality constructs (e.g., interests) distinguish between each other. The pRIM and its simpler form are applied to a career interests assessment containing four‐category MFC items and the measures of interests differentiation are used for both intra‐ and interpersonal comparisons. Simulations are conducted to examine the recovery of the parameters under various conditions. The results show that the parameters of the pRIM can be well recovered, particularly when a complete linking design and a large sample are used. The implications and application of the pRIM in the personality assessment using MFC items are discussed.