Nonparametric Item Response Theory Models Research Articles

The MACE-SCREEN – Screening for Abuse and Neglect Across Developmental Stages

ABSTRACT Few high-quality instruments are available to comprehensively screen for multiple interpersonal traumatic childhood experiences differentiated by developmental stage. One instrument assessing various forms of childhood trauma is the Maltreatment and Abuse Chronology of Exposure (MACE-X, Teicher & Parigger, 2015), which allows for the assessment of age at exposure for every item. The aim of this study is the development of a shorter version, which is suitable for screening. In- and out-patients (n = 719) of a somatic clinic were assessed with the German version of the 75-item MACE. Descriptive statistics, a nonparametric item response theory model (Mokken Scale Analysis), and factor analysis were combined to select items for a screening version. A screener (MACE-SCREEN) with 39 items allocated to five scales was developed. Scales differentiate between abuse types and neglect, direct experience vs. witnessing violence, and the age of the perpetrator (adults vs. peers). The scales showed good internal consistency (Alpha .78 to .91) and the total score a strong correlation with the total score of the long version (r = .96). This screening version is still comprehensive as it assesses exposure to various forms of interpersonal traumatic events, including peer violence and witnessing domestic abuse. As it also allows for the concurrent assessment of age at exposure, it lends itself to assessing exposure across developmental stages.

Using a nonparametric item response theory model to identify patterns of cognitive decline: The Mokken scale analysis.

Cognitive decline, particularly in dementia, presents complex challenges in early detection and diagnosis. While Item Response Theory (IRT) has been instrumental in identifying patterns of cognitive impairment through psychometric tests, its parametric models often require large sample sizes and strict assumptions. This creates a need for more adaptable, less demanding analytical methods. This study aimed to evaluate the effectiveness of Mokken scale analysis (MSA), a nonparametric IRT model, in identifying hierarchical patterns of cognitive impairment from psychometric tests. Using data from 1164 adults over 60 years old, we applied MSA to the orientation subscale of ACE-III. Our analysis involved calculating scalability, monotone homogeneity, invariant item ordering (IIO) and response functions. The MSA effectively retrieved the hierarchical order of cognitive impairment patterns. Most items showed strong scalability and consistent patterns of cognitive performance. However, challenges with IIO were observed, particularly with items having adjacent difficulty parameters. The findings highlight MSA's potential as a practical alternative to parametric IRT models in cognitive impairment research. Its ability to provide valuable insights into patterns of cognitive deterioration, coupled with less stringent requirements, makes it a useful tool for clinicians and researchers.

C-Test construct validity: Evidence from nonparametric item response theory

C-Test is a gap-filling test designed to measure first and second language proficiency. Over the past four decades, researchers have shown the fit of C-Test data to parametric item response theory (IRT) models, but no study so far has shown the fit of C-Tests to nonparametric IRT models. The purpose of this study is to contribute to the ongoing C-Test validation project by providing evidence of fit to the Mokken scale analysis as a widely used nonparametric IRT model. A six-passage C-Test battery was analyzed using the monotone homogeneity model and the double monotonicity model of Mokken. Unidimensionality was evaluated using the automatic item selection procedure. Findings showed that the C-Test passages form a strong unidimensional scale, fit well to the monotone homogeneity model, and, after deleting one item, have a sufficient fit to the double monotonicity model. The findings also indicated that the items form a hierarchy, and persons can be located on an ordinal scale using their C-Test sum scores. Implications of the study for C-Test validity and application are discussed.

Analyzing Polytomous Test Data: A Comparison Between an Information-Based IRT Model and the Generalized Partial Credit Model

Item response theory (IRT) models the relationship between the possible scores on a test item against a test taker’s attainment of the latent trait that the item is intended to measure. In this study, we compare two models for tests with polytomously scored items: the optimal scoring (OS) model, a nonparametric IRT model based on the principles of information theory, and the generalized partial credit (GPC) model, a widely used parametric alternative. We evaluate these models using both simulated and real test data. In the real data examples, the OS model demonstrates superior model fit compared to the GPC model across all analyzed datasets. In our simulation study, the OS model outperforms the GPC model in terms of bias, but at the cost of larger standard errors for the probabilities along the estimated item response functions. Furthermore, we illustrate how surprisal arc length, an IRT scale invariant measure of ability with metric properties, can be used to put scores from vastly different types of IRT models on a common scale. We also demonstrate how arc length can be a viable alternative to sum scores for scoring test takers.

Developmental Language Differences in Children with Autism Spectrum Disorders and Possible Sex Difference.

Developmental difference is a common characteristic of autism spectrum disorder (ASD) with unclear sex differences. The current study included 610 children with ASD, aged between 2 and 7years, with completed language profiles. We used a nonparametric item response theory model called Mokken scale analysis to examine the order of acquisition of developmental language milestones in children with ASD. Our results demonstrated the developmental language differences in the expressive and receptive language dimensions in children with ASD compared with typical developmental sequences. Furthermore, The acquisition of gestures and pragmatics was more impaired in the female subgroup than in the male subgroup. The identified developmental language sequence could help provide a more comprehensive ASD developmental profile.

Advances in nonparametric item response theory for scale construction in quality-of-life research

We introduce the special section on nonparametric item response theory (IRT) in Quality of Life Research. Starting from the well-known Rasch model, we provide a brief overview of nonparametric IRT models and discuss the assumptions, the properties, and the investigation of goodness of fit. We provide references to more detailed texts to help readers getting acquainted with nonparametric IRT models. In addition, we show how the rather diverse papers in the special section fit into the nonparametric IRT framework. Finally, we illustrate the application of nonparametric IRT models using data from a questionnaire measuring activity limitations in walking. The real-data example shows the quality of the scale and its constituent items with respect to dimensionality, local independence, monotonicity, and invariant item ordering.

Detecting Aberrant Response Behavior with Nonparametric Method: Mokken and PerFit Packages in RStudio

ABSTRACT In order to reach valid and reliable test scores, various test theories have been developed, and one of them is nonparametric item response theory (NIRT). Mokken Models are the most widely known NIRT models which are useful for small samples and short tests. Mokken Package is useful for Mokken Scale Analysis. An important issue about validity is whether individual responses reflect individuals’ characteristics truly or not. Examining individuals’ score patterns with person-fit statistics is one way to detect invalid responses. PerFit Package can be used for detecting aberrant response patterns. This article explains and shows how to use Mokken and PerFit packages in RStudio.

Psychometric validation of Czech version of the Sport Motivation Scale.

The Sport Motivation Scale (SMS) is a widely used instrument that assesses motivational processes within sport and exercise environments. The scale has demonstrated validity and reliability in multiple cultures, however, there is yet no empirical evidence regarding its psychometric properties in the Czech population. In this study we therefore set out to examine the reliability and construct validity of the SMS in a sample of Czech university-aged athletes. We first examined the SMS factor structure using a nonparametric item response theory model (Mokken monotone homogeneity model) and identified six items violating the unidimensionality of the particular subscales. Remaining items were then subjected to test of hypothesized seven-factor structure and several different forms of measurement invariance examined based on gender, competition level and type of sport (individual vs team sports). The hypothesized seven-factor fit well and there was sufficient evidence supporting full invariance across the examined groups. All SMS subscales had adequate internal consistencies ranging from 0.66 to 0.89. Results of correlational analysis among the SMS subscales and between the SMS and two outcomes of interest further supported validity of the scale. Observed differences in SMS subscales between males and females, recreational and competitive athletes, as well as between individual and team-based sport activities, comported with prior empirical studies using a self-determination theory framework. In conclusion, results reinforce the utility and performance of the SMS in a sample of Czech university athletes. The SMS may therefore be recommended for measurement of the multidimensional motivational processes taking place in the exercise and sport domain.

The Comparison of Item Parameters Estimated From Parametric and Nonparametric Item Response Theory Models in Case of The Violance of Local Independence Assumption

Item response theory(IRT) has so many advantages than its precedent Classical Test Theory(CTT) such as non-changing item parameters, ability parameter estimations free from the items. However, in order to get these advantages, some assumptions should be met and they are; unidimensionality, normality and local independence. However, it is not always so easy to be met these assumptions by datasets. Especially when the normality of data is not provided, another approach for IRT can be applied, which is Non-Parametric Item Response Theory (NIRT). NIRT provides more flexible methods to scale datasets and it is used when the assumptions of Parametric Item Response Theory (PIRT) are not met at a satisfactory level. The assumption of local independence, is one of the situations in which NIRT can be used more effectively than PIRT. In this study, by using a real dataset, taken from TIMSS 2011, the effect of local dependence on the item parameters was investigated. With this goal, a dataset composed of 1,000 students was formed randomly from the TIMSS 2011 eight grade Mathematic test. Firstly, the item parameters were calculated from data set according to the two approaches without any manipulation. After that, two items were arranged as local dependent by changing the response patterns completely the same and the item parameters have been estimated from each sample by using R program, ltm and mokken packages. Two sets of item parameters estimated from data set were compared and the differences of the parameters were analyzed with statistical test. By this way, the effect of local independence has been analyzed on the item parameters have been decided.

Investigating psychometric properties and dimensional structure of an educational environment measure (DREEM) using Mokken scale analysis \u2013 a pragmatic approach

BackgroundQuestionnaires and surveys are used throughout medical education. Nevertheless, measuring psychological attributes such as perceptions of a phenomenon among individuals may be difficult. The aim of this paper is to introduce the basic principles of Mokken scale analysis (MSA) as a method for the analysis of questionnaire data and to empirically apply MSA to a real-data example.MethodsMSA provides a set of statistical tools for exploring the relationship between items and latent traits. MSA is a scaling method of item selection algorithms used to partition an array of items into scales. It employs various methods to probe the assumptions of two nonparametric item response theory models: the monotone homogeneity model and the double monotonicity model. The background and theoretical framework underlying MSA are outlined in the paper. MSA for polytomous items was applied to a real-life data example of 222 undergraduate students who had completed a 50-item self-administered inventory measuring the educational environment, the Dundee Ready Educational Measure (DREEM).ResultsA pragmatic and parsimonious approach to exploring questionnaires and surveys from an item response theory (IRT) perspective is outlined. The use of MSA to explore the psychometric properties of the Swedish version of the DREEM failed to yield strong support for the scalability and dimensional structure of the instrument.ConclusionsMSA, a class of simple nonparametric IRT models – for which estimates can be easily obtained and whose fit to data is relatively easily investigated – was introduced, presented, and tested. Our real-data example suggests that the psychometric properties of DREEM are not adequately supported. Thus, the empirical application depicted a potential and feasible approach whereby MSA could be used as a valuable method for exploring the behavior of scaled items in response to varying levels of a latent trait in medical education research.

The Genetic Relationship Between Psychological Distress, Somatic Distress, Affective Disorders, and Substance Use in Young Australian Adults: A Multivariate Twin Study.

Psychological distress (PSYCH), somatic distress (SOMA), affective disorders (AD), and substance use (SU) frequently co-occur. The genetic relationship between PSYCH and SOMA, however, remains understudied. We examined the genetic and environmental influences on these two disorders and their comorbid AD and SU using structural equation modeling. Self-reported PSYCH and SOMA were measured in 1,548 twins using the two subscales of a 12-item questionnaire, the Somatic and Psychological Health Report. Its reliability and psychometric properties were examined. Six ADs, involvement of licit and illicit substance, and two SU disorders were obtained from 1,663-2,132 twins using the World Mental Health Composite International Diagnostic Interview and/or from an online adaption of the same. SU phenotypes (heritability: 49-79%) were found to be more heritable than the affective disorder phenotypes (heritability: 32-42%), SOMA (heritability: 25%), and PSYCH (heritability: 23%). We fit separate non-parametric item response theory models for PSYCH, SOMA, AD, and SU. The IRT scores were used as the refined phenotypes for fitting multivariate genetic models. The best-fitting model showed the similar amount of genetic overlap between PSYCH-AD (genetic correlation rG = 0.49) and SOMA-AD (rG =0.53), as well as between PSYCH-SU (rG = 0.23) and SOMA-SU (rG = 0.25). Unique environmental factors explained 53% to 76% of the variance in each of these four phenotypes, whereas additive genetic factors explained 17% to 46% of the variance. The covariance between the four phenotypes was largely explained by unique environmental factors. Common genetic factor had a significant influence on all the four phenotypes, but they explained a moderate portion of the covariance.

Evaluating psychiatric symptoms in Parkinson's Disease by a clinimetric analysis of the Hopkins Symptom Checklist (SCL-90-R)

Although psychiatric comorbidity in Parkinson's Disease (PD) has often been studied, the individual psychiatric symptoms have rarely been evaluated from a clinimetric point of view in an attempt to measure how much the symptoms have been bothering or distressing the PD patients.The current study is therefore aimed at evaluating from a clinimetric viewpoint the severity of psychiatric symptoms affecting PD patients by using the Hopkins Symptom Checklist (SCL-90-R) to show its measurement-driven construct validity (scalability). The conventional nine SCL-90-R subscales (somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideas, and psychoticism), as well as the clinical most valid subscales from the SCL-28 version (depression, anxiety, interpersonal sensitivity, and neurasthenia) were analysed according to a clinimetric approach by comparing PD patients with a control group from a general population study. Scalability was tested by the non-parametric item response theory model by use of a Mokken analysis. Among the various SCL-90-R or SCL-28 subscales we identified from the clinimetric analysis that the somatization, anxiety, phobic anxiety, psychoticism, and neurasthenia (apathy), as well as the SCL-90-R GSI, were the most impaired psychiatric syndromes reaching a clinically significant effect size above 0.80, whereas the total SCL-28 GSI obtained an effect size of just 0.80. Our clinimetric analysis has shown that patients with PD not only are bothered with diverse somatic symptoms, but also with specific secondary psychiatric comorbidities which are clinically severe markers of impairment in the day-to-day function implying a negative cooping approach.

A Nonparametric Multidimensional Latent Class IRT Model in a Bayesian Framework.

We propose a nonparametric item response theory model for dichotomously-scored items in a Bayesian framework. The model is based on a latent class (LC) formulation, and it is multidimensional, with dimensions corresponding to a partition of the items in homogenous groups that are specified on the basis of inequality constraints among the conditional success probabilities given the latent class. Moreover, an innovative system of prior distributions is proposed following the encompassing approach, in which the largest model is the unconstrained LC model. A reversible-jump type algorithm is described for sampling from the joint posterior distribution of the model parameters of the encompassing model. By suitably post-processing its output, we then make inference on the number of dimensions (i.e., number of groups of items measuring the same latent trait) and we cluster items according to the dimensions when unidimensionality is violated. The approach is illustrated by two examples on simulated data and two applications based on educational and quality-of-life data.

Examination of Polytomous Items’ Psychometric Properties According to Nonparametric Item Response Theory Models in Different Test Conditions

Tests used for such purposes as determining educational quality, defining educational needs, hiring an employee, student selection and placement and performing guidance and clinic services have an important place in education and psychology. Of course, they should have certain psychometric features related to test scores' validity and reliability. Various test theories have helped to create more valid and reliable measurements and, as a result, to make better decisions regarding individuals. In education and psychology, Classical Test Theory (CTT) and Item Response Theory (IRT) are both widely used. CTT assumes that an individual's observed score is the total of the true score and the error score, while IRT estimates an individual's ability or latent trait from responses to test items (Embretson & Reise, 2000).When IRT assumptions and model-data fit are ensured, item and ability parameters' invariance occurs; this is known as the most important advantage IRT has over CTT. Item and ability parameters' invariance means estimating ability parameters independently of item sample and estimating item parameters independently of ability sample. IRT's invariance feature makes it very practicable in many applications, for instance, test development, computerized adaptive testing, bias studies, test equating and item mapping (Hambleton & Swaminathan, 1985). IRT is classified under two main categories as parametric IRT (PIRT) and nonparametric IRT (NIRT) (Olivares, 2005; Sijtsma & Molenaar, 2002).To analyse ordered items, such as Likert-type attitude items, partial credit cognitive items or not ordered graded items such as multiple-choice test items, item response models are developed towards polytomous items in IRT (Ostini & Nering, 2006). In these models developed for polytomous items, a non-linear relationship between an individual's latent trait and the possibility of choosing a certain category of item answer is explained (Embretson & Reise, 2000). Graded Response Model (GRM), part of IRT models developed for polytomous items, is often preferred by researchers for applications since it is more useful in presentations, portfolios, essays and Likert-type items with ordered item categories (DeMars, 2010; Ostini & Nering, 2006). To scale tests that consist of polytomous items by making true estimates according to GRM, evaluating PIRT's assumptions and model-data fit is necessary. And to provide these assumptions and model-data fit, large samples are needed. At this point, NIRT models draw attention because they provide a practical advantage in determining psychometric properties of tests with fewer items and respondents (Stout, 2001).NIRT models are defined as statistical scaling methods that require fewer assumptions than PIRT models for measuring persons and items (Stochl, 2007). With their wide application area, NIRT models are used in ordinal scales, applied research areas, sociology, marketing research and health research on quality of life (Sijtsma, 2005). The literature reveals that two models, namely, the Mokken model and nonparametric regression estimation models, are employed. These two models are themselves divided into sub-models. The Mokken model consists of the sub-models Monotone Homogeneity Model (MHM) and the Double Monotonicity Model (DMM). Nonparametric regression estimation models consist of such sub-models as the Kernel Smoothing Approach Model (KSAM), the Isotonic Regression Estimation and the Smoothed Isotonic Regression Estimation models (Lee, 2007; Sijtsma & Molenaar, 2002). Along with theoretical studies being conducted, new sub-models are being added to nonparametric regression estimation models.As a NIRT model, MHM requires unidimensionality, local independence and monotonicity assumptions, and it defines the relationship that latent variables and items with homogeneous (unidimensional) and monotone item characteristic curve (ICC) have (Meijer & Baneke, 2004; Sijtsma & Molenaar, 2002). …

A two-step item response theory procedure for a better measurement of marketing constructs

In order to broaden the set of methods for developing better multi-item marketing scales, this article proposes an item response theory (IRT) procedure that consists of two successive steps. The first exploratory step uses a non-parametric IRT model for dimensionality detection and scale purification. The second confirmatory step employs a parametric IRT model for evaluating, and likely refining, the resulting scale. This two-step IRT procedure also involves a wide range of new parametric and non-parametric IRT-based interpretive tools (that is, goodness-of-fit indices, scale-score reliability coefficients and item response/information functions) that allows scale developers and users to work in a highly informed manner. To illustrate the benefits of the proposed procedure, the article presents an extended application to the topical construct of emotional attachment.

INDICATORS OF EMOTIONAL WELL-BEING IN PSYCHOGERIATRIC CARE

Responses of 1,442 consecutive participants in psychogeriatric day care (mean age 78.8; SD 6.5) to 15 items of a mood questionnaire were analyzed by Mokken scale analysis which is based on nonparametric item response theory models. As from 2002, 825 participants also answered eight self-esteem questions. For the purpose of an exploratory and confirmatory study the sample was split into random halves. The sample represented a broad range of cognitive impairment, from moderately severe to mild dementia. An automated item selection procedure available in the R package mokken revealed a scale for emotional well-being consisting of nine items fitting the monotone homogeneity model of unidimensionality and adequate person separation (Loevingers H=0.37; SE=0.02; Cronbach's coefficient alpha=0.79; SE=0.02). A confirmatory analysis in the second random half of the sample confirmed these results. The scale for emotional well-being consists of the items feeling 'contented', 'healthy', 'tired', 'lonely', 'down', 'in good spirits', 'helpless', 'weak' and 'having faith in the future'. Mokken scale analysis of the eight self-worth items confirmed the unidimensionality and discriminatory power of the self-esteem scale (H=0.41; SE=0.03; Cronbach's alpha=0.80; SE=0.02). Emotional well-being was positively associated with self-worth (Spearman correlation=0.56; 95%-confidence interval [0.49;0.62]). The two scales allow the objective ordering of persons on the latent variables of emotional well-being and self-worth by their test scores. Three case vignettes illustrate application of the indicators in clinical psychogeriatric practice.

Dependent Dirichlet Process Rating Model

Typical item response theory (IRT) rating scale models assume that the rating category threshold parameters are the same over examinees. Unfortunately, such models are inappropriate for rating data that exhibit differential item functioning (DIF). The authors introduce a new Bayesian nonparametric IRT model for rating scale items, which is more appropriate for rating data that contain DIF. The model is an infinite mixture of Rasch partial credit models, with mixture distribution modeled by the local (Dependent) Dirichlet process. The model treats the rating category thresholds as the random parameters that are subject to the mixture, with (stick-breaking) mixture weights that are covariate-dependent. Thus, the model allows the rating category thresholds to differ across items and examinees, while allowing the form of the distribution for the category thresholds to vary flexibly as a function of covariates. The authors illustrate the new model through the analysis of simulated data and real data. The model demonstrated the ability to correctly identify DIF items. Moreover, the model attained better predictive-fit performance than did other commonly used IRT rating models.

The Validity of the WHO-5 as an Early Screening for Apathy in an Elderly Population

Aim. The objective of our study has been to evaluate the WHO-5 as a new early screening instrument for apathy in a group of elderly persons. Methods. The WHO-5 was compared to the Geriatric Depression Scale (GDS-15). The GDS contains five items measuring well-being and ten items measuring depression. The internal validity of the WHO-5 (total score being a sufficient statistic) was evaluated with both parametric and nonparametric item response theory models. The external validity of the WHO-5 and the GDS was evaluated by ROC using depression as index of validity. Results. The item response theory analyses confirmed that the total score of the WHO-5 is a sufficient statistic. The ROC analysis shows an adequate sensitivity (61%) and specificity (84%). The GDS15 and its two subscales obtained low sensitivity (25–42%), but high specificity (90–98%). Conclusion. The WHO-5 was found both internally and externally valid when considering decreased positive well-being to be an early indication of apathy reflecting that the wind has begun to be taken out of the “motivation sail.”

Assessing Dimensionality by Maximizing H Coefficient–Based Objective Functions

Mokken scale analysis can be used for scaling under nonparametric item response theory models. The results may, however, not reflect the underlying dimensionality of data. Various features of Mokken scale analysis—the H coefficient, Mokken scale conditions, and algorithms—may explain this result. In this article, three new H-based objective functions with slight reformulations of Mokken scale analysis in the unidimensional and multidimensional cases are introduced. Deterministic and stochastic nonhierarchical clustering algorithms reduced the probability of obtaining suboptimal solutions. A simulation study investigated whether these methods can determine the dimensionality structure of data sets that vary with respect to item discrimination, item difficulty, number of items per trait, and numbers of observations per test. Furthermore, it was investigated whether deterministic and stochastic algorithms can generate approximately global optimal solutions. The method based on the average within-scale Hi combined with a stochastic nonhierarchical clustering algorithm was the most successful in dimensionality assessment.

Mokken Scale Analysis inR

Mokken scale analysis (MSA) is a scaling procedure for both dichotomous and polytomous items. It consists of an item selection algorithm to partition a set of items into Mokken scales and several methods to check the assumptions of two nonparametric item response theory models: the monotone homogeneity model and the double monotonicity model. First, we present an R package mokken for MSA and explain the procedures. Second, we show how to perform MSA in R using test data obtained with the Adjective Checklist.