Multidimensional Item Response Theory Research Articles

A Method of Empirical Q-Matrix Validation for Multidimensional Item Response Theory

ABSTRACT A Q-matrix is a binary matrix that defines the relationship between items and latent variables and is widely used in diagnostic classification models (DCMs), and can also be adopted in multidimensional item response theory (MIRT) models. The construction process of the Q-matrix is typically carried out by experts in the subject area of the items and statistical procedures can be used to verify its suitability. In DCMs, different approaches have been proposed for validating the Q-matrix through iterative algorithms. This article presents a method of empirical Q-matrix validation for MIRT models. A simulation study and an application to real data of morphology skills of elementary students are conducted to examine the viability of the method. Relevant issues regarding the implementation of the method and the results obtained are discussed.

Co-occurring dominance and ideal point processes: A general IRTree framework for multidimensional item responding

AbstractResponding to rating scale items is a multidimensional process, since not only the substantive trait being measured but also additional personal characteristics can affect the respondents’ category choices. A flexible model class for analyzing such multidimensional responses are IRTree models, in which rating responses are decomposed into a sequence of sub-decisions. Different response processes can be involved in item responding both sequentially across those sub-decisions and as co-occurring processes within sub-decisions. In the previous literature, modeling co-occurring processes has been exclusively limited to dominance models, where higher trait levels are associated with higher expected scores. However, some response processes may rather follow an ideal point rationale, where the expected score depends on the proximity of a person’s trait level to the item’s location. Therefore, we propose a new multidimensional IRT model of co-occurring dominance and ideal point processes (DI-MIRT model) as a flexible framework for parameterizing IRTree sub-decisions with multiple dominance processes, multiple ideal point processes, and combinations of both. The DI-MIRT parameterization opens up new application areas for the IRTree model class and allows the specification of a wide range of theoretical assumptions regarding the cognitive processing of item responding. A simulation study shows that IRTree models with DI-MIRT parameterization provide excellent parameter recovery and accurately reflect co-occurring dominance and ideal point processes. In addition, a clear advantage over traditional IRTree models with purely sequential processes is demonstrated. Two application examples from the field of response style analysis highlight the benefits of the general IRTree framework under real-world conditions.

Smooth Information Criterion for Regularized Estimation of Item Response Models

Item response theory (IRT) models are frequently used to analyze multivariate categorical data from questionnaires or cognitive test data. In order to reduce the model complexity in item response models, regularized estimation is now widely applied, adding a nondifferentiable penalty function like the LASSO or the SCAD penalty to the log-likelihood function in the optimization function. In most applications, regularized estimation repeatedly estimates the IRT model on a grid of regularization parameters λ. The final model is selected for the parameter that minimizes the Akaike or Bayesian information criterion (AIC or BIC). In recent work, it has been proposed to directly minimize a smooth approximation of the AIC or the BIC for regularized estimation. This approach circumvents the repeated estimation of the IRT model. To this end, the computation time is substantially reduced. The adequacy of the new approach is demonstrated by three simulation studies focusing on regularized estimation for IRT models with differential item functioning, multidimensional IRT models with cross-loadings, and the mixed Rasch/two-parameter logistic IRT model. It was found from the simulation studies that the computationally less demanding direct optimization based on the smooth variants of AIC and BIC had comparable or improved performance compared to the ordinarily employed repeated regularized estimation based on AIC or BIC.

Examining Differential Item Functioning from a Multidimensional IRT Perspective.

Differential item functioning (DIF) is a standard analysis for every testing company. Research has demonstrated that DIF can result when test items measure different ability composites, and the groups being examined for DIF exhibit distinct underlying ability distributions on those composite abilities. In this article, we examine DIF from a two-dimensional multidimensional item response theory (MIRT) perspective. We begin by delving into the compensatory MIRT model, illustrating and how items and the composites they measure can be graphically represented. Additionally, we discuss how estimated item parameters can vary based on the underlying latent ability distributions of the examinees. Analytical research highlighting the consequences of ignoring dimensionally and applying unidimensional IRT models, where the two-dimensional latent space is mapped onto a unidimensional, is reviewed. Next, we investigate three different approaches to understanding DIF from a MIRT standpoint: 1. Analytically Uniform and Nonuniform DIF: When two groups of interest have different two-dimensional ability distributions, a unidimensional model is estimated. 2. Accounting for complete latent ability space: We emphasize the importance of considering the entire latent ability space when using DIF conditional approaches, which leads to the mitigation of DIF effects. 3. Scenario-Based DIF: Even when underlying two-dimensional distributions are identical for two groups, differing problem-solving approaches can still lead to DIF. Modern software programs facilitate routine DIF procedures for comparing response data from two identified groups of interest. The real challenge is to identify why DIF could occur with flagged items. Thus, as a closing challenge, we present four items (Appendix A) from a standardized test and invite readers to identify which group was favored by a DIF analysis.

다차원 IRT 모형 적합 혼합형 검사를 위한 공통문항 척도 연계 방법

Under the context of the common-item nonequivalent groups design, item response theory (IRT) scale
 linking methods have been presented for each of the multidimensional extensions of the three-parameter
 logistic (M3PL), graded response (MGR), generalized partial credit (MGPC), and nominal response (MNR)
 models. The common-item multidimensional IRT (MIRT) linking methods that have been proved effective
 for single-format tests include the direct least squares (DLS), item category response function (ICRF), and
 test response function (TRF) methods. This paper extends the three MIRT linking methods to any mixture
 of the exploratory M3PL, MGR, MGPC, and MNR models for mixed-format tests. Each linking method
 extended attempts to find the linking coefficients in the rotation matrix and translation vector that
 constitute a linear transformation between two multidimensional ability scales. Simulations are conducted to
 investigate the performance of the three linking methods under various linking conditions. The results
 indicated that overall, the ICRF method performed best in estimating the rotation matrix and translation
 vector of the linear transformation between the new and true simulation scales. In not a few conditions, the
 DLS method resulted in the second best linking solutions. The TRF method tended to estimate the rotation
 matrix well but the translation vector poorly and thus showed the worst performance among the linking
 methods in some conditions.

HBMIRT: A SAS macro for estimating uni- and multidimensional 1- and 2-parameter item response models in small (and large!) samples.

Item response theory (IRT) has evolved as a standard psychometric approach in recent years, in particular for test construction based on dichotomous (i.e., true/false) items. Unfortunately, large samples are typically needed for item refinement in unidimensional models and even more so in the multidimensional case. However, Bayesian IRT approaches with hierarchical priors have recently been shown to be promising for estimating even complex models in small samples. Still, it may be challenging for applied researchers to set up such IRT models in general purpose or specialized statistical computer programs. Therefore, we developed a user-friendly tool - a SAS macro called HBMIRT - that allows to estimate uni- and multidimensional IRT models with dichotomous items. We explain the capabilities and features of the macro and demonstrate the particular advantages of the implemented hierarchical priors in rather small samples over weakly informative priors and traditional maximum likelihood estimation with the help of a simulation study. The macro can also be used with the online version of SAS OnDemand for Academics that is freely accessible for academic researchers.

A Validation Study of the Extended Relevance Scale using the D3mirt Package for R

ABSTRACT This paper reports a validation study based on descriptive multidimensional item response theory (DMIRT), implemented in the R package D3mirt by using the ERS-C, an extended version of the Relevance subscale from the Moral Foundations Questionnaire including two new items for collectivism (17 items in total). Two latent models are created and investigated in two large samples (N = 1560 and N = 1380 US citizens). The internal investigation indicates a presence of two unidimensions, Compassion and Conformity, and a third within-multidimension of Collectivism in both samples. The external investigation show that Republicans are the highest on Conformity in the models, followed by Independents, and lastly Democrats. Democratic followers are the highest on compassion, followed by Independents and Republicans. No difference between groups on Collectivism were found. Lastly, the models succeeded in correctly identifying the Republican and Democratic voters but struggled to correctly classify the Independent.

How Scoring Approaches Impact Estimates of Growth in the Presence of Survey Item Ceiling Effects.

Survey scores are often the basis for understanding how individuals grow psychologically and socio-emotionally. A known problem with many surveys is that the items are all "easy"-that is, individuals tend to use only the top one or two response categories on the Likert scale. Such an issue could be especially problematic, and lead to ceiling effects, when the same survey is administered repeatedly over time. In this study, we conduct simulation and empirical studies to (a) quantify the impact of these ceiling effects on growth estimates when using typical scoring approaches like sum scores and unidimensional item response theory (IRT) models and (b) examine whether approaches to survey design and scoring, including employing various longitudinal multidimensional IRT (MIRT) models, can mitigate any bias in growth estimates. We show that bias is substantial when using typical scoring approaches and that, while lengthening the survey helps somewhat, using a longitudinal MIRT model with plausible values scoring all but alleviates the issue. Results have implications for scoring surveys in growth studies going forward, as well as understanding how Likert item ceiling effects may be contributing to replication failures.

Variational Estimation for Multidimensional Generalized Partial Credit Model.

Multidimensional item response theory (MIRT) models have generated increasing interest in the psychometrics literature. Efficient approaches for estimating MIRT models with dichotomous responses have been developed, but constructing an equally efficient and robust algorithm for polytomous models has received limited attention. To address this gap, this paper presents a novel Gaussian variational estimation algorithm for the multidimensional generalized partial credit model. The proposed algorithm demonstrates both fast and accurate performance, as illustrated through a series of simulation studies and two real data analyses.

The appearance of multidimensionality: Exploring the case for unidimensionality of the PTSD Checklist for DSM-5 using a firefighter sample.

Rasmussen et al. (2019) described the proliferation of factors for posttraumatic stress disorder (PTSD) measures and raised concerns about the construct validity of factors that include two or three items. In this brief report, we describe how the pattern of covariation among the responses to items of well-established measures, such as the PTSD Checklist for DSM-5 (PCL-5), can give the appearance of multidimensionality. We evaluated whether the structure of the 20-item PCL-5 is unidimensional, using the methods of multidimensional item response theory (MIRT) and the concept of a testlet. These analyses were done using a sample of trauma-exposed urban firefighters. A unidimensional and a bifactor model, which includes a general factor composed of all items and four specific factors mirroring the DSM-5 conceptualization, were evaluated for both Likert-type multiple-category and binary coding system of the PCL-5 item response data. Seven testlets were created from the 20 PCL-5 items following the seven-factor model (Armour et al., 2015) presented in Table 1 of Rasmussen et al. (2019). Findings using the unidimensional nominal item response theory model for the seven testlets indicated that the PCL-5 may be considered unidimensional with a single score representing individual differences on a continuum that ranges from low to high. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Variation of subclinical psychosis across 16 sites in Europe and Brazil: findings from the multi-national EU-GEI study.

Incidence of first-episode psychosis (FEP) varies substantially across geographic regions. Phenotypes of subclinical psychosis (SP), such as psychotic-like experiences (PLEs) and schizotypy, present several similarities with psychosis. We aimed to examine whether SP measures varied across different sites and whether this variation was comparable with FEP incidence within the same areas. We further examined contribution of environmental and genetic factors to SP. We used data from 1497 controls recruited in 16 different sites across 6 countries. Factor scores for several psychopathological dimensions of schizotypy and PLEs were obtained using multidimensional item response theory models. Variation of these scores was assessed using multi-level regression analysis to estimate individual and between-sites variance adjusting for age, sex, education, migrant, employment and relational status, childhood adversity, and cannabis use. In the final model we added local FEP incidence as a second-level variable. Association with genetic liability was examined separately. Schizotypy showed a large between-sites variation with up to 15% of variance attributable to site-level characteristics. Adding local FEP incidence to the model considerably reduced the between-sites unexplained schizotypy variance. PLEs did not show as much variation. Overall, SP was associated with younger age, migrant, unmarried, unemployed and less educated individuals, cannabis use, and childhood adversity. Both phenotypes were associated with genetic liability to schizophrenia. Schizotypy showed substantial between-sites variation, being more represented in areas where FEP incidence is higher. This supports the hypothesis that shared contextual factors shape the between-sites variation of psychosis across the spectrum.

Development and validation of the Convalescence Symptom Assessment Scale for EsophageCtomy patients.

A specific, valid and reliable measure is much needed to dynamically assess the recovery of symptoms in oesophagectomy patients. This study describes developing and validating the Convalescent Symptom Assessment Scale for oesophagectomy patients (CSAS_EC). An instrument development and cross-sectional validation study was conducted. This study consists of two components: instrument development and psychometric tests. In instrument development, the literature review, qualitative interviews, Delphi method expert consultation and face validation were used to develop and refine scale content. In psychometric tests, the clinical test version scale was used to conduct a cross-sectional in the thoracic surgery department from 17 June to 20 November 2022. The Classical Test Theory and Multidimensional Item Response Theory (MIRT) analyses examined psychometric properties. In instrument development, literature review (n = 20), qualitative interviews (n = 21), expert consultation (n = 12) and pre-survey (n = 15) led to the development of the clinical test version scale. In psychometric tests, a total of 331 participants were enrolled. Confirmatory factor analysis and MIRT analysis verified that a model with 28 items in four dimensions was good. The four dimensions were early recovery symptoms, late recovery symptoms, persistent present symptoms and psychosocial symptoms. The Cronbach's α is 0.827. The validity and reliability were demonstrated to be acceptable. The CSAS_EC scale can be used as a tool to evaluate the recovery status of oesophagectomy patients.

Development and initial validation of a creative self-efficacy scale for undergraduates: categorical confirmatory factor analysis and multidimensional item response theory

Creative self-efficacy (CSE) has recently received much attention due to its association with student learning and creativity. To that end, a CSE scale was developed for undergraduates and sources of validity evidence based on scale content, response processes, and internal structure were collected. Score reliability, using categorical omega based on the categorical confirmatory factor analysis model and marginal reliability for response pattern scores based on item response theory (IRT), were estimated. After various revision iterations of the initial 28-item pool by 10 subject matter experts and 18 undergraduates, some items were revised, four items were dropped, and ultimately 24 items were field tested for measuring two hypothesized dimensions of CSE among 602 undergraduates. Categorical confirmatory factor analysis results indicated that the two-dimensional model had better fit. Similarly, between the two competing multidimensional IRT models, the two-dimensional graded response model had the best fit. Categorical omega coefficients and marginal reliability for response pattern scores were, respectively, 0.88 and 0.81 for the two underlying dimensions.

Revalidation of Teachers' Sense of Efficacy Scale using Ghanaian in‐service teachers: Multidimensional item response theory with factor analyses

AbstractTeachers' Sense of Efficacy Scale (TSES), since its development, has gained much popularity because of its adequate psychometric properties. Yet, scholarly information on the soundness of this instrument in the African context has not been documented. Besides, the previous validation studies on the TSES have yielded mixed results with different factor structures, calling for further studies to test its utility and use across different cultures. The purpose of this study was to revalidate the TSES in Ghana using item response theory (IRT) complemented by factor analyses. Through a validation study approach, 693 basic school teachers were purposively sampled to participate in the research by responding to the Teacher Efficacy Scale. The between‐item and within‐item multidimensional IRT analyses combined with factor analysis were performed to understand how the items functioned with intentions to propose a well‐fit instrument within the Ghanaian context. The outcome of this research revealed that the original TSES with a three‐factor structure, 24 items measured on a 9‐point scale did not provide an optimal measure of teachers' self‐efficacy within the Ghanaian context. Rather, a three‐structure scale with 18 items and a 6‐point scale offered a maximal measure of teachers' self‐efficacy. The practical implications of the findings are discussed.

The ABC of happiness: Validation of the tridimensional model of subjective well-being (affect, cognition, and behavior) using Bifactor Polytomous Multidimensional Item Response Theory

BackgroundHappiness is often conceptualized as subjective well-being, which comprises people's evaluations of emotional experiences (i.e., the affective dimension: positive and negative feelings and emotions) and judgements of a self-imposed ideal (i.e., the cognitive dimension: life satisfaction). Recent research has established these two dimensions as primary parts of a higher order factor. However, theoretical, conceptual, and empirical work suggest that people's evaluations of harmony in their life (i.e., the sense of balance and capacity to behave and adapt with both acceptance and flexibility to inter- and intrapersonal circumstances) constitutes a third dimension (i.e., the behavioral dimension). This tridemensional conceptualization of subjective well-being has recently been verified using Unidimensional Item Response Theory (UIRT) and Classical Test Theory (CTT). Here, we use a recently developed and more robust approach that combines these two methods (i.e., Multidimensional Item Response Theory, MIRT) to simultaneously address the complex interactions and multidimensionality behind how people feel, think, and behave in relation to happiness in their life. MethodA total of 435 participants (197 males and 238 females) with an age mean of 44.84 (sd = 13.36) responded to the Positive Affect Negative Affect Schedule (10 positive affect items, 10 negative affect items), the Satisfaction with Life Scale (five items), and the Harmony in life Scale (five items). We used Bifactor-Graded Response MIRT for the main analyses. ResultAt the general level, each of the 30 items had a strong capacity to discriminate between respondents across all three dimensions of subjective well-being. The investigation of different parameters (e.g., marginal slopes, ECV, IECV) strongly reflected the multidimensionality of subjective well-being at the item, the scale, and the model level. Indeed, subjective well-being could explain 64 % of the common variance in the whole model. Moreover, most of the items measuring positive affect (8/10) and life satisfaction (4/5) and all the items measuring harmony in life (5/5) accounted for a larger amount of variance of subjective well-being compared to that of their respective individual dimensions. The negative affect items, however, measured its own individual concept to a lager extent rather than subjective well-being. Thus, suggesting that the experience of negative affect is a more independent dimension within the whole subjective well-being model. We also found that specific items (e.g., “Alert”, “Distressed”, “Irritable”, “I am satisfied with my life”) were the recurrent exceptions in our results. Last but not the least, experiencing high levels in one dimension seems to compensate for low levels in the others and vice versa. ConclusionAs expected, the three subjective well-being dimensions do not work separately. Interestingly, the order and magnitude of the effect by each dimension on subjective well-being mirror how people define happiness in their life: first as harmony, second as satisfaction, third as positive emotions, and fourth, albeit to a much lesser degree, as negative emotions. Ergo, we argue that subjective well-being functions as a complex biopsychosocial adaptive system mirroring our attitude towards life in these three dimensions (A: affective dimension; B: behavioral dimension; C: cognitive dimension). Ergo, researchers and practitioners need to take in to account all three to fully understand, measure, and promote people's experience of the happy life. Moreover, our results also suggest that negative affect, especially regarding high activation unpleasant emotions, need considerable changes and further analyses if it is going to be included as a construct within the affective dimension of a general subjective well-being factor.

Estimation of a Simple Structure in a Multidimensional IRT Model Using Structure Regularization.

We develop a method for estimating a simple matrix for a multidimensional item response theory model. Our proposed method allows each test item to correspond to a single latent trait, making the results easier to interpret. It also enables clustering of test items based on their corresponding latent traits. The basic idea of our approach is to use the prenet (product-based elastic net) penalty, as proposed in factor analysis. For optimization, we show that combining stochastic EM algorithms, proximal gradient methods, and coordinate descent methods efficiently yields solutions. Furthermore, our numerical experiments demonstrate its effectiveness, especially in cases where the number of test subjects is small, compared to methods using the existing L1 penalty.

Investigation of Missing Data Methods in Polytomous Multidimensional Item Response Theory

Bu çalışmada, bazı kayıp veri ile baş etme yöntemlerinin çoklu puanlanan çok boyutlu madde tepki kuramı modellerinden olan çok boyutlu aşamalı tepki modelinden elde edilen madde ve yetenek parametrelerine etkisinin araştırılması amaçlanmıştır. İlk olarak belirlenen parametrelere uygun şekilde iki boyutlu bir referans veri seti üretilmiştir. Üretilen veri setinden tamamen rassal olarak kayıp mekanizmasında %5, %10 ve %20 oranında kayıplar oluşturulmuştur. Eksik veri setleri seri ortalaması atama (SO), regresyon atama (RA) ve beklenti maksimizasyonu (BM) yöntemleri ile tamamlanmış ve tamamlanan veri setlerinden madde ile yetenek parametreleri kestirilmiştir. Kestirilen parametreler, referans veri setinden kestirilen parametreler ile karşılaştırılmıştır. Elde edilen bulgulara göre madde ayırıcılık parametreleri için tüm kayıp oranlarında RA ve BM yöntemlerinin daha düşük RMSE değerleri ürettiği görülmüştür. Kategori sınır kesişim parametrelerinde ise d1 ve d3 için RA ve SO yöntemleri, d2 için ise BM ve RA yöntemleri tüm kayıp oranlarında daha düşük hata üretmiştir. Genel olarak tüm madde parametreleri değerlendirildiğinde, RA yönteminin parametre değerlerinin geri kazanımında oldukça iyi sonuçlar ürettiği söylenebilir. Yetenek parametrelerinin geri kazanımı için ise tüm kayıp oranlarında en düşük RMSE değerleri üreten yöntemin BM olduğu görülmüştür.

Psychometric evaluation of the United States-adapted perceived perioperative competence scale-revised: A national survey

BackgroundNursing competence is integral to ensuring patient safety, especially in high-risk environments such as the operating room. Instruments which facilitate self-assessment of specialty specific nursing competence allow nurses to gain important insights into their practice to facilitate continuous growth in their professional practice. Currently, there are no psychometrically tested tools to assess perioperative competence applicable to the United States context. ObjectiveTest the psychometric properties of the Perceived Perioperative Competence Scale-Revised in the United States context. MethodsA cross-sectional survey design was used. Perioperative nurses were recruited via four professional associations and the survey was administered online. Construct validity of the six-dimensional Perceived Perioperative Competence Scale-Revised was tested using a multidimensional item response theory model known as the graded response model. Measurement invariance was assessed relative to years of perioperative experience. Internal consistency was estimated using McDonald's Omega and Cronbach's alpha reliability coefficients. ResultsResponses from a total of 1,581 participants were analyzed in the psychometric analysis. The six-dimensional graded response model of the Perceived Perioperative Competence Scale-Revised displayed satisfactory model fit for the sample (Chi-square(df) = 5,699.09(725); root mean square error of approximation = 0.066, 90% confidence interval: 0.064, 0.067; comparative fit index = 0.955; Tucker-Lewis index = 0.952; standardized root mean squared residual = 0.045). Scalar invariance was established when assessing the psychometric equivalence of the scale across years of perioperative experience (<10 years, ≥ 10 years) (Chi-square(df) = 5,785.29(1,573); root mean square error of approximation = 0.058, 90% confidence interval: 0.057, 0.060; comparative fit index = 0.959; Tucker–Lewis index = 0.959). Reliability across the six subscales ranged from alpha = 0.87 - 0.94 and Omega = 0.93 - 0.97. ConclusionsResults suggest that the Perceived Perioperative Competence Scale-Revised is suitable to use with perioperative nurses practicing in clinical settings in the United States. Measurement invariance testing indicates the scale is measuring the same construct and is being interpreted in a conceptually similar manner across groups based on years of perioperative experience.

The validity and IRT psychometric analysis of Chinese version of Difficult Doctor-Patient Relationship Questionnaire (DDPRQ-10)

ObjectiveThe doctor-patient relationship (DPR) plays a crucial role in the Chinese healthcare system, functioning to improve medical quality and reduce medical costs. This study examined the psychometric properties of the Chinese version of the Difficult Doctor-Patient Relationship Questionnaire (DDPRQ-10) among general hospital inpatients in China.MethodsThe research recruited 38 resident doctors responsible for 120 participants, and factor analyses were used to assess the construct validity of the scale. Convergent validity was evaluated by examining the correlation between DDPRQ-10 and depressive symptoms, burnout, and self-efficacy, using the Patient Health Questionnaire Depression Scale-9 item (PHQ-9), and the Maslach Burnout Inventory (MBI). Both multidimensional item response theory (MIRT) and unidimensional item response theory (IRT) frameworks were used to estimate the parameters of each item.ResultsThe Chinese version of DDPRQ-10 showed satisfactory internal consistency (Cronbach's alpha = 0.931), and fitted in a modified two-factor model of positive feelings and negative feelings (χ2/df = 1.494, GFI = 0.925, RMSEA = 0.071, SRMR = 0.008, CFI = 0.985, NFI = 0.958, NNFI = 0.980, TLI = 0.980, IFI = 0.986). Significant correlations with PHQ-9 with DDPRQ-10 and both subscales were revealed (r = 0.293 ~ 0.333, p < .001), while DDPRQ-10 score also significantly correlated with doctors’ MBI score (r = -0.467, p < .001). The MIRT model of full scale and IRT models of both subscales showed high discrimination of all items (a = 2.30 ~ 10.18), and the test information within the range of low-quality relationship was relatively high.ConclusionThe Chinese version of DDPRQ-10 displayed satisfactory reliability and validity and thus was appropriate for measuring the DPR in Chinese medical settings.

Detecting differential item functioning with multiple causes: A comparison of three methods

Differential item functioning (DIF) often arises from multiple sources. Within the context of multidimensional item response theory, this study examined DIF items with varying secondary dimensions using the three DIF methods: SIBTEST, Mantel-Haenszel, and logistic regression. The effect of the number of secondary dimensions on DIF detection rates was investigated with several test conditions such as the percentage of DIF items, item loadings, the correlation between primary and secondary dimensions, and DIF magnitude. Results indicate that both the Mantel–Haenszel and logistic regression procedures performed effectively in detecting DIF when multiple secondary dimensions were present.