Local Independence Assumption Research Articles

INDONESIAN VERSION OF SATISFACTION WITH LIFE SCALE, A PSYCHOMETRIC EVALUATION WITH RASCH MODEL

Previous studies on The Satisfaction with Life Scale (SWLS) showed inconsistency, primarily related to the justice principle based on gender when using Confirmatory Factor Analysis (CFA). Therefore, this study aims to evaluate the psychometric property of the Indonesian version of SWLS using the Rasch model. This model can be used as an evaluation technique for psychological instruments. The Rasch model showed a more detailed analysis than CFA, explaining person-item fit statistics, rating scale diagnosis, item calibration, and differential item functioning. The total participants after person fit checking were 1,154 university students who completed an online survey consisting of demographic data and a five-item Indonesian version of SWLS. The Rasch Rating Scale Analysis (RSM) showed that the Indonesian version of SWLS fulfilled unidimensional and local independence assumptions. The items delivered a good item fit index with a five-point rating scale, and there were no gender biases in moderate level and high differential item functioning (DIF). Therefore, the Indonesian version of SWLS is recommended for further research measuring life satisfaction. This research implies that using the Indonesian version of SWLS, especially in Indonesia, can use a five-point rating scale for future research.

Influences of Carry-Over Effects Across Scales on Mediation Analyses

ABSTRACT Many education tests and psychological surveys elicit respondent views of similar constructs across scenarios (e.g., story followed by multiple choice questions) by repeating common statements across scales (one-statement-multiple-scale, OSMS). However, a respondent’s earlier responses to the common statement can affect later responses to it (recursive carry-over effect), which violates the local item independence assumption of most measurement models. Ignoring this carry-over effect can bias both estimated item parameters for later scales and relations among scales. This study investigates the consequences of model misspecification for OSMS questions in mediation analyses via real data from the Teaching and Learning International Survey (TALIS) and simulations. The results showed that failing to consider carry-over effects in a mediation model yielded biased relations between latent variables. Moreover, our MMCEO mediation model corrected the inflated correlations caused by carry-over effects and yielded superior estimates of the mediation effects. We discuss implications for appropriately addressing carry‑over effects.

Pairwise Likelihood Estimation of the 2PL Model with Locally Dependent Item Responses

The local independence assumption is crucial for the consistent estimation of item parameters in item response theory models. This article explores a pairwise likelihood estimation approach for the two-parameter logistic (2PL) model that treats the local dependence structure as a nuisance in the optimization function. Hence, item parameters can be consistently estimated without explicit modeling assumptions of the dependence structure. Two simulation studies demonstrate that the proposed pairwise likelihood estimation approach allows nearly unbiased and consistent item parameter estimation. Our proposed method performs similarly to the marginal maximum likelihood and pairwise likelihood estimation approaches, which also estimate the parameters for the local dependence structure.

On the Identifiability of 3- and 4-Parameter Item Response Theory Models From the Perspective of Knowledge Space Theory.

The present work aims at showing that the identification problems (here meant as both issues of empirical indistinguishability and unidentifiability) of some item response theory models are related to the notion of identifiability in knowledge space theory. Specifically, that the identification problems of the 3- and 4-parameter models are related to the more general issues of forward- and backward-gradedness in all items of the power set, which is the knowledge structure associated with IRT models under the assumption of local independence. As a consequence, the identifiability problem of a 4-parameter model is split into two parts: a first one, which is the result of a trade-off between the left-side added parameters and the remainder of the Item Response Function, e.g., a 2-parameter model, and a second one, which is the already well-known identifiability issue of the 2-parameter model itself. Application of the results to the logistic case appears to provide both a confirmation and a generalization of the current findings in the literature for both fixed- and random-effects IRT logistic models.

A Review of Measurement Tools Developed and Adapted Based on the Rasch Model

It is often observed that the Rasch model is frequently used in determining the psychometric properties of measurement tools because the Rasch model has many advantages in the development and adaptation of measurement instruments. The aim of this study is to evaluate the theses included in the National Thesis Center, which examine the psychometric properties of measurement tools within the framework of the Rasch model, within the scope of the requirements of the Rasch model. In line with this purpose, the model of the research is a document analysis research within the scope of qualitative research. All theses containing the word Rasch in the thesis name and index were examined, and 24 theses in which the measurement tool was developed and adapted within the scope of the Rasch model were found. In order to examine these measurement tools, a coding list was created and the data was analyzed by applying categorical analysis which is one of the content analysis methods. According to the results obtained, it was revealed that in the majority of theses, information was given about unidimensionality, but in half of the theses, no information was given about the local independence assumption. There are studies that do not specify which model is used for polytomous items, and it was observed that model comparison was not performed. It was determined that item model fit was generally tested with different approaches in the theses, and item parameters were generally included. It is among the results that the Person separation index related to reliability was not reported in all studies, and sufficient information was not provided in some studies, even though Differential Item Functioning analyses were performed. In light of these results, it is seen that there is no common systematic approach in the development or adaptation of measurement tools within the framework of the Rasch model in the studies. Therefore, it is recommended that more detailed studies explaining this systematic approach should be conducted.

ASSESSMENT OF THE PSYCHOMETRIC PROPERTIES OF ATTITUDE TOWARDS ASSESSMENT TEST (ATAT)

This study is on the assessment of the psychometric properties of Attitude Towards Assessment Test (ATAT). Four research questions guided the study. An instrumentation research design was adopted. The population consisted of secondary school students in Delta State, Nigeria. The sample size comprised 1,000 students, selected through simple random and non-probability cluster sampling techniques. The test under assessment was developed by Megbele, et al. (2023). The Rasch Rating Scale Model was used to answer research 1, which assessed person and item reliability, item statistics, and ordering of response categories. Two fit indices including the infit and outfit mean square (MNSQ) statistics were used to answer research question 2. The data that were used to answer research questions 1 and 2 were analysed with the aid of the Jmetrik IRT software. The Categorical Confirmatory Factor Analysis (CCFA) was used to answer research question 3 while Chi-Square Goodness of Fit Statistics was used to answer research question 4 on construct validity, evidence of unidimensionality, and local independence respectively. The findings of this study revealed that the three components of the scale had high values of item separation index and reliability as well as an acceptable range of Person separation index and reliability; the difficulty index of items in the test was within an acceptable range; Each of the components of the Attitude Towards Assessment Test (ATAT) (Cognitive, affective and behavioural) had one construct each, which is evidence of unidimensionality for the different components of the Attitude Towards Assessment Test. Based on the findings of the study, it was concluded that all items in the different components that made up the test are reliable, have adequate item difficulty infit, and outfit MNSQ estimates, with evidence of unidimensionality and local independence assumptions. The study recommended that the test can be used by examination bodies for the assessment of students in the affective domain.<p> </p><p><strong> Article visualizations:</strong></p><p><img src="/-counters-/soc/0011/a.php" alt="Hit counter" /></p>

Investigating the quality of life for children with autism spectrum disorder scale using Rasch methodology.

Our purpose in this study was to provide additional psychometric evidence of the Quality of Life for Children with Autism Spectrum Disorder (QOLASD-C) scale. We used Rasch modeling to investigate the QOLASD-C functioning, the characteristics of the items comprising the scale, and the item functioning across subgroups of children with ASD based on gender and race/ethnicity. Results showed that QOLASD-C was unidimensional, met the local independence assumption, and measured quality of life (QOL). The items showed excellent fit to the model and good discriminating ability between low and high QOL. Most items showed a moderate difficulty level. No differential item functioning was observed based on children's gender and race/ethnicity. Implications for research and practice are discussed.

Parameter Estimation of KST-IRT Model under Local Dependence

A mantra often repeated in the introductory material to psychometrics and Item Response Theory (IRT) is that a Rasch model is a probabilistic version of a Guttman scale. The idea comes from the observation that a sigmoidal item response function provides a probabilistic version of the characteristic function that models an item response in the Guttman scale. It appears, however, more difficult to reconcile the assumption of local independence, which traditionally accompanies the Rasch model, with the item dependence existing in a Guttman scale. In recent work, an alternative probabilistic version of a Guttman scale was proposed, combining Knowledge Space Theory (KST) with IRT modeling, here referred to as KST-IRT. The present work has, therefore, a two-fold aim. Firstly, the estimation of the parameters involved in KST-IRT models is discussed. More in detail, two estimation methods based on the Expectation Maximization (EM) procedure are suggested, i.e., Marginal Maximum Likelihood (MML) and Gibbs sampling, and are compared on the basis of simulation studies. Secondly, for a Guttman scale, the estimates of the KST-IRT models are compared with those of the traditional combination of the Rasch model plus local independence under the interchange of the data generation processes. Results show that the KST-IRT approach might be more effective in capturing local dependence as it appears to be more robust under misspecification of the data generation process, but it comes with the price of an increased number of parameters.

The development and validation of the Chinese safety climate scale using the item-response theory approach.

To develop a valid and practical tool to measure the safety climate in China, and further raise awareness of it in Chinese industries, we developed the Chinese safety climate scale (including two subscales at the levels of organization and group separately) based on the work of Huang et al. in 2017. A descriptive survey with the convenience sampling method was conducted in Shanghai Disney Resort, China. A sample of 1,570 employees was involved in the final data analyses. The item response theory (IRT) analyses with graded response models were conducted using. The unidimensionality and local independence assumption were held. The Cronbach's α of organization- and group- level safety climate subscale was 0.912 and 0.937, respectively. The confirmatory factor analysis (CFA) showed good model fit for a one-factor model of the organization-level safety climate subscale, χ2 (df = 20) = 129.158, p < 0.001, CFI = 0.994, TLI = 0.992, NFI = 0.993, IFI = 0.994, RMSEA = 0.059, 90% CI = (0.050, 0.069), and SRMR = 0.048. A one-factor model also fits well for the group-level safety climate subscale, χ2 (df = 44) = 219.727, p < 0.001, CFI = 0.996, TLI = 0.9925, NFI = 0.995, IFI = 0.996, RMSEA = 0.050, 90% CI = (0.044, 0.057), and SRMR = 0.046. Discrimination and difficulty parameters showed that all items effectively spanned the range of the latent trait and could successfully separate participants at different safety climate levels. Items in the organization-level and group-level Chinese safety climate subscales had significantly different discrimination parameters, fitted well with the models, and had a substantive relationship with the latent traits. The Chinese safety climate scale was reliable and valid overall. They can facilitate the research and survey regarding the safety climate in China.

Investigating gender-based differential item functioning on the McLean Screening Instrument for Borderline Personality Disorder (MSI-BPD): An item response theory analysis.

The McLean Screening Instrument for Borderline Personality Disorder (MSI-BPD) is a popular screening tool for identifying people who may have borderline personality disorder (BPD). However, because women are more frequently diagnosed with the disorder than men, it is possible that the MSI-BPD differs in its ability to identify BPD as a function of gender identity. Using item response theory (IRT), we sought to determine if components of the MSI-BPD would demonstrate differential item functioning (DIF), such that one gender identity would be more likely to endorse certain items. Twenty-two thousand thirty-five college undergraduates (14,305 women) aged 18-55 years (M = 18.77, SD = 1.75) were assessed using the MSI-BPD as part of a subject pool screening between 2008 and 2019. The MSI-BPD contains 10 items that are measured dichotomously, and the authors recommend a cut-off of 7 of 10 items endorsed to maximize sensitivity and specificity to BPD. Results suggested that a two-parameter model was the best fit to the data and that unidimensionality and local independence assumptions were met. The following items demonstrated DIF: self-harm/suicidality, affective lability, abandonment, impulsivity, and anger. At equal levels of the latent construct of BPD, women were more likely to endorse self-harm/suicidality, affective lability, and abandonment. Women were more likely to endorse impulsivity at higher levels of BPD, and men were more likely to endorse anger at lower levels of BPD. Ultimately, the effect sizes of these differences were small, however, and likely do not impact the individual's overall outcome on the measure. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Investigation of the Effect of Violation of the Local Independence Assumption in Test Equating on Delta and Bootstrap Equating Errors According to Various Variables

Bu araştırmada doğrusal, eşit yüzdelikli ve polinominal loglinear öndüzgünleşirilmiş eşit yüzdelikli test eşitleme yöntemlerinde hataların belirlenmesinde kullanılan delta ve bootstrap eşitleme hatası kestirme yöntemlerinin örneklem büyüklüğü, madde sayısı ve ikinci boyuta yüklenen madde yüzdesi değişkenleri bakımından incelenmesi amaçlanmıştır. Araştırmada eşitleme yöntemleri farklı koşullarda simülasyon verileri ile kontrollü olarak karşılaştırıldığından araştırma bir simülasyon çalışması niteliğindedir. Simülasyon çalışmalarında elde edilen simülasyon verisinin gerçek yanıtları temsil etmesi için simülasyon verileri PISA 2018 matematik sınavı birinci formundan elde edilen verilere ait dağılımlarından yararlanarak üretilmiştir. Araştırmada, örneklem büyüklüğü (250, 1000, 5000), test uzunluğu (20, 40, 60), ikinci boyuta yüklenen madde oranı (%15-%30-%50) olmak üzere 36 koşul incelenmiştir. Bu koşullar altında 2PLM’ye uyumlu iki kategorili ve 100 replikasyon ile 3600 veri seti elde edilmiştir. Araştırmamızda “random gruplar deseni” kullanılmıştır. Genel olarak örneklem büyüklüğü azaldıkça elde edilen hata miktarlarında artış olduğu en az hata içeren koşulun 5000 örneklem büyüklüğü ve testte yer alan madde sayısının 20 olması durumunda elde edildiği, en iyi performansa sahip eşitleme yönteminin doğrusal eşitleme yöntemi olduğu ve eşitleme hatalarını belirlemede kullanılan yöntem olarak da delta yöntemi olduğu sonucuna ulaşılmıştır. Ayrıca testin tek boyutlu yapısının bozulup çok boyutlu olması durumunda ikinci boyuta yüklenen madde oranı bakımından elde edilen eşitleme hatalarında sistematik bir bulguya rastlanılmadığı, araştırmada ele alınan test eşitleme yöntemleri, eşitleme hatalarını belirlemede kullanılan yöntem, örneklem büyüklüğü ve testte yer alan madde sayısı koşullarına göre değişkenlik gösterdiği tespit edilmiştir.

Modeling local item dependence in C-tests with the loglinear Rasch model

C-tests are gap-filling tests mainly used as rough and economical measures of second-language proficiency for placement and research purposes. A C-test usually consists of several short independent passages where the second half of every other word is deleted. Owing to their interdependent structure, C-test items violate the local independence assumption of IRT models. This poses some problems for IRT analysis of C-tests. A few strategies and psychometric models have been suggested and employed in the literature to circumvent the problem. In this research, a new psychometric model, namely, the loglinear Rasch model, is used for C-tests and the results are compared with the dichotomous Rasch model where local item dependence is ignored. Findings showed that the loglinear Rasch model fits significantly better than the dichotomous Rasch model. Examination of the locally dependent items did not reveal anything as regards their contents. However, it did reveal that 50% of the dependent items were adjacent items. Implications of the study for modeling local dependence in C-tests using different approaches are discussed.

Unique Variable Analysis: A Network Psychometrics Method to Detect Local Dependence

The local independence assumption states that variables are unrelated after conditioning on a latent variable. Common problems that arise from violations of this assumption include model misspecification, biased model parameters, and inaccurate estimates of internal structure. These problems are not limited to latent variable models but also apply to network psychometrics. This paper proposes a novel network psychometric approach to detect locally dependent pairs of variables using network modeling and a graph theory measure called weighted topological overlap (wTO). Using simulation, this approach is compared to contemporary local dependence detection methods such as exploratory structural equation modeling with standardized expected parameter change and a recently developed approach using partial correlations and a resampling procedure. Different approaches to determine local dependence using statistical significance and cutoff values are also compared. Continuous, polytomous (5-point Likert scale), and dichotomous (binary) data were generated with skew across a variety of conditions. Our results indicate that cutoff values work better than significance approaches. Overall, the network psychometrics approaches using wTO with graphical least absolute shrinkage and selector operator with extended Bayesian information criterion and wTO with Bayesian Gaussian graphical model were the best performing local dependence detection methods overall.

Generalized Mantel-Haenszel Estimators for Simultaneous Differential Item Functioning Tests.

The Mantel-Haenszel estimator is one of the most popular techniques for measuring differential item functioning (DIF). A generalization of this estimator is applied to the context of DIF to compare items by taking the covariance of odds ratio estimators between dependent items into account. Unlike the Item Response Theory, the method does not rely on the local item independence assumption which is likely to be violated when one item provides clues about the answer of another item. Furthermore, we use these (co)variance estimators to construct a hypothesis test to assess DIF for multiple items simultaneously. A simulation study is presented to assess the performance of several tests. Finally, the use of these DIF tests is illustrated via application to two real data sets.

Rasch analysis of the Oxford shoulder score in a non-surgical occupational population.

The Oxford shoulder score (OSS) questionnaire for measuring patient perception of shoulder disability, has not tested specifically in a non-surgical population and no study has assessed the OSS with modern psychometrics based on Rasch model (RM). To assess the psychometric properties of the OSS using RM among health-care workers with shoulder disorders and to verify its interest in a non-surgical population. In an occupational health department of a French hospital center, a retrospective review was performed of the medical records from June 2019 to October 2020. Responses to 110 questionnaires were examined from 55 subjects (97% of women). A polytomous Rasch model based on the Partial Credit Model was used. Overall fit was satisfactory, the reliability coefficient was high and an ascending order was observed with the 5 categories of the scale. Analysis of the residuals supports unidimensionality and the local independence assumption. Item performance remained stable across the subgroup examined (DIF measures). Scale to-sample targeting indicated a substantial floor effect, and the mildest impairments were not well discriminated. OSS presents good psychometric qualities. However, it does not clearly discriminate subjects presenting the lowest levels of impairment. Its use in a non-surgical population is questionable.

A Guide to Detecting and Modeling Local Dependence in Latent Class Analysis Models

Latent class analysis (LCA) assigns individuals to mutually exclusive classes based on response patterns to a set of indicators. A primary assumption made is local independence, which suggests class indicators are uncorrelated within each class. When the indicators are correlated and unmodeled, parameter estimates can be severely biased. We provide a comprehensive resource for applied researchers to statistically detect local independence violations and model identified correlated residuals. We explain the local independence assumption and illustrate how to detect and model conditional dependence using maximum likelihood (ML) and Bayesian estimation. For ML, we discuss two detection methods (bivariate residual associations, and the modification index) and one modeling technique (LCA residual associations model). We also demonstrate how to use the restrictive prior strategy to detect and model conditional dependence when using Bayesian estimation. These techniques are illustrated with simulated datasets; code is provided in the online supplemental materials.

Item parameter recovery via traditional 2PL, Testlet and Bi-factor models for Testlet-Based tests

The testlet comprises a set of items based on a common stimulus. When the testlet is used in the tests, there may violate the local independence assumption, and in this case, it would not be appropriate to use traditional item response theory models in the tests in which the testlet is included. When the testlet is discussed, one of the most frequently used models is the testlet response theory (TRT) model. In addition, the bi-factor model and traditional 2PL models are also used for testlet-based tests. This study aims to examine the item parameters estimated by these three calibration models of the data properties produced under different conditions and to compare the performances of the models. For this purpose, data were generated under three conditions: sample size (500, 1000, and 2000), testlet variance (.25, .50, and 1), and testlet size (4 and 10). For each simulation condition, the number of items in the test was fixed at i = 40 and 100 replications were made under each condition. Among these models, it was concluded that the TRT model gave less biased results than the other two models, but the results of the bi-factor model and the TRT were more similar as the sample size increased. Among the examined conditions, it was determined that the most effective variable in parameter recovery was the sample size.

Psychometric approaches to analyzing C-tests

The C-test is a type of gap-filling test designed to efficiently measure second language proficiency. The typical C-test consists of several short paragraphs with the second half of every second word deleted. The words with deleted parts are considered as items nested within the corresponding paragraph. Given this testlet structure, it is commonly taken for granted that the C-test design may violate the local independence assumption. However, this assumption has not been fully investigated in the C-test research to date, including the evaluation of alternative psychometric models (i.e., unidimensional and multidimensional) to calibrate and score the C-test. This study addressed each of these issues using a large data set of responses to an English-language C-test. First, we examined the local item independence assumption via multidimensional item response theory (IRT) models, Yen’s Q3, and Jackknife Slope Index. Second, we evaluated several IRT models to determine optimal approaches to scoring the C-test. The results support an interpretation of unidimensionality for the C-test items within a paragraph, with only minor evidence of local item dependence. Furthermore, the two-parameter logistic (2PL) IRT model was found to be the most appropriate model for calibrating and scoring the C-test. Implications for designing, scoring, and analyzing C-tests are discussed.

Validity Testing and Cultural Adaptation of the eHealth Literacy Questionnaire (eHLQ) Among People With Chronic Diseases in Taiwan: Mixed Methods Study

BackgroundAdvancements in digital technologies seek to promote health and access to services. However, people lacking abilities and confidence to use technology are likely to be left behind, leading to health disparities. In providing digital health services, health care providers need to be aware of users’ diverse electronic health (eHealth) literacy to address their particular needs and ensure equitable uptake and use of digital services. To understand such needs, an instrument that captures users’ knowledge, skills, trust, motivation, and experiences in relation to technology is required. The eHealth Literacy Questionnaire (eHLQ) is a multidimensional tool with 7 scales covering diverse dimensions of eHealth literacy. The tool was simultaneously developed in English and Danish using a grounded and validity-driven approach and has been shown to have strong psychometric properties.ObjectiveThis study aims to translate and culturally adapt the eHLQ for application among Mandarin-speaking people with chronic diseases in Taiwan and then undertake a rigorous set of validity-testing procedures.MethodsThe cross-cultural adaptation of the eHLQ included translation and evaluation of the translations. The measurement properties were assessed using classical test theory and item response theory (IRT) approaches. Content validity, known-group validity, and internal consistency were explored, as well as item characteristic curves (ICCs), item discrimination, and item location/difficulty.ResultsThe adapted version was reviewed, and a recommended forward translation was confirmed through consensus. The tool exhibited good content validity. A total of 420 people with 1 or more chronic diseases participated in a validity-testing survey. The eHLQ showed good internal consistency (Cronbach α=.75-.95). For known-group validity, all 7 eHLQ scales showed strong associations with education. Unidimensionality and local independence assumptions were met except for scale 2. IRT analysis showed that all items demonstrated good discrimination (range 0.27-12.15) and a good range of difficulty (range 0.59-1.67) except for 2 items in scale 7.ConclusionsUsing a rigorous process, the eHLQ was translated from English into a culturally appropriate tool for use in the Mandarin language. Validity testing provided evidence of satisfactory-to-strong psychometric properties of the eHLQ. The 7 scales are likely to be useful research tools for evaluating digital health interventions and for informing the development of health technology products and interventions that equitably suit diverse users’ needs.

An Extension of Testlet-Based Equating to the Polytomous Testlet Response Theory Model.

Educational assessments tests are often constructed using testlets because of the flexibility to test various aspects of the cognitive activities and broad content sampling. However, the violation of the local item independence assumption is inevitable when tests are built using testlet items. In this study, simulations are conducted to evaluate the performance of item response theory models and testlet response theory models for both the dichotomous and polytomous items in the context of equating tests composed of testlets. We also examine the impact of testlet effect, length of testlet items, and sample size on estimating item and person parameters. The results show that more accurate performance of testlet response theory models over item response theory models was consistently observed across the studies, which supports the benefits of using the testlet response theory models in equating for tests composed of testlets. Further, results of the study indicate that when sample size is large, item response theory models performed similarly to testlet response theory models across all studies.