Local Independence Assumption Research Articles

The Usefulness of the Rasch Model for the Refinement of Likert Scale Questionnaires

In this paper the use of the Rasch model is explored as a transparent, systematic and theoretically underpinned response to quality issues that are widely recognised as problematic in the refinement of Likert scale questionnaires. Key issues are the choice of length of scale, the pursuit of a favourable estimate of Cronbach's alpha at the possible expense of construct validity, and the fact that total raw scores arise from ordinal data but are used and interpreted as if measurement had occurred. We use a questionnaire under development for the measurement of perceptions of first-year chemistry students on demonstrator effectiveness to illustrate the process of Rasch analysis and instrument refinement. This process involves investigation of fit of the data to the model, possible violations of the assumption of local independence, and several aspects of item functioning. We identified disordered response categories as the probable reason for misfit in this data set and propose strategies for modification of items so that they can be retained rather than rejected.

The Performance of Local Dependence Measures With Psychological Data

The violation of the assumption of local independence when applying item response theory (IRT) models has been shown to have a negative impact on all estimates obtained from the given model. Numerous indices and statistics have been proposed to aid analysts in the detection of local dependence (LD). A Monte Carlo study was conducted to evaluate the relative performance of selected LD measures in conditions considered typical of studies collecting psychological assessment data. Both the Jackknife Slope Index and likelihood ratio statistic G2 are available across the two IRT models used and displayed adequate to good performance in most simulation conditions. The use of these indices together is the final recommendation for applied analysts. Future research areas are discussed.

The Application of Graded Response Model to the Test Data Violated the Assumption of Local Independence

The Application of Graded Response Model to the Test Data Violated the Assumption of Local Independence

Identifying Local Dependence With a Score Test Statistic Based on the Bifactor Logistic Model

Local dependence (LD) refers to the violation of the local independence assumption of most item response models. Statistics that indicate LD between a pair of items on a test or questionnaire that is being fitted with an item response model can play a useful diagnostic role in applications of item response theory. In this article, a new score test statistic, Sb, based on the bifactor logistic model is described. To compare the performance of Sb with the score test statistic ( St) based on the threshold shift model, and the LD X2 statistic, data were simulated under locally independent, bifactor, and threshold shift conditions. The results summarize the null distributions of all three diagnostic statistics, and their power for various degrees of bifactor and threshold shift LD. Future research directions are discussed, including the straightforward generalization of Sb for polytomous item response models, and the challenges involved in the corresponding generalizations of St and LD X2.

Assessing Fit of Item Response Models Using the Information Matrix Test

The information matrix can equivalently be determined via the expectation of the Hessian matrix or the expectation of the outer product of the score vector. The identity of these two matrices, however, is only valid in case of a correctly specified model. Therefore, differences between the two versions of the observed information matrix indicate model misfit. The equality of both matrices can be tested with the so‐called information matrix test as a general test of misspecification. This test can be adapted to item response models in order to evaluate the fit of single items and the fit of the whole scale. The performance of different versions of the test is compared in a simulation study with existing tests of model fit, among them the test of Orlando and Thissen, the score test of local independence due to Glas and Suarez‐Falcon, and the limited information approach of Maydeu‐Olivares and Joe. In general, the different versions of the information matrix test adhere to the nominal Type I error rate and have high power for detecting misspecified item characteristic curves. Additionally, some versions of the test can be used in order to detect violations of the local independence assumption.

When Should We Use Testlet Model? A Comparison Study of Bayesian Testlet Random-Effects Model and Standard 2-PL Bayesian Model

A testlet is comprised of a group of multiple choice items based on a common stimuli. When a testlet is used, the traditional item response models may not be appropriate due to the violation of the assumption of local independence (LI). A variety of new models have been proposed to analyze response data sets for testlets. Among them, the Bayesian random effects model proposed by Bradlow, Wainer and Wang (1999) is one of the most promising. However, in many situations it is not clear to practitioners whether the traditional IRT methods should still be used instead of a newly proposed testlet model. The objective of the current study is to investigate the effects of model selection in various situations. In simulation 1, simulated response data sets were generated under three simulation factors, which were: testlet variance (0, 0.5, 1, 2); testlet size (2, 5, 10); and test length (20, 40, 60). For each simulation condition, the test structure was determined by fixing the number of examinees as I =2000, and the percentage of testlet items in a test as 50%. Under each condition, 30 replications were generated. Both two-parameter Bayesian testlet random effect model and standard two-parameter Bayesian model were fitted to every dataset using MCMC method. The computer program SCORIGHT was used to conduct all the analysis across different conditions. Two models were compared corresponding to seven criteria: bias, mean absolute error, root mean square error, correlation between estimated and true values, 95% posterior interval width, 95% coverage probability. These indexes were computed for all parameters separately. Simulation 2 compared the two models under two factors: the proportion of independent items (1/3, 1/2, 2/3); test length (20, 30, 40, 60). The data generation, analyze process and criteria mimicked those of simulation 1. The results showed that: (1) The accuracy of the estimation of all parameters under 2-PL Bayesian testlet random-effect model remained stable with varying levels of testlet effect and testlet size. However, the estimate errors of all the parameters under 2-PL Bayesian model increased dramatically as the testlet effect and testlet size became larger. Besides, using Bayesian testlet random-effect model, the error for every parameter was always less than that for 2-PL Bayesian model. It was especially necessary to choose 2-PL Bayesian testlet random-effects model when testlet variance and testlet size were large. (2) Even though, the accuracy of estimation of item parameters in Bayesian testlet random-effect model wasn't affected by test length, the accuracy of ability parameter was. Moreover, as the test got shorter, the errors of all parameters under 2-PL Bayesian model increased dramatically. In all, under short test conditions, even if there was large testlet effect, Bayesian testlet random-effect model couldn't work well, meanwhile, if items were all independent, using Bayesian testlet random-effect model would result in much worse ability estimations than 2-PL Bayesian model. (3) When the proportion of independent items was large, and the test length was larger than 20 items, the estimations of two models didn't show significant differences. In conclusion, 2-PL Bayesian testlet random-effect model is more general. Using the more complex testlet model when items are all independent, will lead almost the same accuracy of the parameter estimations as using the 2-PL Bayesian model. It is better to choose 2-PL Bayesian testlet random-effect model when testlet variance, testlet size, and test length are large. However, when test length is short, even the Bayesian testlet random- effects model couldn't provide accurate estimations of parameters when local dependence happened. So it is important to make sure the test was comprised of enough items before applying a testlet model. We also give some suggestions for practitioners. In the test construction period, first it is better for items to be independent, if not, shorter testlets and larger proportion of independent items should be included. While in the test analysis period, local dependence should be detected first. If evidence shows that there is dependence structure, then an appropriate model should be chosen to avoid estimation errors.

A laboratory study on the reliability estimations of the mini-CEX

Reliability estimations of workplace-based assessments with the mini-CEX are typically based on real-life data. Estimations are based on the assumption of local independence: the object of the measurement should not be influenced by the measurement itself and samples should be completely independent. This is difficult to achieve. Furthermore, the variance caused by the case/patient or by assessor is completely confounded. We have no idea how much each of these factors contribute to the noise in the measurement. The aim of this study was to use a controlled setup that overcomes these difficulties and to estimate the reproducibility of the mini-CEX. Three encounters were videotaped from 21 residents. The patients were the same for all residents. Each encounter was assessed by 3 assessors who assessed all encounters for all residents. This delivered a fully crossed (all random) two-facet generalizability design. A quarter of the total variance was associated with universe score variance (28%). The largest source of variance was the general error term (34%) followed by the main effect of assessors (18%). Generalizability coefficients indicated that an approximate sample of 9 encounters was needed assuming a single different assessor per encounter and assuming different cases per encounter (the usual situation in real practice), 4 encounters when 2 raters were used and 3 encounters when 3 raters are used. Unexplained general error and the leniency/stringency of assessors are the major causes for unreliability in mini-CEX. To optimize reliability rater training might have an effect.

Psychometric assessment of HIV/STI sexual risk scale among MSM: A Rasch model approach

BackgroundLittle research has assessed the degree of severity and ordering of different types of sexual behaviors for HIV/STI infection in a measurement scale. The purpose of this study was to apply the Rasch model on psychometric assessment of an HIV/STI sexual risk scale among men who have sex with men (MSM).MethodsA cross-sectional study using respondent driven sampling was conducted among 351 MSM in Shenzhen, China. The Rasch model was used to examine the psychometric properties of an HIV/STI sexual risk scale including nine types of sexual behaviors.ResultsThe Rasch analysis of the nine items met the unidimensionality and local independence assumption. Although the person reliability was low at 0.35, the item reliability was high at 0.99. The fit statistics provided acceptable infit and outfit values. Item difficulty invariance analysis showed that the item estimates of the risk behavior items were invariant (within error).ConclusionsThe findings suggest that the Rasch model can be utilized for measuring the level of sexual risk for HIV/STI infection as a single latent construct and for establishing the relative degree of severity of each type of sexual behavior in HIV/STI transmission and acquisition among MSM. The measurement scale provides a useful measurement tool to inform, design and evaluate behavioral interventions for HIV/STI infection among MSM.

Non-compliance in surgical patients with herniated lumbar discs

In a recent observational study on a sample of patients with herniated lumbar discs who underwent physiotherapy, the recovery rate of those who chose physiotherapy rather than surgery, as recommended, was not appreciably different from that of the other patients, although their prognosis was worse. To investigate whether this finding was due to a confounding effect induced by unobserved features of the patients, we fit a latent class model for the joint distribution of compliance, participation rate and recovery, given the individual attitude (modelled as latent variable) and a set of covariates. The usual assumption of local independence is relaxed in favour of conditional association between the observed variables given the latent one. The results indicate that the latent classes may be interpreted as dividing the patients according to whether they are highly versus moderately motivated towards physiotherapy: conditionally on the covariates, the more motivated subjects attend a larger number of therapy sessions and benefit much more from the therapy. In addition, the fact that the proportion of less motivated patients among non-compliers is very small may be due to a self-selection mechanism, which may explain the surprisingly good performance of physiotherapy for these patients.

Hidden Markov partition models

We describe an extension of the hidden Markov model in which the manifest process conditionally follows a partition model. The assumption of local independence for the manifest random variable is thus relaxed to arbitrary dependence. The proposed class generalizes different existing models for discrete and continuous time series, and allows for the finest trading off between bias and variance. The models are fit through an EM algorithm, with the usual recursions for hidden Markov models extended at no additional computational cost.

De complexiteit van het concept mondgezondheidgerelateerde levenskwaliteit

The methods uses most often for developing and analyzing questionnaires, such as the explorative factor analysis and Cronbach's alpha, presume that psychological constructs are latent (imperceptible) and that there is a reflective-measurement model with the underlying assumption of local independence. Local independence means that the latent variable explains why the variables observed are related. Many questionnaires for measuring oral health-related quality of life are analyzed as if they were based on a reflective-measurement model assuming local independence. This assumption requires these questionnaires to contain solely items reflecting instead of determining oral health-related quality of life. The tenability of this assumption is questionable.

Assessing Fit of Unidimensional Graded Response Models Using Bayesian Methods

The posterior predictive model checking method is a flexible Bayesian model-checking tool and has recently been used to assess fit of dichotomous IRT models. This paper extended previous research to polytomous IRT models. A simulation study was conducted to explore the performance of posterior predictive model checking in evaluating different aspects of fit for unidimensional graded response models. A variety of discrepancy measures (test-level, item-level, and pair-wise measures) that reflected different threats to applications of graded IRT models to performance assessments were considered. Results showed that posterior predictive model checking exhibited adequate power in detecting different aspects of misfit for graded IRT models when appropriate discrepancy measures were used. Pair-wise measures were found more powerful in detecting violations of the unidimensionality and local independence assumptions.

Classification and generation of disturbance vectors for collision attacks against SHA-1

The main contribution of this paper is to provide a classification of disturbance vectors used in differential collision attacks against $${\tt{SHA}-1}$$ . We show that all published disturbance vectors can be classified into two types of vectors, type-I and type-II. We present a deterministic algorithm which produce efficient disturbance vectors with respect to any given cost function. We define two simple cost functions to evaluate the efficiency of a candidate disturbance vector. Using our algorithm and those cost function we retrieved all previously known vectors and found that the most efficient disturbance vector is the one first reported as Codeword2 by Jutla and Patthak, A matching lower bound on the minimum weight of SHA-1 expansion code. Cryptology ePrint Archive, Report 2005/266, (2005). We also present a statistical evaluation of local collisions’ holding probabilities and show that the common assumption of local collision independence is flawed.

Local Dependence Model in Latent Rank Theory

In test data analysis, it is often important to examine the inter-item dependency structure underlying the test data. However, this structure often varies according to the academic ability level of the examinees. In this paper, a latent rank theory (LRT) model incorporating a Bayesian network model is proposed. Although the conventional LRT model is formulated under the assumption of local independence among test items, the proposed LRT model supposes local dependence among test items. The proposed model can be used to efficiently analyze the data of math and science tests in which inter-item dependency relationships among items are not negligible.

A dimensionally reduced finite mixture model for multilevel data

Recently, different mixture models have been proposed for multilevel data, generally requiring the local independence assumption. In this work, this assumption is relaxed by allowing each mixture component at the lower level of the hierarchical structure to be modeled according to a multivariate Gaussian distribution with a non-diagonal covariance matrix. For high-dimensional problems, this solution can lead to highly parameterized models. In this proposal, the trade-off between model parsimony and flexibility is governed by assuming a latent factor generative model.

Model specification in oral health‐related quality of life research

The aim of this study was to analyze conventional wisdom regarding the construction and analysis of oral health-related quality of life (OHRQoL) questionnaires and to outline statistical complications. Most methods used for developing and analyzing questionnaires, such as factor analysis and Cronbach's alpha, presume psychological constructs to be latent, inferring a reflective measurement model with the underlying assumption of local independence. Local independence implies that the latent variable explains why the variables observed are related. Many OHRQoL questionnaires are analyzed as if they were based on a reflective measurement model; local independence is thus assumed. This assumption requires these questionnaires to consist solely of items that reflect, instead of determine, OHRQoL. The tenability of this assumption is the main topic of the present study. It is argued that OHRQoL questionnaires are a mix of both a formative measurement model and a reflective measurement model, thus violating the assumption of local independence. The implications are discussed.

Locally Dependent Latent Class Models with Covariates: An Application to Under-Age Drinking in the USA

Under-age drinking is a long-standing public health problem in the USA and the identification of underage drinkers suffering alcohol-related problems has been difficult by using diagnostic criteria that were developed in adult populations. For this reason, it is important to characterize patterns of drinking in adolescents that are associated with alcohol-related problems. Latent class analysis is a statistical technique for explaining heterogeneity in individual response patterns in terms of a smaller number of classes. However, the latent class analysis assumption of local independence may not be appropriate when examining behavioural profiles and could have implications for statistical inference. In addition, if covariates are included in the model, non-differential measurement is also assumed. We propose a flexible set of models for local dependence and differential measurement that use easily interpretable odds ratio parameterizations while simultaneously fitting a marginal regression model for the latent class prevalences. Estimation is based on solving a set of second-order estimating equations. This approach requires only specification of the first two moments and allows for the choice of simple 'working' covariance structures. The method is illustrated by using data from a large-scale survey of under-age drinking. This new approach indicates the effectiveness of introducing local dependence and differential measurement into latent class models for selecting substantively interpretable models over more complex models that are deemed empirically superior.

Applying item response theory and computer adaptive testing: the challenges for health outcomes assessment

We review the papers presented at the NCI/DIA conference, to identify areas of controversy and uncertainty, and to highlight those aspects of item response theory (IRT) and computer adaptive testing (CAT) that require theoretical or empirical research in order to justify their application to patient reported outcomes (PROs). IRT and CAT offer exciting potential for the development of a new generation of PRO instruments. However, most of the research into these techniques has been in non-healthcare settings, notably in education. Educational tests are very different from PRO instruments, and consequently problematic issues arise when adapting IRT and CAT to healthcare research. Clinical scales differ appreciably from educational tests, and symptoms have characteristics distinctly different from examination questions. This affects the transferring of IRT technology. Particular areas of concern when applying IRT to PROs include inadequate software, difficulties in selecting models and communicating results, insufficient testing of local independence and other assumptions, and a need of guidelines for estimating sample size requirements. Similar concerns apply to differential item functioning (DIF), which is an important application of IRT. Multidimensional IRT is likely to be advantageous only for closely related PRO dimensions. Although IRT and CAT provide appreciable potential benefits, there is a need for circumspection. Not all PRO scales are necessarily appropriate targets for this methodology. Traditional psychometric methods, and especially qualitative methods, continue to have an important role alongside IRT. Research should be funded to address the specific concerns that have been identified.

Testing for Local Dependence in Rasch’s Multiplicative Gamma Model for Speed Tests

The author considers a latent trait model for the response time on a (set of) pure speed test(s), the multiplicative gamma model (MGM), which is based on the assumption that the test response times are approximately gamma distributed, with known index parameters and scale parameters depending on subject ability and test difficulty parameters. Like any other parametric latent trait model, the MGM is based on strong assumptions. One of these assumptions is local independence. Two statistical tests for checking if the local independence assumption holds are compared, using generated and empirical data.

The Presence and Impact of Local Item Dependence on Objective Structured Clinical Examinations Scores and the Potential Use of the Polytomous, Many-Facet Rasch Model

Objective The purpose of this research project was to extend the research on the robustness of the dichotomous Rasch model to violations of the local independence assumption to the polytomous many-facet Rasch model (MFRM). Candidate scores from oral examinations and objective structured clinical examinations (OSCEs) have been shown to contain variance due to rater error/bias. If the MFRM is robust to local item dependence (LID), then the MFRM could theoretically be applied to medical OSCEs. Methods Five OSCEs were used in the study: 3 chiropractic licensure OSCEs and 2 nursing licensure OSCEs. Items were assigned to split-halves based on common stimulus. Split-half correlations were compared with Spearman-Brown estimates of reliability based on Cronbach α with all items contributing. Two- and 3-facet MFRM analyses were performed, first with individual items contributing and second with station totals contributing. Correlations were estimated between the 2 MFRM estimates. Results Cronbach α estimates with all items contributing were all very high (>.87). Spearman-Brown estimates were all considerably higher than split-half correlations. Correlations between MFRM by items and by stations were all very high (>.993). Conclusions The research project provided evidence that OSCEs violate the local item independence assumption. The project also showed that the MFRM is quite robust to such violations. The authors recommend that the MFRM be applied to OSCEs by station totals for estimates of candidate ability, and by items for item performance measures and quality control programs.