Item Rescoring Research Articles

Rasch analysis of the Unidimensional Self-Efficacy Scale in Neuromuscular Disorders and comparison between sex, age, and diagnoses.

Self-efficacy reflects a person's perceptions of their capabilities for specific tasks and influences motivation and performance. The Unidimensional Self-Efficacy in Neuromuscular Disorders (USE-NM) was modified from the Multiple Sclerosis (MS) USE-MS scale and administered to patients attending a specialist neuromuscular clinic. The aim was to investigate this measure in neuromuscular disorders and to compare between patient sex, age, and diagnosis. The USE-NM was posted to patients recruited from a specialist neuromuscular clinic at the Walton Centre. Responses were subjected to Rasch analysis using RUMM2030 software and descriptive statistics were performed using SPSS version 28. One hundred and ninety-eight patients (56.1% male) grouped by age (<50; 50-59; 60-69; and >69 years) and with varied NM disorders returned the USE-NM. It did not meet the Rasch model expectations due to disordered thresholds of items 6 and 8 ("Sometimes I feel inadequate as a person because of my neuromuscular disorder" and "I feel that my social life would be better if I did not have a neuromuscular disorder"). Following item re-scoring, the modified USE-NM satisfied the Rasch model with a unidimensional scale free from differential item functioning and an overall chi-square probability of 0.146 with good reliability and validity. Post hoc nonparametric testing showed no significant difference in fatigue between sex, age, and neuromuscular diagnoses. The Rasch-modified USE-NM offers a measure of self-efficacy for neuromuscular disorders encountered in a typical specialist clinic. Future considerations could be given to assessing any benefits of multidisciplinary team input, across a specialist neuromuscular service.

Rasch analysis of the UK Functional Assessment Measure in a sample of patients with traumatic brain injury from the UK national clinical database.

To determine whether the UK Functional Assessment Measure (UK FIM+FAM) fits the Rasch model in patients with complex disability following traumatic brain injury. Psychometric evaluation including preliminary exploratory and confirmatory factor analyses followed by Rasch analysis. A multicentre UK national cohort of 1,956 patients admitted for specialist rehabilitation following traumatic brain injury. The suitability of the Partial Credit Model was confirmed by the likelihood-ratio test (χ2 (df86) =7,325.0, p < 0.001). Exploratory and confirmatory factor analyses supported 3 factors (Motor, Communication, Psychosocial). Rasch analysis of the full scale incorporating the 3 factors as super-items resulted in an acceptable overall model fit (χ2 (df24)=36.72, p = 0.05) and strict uni-dimensionality when tested on a sub-sample of n = 320. These results were replicated in a full sample (n = 1,956) showing uni-dimensionality and good reliability with Person Separation Index = 0.81, but item trait interaction was significant due to the large sample size. No significant differential item functioning was observed for any personal factors. Neither uniform re-scoring of items nor exclusion of participants with extreme scores improved the model fit. The UK FIM+FAM scale satisfies the Rasch model reasonably in traumatic brain injury. A conversion table was produced, but its usefulness in clinical practice requires further exploration and clinical translation.

A clinical assessment tool to improve the use of pain relieving treatments in knee osteoarthritis

A clinical assessment tool to improve the use of pain relieving treatments in knee osteoarthritis

Rasch Analysis of the Edmonton Symptom Assessment System

ContextThe Edmonton Symptom Assessment System (ESAS) is a widely used multisymptom assessment tool in cancer and palliative care settings, but its psychometric properties have not been widely tested using modern psychometric methods such as Rasch analysis. ObjectivesTo apply Rasch analysis to the ESAS in a community palliative care setting and determine its suitability for assessing symptom burden in this group. MethodsESAS data collected from 229 patients enrolled in a community hospice service were evaluated using a partial credit Rasch model with RUMM2030 software (RUMM Laboratory Pty, Ltd., Duncraig, WA). Where disordered thresholds were discovered, item rescoring was undertaken. Rasch model fit and differential item functioning were evaluated after each iterative phase. ResultsUniform rescoring was necessary for all 12 items to display ordered thresholds. The best model fit was achieved after item rescoring and combining three pairs of locally dependent items into three superitems (χ2 = 29.56 [27]; P = 0.33) that permitted ordinal-to-interval conversion. ConclusionThe ESAS satisfied unidimensional Rasch model expectations in a 12-item format after minor modifications. This included uniform rescoring of the disordered response categories and creating superitems to improve model fit and clinical utility. The accuracy of the ESAS scores can be improved by using ordinal-to-interval conversion tables published in the article.

Rasch analysis of the UK Functional Assessment Measure in patients with complex disability after stroke.

To determine whether the UK Functional Assessment Measure (UK FIM+FAM) fits the Rasch model in stroke patients with complex disability and, if so, to derive a conversion table of Rasch-transformed interval level scores. The sample included a UK multicentre cohort of 1,318 patients admitted for specialist rehabilitation following a stroke. Rasch analysis was conducted for the 30-item scale including 3 domains of items measuring physical, communication and psychosocial functions. The fit of items to the Rasch model was examined using 3 different analytical approaches referred to as "pathways". The best fit was achieved in the pathway where responses from motor, communication and psychosocial domains were summarized into 3 super-items and where some items were split because of differential item functioning (DIF) relative to left and right hemisphere location (χ2 (10) = 14.48, p = 0.15). Re-scoring of items showing disordered thresholds did not significantly improve the overall model fit. The UK FIM+FAM with domain super-items satisfies expectations of the unidimensional Rasch model without the need for re-scoring. A conversion table was produced to convert the total scale scores into interval-level data based on person estimates of the Rasch model. The clinical benefits of interval-transformed scores require further evaluation.

A Rasch analysis of the Revised Study Process Questionnaire in an Australian osteopathy student cohort

There are a number of questionnaires designed to capture a students’ self-reported learning approach. One example is the Study Process Questionnaire and its modified version, the 2-factor revised Study Process Questionnaire (R-SPQ-2F). Students in the first year of the 2014 and 2015 cohorts in the osteopathy program at (institution) completed the R-SPQ-2F at the commencement of their course. Rasch analysis was employed to evaluate the R-SPQ-2F internal structure. Reliability estimations were calculated for both subscales using Cronbach’s alpha and McDonald’s omega. Both the Deep and Surface subscales failed to fit the Rasch model at initial analysis. Modifications included item rescoring, removal of misfitting responses, and removal of one item from both subscales. After modification, both subscales fit the Rasch model and demonstrated unidimensionality. Reliability estimations were acceptable and consistent with previous literature. The study has provided evidence for the internal structure of the R-SPQ-2F in this cohort.

Assessing Mechanisms of Mindfulness: Improving the Precision of the Nonattachment Scale Using a Rasch Model

Nonattachment, or the lack of possessive and mental fixations and clinging, is considered a key element by which mindfulness cultivates psychological wellbeing. The 30-item Nonattachment Scale (NAS) is a measure of nonattachment, but its item functioning and measurement precision remain to be explored. The present study used a Rasch model to examine 434 participants’ responses to the NAS. Disordered thresholds were corrected by uniform item re-scoring. Satisfactory model fit was achieved after removing four misfitting items and combining locally dependent items into sub-tests. NAS item functioning improved significantly following these minor modifications. In addition, by combining particular response options upon scoring, researchers can utilize our modified version without the need to alter current response formatting, thus offering them greater precision in their measurement of nonattachment. Ordinal-to-interval conversion tables presented in the manuscript further increases precision of our proposed 26-item version of the instrument and enables use of the scale without the need to violate fundamental assumptions of parametric statistics.

Psychometric evaluation of the Revised Illness Perception Questionnaire (IPQ-R) in cancer patients: Confirmatory factor analysis and Rasch analysis

ObjectiveTo provide new insights into the psychometrics of the revised Illness Perception Questionnaire (IPQ-R) in cancer patients. To undertake, for the first time using data from breast, colorectal and prostate cancer patients, a confirmatory factor analysis (CFA) to assess the validity of the IPQ-R's core seven-factor structure. Also, for the first time in any illness group, to undertake Rasch analysis to explore the extent to which the IPQ-R factors form unidimensional scales, with linear measurement properties and no Differential Item Functioning (DIF). MethodsPatients with potentially curable breast, colorectal or prostate cancer, within 6months post-diagnosis, completed the IPQ-R online (N=531). CFA was conducted, including multi-sample analysis, and for each IPQ-R factor fit to the Rasch model was assessed by examining, amongst other things, item fit, DIF and unidimensionality. ResultsThe CFA showed a moderate fit of the data to the IPQ-R model, and stability across diagnosis, although fit was significantly improved following the removal of selected items. All seven factors achieved fit to the Rasch model, and exhibited unidimensionality and minimal DIF, although in most cases this was after some item rescoring and/or deletion. In both analyses, IPQ-R items 12, 18 and 24 were indicated as misfitting and removed. ConclusionGiven the rigorous standard of Rasch measurement, and the generic nature of the IPQ-R, it stood up well to the demands of the Rasch model in this study. Importantly, the results show that with some relatively minor, pragmatic modifications the IPQ-R could possess Rasch-standard measurement in cancer patients.

Comparisons among Designs for Equating Mixed‐Format Tests in Large‐Scale Assessments

In this study we examined variations of the nonequivalent groups equating design for tests containing both multiple‐choice (MC) and constructed‐response (CR) items to determine which design was most effective in producing equivalent scores across the two tests to be equated. Using data from a large‐scale exam, this study investigated the use of anchor CR item rescoring (known as trend scoring) in the context of classical equating methods. Four linking designs were examined: an anchor with only MC items, a mixed‐format anchor test containing both MC and CR items; a mixed‐format anchor test incorporating common CR item rescoring; and an equivalent groups (EG) design with CR item rescoring, thereby avoiding the need for an anchor test. Designs using either MC items alone or a mixed anchor without CR item rescoring resulted in much larger bias than the other two designs. The EG design with trend scoring resulted in the smallest bias, leading to the smallest root mean squared error value.

Rasch analysis of the Beck Depression Inventory-II in a neurological rehabilitation sample

Purpose. To apply Rasch analysis to the Beck Depression Inventory-II (BDI-II) responses of a mixed neurorehabilitation sample to determine its suitability for assessing depression among this group.Method. Three hundred and fifteen in-patients undergoing post-acute specialist in-patient rehabilitation at the Regional Rehabilitation Unit (RRU) of Northwick Park Hospital in North London were administered the Beck Depression Inventory. All patients were administered the BDI-II by a clinical psychologist using a large print version and taking as much time as patients required. Rasch analysis was completed using the RUMM2020 software. Where disordered thresholds were identified item rescoring was undertaken. Each item was also examined for Differential Item Functioning. Where misfit to model expectations was identified items were removed in an iterative fashion. The effect of any deletion upon person estimates was examined. Specific tests of unidimensionality were undertaken at each iterative phase.Results. Rescoring was necessary before Items 1–4, 6, 8–12, 20 and 21 showed ordered thresholds. Three items failed to satisfy model expectations – items 16 (changes in sleeping pattern), 18 (changes in appetite) and 21 (loss of interest in sex) and were deleted – after which good overall fit to the Rasch model was observed. The three items deleted affected person estimates in a significant way such that the 21 item version in this group of patients may be biased because of multidimensionality.Conclusions. The BDI-II in a sample of patients undergoing neurorehabilitation satisfies unidimensional Rasch model expectations in an 18-item format. Some disordering of response categories was evident, but this evidence requires replication before any common rescoring option should be considered.

COMPARISONS AMONG DESIGNS FOR EQUATING CONSTRUCTED‐RESPONSE TESTS

ABSTRACTThis study examined variations of a nonequivalent groups equating design used with constructed‐response (CR) tests to determine which design was most effective in producing equivalent scores across the two tests to be equated. Using data from a large‐scale exam, the study investigated the use of anchor CR item rescoring in the context of classical equating methods. Four linking designs were examined: (a) an anchor set containing common CR items, (b) an anchor set incorporating common CR items rescored, (c) an external multiple‐choice (MC) anchor test, and (d) an equivalent groups design incorporating CR items rescored (no anchor test). The use of CR items without rescoring or the use of an external MC anchor resulted in much larger bias than the other two designs. The use of a rescored CR anchor and the equivalent groups design led to similar levels of equating error.

EQUATING OF MIXED‐FORMAT TESTS IN LARGE‐SCALE ASSESSMENTS

ABSTRACTThis study examined variations of the nonequivalent‐groups equating design for mixed‐format tests—tests containing both multiple‐choice (MC) and constructed‐response (CR) items—to determine which design was most effective in producing equivalent scores across the two tests to be equated. Four linking designs were examined: (a) an anchor with only MC items; (b) a mixed‐format anchor containing both MC and CR items; (c) a mixed‐format anchor incorporating CR item rescoring; and (d) a hybrid combining single‐group and equivalent‐groups designs, thereby avoiding the need for an anchor test. Designs using MC items alone or those using a mixed anchor without CR item rescoring resulted in much larger bias than the other two design approaches. The hybrid design yielded the smallest root mean squared error value.