Rasch Scoring Research Articles

Rasch Analysis, Dimensionality, and Scoring of Aggressive Behavior Inventory for Junior High-School Students

Aggressive behavior often appears in junior high-school students who are entering adolescent development and requires serious attention from many parties. To measure these behavioral tendencies, many assessment instruments have been developed but none of them use Item Response Theory. This study aims to develop and test the validity of the instrument of aggressive behavior of junior high-school students, see the level of suitability of the item and see the level of difficulty of the item and be able to see variable maps of the ability of the person to answer and the ability of items to reveal aggressive behavior. This study used a sample of 360 students with 47 items. The analysis technique used is Rasch analysis to test the reliability, person validity, item validity, and rating validity. The analysis showed that overall the inventory developed is valid and reliable (person reliability is 0.89 and item reliability is 0.98.) The rating scale results using the Andrich Threshold Value indicate that the five choices given are valid for respondents. It can be concluded that the inventory behavior aggressive is valid internally to assess aggressive behavior of junior high-school students and can be used by teachers, school counselors in identifying aggressive behavior.

Rasch Analysis, Dimensionality, and Scoring of the Neuropsychiatric Inventory Irritability and Aggression Subscales in Individuals With Traumatic Brain Injury

ObjectivesTo develop, for versions completed by individuals with traumatic brain injury (TBI) and an observer, a more precise metric for the Neuropsychiatric Inventory (NPI) Irritability and Aggression subscales using all behavioral item ratings for use with individuals with TBI and to address the dimensionality of the represented behavioral domains. DesignRasch and confirmatory factor analyses of retrospective baseline NPI data from 3 treatment studies. SettingPostacute rehabilitation clinic. ParticipantsNPI records (N = 525) consisting of observer ratings (n = 287) and self-ratings (n = 238) by participants with complicated mild, moderate, or severe TBI at least 6 months postinjury. InterventionsNot applicable. Main Outcome MeasuresFrequency and severity ratings from NPI Irritability/Lability and Agitation/Aggression subscales. ResultsConfirmatory factor analyses of both observer and participant ratings showed good fit for either a 1-factor or a 2-factor solution. Consistent with this, the Rasch model also fit the data well with aggression items indicating the more severe end of the construct and irritability items populating the milder end. ConclusionsIrritability and aggression appear to represent different levels of severity of a single construct. The derived Rasch metric offers a measure of this construct based on responses to all specific items that is appropriate for parametric statistical analysis and may be useful in research and clinical assessments of individuals with TBI.

Measuring Longitudinal Gains in Student Learning: A Comparison of Rasch Scoring and Summative Scoring Approaches

This study pioneers a Rasch scoring approach and compares it to a conventional summative approach for measuring longitudinal gains in student learning. In this methodological note, our proposed methodology is demonstrated using an example of rating scales in a student survey as part of a higher education outcome assessment. Such assessments have become increasingly important worldwide for purposes of institutional accreditation and accountability to stakeholders. Data were collected from a longitudinal study by tracking self-reported learning outcomes of individual students in the same cohort who completed the student learning experience questionnaire (SLEQ) in their first and final years. Rasch model was employed for item calibration and latent trait estimation, together with a scaling procedure of concurrent calibration incorporating a randomly equivalent group design and a single group design to measure the gains in self-reported learning outcomes as yielded by repeated measures. The extent to which Rasch scoring compared to the conventional summative scoring method in its sensitivity to change was quantified by a statistical index namely relative performance (RP). Findings indicated greater ability to capture learning outcomes gains from Rasch scoring over the conventional summative scoring method, with RP values ranging from 3 to 17% in the cognitive, social, and value domains of the SLEQ. The Rasch scoring approach and the scaling procedure presented in the study can be readily generalised to studies using rating scales to measure change in student learning in the higher education context. The methodological innovations and contributions of this study are discussed.

Reliability and responsiveness of measures of pain in people with osteoarthritis of the knee: a psychometric evaluation

Purpose: To examine the fit between data from the Short Form McGill Pain Questionnaire (SF-MPQ-2) and the Rasch model, and to explore the reliability and internal responsiveness of measures of pain in people with knee osteoarthritis. Methods: Participants with knee osteoarthritis completed the SF-MPQ-2, Intermittent and Constant Osteoarthritis Pain questionnaire (ICOAP) and painDETECT. Participants were sent the same questionnaires 3 and 6 months later. Results: Fit to the Rasch model was not achieved for the SF-MPQ-2 Total scale. The Continuous subscale yielded adequate fit statistics after splitting item 10 on uniform DIF for gender, and removing item 9. The Intermittent subscale fit the Rasch model after rescoring items. The Neuropathic subscale had relatively good fit to the model. Test–retest reliability was satisfactory for most scales using both original and Rasch scoring ranging from fair to substantial. Effect sizes ranged from 0.13 to 1.79 indicating good internal responsiveness for most scales. Conclusions: These findings support the use of ICOAP subscales as reliable and responsive measure of pain in people with knee osteoarthritis. The MPQ-SF-2 subscales found to be acceptable alternatives.Implications for RehabilitationThe McGill Pain Questionnaire short version 2 is not a unidimensional scale in people with knee osteoarthritis, whereas three of the subscales are unidimensional.The McGill Pain Questionnaire short version 2 Affective subscale does not have good measurement properties for people with knee osteoarthritis.The McGill Pain Questionnaire short version 2 and the Intermittent and Constant Osteoarthritis Pain scales can be used to assess change over time.The painDETECT performs better as a screening measure than as an outcome measure.

A Comparison of Standard Scoring versus Rasch Scoring of the Visual Function Index-14 in Patients with Cataracts

To compare the discriminatory ability and sensitivity to change of the standard summative score of the Visual Function Index (VF)-14 with two alternative Rasch-based scoring systems. A total of 4335 prospective patients with cataracts completed the VF-14 before surgery and 3 months after surgery. Rasch analysis was applied to the VF-14 patient responses before surgery and the VF-14 joint patient responses before and after surgery. To study the discriminatory ability, the VF-14 patient responses were grouped according to the preoperative visual acuity (VA) and the presence of ocular morbidities besides cataracts. For analysis of the sensitivity to change, the overall mean change in VF-14 scores was calculated after surgery, and the patients were grouped according to the presence of other ocular morbidities, postoperative VA gain, and satisfaction with the surgical outcome. The relative precision (RP) index and the effect size were used to compare the different scoring systems. Rasch analysis confirmed the unidimensional structure of the VF-14. All items and scales adjusted well to the model (fit indices range, 0.71-1.34). The RP index for discrimination by ocular morbidity was 0.82 and by preoperative VA level, 1.02. Regarding sensitivity to change, the RP was 2.68 based on ocular morbidity and 1.78 with samples grouped by postoperative VA gain. For longitudinal studies in which change is the relevant outcome, Rasch scores should be used, rather than the traditional score. However, for cross-sectional studies, both scoring systems were similarly precise.

Determinants of Quality of Life for Patients With Kidney Stones

We evaluated various factors relating to quality of life in a population of patients with stones. A total of 155 patients seen at our urology clinic for stones between January and May 2007 were prospectively questioned regarding stone history and were administered the SF-36 questionnaire to assess quality of life. Age, body mass index, American Society of Anesthesiologists score, number of stone episodes, interval from the last stone episode, number of surgical procedures for stones and associated complications, missed days of work and long-term medical treatment were elicited from the patients. Individual SF-36 domains and composite scores were compared to those of the American general population. Univariate and multivariate regression analyses were performed to assess the impact of all covariates on quality of life scores. Patients with stones scored lower than the average American population in 5 of the 8 domains of the SF-36 as well as in the physical composite score. Multivariate regression modeling showed that increasing body mass index and age were the strongest predictors of decreased physical well-being. The number of stone episodes did not influence SF-36 scores, although the number of surgical procedures impacted physical and mental components. The number of ureteroscopy procedures and stent placements primarily impacted mental well-being. Medical therapy, particularly the use of potassium citrate, was associated with more favorable quality of life. Various factors impact quality of life in patients with urolithiasis but the most important are body mass index, age and the number of surgical procedures. Prospective longitudinal studies may further elucidate the determinants of quality of life and they might be used to optimize patient care.

Measurement properties and hierarchical item structure of the Epworth Sleepiness Scale in Parkinson's disease

The aim of this work was to evaluate the measurement properties and hierarchical item structure of the Epworth Sleepiness Scale (ESS) in patients with Parkinson's disease (PD). Data were taken from a cross-sectional study regarding fatigue and sleep-related aspects of PD. One hundred and eighteen consecutive patients with neurologist-diagnosed PD without significant co-morbidities (54% men; mean age, 64 years; mean PD duration, 8.4 years) from four Swedish neurological outpatient clinics participated. The ESS displayed good data quality with few missing items (0-2.5%): good reliability (Cronbach's alpha, 0.84), marginal floor and no ceiling effects (1.7% and 0% respectively), and differentiated between those reporting problems staying awake during the past month and those who did not. Item-total correlations, factor and Rasch analyses indicated that items tap a single underlying construct. Rasch analysis supported basic rating scale assumptions and demonstrated an item hierarchy similar to that previously found in patients with other sleep disorders. Gaps in the levels of sleep propensity covered by ESS items and their response options were identified at the higher and lower ends of the underlying sleepiness continuum. This study provides an evidence base for using the ESS in PD by demonstrating good psychometric properties and a stable hierarchical item structure. However, addition of new items and use of Rasch scoring has potential to further enhance the clinical usefulness of the ESS.

27 SELF-REPORTED FUNCTION SEVERITY SCORING FOR KNEE OSTEOARTHRITIS

Purpose: To determine a short measure of physical function that might be used as a component for determining criteria for total joint replacement, using existing KOOS and WOMAC data from people with knee osteoarthritis (OA). Methods: Rasch analysis was conducted on existing data from populations and medically and surgically (meniscectomy, osteotomy, pre total knee replacement) treated individuals with OA. Fit of the data to the Rasch model was evaluated by model chisquare and item fit statistics (chi-square, size of residual, F-test). Differential item functioning was evaluated by gender, age and country. Unidimensionality was evaluated by factor analysis and residual analysis. Individual data sets were analyzed and then data pooled and re-analyzed for fit to the Rasch model. Once the final set of items was determined by the above procedures, correlations between the physical function scores based on the Rasch scoring and a simple sum score were calculated to determine if the summed score represented the data. A priori, a correlation in excess of 0.80 was set as a criterion. Results: Nine data sets (n=1662) were included with ages ranging from 26 to 101 years (4 sets average age in 50s). The male/female ratio was 964:698. Of the 17 physical and 5 sports items, 10 items fit the Rasch model, hence have interval level properties, are free from differential item functioning and represent a unidimensional scale. Table 1 presents the ten items from easiest (taking off socks) to most difficult (kneeling). The Rasch scores and the sum of the 10 items were correlated (R=0.83).

Rasch scoring of outcomes of total hip replacement

We examined whether there are advantages in terms of outcome assessment of using Rasch methods of scoring the 12-item Oxford Hip Score questionnaire over conventionally summed scores. Data were collected on patients receiving total hip replacement surgery. Three patient groups were created according to surgery type: primary, revision, and re-revision; two groups were created according to satisfaction with surgery: very satisfied and dissatisfied. Analyses were performed to test the relative precision (RP) of Rasch scoring versus conventionally summed scores in discriminating the groups experiencing different types of surgery and level of satisfaction. At the 1-year follow-up, RP ratios favored the Rasch scoring method in both tests of discrimination. Considerable gains in precision were achieved with Rasch scoring methods when groups were compared in a cross-sectional way. Alternative approaches to scoring questionnaires should be investigated to better assess comparisons over time.

MH4: APPLICATION OF THE RASCH MODEL TO THE SF36 MENTAL HEALTH 5 ITEM SCALE (MH5)

OBJECTIVES: To verify whether the SF36 Mental Health 5 item scale (MH5), an instrument commonly used in mental health research, fulfills criteria of unidimensionality and invariant measurement specified by item response theory (IRT). METHODS: As part of a survey of health care needs among university students in Geneva, Switzerland, 1257 respondents (64% of eligible persons) filled the MH5 (version 2). Each item was scored on a 5-level frequency scale (“never” to “all of the time”). We analyzed these data using both the traditional method (summative scoring), and the polytomous one-parameter IRT (or Rasch) model. RESULTS: In traditional analysis, the MH5 scale performed as expected (single factor, Cronbach alpha 0.85, mean 67.1, standard deviation 17.6, range 0–100). Rasch analysis revealed the good fit of all item characteristic curves. Threshold locations for feeling “nervous” (mental health logits: −3.2, −0.7, 2.0, 3.3), “down in the dumps” (−4.0, −1.7, 0.4, 1.5), “calm and peaceful” (−3.0, −0.6, 0.9, 6.0), “downhearted and blue” (−4.2, −1.6, 1.0, 3.0), and “happy” (−3.1, −0.9, 0.7, 4.2) were all ordinal, consistently with theory, and spread widely over a span of 10.2 logits. While classic MH5 scores and Rasch-based scores were closely correlated (r = 0.98), their relationship was S-shaped: the intervals, in mental health logits, between MH5 scores 0–10, 10–20, etc, up to 90–100 were: 2.23, 1.22, 1.00, 0.90, 0.85, 0.83, 0.90, 1.09, 1.54, and 3.21. CONCLUSIONS: The five MH5 items fit the Rasch model very well. Classic summative scoring tends to compress the measurement scale at its extremities, by assigning scores that are too low at the high end, and too high at the low end. Rasch scoring may render MH5 more sensitive to changes among well populations and among the very sick.

A brief outpatient functional assessment measure: validity using Rasch measures.

The Medical Rehabilitation Follow Along (MRFA(TM)) is a brief outpatient functional assessment measure that was developed using Rasch analysis. The MRFA currently has musculoskeletal, neurologic, multiple sclerosis, cardiac, and pulmonary forms. Using Rasch scoring and selected scales, the 31-item musculoskeletal form of the MRFA was compared with and contrasted to a measure of general health status, the Medical Outcomes Trust SF-36. Content, construct, and criterion validity were addressed using scale scores before and after outpatient rehabilitation, as well as therapist ratings of improvement. The results supported the validity of inferences made from the MRFA scales using Rasch measures for persons with musculoskeletal problems. Rasch and raw scoring provided similar results with respect to the validity of the MRFA scales. Implications for the use of Rasch and raw scoring approaches with the MRFA are discussed.