Rasch Rating Scale Analysis Research Articles

Where Does Gender Fit in the Measurement of Self-Control?

Gender’s role in self-control measures has been largely neglected. Although studies show that males have lower self-control than females, rarely have researchers questioned whether items used to measure self-control should be used for both groups. This study uses a Rasch rating scale analysis to assess item functioning of Grasmick et al.’s 24-item self-control scale for males and females. Using a sample of young adults, results indicate that 33% of the scale items showed differential functioning or item bias; that is, after controlling for self-control, females found one third of the items to be either more or less agreeable than males. Once biased items were removed from the scale, males, on average, still had lower self-control than females. In addition, after excluding biased items from the scale, the effect of self-control on criminal behavior and other outcomes was similar to the effect found with the full 24-item scale. Suggestions for future research on Grasmick et al.’s self-control scale are offered, and limitations of the current study are discussed.

Measuring perceived procedural justice and coercion among persons with mental illness in police encounters: the Police Contact Experience Scale

AbstractDespite increased recent attention to improving the quality of encounters between police officers and people with serious mental illness, there are no measures available for assessing how consumers perceive their interactions with police officers. Drawing upon conceptual frameworks developed within social psychology, this study reports the development and testing of a new measure, the Police Contact Experience Scale (PCES), which yields indicators of procedural justice and coercion as evaluated by persons with mental illness in relation to specific encounters with the police. The PCES was administered to 154 individuals with mental illness that had police contact within the prior 12 months. Rasch rating scale analysis supported construct validity of a 10‐item procedural justice and a 5‐item coercion scale. Correlational analysis supported convergent validity for both scales. © 2010 Wiley Periodicals, Inc.

Application of Rasch Analysis to Examine Psychometric Aspects of the Activities-Specific Balance Confidence Scale When Used in a New Cultural Context

Arnadottir SA, Lundin-Olsson L, Gunnarsdottir ED, Fisher AG. Application of Rasch analysis to examine psychometric aspects of the Activities-Specific Balance Confidence Scale when used in a new cultural context. Objective To investigate by using Rasch analysis the psychometric properties of the Activities-Specific Balance Confidence (ABC) Scale when applied in a new Icelandic context. Design Cross-sectional, population-based, random selection from the Icelandic National Registry. Setting Community-based. Participants Icelanders (N=183), 65 to 88 years old, and 48% women. Interventions Not applicable. Main Outcome Measure ABC, an instrument used to evaluate how confident older people are in maintaining balance and remaining steady when moving through the environment. An Icelandic translation of the ABC (ABC-ICE) scale was evaluated by implementing Rasch rating scale analysis to transform ordinal ABC-ICE scores into interval measures and evaluating aspects of validity and reliability of the scale. Results Participants were not able to differentiate reliably between the 11 rating scale categories of the ABC-ICE. Additionally, 3 items failed to show acceptable goodness of fit to the ABC-ICE rating scale model. By collapsing categories and creating a new 5-category scale, only 1 item misfit. Removing that item resulted in a modified version of ABC-ICE with 5 categories and 15 items. Both item goodness-of-fit statistics and principal components analysis supported unidimensionality of the modified ABC-ICE. The ABC-ICE measures reliably separated the sample into at least 4 statistically distinct strata of balance confidence. Finally, the hierarchical order of item difficulties was consistent with theoretic expectations, and the items were reasonably well targeted to the balance confidence of the persons tested. Conclusions Rasch analysis indicated a need to modify the ABC-ICE to improve its psychometric properties. Further studies are needed to determine if similar analyses of other versions of the ABC, including the original one, will yield similar results.

Improving measurement properties of the Patient Health Questionnaire–9 with rating scale analysis.

The goal of this study was to explore the psychometric properties of the Patient Health Questionnaire-9 (PHQ-9; R. L. Spitzer, K. Kroenke, & J. B. W. Williams, 1999). Factor analysis and Rasch rating scale analysis were used to examine the psychometric properties of the PHQ-9. The sample consisted of 202 adults with spinal cord injury (SCI). The PHQ-9 items appear to form a usefully unidimensional scale. One "double-barreled" item, "Moving or speaking so slowly that other people could have noticed or being so fidgety or restless that you've been moving around a lot more than usual," misfit the Rasch model. Category probability curves indicate respondent difficulty in distinguishing between the 2 intermediate rating scale categories: several days and more than half the days. Combining these categories eliminated this problem and resulted in all items fitting the measurement model. The measurement properties of the PHQ-9 can be improved by collapsing rating scale categories and by restructuring several double- and triple-barreled items. Adopting these changes may improve sensitivity in measuring depression after SCI.

Initial Development of the Brief Arab Religious Coping Scale (BARCS)

This project aimed to develop a brief and culturally sensitive measure of positive religious coping that could be useful in research with Arab Americans of both Christian and Muslim affiliations. In Study 1,62 items were generated based on previous religious coping measures and knowledge of religious practices in the Arab community. Seventy-six community respondents completed this pilot questionnaire. Based on Rasch rating scale analysis, 15 items were retained for the Brief Arab Religious Coping Scale (BARCS). In Study 2, the BARCS was completed by an Internet sample of 599 Arab Americans. The measure demonstrated excellent reliability and strong internal validity, although it did not correlate with acculturation stress and depression as expected. Limitations and suggestions for future research are discussed.

Differential item functioning of the functional independence measure in higher performing neurological patients

When comparing outcomes of the Functional Independence Measure (FIM ) between patient groups, item characteristics of the FIM should be consistent across groups. The purpose of this study was to compare item difficulty of the FIM in 3 patient groups with neurological disorders. Patients with stroke (n=295), multiple sclerosis (n=150), and traumatic brain injury (n=88). FIM scores were administered in each group. The FIM consists of a motor domain (13 items) and a cognitive domain (5 items). Rasch rating scale analysis was performed to investigate differences in item difficulty (differential item functioning) between groups. Answering categories of the FIM items were reduced to 3 (from the original 7) because of disordered thresholds and low answering frequencies. Two items of the motor domain ("bladder" and "bowel") did not fit the Rasch model. For 7 out of the 11 fitting motor items, item difficulties were different between groups (i.e. showed differential item functioning). All cognitive items fitted the Rasch model, and 4 out of 5 cognitive items showed differential item functioning. Differential item functioning is present in several items of both the motor and cognitive domain of the FIM. Adjustments for differential item functioning may be required when FIMdata will be compared between groups or will be used in a pooled data analysis.

Psychometric Properties and Developmental Differences in Children’s ADL Item Hierarchy

To investigate the measurement properties of the WeeFIM instrument and examine the developmental differences in motor item difficulty in a pediatric inpatient rehabilitation sample. Database approach, retrospective study, using 814 WeeFIM records from 12 facilities. Rasch rating scale analysis was used for all analyses. Data for the complete sample were used to evaluate the dimensionality and item fit of the WeeFIM instrument. Patients were then divided into three age groups, <3 yrs of age (toddlers), 3-7 yrs of age (preschoolers and kindergartners), and >7 yrs of age (school-age children), and their data were used to compare the order of motor items by level of difficulty within and across different age groups. Principal component analysis of the WeeFIM suggests distinct motor and cognitive scales. Within the motor scale, bowel and bladder items misfit, suggesting they measure a distinct aspect of function. The order of motor item difficulty, an indicator of construct validity, varies across age groups, suggesting that motor tasks present different challenges to children at different developmental stages: toileting is the most difficult, whereas locomotion is the easiest motor item for toddlers. For children >3 yrs of age, eating is easiest and stair climbing is most difficult, followed by tub transfer and bathing. Similar to the adult FIMtrade mark instrument, the WeeFIM instrument has two distinct dimensions. The motor items form a unidimensional construct with acceptable measurement properties. Developmental differences in motor task mastery among children with disabilities are assumed but rarely tested. As evidenced by the age-specific item hierarchies found in this study, developmental differences among children with disabilities mimic that of children without disabilities.

Rating scale analysis of the Berg balance scale

Kornetti DL, Fritz SL, Chiu Y-P, Light KE, Velozo CA. Rating scale analysis of the Berg Balance Scale. Arch Phys Med Rehabil 2004;85:1128–35. Objectives To examine, using Rasch analysis, the rating scale performance of the Berg Balance Scale (BBS) and to describe the 45/56 cutoff score in functional terms. Design Retrospective chart review of BBS scores. Rasch rating scale analysis was performed on these data. Setting Outpatient Veterans Affairs medical center. Participants One hundred (99 men, 1 woman) community-dwelling veterans referred for balance deficits (age range, 64–88y). Interventions Not applicable. Main outcome measure The BBS. Results Condensing item-rating categories allowed the elimination of underutilized categories and constructed categories that better separated people of differing abilities. Rating pivot points were developed for each item to represent a transition between passing and failing. Following pivot-point development and rating scale rescoring, person and item measures became more evenly distributed across the BBS and resulted in changes in item difficulty order. In our sample, functional indicators of a score of at least 45/56 were a rating of passing the item “tandem stance,” as well as passing 2 of the following 3 items: “alternating foot,” “standing on one leg,” and “look behind.” Conclusions Our findings provide direction for improving the rating scale structure for each of the items and establish a connection between the BBS cutoff score of 45/56 and functional ability.

Assessment of Capacity for Myoelectric Control: a new rasch-built measure of prosthetic hand control

To report the results from a Rasch rating scale analysis of the Assessment of Capacity for Myoelectric Control (ACMC) implemented to evaluate internal scale validity, person response validity, separation reliability, targeting and responsiveness of the measures over time. Longitudinal data (18 months) from a prospective study of development of capacity for myoelectric control in children and adults were used for the analysis. A consecutive sample of 75 subjects (43 males, 32 females) with upper limb reduction deficiency or amputation and myoelectric prosthetic hands referred for occupational therapy from September 2000 to March 2002. Participants' ages ranged from 2 to 57 years. Outcome measure was the ACMC. Occupational therapists completed 210 assessments at an arm prosthesis centre in Sweden. A two-faceted rating scale analysis of the data was performed. All 30 ACMC items and 96.2% of participants demonstrated goodness-of-fit to the rating scale model for the ACMC. Separation and SE values suggested adequate reliability of the item and person estimates. The items demonstrated internal scale validity and the participants demonstrated person response validity. The ACMC was well targeted and sensitive enough to detect expected change in ability.

Measurement characteristics of the Functional Independence Measure.

Clinicians and researchers recognize the need for measures offunctional status that possess linear properties and are reliable and valid. Rasch rating scale analysis provides the means for converting raw scores from functional assessment tools to linear measures for which measurement error can be quantified. The extent to which clinicians perceive patients who are undergoing rehabilitation after stroke as similarto other patient groups was investigated using the Functional Independence Measure (FIM). Earlier work demonstrated that the first 13 items of the FIM represent a measure of motor function and that the last 5 items represent a measu re of cognitive function. The FIM was used for patients with stroke in a manner similar to that for most other impairment groups on the motor items. Patients with stroke were, however, unlike many impairment groups in their ratings on the cognitive items. Tables showing raw score to scaled measure conversions are provided for two sets of impairment groups on the motor items and three sets of impairment groups on the cognitive items. Clinicians can be confident that the measures derived from the FIM are linear across the range of the instrument and are attuned to the uniqueness of patients with stroke and other specific impairments.

The Classroom Communication Ease Scale. Development of a self-report questionnaire for mainstreamed deaf students.

This study describes efforts to develop a self-report psychological scale (CCES: Classroom Communication Ease Scale) to provide information about the extent to which mainstreamed deaf students think that they communicate effectively with teachers and peers, as well as how they feel about their communication experiences in mainstreamed classrooms. Data obtained from administration of the CCES to 50 deaf students enrolled in baccalaureate programs at a postsecondary technical institution were subjected to Rasch rating scale analysis. Fit of data to the measurement model resulted in a set of 42 items that empirically define the concept of "communication ease" in its cognitive aspects and a set of 21 items that elaborate the concept further in terms of its affective aspects.

The Student Integration Survey: Development of an early alert assessment and reporting system

This study describes efforts to develop a self-report rating scale (Student Integration Survey: SIS) that provides diagnostic information about students with academic and/or social integration difficulties. Data obtained from administration of the SIS to incoming freshmen at a postsecondary technical institution for the deaf in the fall of 1990 and 1991 were subjected to Rasch rating scale analysis. Fit of data to the measurement model resulted in a set of 28 positive and negative statements that empirically defined the concept of “institutional integration” in terms of their calibration hierarchy. This type of instrument calibration has the potential of providing person measures and item scale values that are invariant over a useful range of comparisons. The early alert reporting system provides an evaluation and diagnosis of observed responses relevant to expected responses for individual respondees. The substance and methods employed may provide a model for other integration assessment efforts.

SCALAR ANALYSIS OF THE TEST OF WRITTEN ENGLISH

The present research was conducted to explore the psychometric characteristics of the Test of Written English (TWE) rating scale. Rasch model scalar analysis methodology was employed with more than 4,000 scored essays across two elicitation prompts to gather the following information about the TWE rating scale and rating process: the position and size of the interval on the overall latent trait that could be attributed to behavioral descriptors accompanying each possible integer scoring step on the TWE scale the standard error of estimate associated with each possible transformed integer rating the fit of rating scale steps and individual rated essays to a unidimensional model of writing ability and, concurrently, the adequacy of such a model, including the proportion of misfitting essays as a portion of all essays analyzed the fit of individual readers to a unidimensional model of writing ability and to the expectations of a chi-square contingency test of independence of readers and ratings assigned, along with information on some characteristics of misfitting readers comparative scalar information for two distinct TWE elicitation prompts, including nonparametric tests of the independence of readers and scale steps assigned and the feasibility of equating of scales. Results suggested that the intervals between TWE scale steps were surprisingly uniform and that the size of the intervals was appropriately larger than the error associated with assignment of individual ratings. The proportion of positively misfitting essays was small (approximately 1% of all essays analyzed) and was approximately equal to the proportion of essays that required adjudication by a third reader. This latter finding, along with the low proportion of misfitting readers detected, provided preliminary evidence of the feasibility of employing Rasch rating scale analysis methodology for the equating of TWE essays prepared across prompts. Some information on characteristics of misfitting readers was presented that could be useful in the reader training process.

Content Validity Revisited

This article describes a process for linking job analysis data to test specifications. Rasch rating scale analysis is used to order the ratings of importance assigned to task and content items by practicing laboratory managers. This ordering produces variables representing the range of task and content items from most to least important. The variables also provide an objective frame of reference for review of the data by the experts. The Rasch calibrations for each task and content item are transformed to percentages based on useful limits. From the transformed percentages, test specifications reflecting practice patterns in the field of laboratory management are developed.