Classical Psychometric Methods Research Articles

Further refinement of the Patient-Reported Impact of Dermatological Diseases (PRIDD) measure using classical test theory and item response theory.

Existing dermatology-specific Patient-Reported Outcome Measures (PROMs) do not fully capture the substantial physical, psychological and social impact of dermatological conditions on patients' lives and are not recommended for use according to the COSMIN criteria. Most were developed with insufficient patient involvement and relied on classical psychometric methods. We are developing the new Patient-Reported Impact of Dermatological Diseases (PRIDD) measure for use in research and clinical practice in partnership with patients. To examine the factor structure of PRIDD, determine the definitive selection of items for each subscale, and establish structural validity and internal consistency through classical and modern psychometric methods. Two cross-sectional online surveys were conducted. Adults (≥ 18 years) worldwide living with a dermatological condition were recruited through the membership network of the International Alliance of Dermatology Patient Organizations (GlobalSkin). They completed the PRIDD questionnaire and a demographics questionnaire via an online survey. We examined missing data and distribution of scores for each item. The factor structure was assessed using confirmatory and exploratory factor analysis (Survey 1). Internal consistency was examined using Cronbach's α. Rasch measurement theory analyses were conducted, including iterative assessment of rating scale function, fit to the Rasch model, unidimensionality, reliability, local dependence, targeting and differential item functioning (DIF) (Surveys 1 and 2). Participants in Surveys 1 and 2 numbered 483 and 504 people, respectively. All items had ≤ 3% missing scores and all five response options were used. A four-factor model showed the best fit. PRIDD and all four subscales were internally consistent but showed some misfit to the Rasch measurement model. Adjustments were made to rectify disordered thresholds, remove misfitting items, local dependency and DIF, and improve targeting. The resulting 16-item version and subscales fit the Rasch model, showed no local dependency or DIF at the test level, and were well targeted. This field test study produced the final PRIDD measure, consisting of 16 items across four domains. The data triangulated and refined the conceptual framework of impact and provide evidence of PRIDD's structural validity and internal consistency. The final step in the development and validation of the PRIDD measure is to test the remaining measurement properties.

The COVID-19 Yorkshire Rehabilitation Scale (C19-YRS): Application and psychometric analysis in a post-COVID-19 syndrome cohort.

As our understanding of the nature and prevalence of post‐coronavirus disease 2019 (COVID‐19) syndrome (PCS) is increasing, a measure of the impact of COVID‐19 could provide valuable insights into patients' perceptions in clinical trials and epidemiological studies as well as routine clinical practice. To evaluate the clinical usefulness and psychometric properties of the COVID‐19 Yorkshire Rehabilitation Scale (C19‐YRS) in patients with PCS, a prospective, observational study of 187 consecutive patients attending a post‐COVID‐19 rehabilitation clinic was conducted. The C19‐YRS was used to record patients' symptoms, functioning, and disability. A global health question was used to measure the overall impact of PCS on health. Classical psychometric methods (data quality, scaling assumptions, targeting, reliability, and validity) were used to assess the C19‐YRS. For the total group, missing data were low, scaling and targeting assumptions were satisfied, and internal consistency was high (Cronbach's α = 0.891). Relationships between the overall perception of health and patients' reports of symptoms, functioning, and disability demonstrated good concordance. This is the first study to examine the psychometric properties of an outcome measure in patients with PCS. In this sample of patients, the C19‐YRS was clinically useful and satisfied standard psychometric criteria, providing preliminary evidence of its suitability as a measure of PCS.

Evaluation of the Psychometric Properties of the Burn Specific Health Scale-Brief: A National Institute on Disability, Independent Living, and Rehabilitation Research Burn Model System Study.

The Burn Specific Health Scale-Brief (BSHS-B) is a commonly used burn-specific health outcome measure that includes 40 items across nine subscales. The objective of this study was to use both classical and modern psychometric methods to evaluate psychometric properties of the BSHS-B. Data were collected postburn injury by a multisite federally funded project tracking long-term outcomes. We examined dimensionality, local dependence, item fit, and functioning of response categories, homogeneity, and floor and ceiling effects. Items were fit to Item Response Theory models for evaluation. A total of 653 adults with burn injury completed the BSHS-B. Factor analyses supported unidimensionality for all subscales, but not for a total score based on all 40 items. All nine of the subscales had significant ceiling effects. Six item pairs displayed local dependence suggesting redundance and 11 items did not fit the Item Response Theory models. At least 15 items have too many response options. Results identified numerous psychometric issues with the BSHS-B. A single summary score should never be used for any purpose. Psychometric properties of the scale need to be improved by removing redundant items, reducing response categories, and modifying or deleting problematic items. Additional conceptual work is needed to, at a minimum, revise the work subscale and optimally to revisit and clearly define the constructs measured by all the subscales. Additional items are needed to address ceiling effects.

Rasch analysis of the patient-rated wrist evaluation questionnaire

BackgroundThe Patient-Rated Wrist Evaluation (PRWE) was developed as a wrist joint specific measure of pain and disability and evidence of sound validity has been accumulated through classical psychometric methods. Rasch analysis (RA) has been endorsed as a newer method for analyzing the clinical measurement properties of self-report outcome measures. The purpose of this study was to evaluate the PRWE using Rasch modeling.MethodsWe employed the Rasch model to assess overall fit, response scaling, individual item fit, differential item functioning (DIF), local dependency, unidimensionality and person separation index (PSI). A convenience sample of 382 patients with distal radius fracture was recruited from the hand and upper limb clinic at large academic healthcare organization, London, Ontario, Canada, 6-month post-injury scores of the PRWE was used. RA was conducted on the 3 subscales (pain, specific activities, and usual activities) of the PRWE separately.ResultsThe pain subscale adequately fit the Rasch model when item 4 “Pain - When it is at its worst” was deleted to eliminate non-uniform DIF by age group, and item 5 “How often do you have pain” was rescored by collapsing into 8 intervals to eliminate disordered thresholds. Uniform DIF for “Use my affected hand to push up from the chair” (by work status) and “Use bathroom tissue with my affected hand” (by injured hand) was addressed by splitting the items for analysis. After background rescoring of 2 items in pain subscale, 2 items in specific activities and 3 items in usual activities, all three subscales of the PRWE were well targeted and had high reliability (PSI = 0.86). These changes provided a unidimensional, interval-level scaled measure.ConclusionLike a previous analysis of the Patient-Rated Wrist and Hand Evaluation, this study found the PRWE could be fit to the Rasch model with rescoring of multiple items. However, the modifications required to achieve fit were not the same across studies, our fit statistics also suggested one of the pain items should be deleted. This study adds to the pool of evidence supporting the PRWE, but cannot confidently provide a Rasch-based scoring algorithm.

Rasch analysis of the Patient Rated Elbow Evaluation questionnaire.

BackgroundThe Patient Rated Elbow Evaluation (PREE) was developed as an elbow joint specific measure of pain and disability and validated with classical psychometric methods. More recently, Rasch analysis has contributed new methods for analyzing the clinical measurement properties of self-report outcome measures. The objective of the study was to determine aspects of validity of the PREE using the Rasch model to assess the overall fit of the PREE data, the response scaling, individual item fit, differential item functioning (DIF), local dependency, unidimensionality and person separation index (PSI).MethodsA convenience sample of 236 patients (Age range 21–79 years; M: F- 97:139) with elbow disorders were recruited from the Roth│McFarlane Hand and Upper Limb Centre, London, Ontario, Canada. The baseline scores of the PREE were used. Rasch analysis was conducted using RUMM 2030 software on the 3 sub scales of the PREE separately.ResultsThe 3 sub scales showed misfit initially with disordered thresholds on17 out of 20 items), uniform DIF was observed for two items (“Carrying a 10lbs object” from specific activities subscale for age group; and “household work” from the usual activities subscale for gender); multidimensionality and local dependency. The Pain subscale satisfied Rasch expectations when item 2 “Pain – At rest” was split for age group, while the usual activities subscale readily stood up to Rasch requirements when the item 2 “household work” was split for gender. The specific activities subscale demonstrated fit to the Rasch model when sub test analysis accounted for local dependency. All three subscales of the PREE were well targeted and had high reliability (PSI >0.80).ConclusionThe three subscales of the PREE appear to be robust when tested against the Rasch model when subject to a few alterations. The value of changing the 0–10 format is questionable given its widespread use; further Rasch-based analysis of whether these findings are stable in other samples is warranted.

Validation of a Parent Report Questionnaire: The Infant Gastrointestinal Symptom Questionnaire.

Objective. To evaluate the reliability and validity of the Infant Gastrointestinal Symptom Questionnaire (IGSQ), a tool to assess feeding tolerance in infants. Methods. Qualitative methods were used to develop IGSQ content across 5 symptom clusters, yielding a 13-item index of parent-reported infant digestion and elimination behaviors over the prior 7 days. Classical psychometric methods evaluated factor structure, interrater and retest reliability, and validity in 4 prospective studies of 836 infants. Results. Interrater and retest reliability were acceptable to good. IGSQ Index score was highly correlated (r = 0.89) with daily parent reports. IGSQ scores were significantly different between infants whose parents planned to switch formulas because of perceived feeding problems and those without parental concerns. Conclusions. The IGSQ is a practical, reliable, and valid method for assessment of infant gastrointestinal-related behaviors. Its use in clinical studies can provide empirical evidence to advance parent education regarding both normal and clinically meaningful feeding-related behaviors.

The World Health Organization Quality of Life instrument for people with intellectual and physical disabilities (WHOQOL-Dis): evidence of validity of the Brazilian version.

BackgroundThe number of people with disabilities in Brazil and worldwide has grown substantially in recent decades. Cross-cultural quality of life instruments can be helpful in the development of interventions designed to meet the needs of this population and contribute to rational allocation of resources. This study sought to provide evidence of validity and reliability the Brazilian Portuguese version of WHOQOL-Dis-D (a cross-cultural, multicentre instrument developed by the WHOQOL-Group for the assessment of quality of life in persons with physical disability – PD) and WHOQOL-Dis-ID (for persons with intellectual disability – ID).MethodsClassical psychometric methods were used to conduct independent analyses of the PD and ID samples. Criterion groups were established for analysis of construct validity. Concurrent validity was assessed in relation to SWLS and BDI-II scores; discriminant validity, in relation to WHODAS-II. Cronbach alpha was used to test the instrument scales and subscales for reliability. The ID subgroup was retested, and test-retest reliability assessed by means of intraclass correlation coefficients and paired Student’s t-test.ResultsA total of 162 (98 females) people with PD and 156 (55 females) people with ID participated in the study. Cronbach alpha was satisfactory across practically all domains and factors in the PD subsample. In IDs, most factors or domains had coefficients higher than 0.70, but four subscales exhibited less satisfactory performance. Evidence of construct and concurrent validity and reliability were obtained.ConclusionsThe analyses presented herein provide satisfactory evidence of the validity and reliability of the instrument and corroborated the factor structure revealed during cross-cultural research. Further studies with larger sample sizes are required to obtain additional evidence of validity and reliability.

Developing CAHPS for cancer care prototype: Progress and next steps.

134 Background: The CAHPS Survey for Cancer Care Prototype (application for CAHPS trademark in process) represents a significant advancement towards helping cancer care providers understand patients’ experiences of care. Although a number of standard commercial surveys are in use, currently none are available that both (1) capture the aspects of care most important to patients and (2) allow comparisons of those aspects of care across treatment centers and modalities. Methods: We developed a survey instrument that incorporated patient, clinician, and other stakeholder input throughout the development process. The survey measures patients’ perceptions of observable provider behaviors that are indicative of high quality care. Field test data were collected from 1,367 of 2,826 eligible patients (48% response rate) treated with surgery, radiation, or medical oncology from 6 accredited cancer treatment centers across the US. Classical psychometric methods were used to evaluate the measurement properties of composite scales per standard CAHPS methodology. Results: Based on a priori conceptualization, patient and stakeholder input, and empirical analyses, we identified 5 composites measuring shared decision-making, exchanging information about treatment, affective communication, access to care, and patient self-management. A confirmatory factor analysis using structural equation modeling showed an excellent fit (CFI > 0.96; RMSEA < 0.05) between the observed data and the structure for these five composites. Cronbach’s alpha for these composites ranged from 0.71-0.88, discriminant validity was good for all scales, and correlations with overall ratings of cancer care providers were moderate (0.31) to high (0.69). Evaluation of the precision with which the composites distinguish among cancer centers indicated that the measures were reliable; however, a second field test with a greater number and variety of centers will be needed to establish a standard sampling approach and confirm these results. Conclusions: We anticipate wide-spread adoption of this measure by cancer care providers to direct quality improvement efforts due to great interest and involvement in this project on the part of stakeholders.

The Cornell-Brown Scale for Quality of Life in Dementia

The objective of this study was to validate a Spanish version of the Cornell-Brown Scale (CBS) for quality of life (QoL) in dementia. One hundred persons with mild-to-moderate dementia (Diagnostic and Statistical Manual of the American Psychiatric Association, 4th edition) were recruited and interviewed at 5 Spanish centers to obtain sociodemographic information, health perceptions, depressive symptoms (Geriatric Depression Scale 15-item version), functional ability (Barthel Index), dementia severity (Mini-Mental State Examination), specific QoL (CBS), and generic QoL (World Health Organization Quality of Life-BREF version). Analysis was carried out using classical psychometric methods. Internal consistency reliability for the CBS was good (0.87). A priori hypotheses about the relationship between CBS and the World Health Organization Quality of Life-BREF version psychological domain and Geriatric Depression Scale 15-item version were confirmed, indicating good construct validity. Regarding contrasting groups' differences, CBS scores significantly differentiated between healthy and unhealthy and depressed and nondepressed participants but not between those with mild and moderate dementia. The exploratory factor analysis showed a 5-factor solution, which accounted for 63.9% of the total variance of CBS. This study is the first to report the factor structure of the CBS. These results indicate that CBS is a useful tool in assessing persons with dementia. Replication of these results with larger samples is advised. The Spanish version of the CBS is reliable and valid in persons with mild/moderate dementia who are living at home and with a known caregiver.

Reliability, construct validity and measurement potential of the ICF comprehensive core set for osteoarthritis

BackgroundThis study aimed to investigate the reliability and construct validity of the International Classification of Functioning, Disability and Health (ICF) Comprehensive Core Set for osteoarthritis (OA) in order to test its possible use as a measuring tool for functioning.Methods100 patients with OA (84 F, 16 M; mean age 63 yr) completed forms including demographic and clinical information besides the Short Form (36) Health Survey (SF-36®) and the Western Ontario and McMaster Universities Index of Osteoarthritis (WOMAC). The ICF Comprehensive Core Set for OA was filled by health professionals. The internal construct validities of "Body Functions-Body structures" (BF-BS), "Activity" (A), "Participation" (P) and "Environmental Factors" (EF) domains were tested by Rasch analysis and reliability by internal consistency and person separation index (PSI). External construct validity was evaluated by correlating the Rasch transformed scores with SF-36 and WOMAC.ResultsIn each scale, some items showing disordered thresholds were rescored, testlets were created to overcome the problem of local dependency and items that did not fit to the Rasch model were deleted. The internal construct validity of the four scales (BF-BS 16 items, A 8 items, P 7 items, EF 13 items) were good [mean item fit (SD) 0.138 (0.921), 0.216 (1.237), 0.759 (0.986) and -0.079 (2.200); person item fit (SD) -0.147 (0.652), -0.241 (0.894), -0.310 (1.187) and -0.491 (1.173) respectively], indicating a single underlying construct for each scale. The scales were free of differential item functioning (DIF) for age, gender, years of education and duration of disease. Reliabilities of the BF-BS, A, P, and EF scales were good with Cronbach's alphas of 0.79, 0.86, 0.88, and 0.83 and PSI's of 0.76, 0.86, 0.87, and 0.71, respectively. Rasch scores of BF-BS, A, and P showed moderate correlations with SF-36 and WOMAC scores where the EF had significant but weak correlations only with SF36-Social Functioning and SF36-Mental Health.ConclusionSince the four different scales derived from BF-BS, A, P, and EF components of the ICF core set for OA were shown to be valid and reliable through a combination of Rasch analysis and classical psychometric methods, these might be used as clinical assessment tools.

Using the WHOQOL-BREF in Persons With Dementia

Increasing evidence suggests that persons with early dementia can give reliable and valid assessments about their own quality of life (QOL), thereby improving accuracy, and reducing the need for proxy informants. The objective of this study was to examine QOL in persons diagnosed with dementia (Diagnostic and Statistical Manual of Mental Disorder-IV) using a battery of subjective assessments including the new World Health Organization Quality of Life (WHOQOL-BREF). Persons with mild-to-moderate dementia (n=104) were recruited and interviewed at 6 Spanish centers to obtain sociodemographic information, health perceptions, depressive symptoms (Geriatric Depression Scale 15-item version), functional ability (Barthel Index), generic QOL (WHOQOL-BREF), and specific QOL (DEMQOL-28). Analysis was performed using classical psychometric methods. Internal consistency reliability for the WHOQOL-BREF domains ranged from moderate (0.54 for social) to good (0.79 for psychological). Test-retest reliability (intraclass correlation) ranged from moderate (0.51 for psychological) to good (0.70 for physical). Associations were confirmed between WHOQOL-BREF domains with DEMQOL-28, Geriatric Depression Scale 15-item version, and Barthel dimensions. With regard to contrasting groups' differences, WHOQOL-BREF scores significantly differentiated between healthy and unhealthy and depressed and nondepressed participants. This study is the first to report on the use of the WHOQOL-BREF in persons with mild-to-moderate dementia. These results indicate that it is a useful tool in assessing these groups, as it includes important dimensions commonly omitted from other dementia measures.

The WHO quality of life (WHOQOL) questionnaire: Spanish development and validation studies

In the 1990s, the World Health Organization (WHO) undertook a project to develop an instrument (the WHOQOL) for measuring quality of life (QoL). The WHOQOL was developed in the framework of a collaborative project involving numerous centers in different cultural settings. This paper describes the psychometric properties of the Spanish WHOQOL during its development. One thousand and eighty-two patients with physical health conditions, persons without any health condition, patients with schizophrenia, and family caregivers of patients with schizophrenia participated on the WHOQOL-100 and WHOQOL-BREF field trials. QoL self-assessment was completed, together with sociodemographic and health status questions. Analysis was performed using classical psychometric methods. Both versions of the WHOQOL showed satisfactory psychometric properties as follows: acceptability, internal consistency, and evidence of convergent and discriminant validity. The WHOQOL-100 and WHOQOL-BREF are suitable to use in patients with different health conditions, including schizophrenia, and in different populations, including caregivers. Spanish field trials are the first to report on use of the WHOQOL in persons with schizophrenia and caregivers. These results indicate that both versions are useful tools in assessing these groups, as the WHOQOL includes important dimensions commonly omitted from other generic QoL measures.

Using the WHOQOL-DIS to Measure Quality of Life in Persons with Physical Disabilities Caused by Neurodegenerative Disorders

Background/Aims: Neurodegenerative disorders (ND) have a major impact on quality of life (QoL) and place a substantial burden on patients, their families and carers; they are the second leading cause of disability. The objective of this study was to examine QoL in persons with ND. Methods: A battery of subjective assessments was used, including the World Health Organization Quality of Life Questionnaire (WHOQOL-BREF) and the World Health Organization Quality of Life – Disability (WHOQOL-DIS). Psychometric properties of the WHOQOL-BREF and WHOQOL-DIS were investigated using classical psychometric methods. Results: Participants (n = 149) were recruited and interviewed at two specialized centers to obtain information on health and disability perceptions, depressive symptoms (Hospital Anxiety and Depression Scale – Depression, HADS-D), Fatigue Assessment Scale (FAS), Satisfaction with Life (SWL), generic QoL (WHOQOL-BREF, WHOQOL-DIS), specific QoL (Multiple Sclerosis Impact Scale, MSIS-29; Parkinson’s Disease Questionnaire, PDQ-39) and sociodemographics. Internal consistency was acceptable, except for the WHOQOL-BREF social (0.67). Associations, using Pearson’s and Spearman’s rho correlations, were confirmed between WHOQOL-BREF and WHOQOL-DIS with MSIS-29, PDQ-39, HADS-D, FAS and SWL. Regarding ‘known group’ differences, Student’s t tests showed that WHOQOL-BREF and WHOQOL-DIS scores significantly discriminated between depressed and nondepressed and those perceiving a more severe impact of the disability on their lives. Conclusion: This study is the first to report on use of the WHOQOL-BREF and WHOQOL-DIS in Spanish persons with ND; they are promising useful tools in assessing persons with ND through the continuum of care, as they include important dimensions commonly omitted from other QoL measures.

Critiche sperimentali all'assunzione di unidimensionalità

- In this paper the assumption that bipolar scales are unidimensional, which is implicit in the - classical theories of personality, is discussed based on a review of the literature on the topic. A comparison between these results, obtained using - classical psychometric methods and those emerged in previous studies conducted by the authors using the methodology of the experimental phenomenology of perception is proposed. These studies have consistently revealed that bipolar scales traditionally assumed as basic structures of human spatial experience, do not lie on the same unidimensional continuum.

The first stage of developing preference-based measures: constructing a health-state classification using Rasch analysis

To set out the methodological process for using Rasch analysis alongside classical psychometric methods in the development of a health-state classification that is amenable to valuation. The overactive bladder questionnaire is used to illustrate a five step process for deriving a reduced health-state classification from an existing non-preference-based health-related quality-of-life instrument. Step I uses factor analysis to establish instrument dimensions, step II excludes items that do not meet the initial validation process and step III uses criteria based on Rasch analysis and other psychometric testing to select the final items for the health-state classification. In step IV, item levels are examined and Rasch analysis is used to explore the possibility of reducing the number of item levels. Step V repeats steps I-IV on alternative data sets in order to validate the selection of items for the health-state classification. The techniques described enable the construction of a five-dimension health-state classification, the OAB-5D, amenable to valuation tasks that will allow the derivation of preference weights. The health-related quality of life of patients with conditions like overactive bladder can be valued and quality adjustment weights estimated for calculation of quality-adjusted life years.

Addiction severity index recent and lifetime summary indexes based on nonparametric item response theory methods.

Baseline Addiction Severity Index (5th ed.; ASI-5) data of 2,142 substance abuse patients were analyzed with two nonparametric item response theory (NIRT) methods: Mokken scaling and conditional covariance techniques. Nine reliable and dimensionally homogeneous Recent Problem indexes emerged in the ASI-5's seven areas, including two each in the Employment/Support and Family/Social Relationships areas. Lifetime Problem indexes were derived for five of the areas--Medical, Drug, Alcohol, Legal, and Psychiatric--but not for the Employment/Support and Family/Social Relationships areas. Correlational analyses conducted on a subsample of 586 patients revealed the indexes for the seven areas to be largely independent. At least moderate correlations were obtained between the Recent and Lifetime indexes within each area where both existed. Concurrent validity analyses conducted on this same subsample found meaningful relationships, except for the Employment/Support area. NIRT-based methods were able to add to findings produced previously by classical psychometric methods and appear to offer promise for the psychometric analysis of complex, mixed-format instruments such as the ASI-5.

The attitudes to ageing questionnaire (AAQ): development and psychometric properties

This paper describes the development of the Attitudes to Ageing Questionnaire (AAQ) which is a self-report measure with which older people themselves can express their attitudes to the process of ageing. The development of the AAQ followed a coherent, logical and empirical process taking full account of relevant gerontological knowledge and modern and classical psychometric analytical methods. Pilot testing with 1,356 participants from 15 centres worldwide refined the scale and provided the basis for a field test. A total of 5,566 participants from 20 centres worldwide contributed to the further development of this new scale with the derivation involving both classical and modern psychometric methods. The result is a 24-item cross-cultural attitudes to ageing questionnaire consisting of a three-factor model encompassing psychological growth, psychosocial loss, and physical change. The three-factor model suggests a way of conceptualizing and measuring successful ageing in individuals. The AAQ provides researchers, clinicians and policy makers with a unique scale to measure the impact of successful ageing interventions. It also provides a vehicle for the measurement of how individuals age across cultures and under different economic, political and social circumstances.

A New Scale for Measuring Adults' Prosocialness

Abstract. In the present study, the authors proposed a novel self-report 16-item scale for assessing individual differences in adult prosocialness and tested its measurement properties by employing an item response theory (IRT) analysis of data collected from a sample of 2,574 Italian adults. Prior work employing classical psychometric methods of analysis had already established the reliability and validity of the instrument. The present study furthered this scrutiny by examining whether the different prosocialness items were equally effective in discriminating people and equally informative; it also examined gender differences in the functioning of the items. The results of IRT analyses strongly supported the measurement effectiveness and sensitivity of the 16 prosocialness items, and findings are discussed for their implications in behavioral assessment research on prosocialness.