Psychometric Properties Of Measure Research Articles

Psychometrics and the neuroscience of individual differences: Internal consistency limits between-subjects effects.

In the clinical neuroscience literature, between-subjects differences in neural activity are presumed to reflect reliable measures-even though the psychometric properties of neural measures are almost never reported. The current article focuses on the critical importance of assessing and reporting internal consistency reliability-the homogeneity of "items" that comprise a neural "score." We demonstrate how variability in the internal consistency of neural measures limits between-subjects (i.e., individual differences) effects. To this end, we utilize error-related brain activity (i.e., the error-related negativity or ERN) in both healthy and generalized anxiety disorder (GAD) participants to demonstrate options for psychometric analyses of neural measures; we examine between-groups differences in internal consistency, between-groups effect sizes, and between-groups discriminability (i.e., ROC analyses)-all as a function of increasing items (i.e., number of trials). Overall, internal consistency should be used to inform experimental design and the choice of neural measures in individual differences research. The internal consistency of neural measures is necessary for interpreting results and guiding progress in clinical neuroscience-and should be routinely reported in all individual differences studies. (PsycINFO Database Record

Psychometric Properties of Two Short Versions of the Inventory of Callous–Unemotional Traits Among Incarcerated Youth

Accumulating evidence shows callous–unemotional (CU) traits to delineate a subgroup of youth with unique etiological underpinnings to their conduct problems. Due to its inclusion as a conduct disorder specifier in Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5), research that evaluates the psychometric properties of measures used to assess this construct is essential. The present study contributes to this area of investigation by examining two recently developed brief versions (10- and 12-item) of the Inventory of Callous–Unemotional Traits (ICU) among a Portuguese sample of incarcerated male juvenile offenders ( N = 221). Each of these shortened versions demonstrated evidence in terms of factor structure, internal consistency, and convergent/discriminant validity with theoretically meaningful constructs. In line with prior research, there was evidence of unique correlations between separate ICU subscales and several outcomes of interest. Study findings indicate that the two shortened versions of the ICU, and particularly the ICU-12, may offer researchers and clinicians a relatively brief and psychometrically sound means of assessing CU traits.

Psychometric Comparison of Dissociative Experiences Scales II and C: A Weak Trauma‐Dissociation Link

SummaryThe debate regarding the relationship between dissociation and trauma has raised questions regarding the validity of measures of dissociation. Dalenberg et al.'s () meta‐analysis included studies using the Dissociative Experiences Scale (DES II), but excluded the DES‐Comparison (DES‐C) scale, claiming that it lacked validity as a measure of dissociation. Lynn et al. () contended that omitting those studies might have skewed the results. In the current study, we compared the psychometric properties of both measures in two nonclinical US adult (student, general population) samples to evaluate the convergent and discriminant validity of the scales. We found support for the DES‐II as a measure of dissociation as well as the validity and reliability of the DES‐C, which compares well to the DES II. Compared with studies in Dalenberg et al., we found lower correlations between trauma and dissociation. No empirical basis exists to exclude studies using the DES‐C in literature reviews. Copyright © 2017 John Wiley & Sons, Ltd.

Análisis psicométrico de un cuestionario para medir el ambiente educativo en una muestra de estudiantes de medicina en Perú

The objective of the study was to determine the psychometric properties of the Dundee Ready Education Environment Measure (DREEM) questionnaire in students of a Peruvian medical school. Participants were first-to-sixth-year students, invited to participate through a web platform. In the analysis the alpha coefficient of Cronbach was used to evaluate internal consistency. In addition, a confirmatory factorial analysis was performed to evaluate the construct validity of the subscales of the DREEM. The adjustment quality indexes used were: Root Mean Square Error Approximation (RMSEA), the Index of Benchmarking (CFI), Tucker-Lewis Index (TLI), and the Standardized Root Mean Residual (SRMR). The results showed that the social self-perception dimension had an alpha of Cronbach of less than 0.70. In addition, the quality indices did not support the five-dimensional structure of the DREEM. In conclusion, the DREEM questionnaire did not perform optimally in this Peruvian sample.

Factor Structure of the Maslach Burnout Inventory–Educators Survey in a Polish-Speaking Sample

This article investigates the psychometric properties of a mainstream burnout measure dedicated to teachers: the Maslach Burnout Inventory–Educators Survey (MBI-ES). The study used data gathered from a random sample of 1,206 primary school teachers in Poland to verify the construct validity of the MBI-ES. Eight alternative measurement models suggested in the literature were tested using confirmatory factor analysis. Contrary to many previous studies, this study did not support the oblique three-factor structure of the MBI-ES. A bifactor model with one general Burnout factor and three specific orthogonal factors of personal accomplishment, depersonalization, and emotional exhaustion showed best fit to the data. Additional analyses supported the measure’s essential unidimensionality. The results yield theoretical implications for construct reconceptualization and practical guidelines for researchers and practitioners.

A systematic review evaluating the psychometric properties of measures of social inclusion.

IntroductionImproving social inclusion opportunities for population health has been identified as a priority area for international policy. There is a need to comprehensively examine and evaluate the quality of psychometric properties of measures of social inclusion that are used to guide social policy and outcomes.ObjectiveTo conduct a systematic review of the literature on all current measures of social inclusion for any population group, to evaluate the quality of the psychometric properties of identified measures, and to evaluate if they capture the construct of social inclusion.MethodsA systematic search was performed using five electronic databases: CINAHL, PsycINFO, Embase, ERIC and Pubmed and grey literature were sourced to identify measures of social inclusion. The psychometric properties of the social inclusion measures were evaluated against the COSMIN taxonomy of measurement properties using pre-set psychometric criteria.ResultsOf the 109 measures identified, twenty-five measures, involving twenty-five studies and one manual met the inclusion criteria. The overall quality of the reviewed measures was variable, with the Social and Community Opportunities Profile-Short, Social Connectedness Scale and the Social Inclusion Scale demonstrating the strongest evidence for sound psychometric quality. The most common domain included in the measures was connectedness (21), followed by participation (19); the domain of citizenship was covered by the least number of measures (10). No single instrument measured all aspects within the three domains of social inclusion. Of the measures with sound psychometric evidence, the Social and Community Opportunities Profile-Short captured the construct of social inclusion best.ConclusionsThe overall quality of the psychometric properties demonstrate that the current suite of available instruments for the measurement of social inclusion are promising but need further refinement. There is a need for a universal working definition of social inclusion as an overarching construct for ongoing research in the area of the psychometric properties of social inclusion instruments.

Psychometric properties of a Power Mobility Caregiver Assistive Technology Outcome Measure.

Caregiver burnout is a serious concern among informal caregivers, especially for those who provide care to individuals with more severe limitations such as power mobility users. The Power Wheelchair Caregiver Assistive Technology Outcome Measure tool measures device specific and overall burden experienced by informal caregivers of power mobility users. A one-month, test-retest study was conducted to examine the reliability, internal consistency, and construct validity of the Power Wheelchair Caregiver Assistive Technology Outcome Measure. Two construct validity measures were administered: the Hospital Anxiety and Depression Scale and the Late Life Disability Index. The test-retest-reliabilities of part 1 (power wheelchair specific burden) and part 2 (general caregiving burden) were 0.769 and 0.843 respectively. Scores on part 1 were moderately and positively correlated with part 2 and with frequency of participation. Scores on part 2 were moderately and negatively correlated with anxiety, depression, and positively with perceived limitation of participation. The strength and direction of these correlations provide support for the construct validity of the measure and suggest part 1 and part 2 provide complementary information. Further testing is needed to assess the clinical utility and responsiveness of the measure.

The psychometric and empirical properties of measures of risk preferences

We examine the psychometric and empirical properties of some commonly used survey-based measures of risk preferences in a population-based sample of 11,000 twins. Using a model that provides a general framework for making inferences about the component of measured risk attitudes that is not due to measurement error, we show that measurement-error adjustment leads to substantially larger estimates of the predictive power of risk attitudes, of the size of the gender gap, and of the magnitude of the sibling correlation. Risk attitudes are predictive of investment decisions, entrepreneurship, and drinking and smoking behaviors; are robustly associated with cognitive ability and personality; and our estimates are often larger than those in the literature. Our results highlight the importance of adjusting for measurement error across a wide range of empirical settings.

Development and Reliability of a Preliminary Foot Osteoarthritis Magnetic Resonance Imaging Score.

Foot osteoarthritis (OA) is very common but underinvestigated musculoskeletal condition and there is little consensus as to common magnetic resonance imaging (MRI) features. The aim of this study was to develop a preliminary foot OA MRI score (FOAMRIS) and evaluate its reliability. This preliminary semiquantitative score included the hindfoot, midfoot, and metatarsophalangeal joints. Joints were scored for joint space narrowing (JSN; 0-3), osteophytes (0-3), joint effusion/synovitis, and bone cysts (present/absent). Erosions and bone marrow lesions (BML) were scored (0-3) and BML were evaluated adjacent to entheses and at sub-tendon sites (present/absent). Additionally, tenosynovitis (0-3) and midfoot ligament pathology (present/absent) were scored. Reliability was evaluated in 15 people with foot pain and MRI-detected OA using 3.0T MRI multi-sequence protocols, and assessed using ICC as an overall score and per anatomical site. Intrareader agreement (ICC) was generally good to excellent across the foot in joint features (JSN 0.90, osteophytes 0.90, effusion/synovitis 0.46, cysts 0.87), bone features (BML 0.83, erosion 0.66, BML entheses 0.66, BML sub-tendon 0.60) and soft tissue features (tenosynovitis 0.83, ligaments 0.77). Interreader agreement was lower for joint features (JSN 0.43, osteophytes 0.27, effusion/synovitis 0.02, cysts 0.48), bone features (BML 0.68, erosion 0.00, BML entheses 0.34, BML sub-tendon 0.13), and soft tissue features (tenosynovitis 0.35, ligaments 0.33). This preliminary FOAMRIS demonstrated good intrareader reliability and fair interreader reliability when assessing the total feature scores. Further development is required in cohorts with a range of pathologies and to assess the psychometric measurement properties.

Test-retest reliability and convergent validity of measures of children's travel behaviours and independent mobility

Abstract Introduction Active transportation (AT) and independent mobility (IM) represent promising avenues for increasing children's physical activity and minimizing car use. However, there are limited data on the psychometric properties of measures of AT and IM. Methods 94 child and parent dyads living in Ottawa (Canada) consented to complete a questionnaire twice, approximately two weeks apart, in English or French language. They were questioned on children's travel to/from places (e.g., school, parks, shops) and on the extent of the child's IM. The weekly volume of AT to/from school was calculated by multiplying the number of active trips by the home-school distance. An IM index consisting of six “mobility licenses” (to travel home from school, travel to other places within walking distance, cross main roads, cycle on main roads, go out after dark, and travel on local buses without adult supervision) was computed. Test-retest reliability and convergent validity between children and parents were assessed with Cohen's kappa or intra-class correlation coefficients (ICC). Results The volume of AT to/from school showed high test-retest reliability and convergent validity in both languages (ICC range=0.81–0.97). Similarly, test-retest reliability was also high for the number of active trips to/from all destinations (ICC range=0.60–0.94) and the IM index (ICC range=0.63–0.88). Convergent validity for trips to/from all destinations was fair in the English language subsample (ICC range=0.22–0.25), but substantial in the French language subsample (ICC range=0.60–0.82). The IM index showed substantial validity at both test and retest and in both languages (ICC range=0.61–0.80). Coefficients were generally lower when examining single destinations or mobility licenses. Conclusion With minor modifications, the child and parent mobility questionnaires can provide valid and reliable estimates of AT to/from a broad range of destinations and IM among English and French speaking grade 4–6 children.

Systematic Review of Measures Used in Pictorial Cigarette Pack Warning Experiments.

We sought to describe characteristics and psychometric properties of measures used in pictorial cigarette pack warning experiments and provide recommendations for future studies. Our systematic review identified 68 pictorial cigarette pack warning experiments conducted between 2000 and 2016 in 22 countries. Two independent coders coded all studies on study features, including sample characteristics, theoretical framework, and constructs assessed. We also coded measurement characteristics, including construct, number of items, source, reliability, and validity. We identified 278 measures representing 61 constructs. The most commonly assessed construct categories were warning reactions (62% of studies) and perceived effectiveness (60%). The most commonly used outcomes were affective reactions (35%), perceived likelihood of harm (22%), intention to quit smoking (22%), perceptions that warnings motivate people to quit smoking (18%), and credibility (16%). Only 4 studies assessed smoking behavior. More than half (54%) of all measures were single items. For multi-item measures, studies reported reliability data 68% of the time (mean α = 0.88, range α = 0.68-0.98). Studies reported sources of measures only 33% of the time and rarely reported validity data. Of 68 studies, 37 (54%) mentioned a theory as informing the study. Our review found great variability in constructs and measures used to evaluate the impact of cigarette pack pictorial warnings. Many measures were single items with unknown psychometric properties. Recommendations for future studies include a greater emphasis on theoretical models that inform measurement, use of reliable and validated (preferably multi-item) measures, and better reporting of measure sources. Robust and consistent measurement is important for building a strong, cumulative evidence base to support pictorial cigarette pack warning policies. This systematic review of experimental studies of pictorial cigarette warnings demonstrates the need for standardized, theory-based measures.

Reliability and validity of advanced theory-of-mind measures in middle childhood and adolescence.

Although theory-of-mind (ToM) development is well documented for early childhood, there is increasing research investigating changes in ToM reasoning in middle childhood and adolescence. However, the psychometric properties of most advanced ToM measures for use with older children and adolescents have not been firmly established. We report on the reliability and validity of widely used, conventional measures of advanced ToM with this age group. Notable issues with both reliability and validity of several of the measures were evident in the findings. With regard to construct validity, results do not reveal a clear empirical commonality between tasks, and, after accounting for comprehension, developmental trends were evident in only one of the tasks investigated. Statement of contribution What is already known on this subject? Second-order false belief tasks have acceptable internal consistency. The Eyes Test has poor internal consistency. Validity of advanced theory-of-mind tasks is often based on the ability to distinguish clinical from typical groups. What does this study add? This study examines internal consistency across six widely used advanced theory-of-mind tasks. It investigates validity of tasks based on comprehension of items by typically developing individuals. It further assesses construct validity, or commonality between tasks.

PROMIS Sleep Disturbance and Sleep-Related Impairment in Adolescents: Examining Psychometrics Using Self-Report and Actigraphy.

The National Institutes of Health Patient-Reported Outcomes Measurement Information System (PROMIS) has self-reported health measures available for both pediatric and adult populations, but no pediatric measures are available currently in the sleep domains. The purpose of this observational study was to perform preliminary validation studies on age-appropriate, self-reported sleep measures in healthy adolescents. This study examined 25 healthy adolescents' self-reported daytime sleepiness, sleep disturbance, sleep-related impairment, and sleep patterns. Healthy adolescents completed a physical exam at the National Institutes of Health Clinical Center (Bethesda, MD), had no chronic medical conditions, and were not taking any chronic medications. The Cleveland Adolescent Sleepiness Questionnaire (CASQ), PROMIS Sleep Disturbance (v. 1.0; 8a), and PROMIS Sleep-Related Impairment (v. 1.0; 8b) questionnaires were completed, and sleep patterns were assessed using actigraphy. Total scores on the three sleep questionnaires were correlated (all Spearman's r > .70, p < .001). Total sleep time determined by actigraphy was negatively correlated with the CASQ (p = .01), PROMIS Sleep Disturbance (p = .02), and PROMIS Sleep-Related Impairment (p = .02). The field of pediatric sleep is rapidly expanding, and researchers and clinicians will benefit from well-designed, psychometrically sound sleep questionnaires. Findings suggest the potential research and clinical utility of adult versions of PROMIS sleep measures in adolescents. Future studies should include larger, more diverse samples and explore additional psychometric properties of PROMIS sleep measures to provide age-appropriate, validated, and reliable measures of sleep in adolescents.

Fear of adverse mental health treatment experiences: Initial psychometric properties of a brief self-report measure.

Many are apprehensive about mental health care, which potentially affects engagement in recovery processes as well as health outcomes. This article introduces a tool to assess fear of adverse mental health treatment experiences from the client's perspective. In a sample of 656 adults receiving mental health services at community agencies, this study is an initial exploration into the validity of a scale assessing fears associated with commonly experienced coercive or disorganized interventions. Factor analyses supported the construct validity of the 10-item Fear of Adverse Treatment Experiences Scale. It significantly discriminates based upon service characteristics, gender, history of victimization, and past experiences with coercive or disorganized interventions, with higher levels of fear reported by users of traditional mental health services, former inpatients who had their voluntary admission status changed, males, people with history of childhood abuse, and people with certain forms of criminal justice involvement. (PsycINFO Database Record

Test-Retest Reliability of Common Measures of Eating Disorder Symptoms in Men Versus Women.

Approximately 10% to 30% of individuals with eating disorders (EDs) are male, yet because measures often have not been tested among male participants, it is unclear whether the psychometric properties of ED measures are equivalent between sexes. The purpose of this study was to compare the test-retest reliability of common ED measures in men versus women. Participants ( N = 227; 58.1% female) completed self-report measures of body dissatisfaction, restrained eating, disinhibited eating, bulimic symptoms, and desire-for-muscularity at baseline and 2-to-4 weeks later. Intraclass correlations were used to compute retest correlations. Spearman's rho was used to compute retest correlations for skewed and kurtotic variables. We compared 95% confidence intervals for intraclass correlation coefficients to determine whether measures differed in reliability between sexes. Most ED measures had at least acceptable test-retest reliabilities. However, few measures of disinhibited and binge eating demonstrated good reliability in men. Results highlight the utility of several ED measures for assessing symptom change over time, and the need for additional research to identify and correct for sources of gender unreliability among ED self-report measures in men-particularly for assessing constructs that include binge-eating behavior.

Shoulder proprioception: How is it measured and is it reliable? A systematic review

Study DesignSystematic review. IntroductionConstituents of proprioception include our awareness of the position (joint position sense [JPS]) and motion (kinesthesia) of our limbs in space. Proprioceptive deficits are associated with musculoskeletal disorders but remain a challenge to quantify, particularly at the shoulder. Purpose of the StudyTo report the psychometric values of validity, reliability, and responsiveness for shoulder JPS and/or kinesthesia protocols. MethodsA review of 5 databases was conducted from inception to July 2016 for studies reporting a psychometric property of a shoulder proprioception protocol. The included studies were evaluated using the QualSyst checklist and COSMIN 4-point scale. ResultsTwenty-one studies were included, yielding 407 participants and 553 evaluated shoulders (n). The included studies support excellent methodological scores using the QualSyst checklist (88.1 ± 9.9%) and good psychometric scores with the COSMIN for reliability (71.1%) and moderate-to-low quality score (50%) for criterion validity. Weighted average intraclass correlation coefficients (ICCs) for intrarater reliability were highest for passive JPS and kinesthesia, ICC = 0.92 ± 0.07 (n = 214) and ICC = 0.92 ± 0.04 (n = 74), respectively. The most reliable movement and tool are internal rotation at 90° of abduction, ICC = 0.88 ± 0.01 (n = 53), and the dynamometer, ICC = 0.92 ± 0.88 (n = 225). Only 2 studies quantify an aspect of validity and no responsiveness indices were reported among the included studies. ConclusionBased on the results of the included studies, the evaluation of shoulder proprioception is most reliable when using a passive protocol with an isokinetic dynamometer for internal rotation at 90° of shoulder abduction. Standardized protocols addressing the psychometric properties of shoulder proprioception measures are needed. Level of EvidenceLevel 1a: systematic review.

Adaptation to European Spanish and psychometric properties of the Patient Activation Measure 13 in patients with chronic diseases.

The 13-item Patient Activation Measure (PAM-13) is an instrument that assesses people's knowledge, skills and confidence for self-management of their health and health care. Scores on the PAM-13 have been shown to predict adherence to health behaviours, health-related outcomes and health care costs. To develop a European Spanish adaptation of the original PAM-13 and to examine its psychometric properties in a sample of chronic patients. The PAM-13 was forward-backward translated and then completed by chronic patients attending a primary health care centre. Data were analysed with a Rasch model. We assessed the functioning of the rating scale, its reliability, the item goodness-of-fit, differential item functioning (DIF), local dependence, unidimensionality and correlation analysis. A total of 208 patients (80%) completed the questionnaire. Data showed a fit to the Rasch model. More than 50% of patients endorsed all the items. Item rank for the Spanish sample was similar to the original, with few differences. We found significant differences (P < 0.05) in PAM-13 measures according to adherence to prescribed medicines and positive correlations with self-efficacy and physical quality of life. The European Spanish PAM-13 is a reliable and valid instrument for assessing activation in patients with chronic disease in Spain. We suggest adding new items to the instrument so as to examine patients with higher activation levels in the future. Further studies are needed to evaluate the usefulness of this new Spanish PAM-13 in different settings and populations, as well as to examine the relationship between PAM-13 scores and other health-related outcomes.

From the Editors

From the Editors

Measuring Inpatient Rehabilitation Facility Quality of Care: Discharge Self-Care Functional Status Quality Measure

ObjectiveTo describe the calculation and psychometric properties of the discharge self-care functional status quality measure implemented in the Centers for Medicare & Medicaid Services' (CMS) Inpatient Rehabilitation Facility (IRF) Quality Reporting Program on October 1, 2016. DesignMedicare fee-for-service (FFS) patients from 38 IRFs that participated in the CMS Post-Acute Care Payment Reform Demonstration were included in this cohort study. Data came from the Continuity Assessment Record and Evaluation Item Set, IRF–Patient Assessment Instrument, and Medicare claims. For each patient, we calculated an expected discharge self-care score, risk-adjusted for demographic and baseline clinical characteristics. The performance score of each IRF equaled the percentage of patient stays where the observed discharge self-care score met or exceeded the expected score. We assessed the measure's discriminatory ability across IRFs and reliability. SettingIRFs. ParticipantsMedicare FFS patients aged ≥21 years (N=4769). InterventionsNot applicable. Main Outcome MeasuresFacility-level discharge self-care quality measure performance score. ResultsA total of 4769 patient stays were included; 57% of stays were in women, and 12.1% were in patients aged <65 years. Stroke was the most common diagnosis (21.8%). The mean±SD performance score was 55.1%±16.6% (range, 25.8%–100%). About 54% of IRFs had scores significantly different from the percentage of stays that met or exceeded the expected discharge self-care score in the overall demonstration sample. The quality measure showed strong reliability, with intraclass correlation coefficients of .91. ConclusionsThe discharge self-care quality measure showed strong discriminatory ability and reliability, representing an important initial step in evaluation of IRF self-care outcomes. A wide range in performance scores suggested a gap in quality of care across IRFs. Future work should include testing the measure with nationwide data from all IRFs.

Positive Psychological Assessment and Early-Stage Dementia

ABSTRACTObjectives: There are no known positive psychological (PP) measures adapted/validated for use with people in the early stages of dementia. Objectives were to: (1) develop/pilot an accessible protocol for administering PP measures in this population; (2) examine their psychometric properties; and (3) provide preliminary observations about how people with early-stage dementia perceive various aspects of PP as it pertains to living with dementia.Methods: Five PP measures were adapted/administered to 36 people with early-stage dementia. Constructs of interests were gratitude, life satisfaction, meaning in life, optimism, and resilience. Psychometric data were calculated and response patterns analyzed.Results: Participants easily completed the adapted measures using the accessible protocol. Preliminary evidence of reliability and validity were yielded. Response patterns suggested a range of PP perceptions and experiences among participants.Conclusions: This study represents a step towards validating the psychometric properties of several PP measures adapted for use with people in the early stages of dementia and provides insight into how PP works in this population.Clinical Implications: PP assessment may assist clinicians and researchers to better understand PP functioning in this population potentially contributing to effective strengths based psychosocial interventions. Recommendations for clinical practice and future research are provided.