Scale Score Reliability Research Articles

Validation of a Six-Item Male Body Image Concerns Scale (MBICS)

Elevated body image concerns may be a risk factor for eating disorders among males and contribute to a range of other mental health problems. This study tested a 6-item measure of general male body image concerns in two studies with adolescent males ages 14–18 (total N = 122). The measure showed strong convergent validity, scale score reliability, and test-retest reliability, and was significantly correlated with the number of episodes of binge eating in the past month. A short scale will relieve participant burden and provide a useful research tool for studies with males at risk for or with eating disorders.

Comparing sport motivation scales: A response to Pelletier et al.

ObjectivesPelletier, Rocchi, Vallerand, Deci, and Ryan (2013) proposed a new version of the Sport Motivation Scale (SMS; Pelletier, Fortier, Vallerand, Tuson, & Blais, 1995) as a measure of different types of behavioral regulations in sport, as outlined in self-determination theory (Ryan & Deci, 2000). They examined various aspects of reliability and validity of scale scores, and concluded that the new scale performs better than the original version. They also claimed that the SMS-II is superior to other measures of motivation in sport, including the Behavioral Regulation in Sport Questionnaire (BRSQ) developed by Lonsdale, Hodge, and Rose (2008). By comparing the evidence presented in papers by Pelletier et al. and Lonsdale et al., our objective was to examine the relative merits and shortcomings of the two measures and suggest directions for future research into sport motivation measurement. ConclusionsBoth the SMS-II and BRSQ have shown relative strengths and weaknesses. Overall, the construct validity evidence of scores derived from the two measures was similar. There is insufficient information to support the claim that one scale is superior to the other. Researchers are encouraged to make direct comparisons by administering both measures to the same group of participants in future studies.

Measuring parent perceptions of school climate.

Parents' attitudes about their children's schools matter. Their views can shape their children's attitudes about school, affect their levels of family-school engagement, and influence their residential and school enrollment decisions. In this article, we describe the development of a survey scale to assess parent perceptions of the climate of their child's school. Our comprehensive scale development process incorporated feedback from academics and potential respondents from the outset of the design process to enhance scale quality. We conducted 3 studies with national samples of parents (n = 385; n = 253; n = 266) to gather evidence of scale score reliability and valid score inferences based on convergent and discriminant validity. Through confirmatory factor analysis, we identified a theoretically grounded factor structure that fit the data well but found no evidence that parental response patterns distinguish between academic and social elements of school climate. Furthermore, we found that parents of younger children, on average, had a more positive perception of the school's climate than did parents of older children. We conclude by discussing how researchers and Pre-K-12 schools and districts can use the scale to aid school improvement efforts.

On the Inappropriateness of Using Items to Calculate Total Scale Score Reliability via Coefficient Alpha for Multidimensional Scales

Researchers have the implicit option of calculating internal consistency reliability (coefficient α) for total scale scores derived from multidimensional inventories based on either the inter-item correlation matrix (item unit-level) or the inter-subscale correlation matrix (subscale unit-level). It is demonstrated that item unit-level and subscale unit-level reliability estimates often diverge substantially in practice. Specifically, the item unit-level reliability estimation is often larger than the corresponding subscale unit-level estimate. It is recommended that if researchers calculate total scale score reliability at the item unit-level, then a model-based approach to the estimation of internal consistency reliability (i.e., omega hierarchical) should be applied, when the underlying model is multidimensional. If omega hierarchical cannot be applied for any particular reason, it is recommended that total scale score reliabilities be calculated at the subscale unit-level of analysis, not the item unit-level.

PedsQL™ multidimensional fatigue scale in sickle cell disease: Feasibility, reliability, and validity

Sickle cell disease (SCD) is an inherited blood disorder characterized by a chronic hemolytic anemia that can contribute to fatigue and global cognitive impairment in patients. The study objective was to report on the feasibility, reliability, and validity of the PedsQL™ Multidimensional Fatigue Scale in SCD for pediatric patient self-report ages 5-18 years and parent proxy-report for ages 2-18 years. This was a cross-sectional multi-site study whereby 240 pediatric patients with SCD and 303 parents completed the 18-item PedsQL™ Multidimensional Fatigue Scale. Participants also completed the PedsQL™ 4.0 Generic Core Scales. The PedsQL™ Multidimensional Fatigue Scale evidenced excellent feasibility, excellent reliability for the Total Scale Scores (patient self-report α = 0.90; parent proxy-report α = 0.95), and acceptable reliability for the three individual scales (patient self-report α = 0.77-0.84; parent proxy-report α = 0.90-0.97). Intercorrelations of the PedsQL™ Multidimensional Fatigue Scale with the PedsQL™ Generic Core Scales were predominantly in the large (≥0.50) range, supporting construct validity. PedsQL™ Multidimensional Fatigue Scale Scores were significantly worse with large effects sizes (≥0.80) for patients with SCD than for a comparison sample of healthy children, supporting known-groups discriminant validity. Confirmatory factor analysis demonstrated an acceptable to excellent model fit in SCD. The PedsQL™ Multidimensional Fatigue Scale demonstrated acceptable to excellent measurement properties in SCD. The results demonstrate the relative severity of fatigue symptoms in pediatric patients with SCD, indicating the potential clinical utility of multidimensional assessment of fatigue in patients with SCD in clinical research and practice.

Psychometric properties of the Iranian version of the Pediatric Quality of Life Inventory™ Short Form 15 Generic Core Scales

The objective of this study was to assess the feasibility, reliability and validity of the Iranian version of the Pediatric Quality of Life Inventory™ 4.0 Short Form 15 Generic Core Scales (PedsQL™ 4.0 SF15) in a sample of Iranian schoolchildren and children with chronic health conditions. A total of 1236 schoolchildren and 1183 parents of schoolchildren participated, and 52 children with chronic health conditions and their parents were recruited from an outpatient clinic. Feasibility, internal consistency reliability, test-retest reliability, known-groups discriminant validity and construct validity of the PedsQL™ 4.0 SF15 were determined. The Iranian version of PedsQL™ 4.0 SF15 evidenced minimal missing responses for child self-report and parent proxy-report (0.4% and 0.6%, respectively), generally demonstrated no significant floor or ceiling effects, and achieved acceptable internal consistency reliability for the Total Scale Score (α = 0.82 child self-report, 0.84 parent proxy-report) and acceptable test-retest reliability. Schoolchildren and their parents reported significantly higher PedsQL™ 4.0 SF15 scores than paediatric patients with chronic health conditions. Child self-report and parent proxy-report showed poor to good agreement. A four-factor model was confirmed among child self-report and parent proxy-report in the confirmatory factor analyses. This study supports the feasibility, reliability and validity of the Iranian version of the PedsQL™ 4.0 SF15 among Iranian children and their parents.

Coefficient Alpha and Reliability of Scale Scores

The majority of large-scale assessments develop various score scales that are either linear or nonlinear transformations of raw scores for better interpretations and uses of assessment results. The current formula for coefficient alpha (α; the commonly used reliability coefficient) only provides internal consistency reliability estimates of raw scores. This article presents a general form of α and extends its use to estimate internal consistency reliability for nonlinear scale scores (used for relative decisions). The article also examines this estimator of reliability using different score scales with real data sets of both dichotomously scored and polytomously scored items. Different score scales show different estimates of reliability. The effects of transformation functions on reliability of different score scales are also explored.

Willing and Able: A Closer Look at Pain Willingness and Activity Engagement on the Chronic Pain Acceptance Questionnaire (CPAQ-8)

An 8-item version of the Chronic Pain Acceptance Questionnaire (CPAQ-8) has recently been proposed and validated. The aims of this study were to further investigate the reliability and validity of the CPAQ-8 in a new sample. Questionnaires were completed by 550 people with chronic pain (478 online survey, 72 paper survey). A demographic and pain history questionnaire was administered along with the CPAQ-8 and measures of pain self-efficacy, pain catastrophizing, psychological flexibility in pain, anxiety, and mood. In addition, 105 respondents completed the CPAQ-8 within 6 weeks to provide test-retest reliability data. The 2-factor structure of the CPAQ-8 (Activity Engagement [AE] and Pain Willingness [PW]) was confirmed and had reasonable-to-good scale score reliability and test-retest reliability. Pain acceptance as measured by the CPAQ-8 was associated with less depression, anxiety, pain interference, fear of reinjury, pain catastrophizing, and psychological inflexibility in pain, and higher levels of satisfaction with life, pain self-efficacy, and general acceptance. Furthermore, pain acceptance fully mediated the relationship between reported pain severity and emotional distress (anxiety and depression) and partially mediated the relationship between pain severity and pain interference in a structural equation model. The test-retest reliability after 4 to 6 weeks ranged from .68 for PW to .86 for AE; the overall score correlation was .81. We conclude that the CPAQ-8 is a reliable and valid measure of pain acceptance and that the 2 subscales of the measure each make an individual contribution to the prediction of adjustment in people with chronic pain. PerspectiveThe present study provides further evidence for the reliability and validity of the CPAQ-8. Support was found for the 2 related subscales, PW and AE, which appear to work in synergy to influence levels of pain interference and emotional distress in people living with chronic pain.

PedsQL™ sickle cell disease module: Feasibility, reliability, and validity

Sickle cell disease (SCD) is an inherited chronic disease that is characterized by complications such as recurrent painful vaso-occlusive events that require frequent hospitalizations and contribute to early mortality. The objective of the study was to report on the initial measurement properties of the new PedsQL™ SCD Module for pediatric patient self-report ages 5-18 years and parent proxy-report for ages 2-18 years. The 43-item PedsQL™ SCD Module was completed in a multisite study by 243 pediatric patients with SCD and 313 parents. Participants also completed the PedsQL™ 4.0 Generic Core Scales and PedsQL™ Multidimensional Fatigue Scale. The PedsQL™ SCD Module Scales evidenced excellent feasibility, excellent reliability for the Total Scale Scores (patient self-report α = 0.95; parent proxy-report α = 0.97), and good reliability for the nine individual scales (patient self-report α = 0.69-0.90; parent proxy-report α = 0.83-0.97). Intercorrelations with the PedsQL™ Generic Core Scales and PedsQL™ Multidimensional Fatigue Scales were medium (0.30) to large (0.50) range, supporting construct validity. PedsQL™ SCD Module Scale Scores were generally worse for patients with severe versus mild disease. Confirmatory factor analysis demonstrated an acceptable to excellent model fit. The PedsQL™ SCD Module demonstrated acceptable measurement properties. The PedsQL™ SCD Module may be utilized in the evaluation of SCD-specific health-related quality of life in clinical research and practice. In conjunction with the PedsQL™ Generic Core Scales and the PedsQL™ Multidimensional Fatigue Scale, the PedsQL™ SCD Module will facilitate the understanding of the health and well-being of children with SCD.

Assessing public speaking fear with the short form of the Personal Report of Confidence as a Speaker scale: confirmatory factor analyses among a French-speaking community sample

Background:The main aim of this study was to assess the reliability and structural validity of the French version of the 12-item version of the Personal Report of Confidence as Speaker (PRCS), one of the most promising measurements of public speaking fear.Methods:A total of 611 French-speaking volunteers were administered the French versions of the short PRCS, the Liebowitz Social Anxiety Scale, the Fear of Negative Evaluation scale, as well as the Trait version of the Spielberger State-Trait Anxiety Inventory and the Beck Depression Inventory-II, which assess the level of anxious and depressive symptoms, respectively.Results:Regarding its structural validity, confirmatory factor analyses indicated a single-factor solution, as implied by the original version. Good scale reliability (Cronbach’s alpha = 0.86) was observed. The item discrimination analysis suggested that all the items contribute to the overall scale score reliability. The French version of the short PRCS showed significant correlations with the Liebowitz Social Anxiety Scale (r = 0.522), the Fear of Negative Evaluation scale (r = 0.414), the Spielberger State-Trait Anxiety Inventory (r = 0.516), and the Beck Depression Inventory-II (r = 0.361).Conclusion:The French version of the short PRCS is a reliable and valid measure for the evaluation of the fear of public speaking among a French-speaking sample. These findings have critical consequences for the measurement of psychological and pharmacological treatment effectiveness in public speaking fear among a French-speaking sample.

Assessing dispositional empathy in adults: A French validation of the Interpersonal Reactivity Index (IRI).

The goal of this study was to validate a French version of the Interpersonal Reactivity Index (IRI), a self-report questionnaire comprised of four subscales assessing affective (empathic concern and personal distress) and cognitive (fantasy and perspective taking) components of empathy. To accomplish this, 322 adults (18 to 89 years) completed the French version of the IRI (F-IRI). A confirmatory factor analysis confirmed the four-factor structure of the original IRI. The F-IRI showed good scale score reliability, test–retest reliability, and convergent validity, tested with the French version of the Empathy Quotient. These findings confirmed the reliability and validity of the F-IRI and suggest that the F-IRI is a useful instrument to measure self-reported empathy. In addition, we observed sex and age differences consistent with findings in the literature. Women reported higher scores in empathic concern and fantasy than men. Older adults reported less personal distress and less fantasy. (PsycINFO Database Record (c) 2013 APA, all rights reserved). (journal abstract)

Effects of Augmenting Response Options on MMPI–2 RC Scale Psychometrics

A number of empirical investigations indicate that tests with a greater number of response options tend to yield better psychometric performance. We hypothesized that a version of the MMPI–2 with a polytomous response format would outperform the standard dichotomous format in terms of observed score reliability and validity. Two versions of the MMPI–2 RC scales were administered consecutively in counterbalanced order to 199 undergraduate students attending a large Midwestern university: the standard true–false version, and an experimental version containing 4 response options (very true, mainly true, slightly true, and false, not at all true). After participants completed both versions, 2 scales from the Multidimensional Personality Questionnaire (MPQ) were administered to assess differences in convergent validity. Results showed enhancements in reliability for all RC scale scores and increases in the convergent validity of scores. Directions for further investigation and potential implications for future test development are discussed.

Development of Sport Courage Scale

While theory and practice of sport have much to say about fear, stress and anxiety, they have little to say about courage. Therefore, the purpose of this study was to develop a Sport Courage Scale. Data were collected from two groups of male and female athletes aged from 13 to 22 in different individual and team sports. The first set of data (N = 380) was analyzed by exploratory factor analysis, and the second set of data (N = 388) was analyzed by confirmatory factor analysis. Analyses revealed a 5-factor structure of Sport Courage Scale that supported factorial validity and reliability of scale scores. These factors were labelled: “Determination”, “Mastery”, “Assertiveness”, “Venturesome”, and “Self-Sacrifice Behaviour”. Finally, evidence of test-retest reliability of scale scores was supported based on responses from 75 athletes. However, more research is needed to further improve the Sport Courage Scale.

Development and validation of the Vicarious Distress Questionnaire.

The ability to empathize with other people’s feelings of distress has been widely studied in psychology. However, no questionnaire to date has been developed in order to measure such distress responses and their behavioural correlates. Three studies describe the development and the validation of the Vicarious Distress Questionnaire (VDQ), which is a self-report measure that assesses participants’ distress responses as well as their consequences (support or avoidance) in response to another’s distress. In Study 1, we developed items and assessed the factor structure of the VDQ. Study 2 presented a confirmatory factor analysis that supported the three-factor model (Distress, Support, and Avoidance factors) and showed good scale score reliability. Study 3 showed significant correlations among the factors of the VDQ and measures of affective empathy, anxiety disorders, alexithymia, and mood disorders. The Distress factor of the VDQ also showed satisfactory discriminant validity with the cognitive factors of empathy (i.e., Perspective Taking and Fantasy). Taken together, these results provide comprehensive support for the validity and reliability of the VDQ.

Gender Role Conflict Scale: Validation for a Sample of Gay Men and Lesbian Women

This study tested the oblique four-factor model of the Gender Role Conflict Scale for a sample of gay men and lesbian women residing in the United States. 400 gay men and 292 lesbian women recruited from university and college gay, lesbian, bisexual, and transgender LISTSERVs participated. The internal consistency reliability of the Gender Role Conflict Scale scores was high, but low means on the expressive dimension of gender role conflict were noted. The results of two separate sets of confirmatory factor analyses indicated that the four-factor model's fit to the data could be enhanced for both groups by applying item parceling to lower the number of indicators per factor, suggesting that the actual structural validity of the Gender Role Conflict Scale may be better than suggested by the reported fit indices.

The Revised Child Anxiety and Depression Scale-Short Version: Scale reduction via exploratory bifactor modeling of the broad anxiety factor.

Using a school-based (N = 1,060) and clinic-referred (N = 303) youth sample, the authors developed a 25-item shortened version of the Revised Child Anxiety and Depression Scale (RCADS) using Schmid-Leiman exploratory bifactor analysis to reduce client burden and administration time and thus improve the transportability characteristics of this youth anxiety and depression measure. Results revealed that all anxiety items primarily reflected a single "broad anxiety" dimension, which informed the development of a reduced 15-item Anxiety Total scale. Although specific DSM-oriented anxiety subscales were not included in this version, the items comprising the Anxiety Total scale were evenly pulled from the 5 anxiety-related content domains from the original RCADS. The resultant 15-item Anxiety Total scale evidenced significant correspondence with anxiety diagnostic groups based on structured clinical interviews. The scores from the 10-item Depression Total scale (retained from the original version) were also associated with acceptable reliability in the clinic-referred and school-based samples (α = .80 and .79, respectively); this is in contrast to the alternate 5-item shortened RCADS Depression Total scale previously developed by Muris, Meesters, and Schouten (2002), which evidenced depression scores of unacceptable reliability (α = .63). The shortened RCADS developed in the present study thus balances efficiency, breadth, and scale score reliability in a way that is potentially useful for repeated measurement in clinical settings as well as wide-scale screenings that assess anxiety and depressive problems. These future applications are discussed, as are recommendations for continued use of exploratory bifactor modeling in scale development.

Psychometric properties and construct validity of the Muscle Appearance Satisfaction Scale among Hungarian men

Limited studies have evaluated the psychometric properties of the Muscle Appearance Satisfaction Scale (MASS), a measure of muscle dysmorphia, in different cultures and languages. The aims were to examine the psychometric properties of the Hungarian version of the MASS (MASS-HU), and to investigate its relationship with self-esteem and exercise-related variables. Two independent samples of male weight lifters ( ns = 289 and 43), and a sample of undergraduates ( n = 240) completed the MASS, Eating Disorder Inventory, and Rosenberg Self-esteem Scale. Exploratory factor analysis supported the original five-factor structure of the MASS only in the weight lifter sample. The MASS-HU had excellent scale score reliability and good test–retest reliability. The construct validity of the MASS-HU was tested with multivariate regression analyses which indicated an inverse relationship between self-esteem and muscle dysmorphia. The 18-item MASS-HU was found to be a useful measure for the assessment of muscle dysmorphia among male weight lifters.

Cross-validation of the very short form of the Physical Self-Inventory (PSI-VS): Invariance across genders, age groups, ethnicities and weight statuses

In a recent review of various physical self-concept instruments, Marsh and Cheng (in press) noted that the very short 12-item version of the French Physical Self-Inventory (PSI-VS) represents an important contribution to applied research but that further research was needed to investigate the robustness of its psychometric properties in new and diversified samples. The present study was designed to answer these questions based on a sample of 1103 normally achieving French adolescents. The results show that the PSI-VS measurement model is quite robust and fully invariant across subgroups of students formed according to gender, weight, age and ethnicity. The results also confirm the convergent validity and scale score reliability of the PSI-VS subscales.

Development and Validation of a Scale Measuring Attitudes Toward Non-Drinkers

The idea that individuals drink alcohol to fit in with their peers has been investigated by many researchers. However, the related concept that consumption of alcohol may serve as a means of avoiding the social costs associated with being a non-drinker has received little attention. Three studies (N = 94, 148, 236) are outlined, which detail the construction and preliminary validation of the Regan Attitudes toward Non-Drinkers Scale (RANDS). Results indicated that scale score reliability for the RANDS was good (α values range from .82 to .89) with exploratory and confirmatory factor analyses suggesting that the scale possesses a unidimensional factor structure. Importantly, scores on the RANDS emerged as a stronger predictor of self-reported yearly alcohol consumption and binge-drinking than indicants commonly assessed in alcohol use and abuse research in adolescents and young adults (e.g., peer pressure). Limitations of these studies and directions for future research are outlined.

The PedsQL™ Infant Scales: feasibility, internal consistency reliability, and validity in healthy and ill infants

The PedsQL™ (Pediatric Quality of Life Inventory™) is a modular instrument designed to measure health-related quality of life (HRQOL) and disease-specific symptoms in children and adolescents ages 2-18. The new PedsQL™ Infant Scales were designed as a generic HRQOL instrument specifically for healthy and ill infants ages 1-24 months. The objective of this study was to report on the initial feasibility, internal consistency reliability, and validity of the PedsQL™ Infant Scales in healthy, acutely ill, and chronically ill infants. The 36-item (ages 1-12 months) and 45-item (ages 13-24 months) PedsQL™ Infant Scales (Physical Functioning, Physical Symptoms, Emotional Functioning, Social Functioning, Cognitive Functioning) were completed by 683 parents of healthy, acutely ill, and chronically ill infants. The PedsQL™ Infant Scales evidenced minimal missing responses, achieved excellent internal consistency reliability for the Total Scale Scores (α = 0.92), distinguished between healthy infants and acutely and chronically ill infants, and demonstrated a confirmatory factor structure largely consistent with the a priori conceptual model. The results demonstrate the initial measurement properties of the PedsQL™ Infant Scales in healthy and ill infants. The findings suggest that the PedsQL™ Infant Scales may be utilized in the evaluation of generic HRQOL in infants ages 1-24 months.