Strong Psychometric Evidence Research Articles

Measurement invariance of the Youth Self-Report across youth who have committed sexual and nonsexual offenses.

Justice-involved youth experience high rates of mental health problems that require proper screening and assessment in order to effectively intervene. The Youth Self-Report (YSR) is a general psychopathology rating scale that measures several dimensions of psychopathology and is commonly used in clinical assessments, including with justice-involved youth. Yet, the underlying factor structure of the YSR has not been examined specifically in a sample of justice-involved youth. We examined the factor structure of the YSR using confirmatory factor analysis with a sample of 961 male youth involved with the justice system (12-18 years of age). Measurement invariance of the YSR was also examined across groups of youth who committed a sexual offence and those who committed a nonsexual offence. The eight-factor model presented with optimal fit to the data, consistent with previous research with nonjustice involved samples, and the model demonstrated strong measurement invariance across youth who committed both types of offenses (sexual and nonsexual). Youth who committed nonsexual offenses reported significantly higher degrees of rule-breaking behavior and lower degrees of social problems than youth who committed sexual offenses. The current findings provide strong psychometric evidence that supports the use of the YSR with justice-involved male youth. As such, clinicians and researchers can be confident in using the YSR as a mental health screening tool with male youth involved with the justice system who have committed various offenses. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Attitudes toward osteopathic medicine scale: development and psychometrics.

To develop a valid and reliable instrument for measuring attitudes toward osteopathic medicine. Participants included 5,669 first-year students from 33 U.S. colleges of osteopathic medicine, who completed an online survey at the beginning of the 2019-2020 academic year. Using data from the nationwide Project in Osteopathic Medical Education and Empathy, we developed a 13-item instrument: Attitudes Toward Osteopathic Medicine Scale (ATOMS) and demonstrated the validity and reliability of its scores. The social desirability response bias was controlled in statistical analyses. The corrected item-total score correlations were all positive and statistically significant, and the effect sizes of item discrimination indices were large. Cronbach's coefficient alpha reliability was 0.83. Construct validity, corroborating face and content validity of the ATOMS, was supported by three components, emerged from factor analysis: "Perspectives on Osteopathic Medicine," "Osteopathic Diagnosis and Treatment," and "Holistic-Integrative Care." Correlations between ATOMS scores and scores of cognitive empathy, emotional empathy; orientation toward interprofessional collaboration; lifelong learning; and burnout were statistically significant in the expected direction, providing validity evidence for the ATOMS. Using the method of contrasted groups, significant differences in the ATOMS scores were found by gender, ethnicity, academic background, and career interest in the expected direction, supporting the validity of the ATOMS scores. National norms were developed to assess individual scores alongside national percentile ranks. The ATOMS, developed in a nationwide study, supported by strong psychometric evidence for measuring orientation toward osteopathic medicine, has implications for the assessment of osteopathic medical education, patient outcomes, and admission decisions.

Use and detailed metric properties of patient-reported outcome measures for rheumatoid arthritis: a systematic review covering two decades

IntroductionThe importance of patient-reported outcome measures (PROMs) for rheumatoid arthritis (RA) clinical studies has been recognised for many years. The current study aims to describe the RA PROMs used over...

Psychometric properties of the Porto Saudade Scale

Saudade involves psychosocial reactions to several circumstances involving deprivation from loved people and/or familiar locations. This work concerns the development and preliminary validation of an instrument to assess general disposition to saudade, the Porto Saudade Scale (PSS). Three studies were conducted which indicated that the PSS has favorable psychometric properties. Exploratory factor analysis of the PSS demonstrated a single latent saudade factor. The confirmatory factor analysis showed that the single latent model of the PSS is adequate and possesses good internal consistency. As expected, the data also supported the convergent, discriminant, and external validity of the PSS. Saudade was more frequently reported among females than among males. These results suggest the validity of the PSS and emphasize that it is a brief measure with strong psychometric evidence for assessing saudade.

Instruments for assessing organisational capacity for use of evidence in health sector policy making: a systematic scoping review

Background Assessing individual and organisational capacity for evidence use is essential for developing evidence-based strategies. Aims and objectives This study aimed to assess the psychometric and practical properties of existing tools to assess the capacity to use evidence at the individual and organisational levels of health policy and identify the best instruments. Methods A systematic review of the databases of ISI Web of Science, Embase, Scopus, and PubMed was conducted up to 6 June 2018. Search engines, websites of key organisations, and the reference lists of selected articles were also used to find relevant studies. The search strategy for each database was written individually. Psychometric properties were assessed using the Standards for Educational and Psychological Testing (SEPT) and pragmatic properties were assessed using the protocol proposed by Lewis et al (2018). Simple statistics were used to describe the psychometric and pragmatic properties of the identified instruments. Results Overall, 16 instruments were identified. SEER had the highest validity score. Reliability was estimated for 38% of the instruments. Responsiveness was assessed in only 19% of the studies. The results showed that internal consistency was reported for 40% of the articles using statistical analysis. Pragmatic properties of the studied instruments were verified; 75% usefulness, 56% actionability, 50% sensitive to change and 56% user-friendly, 19% compatible, 38% feasibility. Discussion and conclusions There are few instruments with strong psychometric evidence, and without high-quality instruments, it would be difficult to determine the factors that affect implementation. Therefore, special attention is needed for the systematic development of instruments and their reporting standards.

An item response theory analysis of an item pool for the recovering quality of life (ReQoL) measure

PurposeReQoL-10 and ReQoL-20 have been developed for use as outcome measures with individuals aged 16 and over, experiencing mental health difficulties. This paper reports modelling results from the item response theory (IRT) analyses that were used for item reduction.MethodsFrom several stages of preparatory work including focus groups and a previous psychometric survey, a pool of items was developed. After confirming that the ReQoL item pool was sufficiently unidimensional for scoring, IRT model parameters were estimated using Samejima’s Graded Response Model (GRM). All 39 mental health items were evaluated with respect to item fit and differential item function regarding age, gender, ethnicity, and diagnosis. Scales were evaluated regarding overall measurement precision and known-groups validity (by care setting type and self-rating of overall mental health).ResultsThe study recruited 4266 participants with a wide range of mental health diagnoses from multiple settings. The IRT parameters demonstrated excellent coverage of the latent construct with the centres of item information functions ranging from − 0.98 to 0.21 and with discrimination slope parameters from 1.4 to 3.6. We identified only two poorly fitting items and no evidence of differential item functioning of concern. Scales showed excellent measurement precision and known-groups validity.ConclusionThe results from the IRT analyses confirm the robust structure properties and internal construct validity of the ReQoL instruments. The strong psychometric evidence generated guided item selection for the final versions of the ReQoL measures.

High-level motor skills assessment for ambulant children with cerebral palsy: a systematic review and decision tree.

To examine the psychometric evidence for high-level motor skills assessment tools for ambulant, school-aged children with cerebral palsy (CP). We searched five databases for population (children with CP aged 5-18y in Gross Motor Function Classification System levels I and II), assessment focus (high-level motor skills), and psychometric evidence. We evaluated evidence strength using the number of studies, quality, and conduct according to COnsensus-based Standards for the selection of health status Measurement INstruments checklists. Eleven assessments (39 studies) met the criteria. Seven high-level motor skills assessment items (Muscle Power Sprint Test, 10m Shuttle Run Test, 10×5m Sprint Test, vertical jump, standing broad jump, seated throw, and Timed Up and Down Stairs) had strong validity and responsiveness evidence. Jumping items and seated throw lacked reliability data. Four high-level motor skills assessment batteries (Functional Strength Measure in CP, Gross Motor Function Measure-Challenge, Peabody Developmental Motor Scale, and Test of Gross Motor Development, Second Edition) had moderate-to-strong validity and/or reliability evidence. Responsiveness data were only available for the Gross Motor Function Measure-Challenge battery. The decision tree was developed with five levels: clinical feasibility, relevance, tool design, clinical utility, and psychometric properties. High-level motor skills assessment tools have strong psychometric evidence for ambulant, school-aged children with CP. The decision tree can assist clinicians and researchers in identifying appropriate tools to measure high-level motor skills. An evidence-based decision tree guides the selection of appropriate high-level motor skills assessment tools. Seven high-level motor skills assessment items have strong psychometric evidence and clinical utility for ambulant children with cerebral palsy. Four high-level motor skills assessment batteries with recreation and mobility items have emerging psychometric evidence in this population.

A Systematic Review of Critical Thinking Instruments for Use in Dental Education.

Critical thinking is widely recognized as an essential competency in dental education, but there is little agreement on how it should be assessed. The aim of this systematic review was to determine the availability of instruments that could be used to measure critical thinking in dental students and to evaluate psychometric evidence to support their use. In January 2017, an electronic search of both the medical and education literature was performed on nine databases. The search included both keyword and Medical Subject Heading terms for critical thinking, higher education/health sciences education, measurement/assessment, and reproducibility of results. The grey literature was included in the search. The search produced 2,977 unique records. From the title and abstract review, 183 articles were selected for further review, which resulted in 36 articles for data extraction. Ten of these studies sought to evaluate psychometric properties of the instruments used and were subjected to quality assessment. Seven assessment instruments were identified. Of these, three instruments that have not been widely used nor tested in health professions students showed evidence of psychometric strength and appeared to have potential for use in dental education. Further research should focus on the three critical thinking instruments with strong psychometric evidence, with the aim of establishing validity and reliability in the context of dental education.

Common Data Elements in the Assessment of Military-Related PTSD Research Applied in the Consortium to Alleviate PTSD.

Driven by the need to share data, sufficiently power studies, and allow for cross-study comparisons of medical and psychiatric diseases, the President's National Research Action Plan issued in 2013 called for the use of state-of-the-art common data elements (CDEs) for research studies. CDEs are variables measured across independent studies that facilitate methodologically sound data aggregation and study replication. Researchers in the field of military-related post-traumatic stress disorder (PTSD) have suggested applicable CDEs; however, to date, these recommendations have been conceptual and not field-tested. The Consortium to Alleviate PTSD (CAP) - an interdisciplinary and multi-institutional, military-related PTSD research consortium funded by the Departments of Defense and Veterans Affairs - generated and applied CDEs that can be used to combine data from disparate studies to improve the methodological and statistical capabilities of study findings. We provide a description and rationale for the CAP CDEs and details about administration with two main goals: (1) to encourage military-related PTSD researchers to use these measures in future studies and (2) to facilitate comparison, replication, and data aggregation. The CAP compiled mandated (core) and optional CDEs based on the following criteria: (1) construct applicability to military-related PTSD; (2) precedence (use) in prior, related research; (3) published and strong psychometric evidence; (4) no cost (public domain); and (5) brevity, to limit participant burden. We provided descriptive statistics and internal consistency reliabilities for mandated measures from an initial cohort of around 400 participants enrolled in CAP studies. Mandated CDEs in the CAP were found to have very good internal consistency reliability. Although further research is needed to determine the incremental validity of these CDEs, preliminary analyses indicated that each mandated measure has very good internal consistency reliability. Investigators designing military-related PTSD research should consider using these field-tested CDEs to facilitate future data aggregation. Feedback based on empirical evidence or practical concerns to improve these CDEs is welcome.

Adolescent and Adult Time Inventory-Time Attitude Scales: Validity and Contributions to Physical Activity and Self-Concept in Spanish Adolescents

In this study, we examined the psychometric properties for the time attitude subscales of the Adolescent and Adult Time Inventory-Time Attitude Scale–Spanish (AATI-TA–Spanish). We sought to determine the internal consistency, structural validity, gender and age invariance, and convergent validity of scores on the six subscales: Past Positive, Past Negative, Present Positive, Present Negative, Future Positive, and Future Negative. Participants were 789 Spanish adolescents. Reliability estimates indicated high internal consistency across the subscales. Confirmatory factor analyses provided strong support for the theorized 6-factor structure. Configural, metric, and scalar invariance was demonstrated for gender and early and late adolescent groups. Convergent validity was shown for physical activity and self-concept. Overall, results provided strong psychometric evidence that Adolescent and Adult Time Inventory-Time Attitude Scale–Spanish scores may be used effectively with Spanish adolescents.

Internal brand orientation: conceptualisation, scale development and validation

ABSTRACTInternal branding is increasingly acknowledged as a potential route to acquiring sustainable competitive advantage. However, to date no systematic attempt has been made to operationalise from an organisational perspective all imperatives for achieving brand internalisation. To address this omission, we introduce internal brand orientation as a new construct that provides a measurable and more succinct picture of internal branding. Following the literature in measurement theory, we report a series of complementary studies and develop a scale which comprises two dimensions: top management brand commitment and shared brand values. The new scale demonstrates strong psychometric evidence of reliability and validity, facilitating future research in the area. Findings can be used as a guidance by business executives in their effort to turn employees into brand ambassadors and motivate everyone in the organisation towards brand supportive behaviours.

Validity of Brief Cognitive Assessment Tool modifications for older adults with visual and motor limitations

ABSTRACTIntroduction: The prevalence of cognitive, sensory, and physical impairments is expected to grow alongside increasing life expectancy. These chronic conditions pose challenges for geriatric assessment. We examined whether Brief Cognitive Assessment Tool (BCAT) modifications to accommodate visual and motor limitations would retain strong validity for identifying mild cognitive impairment (MCI) and dementia.Method: Psychometric analyses were performed on archival data (N = 458) from community-dwelling older adults and residents of assisted living facilities and nursing homes in Maryland, USA (age ≥ 50 years). Participants completed a brief testing battery, including the BCAT, and were assigned a cognitive diagnosis (normal cognition, MCI, mild dementia, moderate dementia, severe dementia) by licensed clinical psychologists.Results: Receiver operator characteristic curve analyses evidenced excellent diagnostic validity of BCAT modification cutoffs for identifying the cognitive categories. Contextual memory and executive control factors, which explained over 80% of variance in cognitive diagnoses, may account for the preservation of validity despite BCAT modifications.Conclusions: The results indicate strong psychometric evidence for the BCAT modifications and provide cutoffs for identifying MCI and staging dementia. For clinicians, the score guidelines are preferable to the guesswork involved with adjusting cutoffs to accommodate visual and motor limitations.

A Study on The Validity and Reliability of The Personality Belief Questionnaire-Short Form in Turkish Community Sample

Objective: This study aims to test psychometric properties of The Personality Belief Questionnaire-Short Turkish Form (PBQ-STF) in the community sample and to determine the validity and reliability of the Borderline Personality Disorder subscale, which was not included in the previous Turkish adaptation studies. Method: PBQ-STF was administered to a sample of 650 participants (in the age range of 18-66 years, 362 women (55.7%) and 288 men (44.3%)). Test-retest reliability was applied on a group of 30 person from sample with interval four weeks. For testing convergent validity, Beck Depression Inventory (n=108), State-Trait Anxiety Inventory (n=100) and Dysfunctional Attitude Scale (n=108) were also employed. Results: Cronbach’s alpha values varied between .70-.84. Test-retest reliabilities were .77-.90. PBQ-SF’s all subscales were correlated each other positively. In the exploratory factor analysis (EFA), PBQ-STF subscales revealed two factors explained 70,89% of the total variance. In the confirmatory factor analysis (CFA), it was seen adaptive statistical values of scale were at acceptable/agreeable level. Conclusion: According to the results obtained from the analyzes made, it was determined that PBQ-STF has strong psychometric evidence and is a reliable and valid scale that can be used in evaluating personality beliefs.

Child and adolescent self-report symptom measurement in pediatric oncology research: a systematic literature review.

Previous work in pediatric oncology has found that clinicians and parents tend to under-report the frequency and severity of treatment-related symptoms compared to child self-report. As such, there is a need to identify high-quality self-report instruments to be used in pediatric oncology research studies. This study's objective was to conduct a systematic literature review of existing English language instruments used to measure self-reported symptoms in children and adolescents undergoing cancer treatment. A comprehensive literature search was conducted in MEDLINE/PubMed, EMBASE, CINAHL, and PsycINFO to identify relevant articles published through November 10, 2016. Using pre-specified inclusion/exclusion criteria, six trained reviewers carefully screened abstracts and full-text articles for eligibility. There were 7738 non-duplicate articles identified in the literature search. Forty articles met our eligibility criteria, and within these articles, there were 38 self-report English symptom instruments. Most studies evaluated only cross-sectional psychometric properties, such as reliability or validity. Ten studies assessed an instrument's responsiveness or ability to detect changes in symptoms over time. Eight instruments met our criteria for use in future longitudinal pediatric oncology studies. This systematic review aids pediatric oncology researchers in identifying and selecting appropriate symptom measures with strong psychometric evidence for their studies. Enhancing the child's voice in pediatric oncology research studies allows us to better understand the impact of cancer and its treatment on the lives of children.

Severity of Nasal Inflammatory Disease Questionnaire for Canine Idiopathic Rhinitis Control: Instrument Development and Initial Validity Evidence.

BackgroundEffective treatments are needed for idiopathic chronic rhinitis in dogs, but assessment of efficacy requires a practical, quantifiable method for assessing severity of disease.ObjectivesTo develop and perform initial validity and reliability testing of an owner‐completed questionnaire for assessing clinical signs and dog and owner quality of life (QOL) in canine chronic rhinitis.AnimalsTwenty‐two dogs with histopathologically confirmed chronic rhinitis and 72 healthy dogs.MethodsIn this prospective study, an online questionnaire was created based on literature review and feedback from veterinarians, veterinary internists with respiratory expertise, and owners of dogs with rhinitis. Owners of affected dogs completed the questionnaire twice, 1 week apart, to test reliability. Healthy dogs were assessed once. Data were analyzed using the Rasch Rating Scale Model, and results were interpreted using Messick's framework for evaluating construct validity evidence.ResultsInitial item generation resulted in 5 domains: nasal signs, paranasal signs, global rhinitis severity, and dog's and owner's QOL. A 25‐item questionnaire was developed using 5‐point Likert‐type scales. No respondent found the questionnaire difficult to complete. Strong psychometric evidence was available to support the substantive, generalizability, content, and structural aspects of construct validity. Statistical differences were found between responses for affected and control dogs for all but 2 items. These items were eliminated, resulting in the 23‐item Severity of Nasal Inflammatory Disease (SNIFLD) questionnaire.Conclusions and Clinical ImportanceThe SNIFLD questionnaire provides a mechanism for repeated assessments of disease severity in dogs with chronic rhinitis.

Psychometric properties of time attitude scores in young, middle, and older adult samples

In an effort to provide a measure of time perspective that can be used across the life-span, we examined the psychometric properties of the Adolescent Time Inventory-Time Attitude Scale (ATI-TA; Mello & Worrell, 2007) in three independent samples of young (N=388), middle (N=201), and older adults (N=189). Results provided strong psychometric evidence that the ATI-TA can be used appropriately with individuals across adolescence and adulthood. Specifically, internal consistency estimates indicated that scores on the six subscales (Past Positive, Past Negative, Present Positive, Present Negative, Future Positive, & Future Negative) were reliable across the three samples. Confirmatory factor analyses showed that the theorized six-factor structure had acceptable fit and fit the data better than alternate models. Subsequent analyses provided support for invariance across young, middle, and older adults. Overall, these results show that the ATI-TA yields reliable scores and a valid structure across adulthood and can be used to measure time perspective throughout the life-span.

Measurement properties of painDETECT by average pain severity.

BackgroundSince the burden of neuropathic pain (NeP) increases with pain severity, it is important to characterize and quantify pain severity when identifying NeP patients. This study evaluated whether painDETECT, a screening questionnaire to identify patients with NeP, can distinguish pain severity.Materials and methodsSubjects (n=614, 55.4% male, 71.8% white, mean age 55.5 years) with confirmed NeP were identified during office visits to US community-based physicians. The Brief Pain Inventory – Short Form stratified subjects by mild (score 0–3, n=110), moderate (score 4–6, n=297), and severe (score 7–10, n=207) average pain. Scores on the nine-item painDETECT (seven pain-symptom items, one pain-course item, one pain-irradiation item) range from −1 to 38 (worst NeP); the seven-item painDETECT scores (only pain symptoms) range from 0 to 35. The ability of painDETECT to discriminate average pain-severity levels, based on the average pain item from the Brief Pain Inventory – Short Form (0–10 scale), was evaluated using analysis of variance or covariance models to obtain unadjusted and adjusted (age, sex, race, ethnicity, time since NeP diagnosis, number of comorbidities) mean painDETECT scores. Cumulative distribution functions on painDETECT scores by average pain severity were compared (Kolmogorov–Smirnov test). Cronbach’s alpha assessed internal consistency reliability.ResultsUnadjusted mean scores were 15.2 for mild, 19.8 for moderate, and 24.0 for severe pain for the nine items, and 14.3, 18.6, and 22.7, respectively, for the seven items. Adjusted nine-item mean scores for mild, moderate, and severe pain were 17.3, 21.3, and 25.3, respectively; adjusted seven-item mean scores were 16.4, 20.1, and 24.0, respectively. All pair-wise comparisons of scores between pain-severity groups showed sizable and statistically significant differences (P<0.0001). Cumulative distribution functions showed distinct separation between severity (P<0.0001). Cronbach’s alphas were 0.76 and 0.80 for the nine- and seven-item scales, respectively.ConclusionThis study provides strong psychometric evidence on the validity and reliability of painDETECT for distinguishing average pain severity in patients with NeP.

Development of the Social Efficacy and Social Outcome Expectations Scale

The current study developed an 18-item scale measuring individuals’ social expectations in relationships related to their efficacy expectations (Subscale 1) and outcome expectations (Subscale 2) based on Bandura’s self-efficacy theory. Results from exploratory and confirmatory factor analyses, using an undergraduate sample (N = 486), revealed strong psychometric evidence for this scale.

Psychometric Considerations for Alternate Assessments Based on Modified Academic Achievement Standards

Because of the unique nature of the students eligible for alternate assessments based on modified academic achievement standards, their varied access to the general education curriculum, and their unique learning needs, innovative psychometric thinking and practice is needed to assure high technical quality of alternate assessments. Indeed, we at least must marshal state-of-the-art procedures to secure strong psychometric evidence to support appropriate and meaningful design and use of these important assessments. The authors contributing work to this special issue, Alternate Assessments Based on Modified Academic Achievement Standards, address important issues and provide guidance to policymakers, test developers, and educators. They also each raise important technical quality issues. This article offers a brief review of such psychometric considerations, in light of the work and comments of the special issue authors.