Evidence Of Convergent Validity Research Articles

The Heads Up Checkup Digital Mental Health & Behavioral Risk Screening System: Clinical Inter‐Rater Reliability for Identifying Youth in Crisis

ABSTRACTThe present study evaluated the inter‐rater reliability of the Heads Up Checkup (HCU), a brief digital mental health and behavioral adaptive screening system designed for use in primary care and diverse school settings. Two independent licensed clinical psychologists reviewed a random sample of 30 (N = 30) HCU clinical screening reports of 13−14 year old adolescents drawn from a larger sample (N = 846) enrolled in a public middle school in California. Results showed strong inter‐rater agreement (Fleiss kappa = 0.93) between clinician ratings and the screener's priority risk index (HPI) in identifying students “in crisis.” In addition, clinicians' ratings of confidence in their priority judgments were found to be significantly higher for the “in crisis” cases. Reasonable evidence of convergent validity emerged due to a strong relationship between clinician ratings of psychological distress and the HPI. Overall findings suggest that as an online universal school‐based screener, the HCU has valid utility for identifying young adolescents “in crisis” which can translate into timely interventions and pragmatic real‐world therapeutic solutions. Future research directions with respect to the refinement of the HCU's measurement characteristics and its feasibility as an online screener at the population‐level in schools are discussed.

Creating a Psychological Test in a Few Seconds

Abstract: The development of Situational Judgment Tests (SJTs) is a multi-step procedure that often requires substantial resources and time. ChatGPT as a large language model has already proven helpful for many complex tasks, including the development of psychological questionnaires. However, ChatGPT has so far not been tested for its ability to develop SJT items, which could speed up SJT development and thus contribute to the dissemination of psychometrically well-designed SJTs in practice and also support research requiring specific SJTs or SJT versions. Thus, the current study ( N = 419) examined whether ChatGPT (2023, Feb 13 version using ChatGPT-3.5) can create SJT items, assessing a facet of personality (i.e., gregariousness), that show similar psychometric properties (examined through confirmatory factor, reliability, and correlational analyses) as human-created SJT items. Results revealed that the measurement model for all eleven SJT items that were created by ChatGPT showed a poor model fit, while a reduced set of eight items yielded an acceptable fit. Reliability estimates as well as convergent and discriminant validity evidence were similar for the human-created and the ChatGPT-created SJT. Implications as well as potential boundary conditions of SJT development through ChatGPT are discussed.

A psychometric examination of the cognitive flexibility scale and its association with Orthorexia Nervosa

BackgroundThe Cognitive Flexibility Scale (CFS) is a 12-item self-report questionnaire designed to assess cognitive flexibility. Despite its widespread use, its psychometric properties have yet to be evaluated in the Italian context. Furthermore, while cognitive flexibility has emerged as a central correlate in Feeding and Eating Disorders, only a limited number of studies have investigated its association with Orthorexia Nervosa (ON), a clinical condition characterized by a pathological fixation with healthy eating. The present study aimed to fill these two knowledge gaps.MethodsA total of 803 participants (Mage = 33.89, SD = 9.44; 68.6% females) were enrolled in the investigation. The sample was randomly split into two subsamples: the first one for examining the psychometric properties of the CFS, and the second one for evaluating its association with ON symptoms. Participants completed self-report questionnaires assessing the constructs under investigation.ResultsConfirmatory factor analysis revealed a global dimension of cognitive flexibility, alongside a method factor accounting for covariance arising by reverse-worded items. The CFS, which was factorially invariant across genders, yielded a reliable total score (ω = 0.834) and provided evidence of convergent and criterion-related validity. Importantly, structural equation modelling showed that cognitive flexibility was negatively associated with emotional distress resulting from violations of orthorexic dietary rules (β = -0.279, p < .001).ConclusionsThe CFS demonstrated to be a psychometrically robust instrument in the Italian context. Moreover, cognitive flexibility may be an important treatment target for mitigating the distress derived from ON symptoms, informing the development of future therapeutic approaches.

Assessing the psychometric properties of the Biphasic Sleep Scale (BiSS): A novel 16‐item self‐report measure

SummaryBiphasic sleep, characterized by nighttime sleep plus daytime napping, has demonstrated some cognitive, health and performance impacts when compared with consolidated monophasic sleep. This motivated the development and validation of the Biphasic Sleep Scale, reported in this paper. Scale development involved a literature review, expert input and individual interviews. The 16‐item Biphasic Sleep Scale was then administered to an international online sample (n = 6965) alongside well‐established validated sleep scales. To ensure a robust evaluation of the Biphasic Sleep Scale, the sample was divided into two parts: with 15% (n = 1000) of the participants allocated to the exploratory analytic phase; and the remaining 85% (n = 5965) reserved for confirmatory analyses. Psychometric evaluation included both exploratory and confirmatory factor analysis, reliability analysis, correlations, network analysis, and item response theory. Exploratory factor analysis indicated a three‐factor structure assessing daytime napping as to likelihood, consequences and effect on nighttime sleep. Confirmatory factor analysis largely confirmed this model with no sex invariance. The three‐factor structure showed adequate fit. The Biphasic Sleep Scale demonstrated good internal consistency (α = 0.88, Ω = 0.89). Network analysis revealed varying centrality and connectivity of items. Item response theory found items covering a range of biphasic sleep levels. Significant positive correlations with sleep criteria provided evidence for convergent validity. Further testing is warranted to confirm the factor structure, refine model parsimony, and establish clinical utility. With additional validation, it is hoped that the Biphasic Sleep Scale will become a widely utilized tool.

Parental Acceptance and Understanding of Autistic Children (PAUACS) - an Instrument Development Study.

Currently there are no instruments designed to assess parents' acceptance and understanding of their autistic child. We aimed to develop and evaluate the reliability and validity of a parent-report scale assessing parents' acceptance and understanding of their autistic child - the Parental Acceptance and Understanding of Autistic Children Scale (PAUACS). A total of 158 parents (74 non-autistic, 42 autistic, 42 questioning; mean age 42.69 years) of autistic children (mean age 10.80 years) completed an online survey comprising the prototype PAUACS as well as validated measures of parental sensitivity, neurodiversity affirming attitudes, autistic traits, mental health, and child adjustment and family experience. A subsample of participants (n = 97; 61.4%) completed the PAUACS questionnaire a second time, 2 weeks later, to assess for test-retest reliability. The final 30-item scale demonstrated excellent internal reliability (α = 0.89) and test-retest reliability (intraclass correlation = 0.92). Exploratory factor analysis revealed a clean structure comprising four distinct factors: Understanding (α = 0.86), Innate (α = 0.74), Acceptance (α = 0.82), and Expectations (α = 0.73). Overall, the PAUACS demonstrates good construct validity. Preliminary evidence of convergent validity and divergent validity was demonstrated. Preliminary evidence suggests PAUACS is a reliable and valid tool in assessing parents' acceptance and understanding of autistic children.

QOL-23. MEASURING BODY FUNCTIONALITY APPRECIATION IN PATIENTS WITH PRIMARY BRAIN TUMORS

Abstract BACKGROUND Body image is a multifaceted construct consisting of thoughts, feelings, and perceptions about one’s body and its functioning. Perception of one’s own physical functioning and capabilities may be an important consideration in neuro-oncology given the sequalae of brain tumors and associated treatment; however, it is understudied in this population. One brief measure of body image, the Functionality Appreciation Scale (FAS) is a comprehensive tool, focused on one’s perceived physical functioning and capabilities, rather than perceived appearance. We aimed to examine the psychometric properties of the recently developed Functionality Appreciation Scale (FAS) among patients with primary brain tumors. METHODS Adult patients with primary brain tumors (N=237) completed the FAS and other measures of psychological distress as well as medical and demographic characteristics. An exploratory variance factor analysis (EFA) using principal axis factoring and a Promax rotation was conducted. Convergent validity was examined via correlations to measures of psychological distress. RESULTS The EFA supported the single-factor structure, which accounted for 62.71% of the variance with all items loading significantly onto this factor. The FAS had good internal consistency (Cronbach’s a=.90). Convergent validity was demonstrated by negative associations (ps&amp;lt;.05) with symptoms of depression (r=-.30), generalized anxiety (r=-.21), and death anxiety (r=-.17). CONCLUSION Results confirmed the single factor structure of the FAS in primary brain tumor patients. Furthermore, the expected negative relationships with measures of psychological distress provide evidence of convergent validity. The FAS is valid and reliable in patients with primary brain tumors. Our findings suggest that body functionality and appreciation may be important to target in psychosocial interventions for those with primary brain tumors.

Developing a Scale Measuring Comprehensive Sex Education Attitudes in K-12 Schools (CSEA-K12)

ABSTRACT Comprehensive sex education (CSE) is linked to several favorable health outcomes yet is consistently underfunded and under-implemented. This is likely due to the perceived controversy of CSE, but research indicates that CSE in the K-12 system is widely supported in most areas of the U.S. Legislators, school board members, and funding agencies need a more thorough and accurate understanding of their communities’ attitudes toward CSE. An attitudinal scale for the general adult population could help generate research that would give these stakeholders the confidence they need to make decisions about CSE in their local areas. The current study created this needed scale using a structured 6-step process that integrated several methods establishing validity and reliability. Scale structure was tested using both exploratory and confirmatory factor analyses, in addition to invariance testing and validity checks. The scale performed similarly across several groups and met predictions for known-groups and predictive validity, while showing some evidence for convergent validity as well. However, although findings indicate that the one-factor model of the CSEA-K12 was the best fitting model, the fit was weaker than desired. Suggestions for future work further addressing internal consistency of the scale are noted.

(MIS)measuring cognitive load and arousal in deception: A multitrait–multimethod analysis

AbstractPurposeCognitive load and arousal are cornerstones of many deception detection strategies and theories; in turn, their effective measurement is critical. However, fundamental criteria for establishing the quality and accuracy of measures have largely been overlooked. In this study, we examined the reliability and construct validity of common cognitive load and arousal measures.MethodWe obtained three independent secondary datasets in which participants (N = 238) had lied or told the truth about witnessing a suspicious event. Using a multitrait–multimethod analysis, we assessed three measures of their cognitive load and arousal: participants' self‐reports, trained coders' observations, and objective behaviours.ResultsAlthough all measures were reliable, they achieved differing levels of validation. Specifically, measures of cognitive load showed evidence of convergent validity, but not discriminant validity. There was no empirical support for the construct validity of arousal measures.ConclusionsThese findings suggest that inconsistencies in the diagnosticity of cues to deception and theory support may be attributable to the measures employed. Researchers may not be assessing constructs of interest, particularly in the case of arousal.

Leadership, management, and team practices in research labs: Development and validation of two new measures.

Effective leadership and management practices contribute to responsible, high-quality research and the well-being of team members. We describe the development and initial validation of a measure assessing principal investigators' leadership and management practices and a measure of research team practices. Using a cross-sectional survey design, 570 postdoctoral researchers funded by the National Institutes of Health reported on the perceived behaviors of their principal investigator (PI) and the research team. The PI leadership and management items factored into two dimensions: fostering relationships and directing research. Correlations of these new scales with existing, validated measures of ethical leadership and general leader behavior provided evidence of convergent validity. Providing evidence for criterion-related validity, scores on the new measures predicted lab climate for research ethics, self-reported productivity, and job satisfaction. Research team practices provided additional predictive value beyond leadership and management behaviors. This study provides construct validity evidence for the new Leadership and Management in Science (LAMPS) Measure and the Research Team Practices (RTP) Measure. Qualitative responses to an open-ended item reinforced the importance of relationships and directive supervision for a positive environment. These measures can be useful tools for future research or may be useful for PIs seeking feedback about their practices.

Adaptation and Validation of the Nursing Attitude Questionnaire in a Spanish Population.

Nursing, as an essential component of contemporary healthcare systems, has undergone significant changes in recent decades, resulting in an increase in research across various fields, such as mental health, well-being, and the development of the discipline itself. Currently, gender stereotypes persist, impacting the social image of the profession and influencing the professional identity of nurses and consequently, their work performance and the quality of care provided. Both public and internal perceptions of the profession are critical factors in these dynamics. to adapt and validate the Nursing Attitude Questionnaire in a Spanish sample, as it has only been validated in two languages to date. This study presents descriptive and validity analyses of several measures utilized, such as Nursing Image (NIS), Medical Empathy (JSE-HP), Professional Values (NPVS) and Communication Skills (HP-CSS). Confirmatory factor model analyses indicated that a 21-item hierarchical model provided a superior fit (χ2(df = 1.65, NFI = 0.954, SRMR = 0.065), suggesting the presence of a general factor accounting for five first-order factors. Internal consistency was deemed adequate for the total measures (α = 0.789), though variable for the subscales. Correlations among measures provided evidence of convergent and discriminant validity, highlighting consistent correlation patterns between Attitude toward Nursing and other variables. In addition, significant differences were observed in the Professionalism subscale according to sex, albeit with a medium-low effect size. Scales are presented to facilitate future interpretation of the results in similar samples. The scale validated in this study exhibited overall good psychometric performance, although certain limitations were identified within the subscales. The established scales represent a novel contribution, providing a valuable tool for the comparison of similar samples.

Spanish Validation of the Brief Pornography Screen Within a Clinical Sample of Individuals with Gambling Disorder.

Problematic pornography use (PPU) shares characteristics with behavioral addictions, such as gambling disorder (GD), and PPU and GD may frequently co-occur. In order to fill existing gaps in screening instruments for PPU validated in GD, the study examined the psychometric properties of the Spanish version of the Brief Pornography Screen (BPS) within a clinical sample of individuals with GD. A total of 200 individuals seeking treatment for GD were consecutively recruited. The psychometric properties of the BPS were examined by performing confirmatory factor analysis (CFA) and testing for evidence of convergent and discriminant validity. The CFA supported a one-factor solution showing high internal consistency (α = 0.81). The BPS showed convergent validity and correlated with psychological distress, general psychopathology, impulsive tendencies, and low self-directedness. The BPS demonstrated adequate psychometric properties and is therefore recommended as a brief screening tool for identifying probable PPU in individuals seeking treatment for GD.

The scrupulosity obsessions and compulsions scale: A measurement of scrupulosity within an OCD framework

Scrupulosity is a psychological construct defined by intrusive thoughts and compulsive behaviors centered on religion or morality. Scrupulosity, in its severe form, may manifest as obsessive-compulsive disorder, and is related to negative outcomes. There are relatively few scales measuring scrupulosity; those that exist have serious limitations. This study addresses these limitations by creating and psychometrically validating the new Scrupulosity Obsessions and Compulsions Scale (SOCS). To determine final items, an exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were conducted using university (n = 1345) and community (n = 496) samples and scale forms were assessed for concurrent, convergent, discriminant, and incremental validity. The initial EFA resulted in 10 items, consisting of one factor of compulsion items and one factor of obsession items. These 10 items accounted for 59.87% of the common variance, and a bifactor model demonstrated acceptable fit (χ2(25) = 113.495, p < .001, CFI = .978, TLI = .960, RMSEA = .051 (90% CI [.042, .061]), and SRMR = .027). The SOCS demonstrated evidence of concurrent, convergent, discriminant, and incremental validity with relation to scrupulosity, OCD, anxiety, religiosity, and male role norms. Both subscales met reliability standards α = .85 and α = .83. The SOCS provides a concise method to measure scrupulosity's nature.

Construction, evidence of validity and reliability of a Teacher Socioemotional Competencies Scale - ECSED

Social-emotional competencies are complex actions that include skills, knowledge and attitudes to maintain effective and satisfactory social interactions. As well as, managing one's own and other people's emotions in different environments. They are vital for healthy coexistence in higher education. The purpose of this research was to establish the evidence of validity and reliability for the scale of socioemotional competencies of university teachers. Scale constructed under the pentagonal model approach of Bizquerra and Pérez (2007). The study is cross-sectional and instrumental. A non-probabilistic convenience sample of 237 teachers from public and private universities in Lima (Peru), from different professional careers, aged between 24 and 70 years, male and female, was used. Satisfactory evidence of content-based validity was found for the criteria of clarity, pertinence, and relevance for each of the items using Aiken's V technique. For validity based on the internal structure of the construct, a Penta dimensional model and a bi-factor model were contrasted by confirmatory factor analysis, with better fit values for the second model (CFI and TLI &gt; 0.95, RMSEA and SRMR &lt; 0.07, = ω h = 0.90, ECV and PUC &gt; 0.95), and favorable evidence of convergent and discriminant validity was also found. For both the overall score and the scores for each factor, there were coefficients denoting high reliability (ordinal α and ω &gt; 0.90).

Fears and Worries at Nighttime in Young Children: Development and Psychometric Validation of a Parent-Report Measure (FAWN-YC).

This paper outlines the development and psychometric evaluation of the Fears and Worries at Nighttime-Young Children (FAWN-YC) scale; a parent-rated measure for children aged 3-5years. Based on previous literature, it was hypothesised that the measure would be represented by a six-factor solution, with four clusters of fear types and two behavioural manifestations of fears. Exploratory factor analysis (EFA; N = 436) and confirmatory factor analysis (CFA; N = 383), resulted in a final 17 items that loaded onto 3 factors: Nighttime Fear Focus (8 items, α = 0.92), Bedtime/Sleep Avoidance and Interference (5 items, α = 0.90), and Dark Fear (4 items, α = 0.88). Evidence of convergent validity was found through strong associations between the total score and subscales of the FAWN-YC with measures of child anxiety, fear, sleep, externalizing and conduct problems. Furthermore, there was support for divergent validity (through a very weak to no relationship with a measure of prosocial behaviours), and evidence for temporal stability was also established with 2-week test-retest reliability. Overall, the results provide strong preliminary evidence for the reliability and validity of the FAWN-YC total score and subscales. Implications for the use of the measure in research and clinical practice are discussed.

Evidence of Internal Structure Validity, Measurement Invariance, Convergent Validity, Clinical Validity, and Reliability of the Woman Abuse Screening Tool in Peruvian Women.

Intimate partner violence (IPV) is the most common type of violence committed against women and results in serious personal, familial, social, and economic consequences; thus, there is a need to detect IPV early. One test for detecting IPV is the Woman Abuse Screening Tool (WAST). The objective of the study was to obtain evidence of the validity of the WAST in terms of its internal structure, measurement invariance, convergent validity, clinical validity, and reliability. A total of 670 women who have or have had a heterosexual relationship participated in the study. The first stage included 513 adult women who partook of health services (M age = 35.5 years, SD age = 10 years). The second stage included 160 women, 80 diagnosed with IPV (M age = 34.2 years, SD age = 11.8 years) and 80 who were not (M age = 31.8 years, SD age = 11.5 years). We used multivariate and receiver operating characteristic curve analyses. We found evidence that the WAST demonstrates a one-dimensional structure; evidence of measurement invariance regarding cohabitation with the partner, length of the relationship, and age; evidence of convergent validity in terms of correlations with emotional dependence, self-compassion, expressive suppression, anxiety, and depression; and evidence of clinical validity in terms of the high probability of detecting positive cases of IPV. These properties support the use of the WAST for detecting possible cases of IPV, which will allow timely intervention. This instrument can also be used in larger studies on IPV in the Spanish-speaking population.

Testing the convergent validity, domain generality, and temporal stability of selected measures of people’s tendency to explore

Given the ubiquity of exploration in everyday life, researchers from many disciplines have developed methods to measure exploratory behaviour. There are therefore many ways to quantify and measure exploration. However, it remains unclear whether the different measures (i) have convergent validity relative to one another, (ii) capture a domain general tendency, and (iii) capture a tendency that is stable across time. In a sample of 678 participants, we found very little evidence of convergent validity for the behavioural measures (Hypothesis 1); most of the behavioural measures lacked sufficient convergent validity with one another or with the self-reports. In psychometric modelling analyses, we could not identify a good fitting model with an assumed general tendency to explore (Hypothesis 2); the best fitting model suggested that the different behavioural measures capture behaviours that are specific to the tasks. In a subsample of 254 participants who completed the study a second time, we found that the measures had stability across an 1 month timespan (Hypothesis 3). Therefore, although there were stable individual differences in how people approached each task across time, there was no generalizability across tasks, and drawing broad conclusions about exploratory behaviour from studies using these tasks may be problematic. The Stage 1 protocol for this Registered Report was accepted in principle on 2nd December 2022 https://doi.org/10.6084/m9.figshare.21717407.v1. The protocol, as accepted by the journal, can be found at https://doi.org/10.17605/OSF.IO/64QJU.

Escala de Invalidación Emocional Percibida (PIES): Análisis psicométrico de la primera versión en español en adultos peruanos

Perceived emotion invalidation is a construct that is currently relevant due to its implications on mental health domains. The aim of this study was to linguistically adapt and to analyze the psychometric properties of the first Spanish-version of the Perceived Invalidation of Emotion Scale (PIES). This test was translated and reviewed by 10 experts to guarantee evidence of content validity. Confirmatory factor analysis verified the original two-dimensional proposal of the model. The study involved 411 adults (77.6% women, 22.4% men) with a mean age of 25.52 years (SD=6.64). Evidence of convergent and concurrent validity were found by associating scores with theoretically related constructs. In addition, the scores for both dimensions showed adequate levels of internal consistency. It is concluded that PIES complies with current psychometric standards and can be used in Peruvian samples similar to the one in this study.

Are narrative story stem methods valid in “non-Western” contexts? A systematic review

Narrative story stem techniques (NSSTs) offer insight into attachment and other representational aspects of preschool to young school aged children’s inner lives. While the method moved into the academic and clinical mainstream some 35 years ago, their applicability to “non-Western” contexts remains little understood. This synthesis comprises 31 NSST studies of samples from parts of Africa, East Asia, Latin America, and the Middle East, and from US and UK ethnocultural minoritized backgrounds. In the reviewed studies, three specific NSSTs dominated, story stems were used most to evaluate attachment, and some were clinically focused. However, there was also a strong cultural focus and over half of samples were socioeconomically disadvantaged. Studies revealed both universal and culturally specific features of NSSTs. Attachment distributions were as expected, given the high clinical risk in pooled samples (49% secure, 19% avoidant, 12% ambivalent, 20% disorganized), including by clinical and socioeconomic risk status. Gender differences were similar to “Western” findings. However, the growing evidence for convergent validity across cultural groups is tempered by low reporting of psychometrics. Narratives may sometimes reflect children’s unintended interpretations of the task and therefore not activate internal representations, or may reflect reality but lack equivalent meaning in coding schemes. We discuss how researchers and clinicians can enhance the validity of NSSTs by considering the role of culture in the sense-making process. Pending further validation work, NSSTs have the added potential to give a voice to young children from underrepresented backgrounds.

The development of an implicit measure of treatment expectations

ABSTRACT In three preregistered studies, we investigated whether implicit treatment expectations, using a relational implicit measure (the MT-PEP), vary between participants provided opposing information about novel medical treatments (Studies 1 and 2) or who responded based on normative beliefs toward common over-the-counter drugs (Study 3). The studies revealed large Cohen’s d effect sizes of both novel and well-known treatment information within the implicit measure. The studies also provide evidence of convergent validity, with MT-PEP scores associated with explicit beliefs about medicine and over-the-counter drug familiarity. Implicit treatment expectations can be assessed and offer a novel tool for the intersection of social psychology and medicine.

Confirmatory Factor Analysis and Propensity to Cheat Scale Validation in the Ethiopian Public Higher Education Institutions

Background This study was primarily intended to develop and validate a comprehensive and psychometrically acceptable measure of students’ propensity to cheat (PTC) behavior among undergraduate students in Ethiopian universities by assessing their engagement in different types of cheating behavior. Methods The present study employed an explanatory research design using a questionnaire based on the Propensity to Cheat Scale (PCS). The questionnaire was administered to 500 university students (male = 367 [73.4%]; female = 133 [26.6%]) selected from three Ethiopian public universities between November and January 2022. In order to measure the underlying variables of propensity towards cheating, a factor model is developed using exploratory factor analysis, and confirmatory factor analysis was employed to validate the students’ perceived PTC. The internal consistency of the PTC scale was assessed using reliability analysis, and validity evaluations were conducted to confirm the scale’s discriminant and convergent validity. Results Confirmatory factor analysis (CFA) results revealed a good fit to the data, and the internal consistency of the PCS was found to be strong, providing a reliable measure of students’ propensity for cheating. Validity evaluations, including discriminant validity and convergent validity, confirmed the validity of the scale. The average variance extracted (AVE) and composite reliability values also supported the scale’s convergent validity. The multidimensional concept of the PTC was supported by a four-factor solution consisting of 26 reliable and valid items. Conclusion The findings of the study demonstrate that the scale has also provided sufficient evidence of convergent and discriminant validity. By establishing discriminant and convergent validity, as well as reliability, through different validation procedures, the study has provided strong evidence for the effectiveness of the PCS as an instrument for determining whether university students are likely to engage in cheating behavior.