Initial Construct Validity Research Articles

Bridging the Gap: Virtual Reality as a Tool for De-Escalation Training in Disability Support Settings

Managing complex behaviors in disability support settings requires competent de-escalation skills. However, the current training methods often lack sufficient opportunities for realistic practice. This study details a three-stage development and evaluation of a virtual reality (VR) application for disability support staff to safely build de-escalation skills through simulated interactions. The first phase involved creating VR prototype scenarios depicting escalations with adolescent clients. Next, 12 disability support experts conducted content validation by refining the scenarios to confirm appropriateness and realism. Finally, a pilot study tested the tool’s usability and examined the initial construct validity in 20 participants. The prototype achieved high usability ratings (average 81.0 (SD 10.7) on the System Usability Scale). Additionally, a positive correlation between VR performance and empathy levels, as measured using the Toronto Empathy Questionnaire, was found (Pearson’s r = 0.487, p = 0.035). The promising results highlight the VR application’s potential as a transformative training tool. Future research should expand scenario diversity and compare VR with traditional methods to establish its efficacy in diverse settings and offer a path to enhance staff and student capabilities in challenging environments.

Faculty verbal messages: scale development and initial validation

IntroductionThis study set out to create a scale of messages faculty use when communicating with students and provide initial validation evidence for that scale.MethodsThree studies were used to create and initially validate the scale. Study one used focus groups to have students generate a list of 65 messages. Study 2, a survey with exploratory factor analysis, yielded two factors: affirming (items) and disconfirming (items) messages. Study 3 confirmed the factor structure and provided initial construct validation based upon the relationship between messages and relational distance, motives to communicate, learner empowerment, and student motivation.ResultsWithdrawal mediates the relationship between faculty verbal messages and learner empowerment. Faculty verbal messages directly, without mediation, predict student motivation.DiscussionFaculty verbal messages matter inside and outside the classroom. Both affirming and discomfiting messages can have a positive effect on students’ learning/motivation as well as the instructor-student relationship.

Polish Adaptation and Validation of the Penn State Worry Questionnaire for Children (PSWQ-C): Factor Structure, Reliability and Validity

This study presents the Polish version of the Penn State Worry Questionnaire for Children (PSWQ-C). The validation of the PSWQ-C in Poland was conducted through two studies. Study 1 examined the initial factor structure, internal consistency, and construct validity of the PSWQ-C in two samples of school students (N = 819) aged 8–19 years (M = 14.63, SD = 3.09). Study 2 involved 620 participants aged 8–19 years (M = 14.35, SD = 3.42). We conducted Confirmatory Factor Analysis to compare the initial factor structure with different models mentioned in the literature. The results suggest that the scale has good psychometric properties, including good reliability and acceptable validity. The factor structure that showed the best fit to the data consisted of one general worry factor and two method factors representing wording effects. This study is part of a broader discussion on the factor structure of the PSWQ-C. The results suggest that the questionnaire is homogeneous and measures one latent variable, worry, despite a bifactorial model being the best fit for the data. The two sub-factors do not have any psychological meaning (“non-worrying”), but are only an effect related to the positive or negative wording of the scale’s statements.

The iBelong Scale: Construction and validation of a measure of racial-ethnic-cultural belonging.

The purpose of this study was to develop a measure of racial-ethnic-cultural (REC) belonging for Black, Indigenous, and other people of color (BIPOC). The iBelong Scale was designed to assess the multidimensional components of REC belonging across diverse BIPOC groups. The scale was constructed based on a grounded conceptual framework of REC belonging, and the initial pool of items received feedback from community members and content experts. Validation of the iBelong Scale comprised data from 808 BIPOC participants collected across three interrelated studies for the purposes of initial validation, construct validity, and test-retest reliability. In Study 1, an exploratory factor analysis yielded a 25-item scale with five factors: (a) Home, (b) Connection, (c) Authenticity, (d) REC Thriving, and (e) Self-Definition. Results from Study 2's confirmatory factor analysis indicated that the five-factor model was an acceptable fit of the data and the best fit among competing models. Measurement invariance was assessed, and results indicated that the iBelong Scale measures REC belonging similarly across differing BIPOC groups. The iBelong Scale was related to a range of measures in theoretically expected ways, including positive associations with general belonging, racial/ethnic identity, and general well-being, and negative associations with loneliness and REC nonbelonging. Findings from Study 3 indicated the test-retest reliability of the iBelong Scale over a 2-week period. Limitations of the studies and implications for research and practice are discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Validation of a Novel Neurologic Assessment Test for Healthcare Staff.

BACKGROUND: To measure the effectiveness of an educational intervention, it is essential to develop high-quality, validated tools to assess a change in knowledge or skills after an intervention. An identified gap within the field of neurology is the lack of a universal test to examine knowledge of neurological assessment. METHODS: This instrument development study was designed to determine whether neuroscience knowledge as demonstrated in a Neurologic Assessment Test (NAT) was normally distributed across healthcare professionals who treat patients with neurologic illness. The variables of time, knowledge, accuracy, and confidence were individually explored and analyzed in SAS. RESULTS: The mean (standard deviation) time spent by 135 participants to complete the NAT was 12.9 (3.2) minutes. The mean knowledge score was 39.5 (18.2), mean accuracy was 46.0 (15.7), and mean confidence was 84.4 (24.4). Despite comparatively small standard deviations, Shapiro-Wilk scores indicate that the time spent, knowledge, accuracy, and confidence are nonnormally distributed ( P < .0001). The Cronbach α was 0.7816 considering all 3 measures (knowledge, accuracy, and confidence); this improved to an α of 0.8943 when only knowledge and accuracy were included in the model. The amount of time spent was positively associated with higher accuracy ( r2 = 0.04, P < .05), higher knowledge was positively associated with higher accuracy ( r2 = 0.6543, P < .0001), and higher knowledge was positively associated with higher confidence ( r2 = 0.4348, P < .0001). CONCLUSION: The scores for knowledge, confidence, and accuracy each had a slightly skewed distribution around a point estimate with a standard deviation smaller than the mean. This suggests initial content validity in the NAT. There is adequate initial construct validity to support using the NAT as an outcome measure for projects that measure change in knowledge. Although improvements can be made, the NAT does have adequate construct and content validity for initial use.

Validation of a Tool-Based Visual Anorectal Examination Advanced Simulator for the Early Detection of Colorectal Cancer.

Rectal examination through proctoscopy or rigid sigmoidoscopy is a common investigation in clinical practice. It is an important diagnostic tool for the workup and management of anorectal pathologies. Performing the examination can be daunting not only for patients but also for junior doctors. There are associated risks with the procedure, such as pain, diagnostic failure, and perforation of the bowel. Simulation-based training is recognised as an important adjunct in clinical education. It allows students and doctors to practice skills and techniques at their own pace in a risk-free environment. These skills can then be transferred to and developed further in clinical practice. There is extensive research published regarding the role of simulation-based training in endoscopy, however, we identified no published study regarding simulation-based training in rigid sigmoidoscopy or proctoscopy. This study aims to establish the initial face, content, and construct validity of a tool-based visual anorectal examination advanced simulator model for proctoscopy and rigid sigmoidoscopy. This innovative, highly realistic simulated environment aims to enhance the training of healthcare professionals and improve the efficiency of detecting and diagnosing distal colorectal disease.

Carceral Justification: Scale Development and Initial Validation

Black-led social movements have demanded sweeping changes to the criminal legal system by advancing abolitionist goals. Public attitudes about the criminal legal system influence what policy agendas are developed, legitimated, and implemented. In order to assess influences on such attitudes, there is a need for research that examines the underlying carceral logics people use to inform their opinions toward anti-carceral policy proposals. Applying System Justification Theory and a framework of carceral logics, we developed the Carceral Justification Scale (CJS) using data collected from 1,394 Alameda County, California registered voters. Items were developed via literature review, qualitative analysis of canvasser conversations, and expert review. Exploratory (N = 461) and confirmatory (N = 463) factor analyses suggested an oblique 2-factor structure and produced 6 items with the following factors: (a) System Works, and (b) System Necessary. Internal consistency estimates were .71 and above and the scales accounted for 28% and 27% of variance, respectively. Initial construct validity was established as CJS scores were associated with racial resentment, system justification, political ideology, and anti-Black bias awareness in ways consistent with theory. This measure is useful for the evaluation of community practice political interventions that aim to increase support for abolitionist policies.

Engaging classroom observation: A brief measure of active learning in the college classroom

The purpose of this study was to develop a valid, reliable, and brief measure of active learning in college classrooms that is cheap and easy to complete and yields results that faculty can easily use to inform their development as instructors. Initial construct and face validity was achieved by modifying existing instruments and creating a draft of a brief measure of active learning for external expert review. Following the suggested revisions, the engaging classroom observation was then piloted and revised as necessary. Reliability was tested and measures of internal consistency and interrater reliability were acceptable. A principal component analysis showed two components that were moderately correlated, which indicated the potential they could be combined. An Exploratory Factor Analysis confirmed the instrument is measuring one factor, which we propose as active learning. This study is significant because it offers a brief instrument based on students’ perceptions that can be used formatively by faculty.

Development and validation of a reasons for electronic cigarette use questionnaire.

People use electronic cigarettes (e-cigarettes) for many reasons, but currently there are no comprehensive assessments of the motivations for tobacco vaping. The aim of the present study is to develop and test the initial construct validity of a new measure to assess reasons for e-cigarette use. We developed a 56-item measure based on the e-cigarette literature. This measure, along with demographic and tobacco use questions, was administered to adults who self-identified as past or present e-cigarette users on the Prolific crowdsourcing platform. The sample (n = 965) was randomly assigned into two analytic groups for exploratory factor analysis (EFA; n = 484) and confirmatory factor analysis (CFA; n = 481). The sample ranged from 19 to 77 (M = 36.6; SD = 11.5) years old, and 42.2% identified as women, 74.6% as White, 7.2% as African American, 4.7% as Asian/Pacific Islander, and 5.1% Hispanic/Latino. After removing highly correlated items and nonloading items on the EFA, the 56-item scale was reduced to 47 items across eight factors. The eight subscales assessing various motivation domains of e-cigarette use included social influence, alternative to cigarettes, pleasurable effects, harm reduction, dependence, cessation, weight/appetite, and smell/flavor domains. Cronbach's α coefficients and preliminary analyses of differential motivation based on sex, age, and daily smoker status are presented. This study demonstrates the construct validity for the first comprehensive measure tested to assess reasons for e-cigarette use. This measure has potential to become a valuable assessment for researchers examining factors contributing to tobacco vaping among a variety of populations and settings. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Measuring Shyness in Children Online

During the pandemic, developmental psychology research labs had to adapt to virtual testing methods. Many labs continue to conduct online studies even as restrictions have been lifted, as these methods extend the geographical range of participating families. However, online testing methods cannot always match in-person procedures. For example, psychologists often measure behaviours that indicate shyness in children during in-person laboratory visits that include contexts such as giving a public speech or interacting with confederates. To adapt this method to a virtual environment, Dr. MacGowan has recently innovated a new procedure for studying shyness within an online, synchronous testing session.&#x0D; Dr. MacGowan's ongoing eProsocial study evaluates children’s behavioral shyness during synchronous sessions on Zoom. This is accomplished by having an experimenter posed as a child confederate named “Sam” log on with a gender-matched profile picture of a child; participants are told that Sam is the next child scheduled for a testing session. The lead experimenter leaves the participant to “chat” with Sam and the disguised researcher uses 7 pre-recorded audio clips of a real child’s voice to facilitate conversations with the participant (e.g., “What grade are you in?”). When the lead experimenter returns, they perform a manipulation check to gauge if the child is suspicious of Sam’s existence by asking “What did you think of Sam?”.&#x0D; A total of 194 sessions have been conducted to date, and only one participant has indicated doubt in the existence of Sam during manipulation checks. These results exhibit initial construct validity. To further examine validity, research assistants will code shyness behaviours, such as latency to respond, and compare this data to caregiver shyness reports. Dr MacGowan’s approach to measuring shyness in an online setting is promising, and other developmental researchers should consider replicating this paradigm in future studies.

Development and Validation of the Military Minority Stress Scale.

Despite affecting nearly 3% of active-duty service members, little is known about how LGBT-related stress experiences may relate to health outcomes. Thus, the present study sought to create a Military Minority Stress Scale and assess its initial reliability and construct validity in a cross-sectional study of active-duty LGBT service members (N = 248). Associations between 47 candidate items and health outcomes of interest were analyzed to retain those with substantial betas. Item response theory analyzes, reliability testing, invariance testing, and exploratory factor analysis were performed. Construct validity of the final measure was assessed through associations between the sum score of the final measure and the health outcomes. The final 13-item measure demonstrated an excellent reliability (ω = 0.95). Bivariate linear regressions showed significant associations between the sum score of the measure and overall health (β = -0.26, p < 0.001), overall mental health (β = -0.34, p < 0.001), physical health (β = 0.45, p < 0.001), life satisfaction (β = -0.24, p < 0.001), anxiety (β = 0.34, p < 0.001), depressive symptoms (β = 0.37, p < 0.001), suicidality (β = 0.26, p < 0.001), and PTSD (β = 0.42, p < 0.001), respectively. This study provides the first evidence that minority stressors in the military setting can be operationalized and measured. They appear to have a role in the health of LGBT service members and may explain the continued health disparities experienced by this population. Little is known regarding the experiences of LGBT active-duty service members, including experiences of discrimination. Understanding these experiences and their associated health outcomes during military service may therefore help and guide further etiological studies and intervention development.

Digital Flourishing: Conceptualizing and Assessing Positive Perceptions of Mediated Social Interactions

Recent research started to apply concepts of well-being to the context of computer mediated communication (e.g., social media, instant messaging). While much research investigates negative perceptions of mediated social interactions (e.g., “problematic” or “addictive” social media use), a multi-dimensional measure that taps into users? positive perceptions is sorely lacking. The present research therefore develops the first comprehensive measure of digital flourishing, defined as positive perceptions of mediated social interactions. Building on a qualitative pre-study that aided the construction of the Digital Flourishing Scale (DFS), Study 1 (N = 474) employed exploratory factor analysis to reveal five subdimensions of digital flourishing. The preregistered Study 2 (N = 438) confirmed these five dimensions, yielding five reliable items per subscale and initial construct validity with three psychological needs from self-determination theory (SDT; competence, autonomy, relatedness) which were used as an underlying well-being framework for the development of the DFS. The preregistered Study 3 generated further construct validity by directly relating DFS to well-being. The scale is relevant for researchers and practitioners alike to better understand how users perceive their mediated interactions to impact mental health and well-being.

Testing of a novel questionnaire of Household Exposure to Wood Smoke.

Household air pollution from wood smoke (WS), contributes to adverse health effects in both low- and high-income countries. However, measurement of WS exposure has been limited to expensive in-home monitoring and lengthy face-to-face interviews. This paper reports on the development and testing of a novel, self-report nine-item measure of WS exposure, called the Household Exposure to Wood Smoke (HEWS). A sample of 149 individuals using household wood stoves for heating from western states in the U.S., completed the HEWS during the winter months (November to March) of 2013 through 2016 with 30 subjects having in-home particle monitoring. Hard copy or online surveys were completed. Cronbach's alpha (α), intraclass correlations (ICC), exploratory factor analysis (EFA) and tests of associations were done to evaluate reliability and validity of the HEWS. Based on initial analysis, only 9 of the 12 items were retained and entered in the EFA. The EFA did not support a unitary scale as the 9 items demonstrated a 3-factor solution (WS exposure duration, proximity, and intensity) with Cronbach's α of 0.79, 0.91, and 0.62, respectively. ICC was 0.86 of the combined items with single items ranging from 0.46 to 0.95. WS intensity was associated with symptoms and levoglucosan levels, while WS duration was associated with stove and flume maintenance. The three-dimensional HEWS demonstrated internal consistency and test-retest reliability, structural validity, and initial criterion and construct validity.

Engagement in the preschool classroom: Brief measures for use with children from ethno-racially diverse and low-income backgrounds

Children's behavioral engagement in preschool is a strong predictor of their academic achievement both concurrently and into the later school years across socioeconomic and ethnic groups (Chien et al., 2010; Fuhs et al., 2013; Hamre & Pianta, 2001; Kim & Suarez-Orozco, 2015; Ladd & Dinella, 2009). The present measurement development study was conducted within the context of a larger intervention research program in partnership with an urban Head Start program in the northeastern United States. Forty-one teachers and 456 children participated. Two new measures were developed and tested for use in preschool classrooms– a teacher rating scale and an independent observation instrument. Although many of the assessments that result from commonly employed systematic observation methods provide a detailed and nuanced description of children's engagement in the classroom, they often require intensive training and are costly to implement. The present study sought to provide the early childhood education field with two brief and more cost-effective measures. Findings revealed initial construct validity evidence for the two measures, which were found to be reliable and to relate modestly with one another as well as to an external criterion measure of children's science knowledge, which was the focus of the larger intervention. Findings also revealed significant relations between children's engagement and child and family demographic characteristics, perhaps attributable to the unique context of Head Start.

Exploratory Factor Analysis of Environmental Awareness Assessment based on Local Wisdom by Using Summated Rating and Likert Scale

There is no standardized instrument to measure environmental awareness based on local wisdom. Therefore, the purpose of this study is to conduct an Exploratory factor analysis (EFA) on an environmental awareness assessment based on local wisdom. This study has a non-experimental design with the ex-post facto type. The Likert scale environmental awareness is transformed into a summated rating scale. Furthermore, the initial construct validation was carried out using the EFA test. Based on the results of the EFA, it was found that the EA which previously consisted of 18 indicators extracted based on eigen values and fixed factors became 8 factors with 14 indicators.

NCOG-39. DISCOVERY OF CLINICAL AND DEMOGRAPHIC DETERMINANTS OF SYMPTOM BURDEN IN PRIMARY BRAIN TUMOR PATIENTS USING NETWORK ANALYSIS AND UNSUPERVISED CLUSTERING

Abstract BACKGROUND Precision health approaches to managing symptom burden in primary brain tumor (PBT) patients are imperative to improving patient outcomes and quality of life. Network Analysis (NA) identifies complex symptom co-severity patterns across large patient cohorts. Unsupervised clustering unbiasedly stratifies patients into clinically relevant subgroups based on symptom patterns. This is the first study to use NA and unsupervised clustering to explore PBT patients’ clinical and demographic determinants of symptom burden. METHODS Symptom severity data reported using the MDASI-BT from a two-institutional cohort of 1,128 PBT patients was analyzed. Gaussian Graphical Model networks were constructed for the entire cohort and for subgroups identified by unsupervised clustering. Network characteristics were analyzed and compared using permutation-based statistical tests. RESULTS NA on the entire cohort revealed that the majority of PBT patients experience symptoms on four core dimensions that drive overall symptom burden: cognitive, physical, focal neurologic, and affective. These dimensions substantially overlap with factor groupings defined during initial construct validation of the MDASI-BT. Fatigue/drowsiness scored the highest in all network centrality measures, indicating a pivotal role in the symptom experience. Unsupervised clustering identified four patient subgroups: PC1 (n = 683), PC2 (n = 244), PC3 (n = 92), and PC4 (n = 109). Moderately accurate networks could be constructed for PC1 and PC2, but not for PC3 and PC4 due to their small size. The PC1 network closely resembles the all-patient network, and these patients have the highest interference scores among the subgroups with fatigue/drowsiness as the primary driver. PC2 represents an older subgroup in which cognitive symptoms drive symptom burden. CONCLUSIONS This novel study identified clinically relevant subgroups of patients with unique symptom burdens. With further validation, our approach may inform more personalized and effective symptom management by identifying symptoms to prioritize for targeting in individual patients.

Psychological resilience: Predictors and measurement among Israel Defense Force combat officer candidates

ABSTRACT The goal of the current study was to develop a resilience selection tool for combat officers that would augment the test battery at the Israel Defense Force (IDF) Officer Selection Center (OSC). To enable this goal, we sought to explore the underlying structure of resilience in a diverse number of resilience predictors. Participants were 1,533 male candidates at the IDF OSC. Using Exploratory Factor Analysis followed by Confirmatory Factor Analysis, 159 items from 13 separate resilience scales were refined into a single, 18-item psychological resilience scale. The new scale emphasizes self-efficacy, competency and overall positive self-perception. The new scale is tailored specifically to military combat officers. Initial construct validity was demonstrated against Big Five dimensions and psychological evaluations at IDF OSC. Predictive validity was demonstrated against command and peer evaluations from IDF Officer Training School (OTS). Medium to strong positive correlations were found between the resilience scale and Big Five dimensions of Extroversion and Conscientiousness, as well as a medium negative correlation with Neuroticism. Low but significant correlations were found between the resilience measure and psychological evaluations of combat officer vocational congruence. Low but significant correlations were also found with command and peer evaluations from Officer Training School. Resilience scale scores were found to predict OTS commander evaluations, but not OTS peer evaluations, beyond standard OSC scores when controlling for demographic variables. The theoretical importance of findings as well as implications for military officer selection are discussed.

Analysis of a Repetitive Language Coding System: Comparisons between Fragile X Syndrome, Autism, and Down Syndrome.

Expressive language sampling (ELS) is a frequently used tool for language analysis, as it can be used across widely ranging cognitive and language abilities. ELS can also evaluate pragmatic language, including excessive self-repetition, which is challenging to assess with traditional standardized assessments. This study explored how a well-established ELS protocol can assess three types of linguistic self-repetition in three neurodevelopmental disabilities: fragile X syndrome (FXS), autism spectrum disorder (ASD), and Down syndrome (DS). We examined its ability to differentiate between these disorders, the relationships between repetitive language and other participant characteristics, and initial construct validity. We found that the groups with FXS and ASD differed significantly on each of the three repetitive language measure, and that the group with DS differed from either ASD or FXS on two. Cognitive ability was significantly related to phrase repetition in the group with ASD. When the groups were combined, there was evidence of convergent and divergent validity. This study extends previous research on ELS and supports its use as a means to characterize pragmatic language. It also provides information about the relationships between repetitive language and other phenotypic characteristics.

Inventory of Dating Stress: Initial psychometric evaluation and construct validity

There is limited research on dating stress, or stressful experiences related to the dating process (e.g., inconsistent communication, sudden lack of response, and rude behavior). Little attention has been given to classifying stressors involved with the pursuit of potential partners from initial contact to relationship formation. In the current study, we developed a novel measure for such experiences, the Inventory of Dating Stress (IDS). We investigated the factor structure and preliminary construct validity for the IDS in an online sample of adults ( n = 478). Results revealed a reliable four-factor structure (Mixed Signals, Mismatch, Ambiguous Rejection, and Harassment) across 18 items and the IDS demonstrated initial construct validity. Overall, the current study offers evidence of preliminary psychometric support for the IDS as a measure of dating stress.

Humor Structure Appreciation Scale: Analysis of the psychometric properties of the Czech version

Objectives. The aim of the study was to assess the psychometric properties of a Czech adaptation of the Humor Structure Appreciation Scale (HSAS) which measures appreciation of two structural dimensions of humor: incongruity-resolution (INC-RES), and nonsense (NON). Differences in humor structure appreciation were investigated between a Czech and a Macedonian sample, and measurement invariance was tested.Participants and setting. Two online studies were conducted. After a back-translation of the scale, the first proposal of the Czech version of the HSAS was given to 145 participants from the Czech Republic. In the second study, the HSAS-CZ was completed by a Czech sample of 346 subjects. A measure of the need for closure was also completed to assess construct validity. The Macedonian sample comprised of 562 subjects filled in only the original version of the HSAS.Results. Principal component analysis showed that the sum of variance explained by the two dimensions was 60.8%, and the factor loadings were satisfactory. The second study confirmed the factor structure of the scale. The results of the Confirmatory factor analyses showed that the two-factor model had an acceptable fit (RMSEA = .053 [.034, .072], CFI = .986, TLI = .981). The standardized factor loadings were high for INC-RES (λ = .62 to .79), as well as for NON humor (λ = .57 to 80). Initial construct validity was obtained. The HSAS-CZ scales were theoretically related to the need for closure. The jokes were found funnier by the Macedonian sample in comparison to the Czech by one third to one half standard deviation. Finally, a strict measurement invariance model comparing the Czech and Macedonian versions of the test showed an acceptable fit. However, significant differences were found in factor loadings as well as in residual variances.Limitations. The main limitation of the study is the use of convenience samples.