Evidence Of External Validity Research Articles

Conceptualization and validation of the TILT questionnaire: relationship with IGD and life satisfaction.

Currently, the study of esports is growing within the field of psychology. Among the different variables attracting interest - including stress or psychological factors associated with performance - an emerging concept known as tilt is gaining prominence in the literature. However, this construct has yet to be operationalized or defined. Thus, the present study aims to address this gap by defining and conceptualizing TILT while devising and validating a questionnaire to measure the construct in esports players. The initial phase of the study comprised 27 interviews conducted with professional players (n = 6), semi-professionals (n = 8), amateurs (n = 8), and coaches (n = 5) to characterize the concept of tilt. Following these interviews, a definition of tilt was formulated, and a panel of five experts in sports psychology and esports proposed a comprehensive set of 53 items. A total of 488 participants (278 males, 210 females), aged 18-50 (mean age = 26.9 years, SD = 7.57), completed the survey, including the 53 tilt items, a questionnaire measuring toxic behavior, and the Internet Gaming Disorder Scale-Short Form (IGDS9-SF). The tilt construct is primarily characterized as a state of frustration escalating into anger, resulting in diminished performance, attention, and recurring negative thoughts about errors. Its onset typically coincides with stressful situations, persisting for approximately 30 min. Through an Exploratory Factor Analysis (EFA), 18 items were retained and categorized into two factors: Causes (7 Items) and Consequences (11 Items) of tilt. The entire questionnaire yielded a Cronbach's α of 0.922, with the first and second factors showing values of 0.854 and 0.890, respectively. Confirmatory factor analysis (CFA) revealed an acceptable fit for the 2-factor solution. Correlations with related constructs, such as Toxic Behavior and IGD, provided preliminary evidence of external validity. Empirical evidence for the validity and internal consistency of the Tilt Scale is robust, indicating its potential utility in future research on the psychological experiences of esports players.

The price is Heavy: How the haptic sensation of weight influences preference for partitioned versus combined pricing

This research makes a novel proposition that the haptic sensation of weight can moderate the effect of price framing on consumer decisions. Participants who experienced heaviness (lightness) preferred a target product presented in terms of combined (partitioned) versus partitioned (combined) pricing frames. This effect was further mediated by ease of processing. Four studies (three laboratory and one field experiment) were conducted to test the key hypotheses and provide evidence for causality and external validity. Across the four studies, the haptic sensation of weight was manipulated through semantics (Studies 1, 3), embodied experience (Study 2), and part of the product experience (Study 4). In studies 1, 2 and 3, the haptic experience of weight was independent of the target product evaluation, whereas in the fourth study, the weight experience was part of the target product evaluation itself. The findings across the studies are consistent, robust, and have theoretical and managerial implications.

Hypertrophic cardiomyopathy detection with artificial intelligence electrocardiography in international cohorts: an external validation study.

Recently, deep learning artificial intelligence (AI) models have been trained to detect cardiovascular conditions, including hypertrophic cardiomyopathy (HCM), from the 12-lead electrocardiogram (ECG). In this external validation study, we sought to assess the performance of an AI-ECG algorithm for detecting HCM in diverse international cohorts. A convolutional neural network-based AI-ECG algorithm was developed previously in a single-centre North American HCM cohort (Mayo Clinic). This algorithm was applied to the raw 12-lead ECG data of patients with HCM and non-HCM controls from three external cohorts (Bern, Switzerland; Oxford, UK; and Seoul, South Korea). The algorithm's ability to distinguish HCM vs. non-HCM status from the ECG alone was examined. A total of 773 patients with HCM and 3867 non-HCM controls were included across three sites in the merged external validation cohort. The HCM study sample comprised 54.6% East Asian, 43.2% White, and 2.2% Black patients. Median AI-ECG probabilities of HCM were 85% for patients with HCM and 0.3% for controls (P < 0.001). Overall, the AI-ECG algorithm had an area under the receiver operating characteristic curve (AUC) of 0.922 [95% confidence interval (CI) 0.910-0.934], with diagnostic accuracy 86.9%, sensitivity 82.8%, and specificity 87.7% for HCM detection. In age- and sex-matched analysis (case-control ratio 1:2), the AUC was 0.921 (95% CI 0.909-0.934) with accuracy 88.5%, sensitivity 82.8%, and specificity 90.4%. The AI-ECG algorithm determined HCM status from the 12-lead ECG with high accuracy in diverse international cohorts, providing evidence for external validity. The value of this algorithm in improving HCM detection in clinical practice and screening settings requires prospective evaluation.

Development and validation of the Driver Attention Regulation Scale: A measure of the perceived ability to regulation attention on the road

The ability to regulate attention while driving is a vital skill for remaining safe on the road. However, there is no current self-report method that measures such a construct, potentially limiting research in the area of attention and distraction. This study aimed to develop and validate a new measure, the Driver Attention Regulation Scale (DARS), that can be used to identify how drivers perceive their ability to regulate attention while driving. To achieve this aim, two online surveys were shared with unique samples of Australian drivers. Following the development of an initial item pool, principal components analysis and follow up confirmatory factor analysis were used to test the structure of the items and revealed that three factors emerged: attentional presence (e.g., focus; vigilance), attentional flexibility (e.g., awareness; shifting), and attentional capacity (e.g., processing speed; cognitive resources). Each of the subscales (α = 0.75–0.88) and the total scale (α = 0.87–0.91) exhibited a good internal consistency across the samples. Evidence of external validity was also demonstrated, as the DARS was positively associated with generalised attention regulation measures, and driver confidence, but negatively associated with emotion regulation difficulties, driver stress, and driving errors/lapses. Mediation analyses also indicated that the attentional constructs measured by the DARS may indirectly influence driving behaviours, via their effect on driver stress and confidence. Overall, the DARS has demonstrated promise as a new measure of driving-related attentional skills that can be used in future research to better understand how situational factors may impact drivers’ confidence, and also how confidence may influence driving behaviours (e.g., speeding).

Measuring mental health action competencies in school teachers: internal and external validity evidence.

Mental health literacy is receiving increasing research attention due to growing concerns for mental health globally. Among children, teachers have recently been recognized as playing a vital role in the recognition and reporting of potential mental health issues. A nationally sampled cross-section of teachers was surveyed to examine the discriminant validity of the mental health literacy measure across levels of teaching. A survey collected a total of n = 369 teacher responses in Switzerland (Kindergarten = 76, Primary = 210, Secondary = 83). Item response theory (IRT) analyses were conducted. Inspection of psychometric properties indicated removal of two weak performing items. The 15-item measure exhibited a significant mean difference, such that class-responsibility function scored higher (M = 2.86, SD = .45) than non-responsible function (M = 2.68, SD = .45) teachers [t(309) = -2.20, p = .01]. It also exhibited a significant mean difference, such that more subjective experienced scored higher (M = 2.86, SD = .45) than less subjective experienced (M = 2.68, SD = .45) teachers [t(210) = -8.66, p < .01]. Hypotheses regarding age and role tenure were in the expected direction, but non-significant. The MHL measure for teachers demonstrated sound measurement properties supporting usage across teaching levels.

Psychometric Evaluation of the School Climate and School Identification Measure-Student on Chilean Students: A Bifactor Model Approach.

School climate is a relevant construct for understanding social relations at school. The SCASIM-St has been widely defined as a multidimensional construct; however, new factor structures have not been explored through evidence that allows for interpreting school climate scores from an approach that respects the multidimensionality of the scale and, at the same time, allows for identifying the degree of essential unidimensionality in the data. Consequently, the objective was to analyze the psychometric properties of the SCASIM-St from a bifactor model approach, evaluating the influence of a general school climate factor versus five specific factors. The study involved 1860 students of both sexes (42% males and 58% females), with an average age of 16.63 years (SD = 0.664), from 17 secondary schools in Chile. The results obtained by a confirmatory factor analysis provided evidence that the best model was the bifactor model for the 38 items, with one general factor and five specific factors. The Explained Common Variance (ECV) values and reliability levels by hierarchical omega accounted for a strong general school climate factor with high levels of reliability. Evidence of external criterion validity, assessed through the attitude toward authority scale (AIA-A), showed a theoretically expected and significant relationship between the factors of both instruments. This study confirmed the psychometric robustness of the SCASIM-St scale by means of a bifactor model, allowing for a new, essentially unidimensional interpretation of the scale scores and providing an instrument to measure school climate in Chile.

Escala de Avaliação do Contexto de Trabalho Docente Remoto: Construção e Validação

Abstract The research aimed to develop and present initial validity evidence for the Remote Teaching Work Context Assessment Scale (TRWCAS). We hypothesized that the instrument would include factors related to work organization, working conditions, and socio-professional relationships. Teachers (n=15,176) completed a self-administered questionnaire. The sample was randomly divided into two groups (sample 1 and sample 2). The questionnaire included the TRWCAS, Self-Report Questionnaire-20 (SRQ-20), and demographic variables. We conducted Exploratory Factor Analysis (EFA) on sample 1 and Confirmatory Factor Analysis (CFA) on sample 2. An initial three-factor solution was found, explaining 55.8% of the total variance. The model in the CFA showed adequate fit indices (RMSEA=.067, 90% CI: .066 - .068, SRMR=.052, CFI=.985, TLI=.984). Evidence of external validity was obtained based on the correlation of the measure with common mental disorders.

Psychometric Validation of the Tactical Assessment Instrument in Football for Use in Physical Education and with Youth Sport Teams.

Recently, the Tactical Assessment Instrument in Football (TAIS) was developed to address football's tactical requirements in terms of game phases, learners' roles, and organizational levels of play. In this study, we sought to evaluate the psychometric properties of this existing instrument in the Physical Education (PE) context and for youth sport teams (ages 8-12years). Our research design involved: (a) participants playing 37 football games, (b) assessing participants via observation with the TAIS and (c) conducting psychometric analysis on the obtained assessment data. For the psychometric analysis, we carried out our work in four phases: (a) a statistical analysis of the evaluation criteria, (b) analysis of the instrument's structural dimensions, (c) internal item reliability analysis, and (d) provision of evidence for external validity. Participants were 592 children (156 girls, 436 boys; ages 8-12years) from 74 PE classes or sport teams (from three institutional contexts - schools, community-based sports, and sports clubs) in Spain. We selected 12 tactical criteria and grouped them into four theoretical dimensions (appropriate attack, inappropriate attack, appropriate defense, inappropriate defense) to confirm the instrument's structural dimensions. The Cronbach alpha and Omega McDonald coefficients were greater than .70. The TAIS discriminated between participants from the three institutional contexts, offering evidence of the instrument's external validity. Thus, the TAIS now has sufficient psychometric support for assessing learning of football tactics in PE and youth sport teams. This instrument facilitates an ecological assessment of youth players' understanding of football tactics through four theoretical tactical dimensions and 12 criteria involving information about four learners' roles (both attack and defense, on- and off-the-ball) and three organizational tactical levels of play (whole team, small groups of learners, and individual learners).

Cultural adaptation to Spanish and psychometric analysis of the Woman Abuse Screening Tool (WAST) in Peruvian adult women

BACKGROUND: Partner violence is a public health problem. In Peru, women with spouses are vulnerable to suffering it. The Woman Abuse Screening Tool (WAST) is a widely recommended instrument for the early identification of these cases. Therefore, our objective was to adapt and analyse the psychometric properties of the WAST scale in adult Peruvian women. METHODOLOGY: This was an instrumental study, with a sample of 628 women who were in a marital condition of cohabitation or married at the time of the study. The mean age was 35 years (M = 35.58; SD = 10.92) and on average they were living with their partner for 9 years (M = 9.93; SD = 8.41). This sample was selected by non-probabilistic snowball sampling. The Woman Abuse Screening Tool (WAST), Generalized Anxiety Disorder-7 (GAD-7) and Patient Health Questionnaire (PHQ-9) were used. The validity of the internal structure was verified with confirmatory factor analysis (CFA), convergent validity by correlation coefficients between the WAST, GAD-7 and PHQ-9, while reliability was verified using Cronbach’s alpha and McDonald’s Omega coefficient. RESULTS: Factorial analysis indicated that the one-dimensional model showed adequate goodness-of-fit indices (CFI = 0.998; TLI=0.991; RMSEA=0.103; SRMR=0.075), and reliability coefficients (α = 0.899 and ω = 0.916) were optimal. Additionally, the WAST showed strong external validity based on the relationship with the PHQ-9 and GAD-7 scores (Rho &gt; 0.50). CONCLUSIONS: The Woman Abuse Screening Tool (WAST) scale demonstrated solid evidence of internal and external validity and optimal reliability in adult Peruvian women. Its use will benefit the early identification of victims of intimate partner violence in primary health care and further research.

Internal and External validity evidence for Level of Personality Functioning Scale – Brief Form 2.0

Internal and External validity evidence for Level of Personality Functioning Scale – Brief Form 2.0

Development of the PREMIUM computerized adaptive testing for measuring the access and care coordination for patients with severe mental illness

Severe mental illness (SMI) patients often have complex health needs, which makes it difficult to access and coordinate their care. This study aimed to develop a computerized adaptive testing (CAT) tool, PREMIUM CAT-ACC, to measure SMI patients' experience with access and care coordination. This multicenter and cross-sectional study included 496 adult in- and out-patients with SMI (i.e., schizophrenia, bipolar disorder, or major depressive disorder). Psychometric analysis of the 13-item bank showed adequate properties, with preliminary evidence of external validity and no substantial differential item functioning for sex, age, care setting, and diagnosis, making it suitable for CAT administration. A post-hoc CAT simulation demonstrated that the tool was efficient and accurate, with an average of seven items, compared to the full item bank administration. Its use by clinicians can contribute to optimizing patient care pathways and transitioning towards more person-centered healthcare.

The validity and reliability of the Turkish version of revised metacognitive awareness of reading strategies inventory

This study aims to adapt the revised Metacognitive Awareness of Reading Strategies Inventory (MARSI- R) into Turkish. MARSI-R is a self-report instrument designed to assess students’ metacognitive awareness of reading strategies and perceived strategy use when reading school-related materials. 525 students (65% female, 35% male, Mage = 13 years old.) from multiple school types and degrees participated in this study. A stepwise validation procedure was used to translate and produce a Turkish version of the inventory. Evidence of structural and external aspects of validity for the inventory was collected. The 15-item inventory had a three-factor solution (global reading strategies, problem-solving strategies, and support reading strategies), as supported by confirmatory factor analysis. Turkish version scores were positively correlated with students' perceived reading ability, which provides evidence of MARSI-R's external validity. The coefficient of stability was calculated using data from 85 students who took the Turkish version of the MARSI-R twice in a five-week interval. The study’s overall results provided evidence of the reliability and validity of the inventory. According to the results presented in this study, the Turkish version of the inventory can be implemented to assess the students’ metacognitive awareness of reading strategies and perceived strategy use. The findings show that the adapted inventory can be used to obtain valid and reliable results for Turkish lower and upper secondary school students.

The Self-Efficacy to Regulate Physical Activity Scale: Exploring the Factor Structure, Temporal Invariance, and External Validity in a Latent Variable Framework

ABSTRACT The purpose of this study was to explore the validity of evidence for self-efficacy to regulate physical activity scale (SERPA) measurement using an exploratory latent variable approach. The objectives were to explore the dimensionality, temporal invariance, and external validity of scores produced by the SERPA, a modified version of the barriers self-efficacy scale. Data (N baseline = 461 and N 30 days post-baseline = 427) from the Well-Being and Physical Activity Study (ClinicalTrials.gov, identifier: NCT03194854), which deployed the Fun For Wellness intervention, were analyzed. A two-dimensional factor structure explained responses to the SERPA at baseline. There was evidence for at least partial strict temporal measurement invariance for this two-dimensional structure. There was evidence for external validity of the SERPA measurement as evidenced by group assignment (i.e. the intervention) exerting a direct effect on the proposed two-dimensional structure of latent self-efficacy to regulate physical activity in adults with obesity at 30 days post-baseline.

Validation of the Violent Ideations Scale (VIS) in Spain.

Research has shown that violent ideations (VIs) may play a key role in aggression and violence. However, there is no tool to measure this construct adapted to the Spanish language so far. The current study aims to translate and evaluate the psychometric properties of the Violent Ideations Scale (VIS) in European Spanish. This study counts with a convenience sample of 752 native Spaniards or Spanish speakers residing in Spain, between 18- and 75-years old, 53% identified as female and 47% identified as male. The dimensionality, gender measurement invariance, concurrent validity and reliability of scores were assessed. Initial evidence of external validity was collected. Results confirmed the unidimensionality of the instrument. Measurement invariance held across females and males and reliabilities were high. VIS-ES scores correlated with aggressive behavior during the previous month. The VIS-ES seems suitable for use in non-clinical Spanish speaking samples.

Short-term and long-term survival in patients with prevalent haemodialysis—an integrated prognostic model: external validation

ObjectivesPrognostic tools with evidence for external validity in routine clinical practice are needed to align care with patients’ preferences and deliver timely supportive services. Current models have limited, if any,...

Learning Approaches: Cross-Cultural Differences (Spain-Argentina) and Academic Achievement in College Students.

Learning approaches describe the students' degree of cognitive commitment to learning in diverse types of academic tasks and educational environments. Even though from a micro-level perspective different profiles of approaches have been identified in high-achievement undergraduates attending several majors, such profiles have not been examined from a macro-level approach in terms of distinct educational cultures. Therefore, the research involved two studies conducted on undergraduates from Argentina and Spain: The first one was aimed at analyzing the psychometric features of the Approaches and Study Skills Inventory for Students (ASSIST) whereas the second was focused on examining the learning approaches profiles of high and low achievers attending the same major (Psychology) in two different educational cultures (Spain and Argentina). The scale's original internal structure, examined on a sample of 400 participants (50% Spanish), was verified except for one item, which was fatherly eliminated. The resulting structure was tested and proven verified in a new sample (N = 1,334; 58.3% Spanish) by confirmatory factor analysis, factorial invariance, and internal consistency studies. External validity evidence was examined as well. Additionally, norms to be used in the professional field were calculated.Profiles of learning approaches by academic achievement from each country were examined by latent class analysis. In both cases, high achievers reported higher and more frequent use of the Deep and Strategic approaches and lower and less frequent usage of the Surface one. Further studies should replicate these analyses in undergraduates attending other majors in order to test the hypothesis sustaining these findings' generalization.

The Presentation of Interest Scale on Reflective Thinking and its Internal and External Validity

Studies on prediction recognize the presence of a large number of predictors of school-academic performance, as well as the existence of well-established predictors, nevertheless some psychoeducational variables remain little explored, even though their great potential. One of these is reflective thinking. The Laboratory for Research on Cognitive Architecture (LAICO) at the Federal University of Minas Gerais brought forth a contribution to reflective thinking. This is the development of the Interest Scale on Reflective Thinking which measures the interest of people in thinking reflectively. In this paper we present the Interest Scale on Reflective Thinking where it is shown in detail its structure and content. We also show evidence of structural validity, external validity and reliability of the scale.

Psychometric Properties of the Sport Multidimensional Perfectionism Scale-2 in Czech Adolescent Athletes: An Exploratory Approach

This study presents a thorough examination of the psychometric properties of the Czech version of the Sport Multidimensional Perfectionism Scale-2 in adolescent athletes. Factorial structure, external validity, and internal consistency were evaluated using a cross-sectional design. In addition, three-month temporal stability was also assessed. The cross-sectional study sample consisted of 243 respondents (50.2% females; mean age 16.40 years). Factorial structure of the Sport-MPS-2 was examined using exploratory factor analysis, evidence for external validity was based on bivariate and canonical correlations with Big Five Inventory-2 (short) and Sport Anxiety Scale-2. The longitudinal part of the study was performed on a subsample of 96 participants. Factorial structure in our sample fully corresponded to the structure suggested by the authors of the instrument. Internal consistencies of subscales were satisfactory. Regarding external validity, Doubts about Actions and Concern over Mistakes were positively related to sport anxiety, and positive relationship was found also between Concern over Mistakes and Negative Emotionality. No substantial relationships were found in case of subscales constituting perfectionistic strivings. Temporal stability of Sport-MPS-2 over three months was satisfactory. The Czech version of Sport-MPS-2 can be considered a reliable instrument for assessing perfectionism in sport in the adolescent population.

Study of the Psychometric Properties of the Social Self-Efficacy Scale with Spanish Adolescents by Gender, Age and Family Socioeconomic Level.

Social self-efficacy has been shown to be a key resource for adolescents’ social experiences with peers and a predictor of prosocial behaviour among adolescents. However, differences by gender, age and socioeconomic level have previously been found in social self-efficacy. The objective of this study is to assess the psychometric properties of the subscale of social self-efficacy from the Self-Efficacy Questionnaire for Children (SEQ-C) developed by Muris (2001) in a representative sample of Spanish adolescents while considering gender, age and socioeconomic level differences. In general, the results showed good psychometric properties and a one-dimensional structure with high internal consistency, adequate explained variance and evidence of external validity for the subscale. Furthermore, the invariance analysis demonstrated that the social self-efficacy subscale shows no bias when used with populations of adolescents who differ by gender, age and socioeconomic level. The results indicate that the Spanish version of the social self-efficacy subscale of the SEQ-C is an adequate measurement instrument for assessing adolescents’ perception of their own social skills.

What People Say and How People Play: Mixed Methods and Multiple Measures of Fishing Behavior with Implications for External Validity

Does resource extraction during experimental economic games reflect resource use and behavior in other contexts? This question about external validity is central to determining the inferences that can be drawn from experimental evidence to broader sets of circumstances. Building on previous studies of external validity, which often raise concerns about the generalizability of experimental economic games to other contexts, we instead highlight parallelism between behavior inside and outside of games and its implications for applied research. Here we present the results from a framed multiphase common-pool resource game played with fishermen from a fishing community in Baja California Sur to explore the associations between game play and both self-reported and observed behavioral measures. Participants played an experimental economic game that incorporated both fisheries-specific resource dynamics and the current policy context of local fisheries management. Game administration followed two years of interviews, participant observation, and household surveys in the community. In addition to providing an in-depth understanding of the broader social-ecological context, this allowed us to compare similar game, self-reported, and observed behaviors for the same individuals. Overall, we found little evidence for external validity in terms of parallelism. None of the tests for association between fishing behaviors in the game and survey met the study-wide significance threshold. Only one association—between game behavior and conservation values—was significant at the single-test level. Based on our results, we caution against relying solely on experimental economic games as proxies for behaviors regarding resource use when developing policies. Collecting multiple or iterative measures of behavior using mixed methods is a necessary precaution, especially if the goal is to make specific policy recommendations.