Evidence Of Validity Research Articles

Construction and validation of a test for verbal auditory screening of cognitive alterations (CAVAC)

The global burden of dementia will continue to increase and Latin America will be one of the most affected regions. It is urgent to have new and better cognitive screening tools. There are few screenings composed entirely of auditory-verbal items and even more so for Argentine culture. The objective was to build and validate a test for verbal auditory screening of cognitive alterations (CAVAC) in Argentina. Evidence based in test content was analyzed through 16 experts collaborated. In addition, evidence based on relationships to other variables, internal consistency and the ability to discriminate between cognitively healthy subjects, Mild Cognitive Impairment (MCI) and dementia were investigated. The CAVAC was applied to 316 Argentines (M age = 68,5). Volunteers from the general population and patients from neurology services from two Argentine hospitals participated. The CAVAC test explores orientation/attention, memory/learning, language and executive functions, and is administered in 10 minutes with a score of 0 to 30. Aiken's V coefficient was satisfactory for all items. The CAVAC achieved a high correlation with the Mini Mental (0.762) and an acceptable Cronbach's alpha (0.746). The CAVAC score differs between controls, mild cognitive impairment (MCI) and dementia, and the ROC curves reveal adequate area values ??to discriminate MCI and dementia (>0.7), and cut-off points with good sensitivity-specificity. A new auditory-verbal test is made available for the screening of cognitive alterations with acceptable evidence of validity, good reliability, with the capacity to identify MCI and dementia, and free of charge for use.

What constitutes valid evidence of causation? Gas stoves and childhood asthma revisited

What constitutes valid evidence of causation? Gas stoves and childhood asthma revisited

Courage Measure (CM) – Cross-Cultural Adaptation and Psychometric Properties for Brazilian Adolescents

Abstract: The objective of the study was to adapt the Courage Scale and estimate validity evidence based on content and internal structure. The study included 510 young people, between 14 and 19 years old. The results of the content validity coefficient suggested the adequacy of the content. Confirmatory factor analysis (CFA) was used to test three factor models, namely one-factor, two-factor, and the random intercept model. The results indicated that it is possible to measure courage through a factor, when the acquiescent response style is controlled, corroborating the theoretical perspective, given that the random intercept model adjusted well to the sample data. In other words, the findings indicate that the scale is capable of measuring courage in the Brazilian context and that acquiescence control is an important method in evaluating the construct.

Best practices for your confirmatory factor analysis: A JASP and lavaan tutorial.

Confirmatory factor analysis (CFA) is a fundamental method for evaluating the internal structural validity of measurement instruments. In most CFA applications, the measurement model serves as a means to an end rather than an end in itself. To select the appropriate model, prior validity evidence is crucial, and items are typically assessed on an ordinal scale, which has been used in the applied social sciences. However, textbooks on structural equation modeling (SEM) often overlook this specific case, focusing on applications estimable using maximum likelihood (ML) instead. Unfortunately, several popular commercial SEM software packages lack suitable solutions for handling this 'typical CFA', leading to confusion and suboptimal decision-making when conducting CFA in this context. This article conceptually contributes to this ongoing discussion by presenting a set of guidelines for conducting a typical CFA, drawing from recent empirical research. We provide a practical contribution by introducing and developing a tutorial example within the JASP and lavaansoftware platforms. Supplementary materials such as videos, files, and scripts are freely available.

Validity Evidence Of Step Count Cut-points To Determine The Moderate-to-vigorous Physical Activity Levels

Validity Evidence Of Step Count Cut-points To Determine The Moderate-to-vigorous Physical Activity Levels

Reliability and Validity Evidences of Tej Emotional and Behavioral Problem Checklist (TEJ-CL) for Child Mental Health Assessment in Nepal

Introduction: The Tej Emotional and Behavioral Problem Checklist (TEJ-CL) was developed in Nepalese context to aid assessment of childhood emotional and problems of children. This study aimed to evaluate TEJ-CL's factor structure, reliability, and validity evidences as an add-on and symptom monitoring test.Methods: This cross-sectional validation study included guardians of 320 children (age 5 – 17 years) from tertiary mental health centers in Kathmandu as referred group, along with 601 children from two schools (private and community) in Kathmandu as non-referred group.Study was conducted after obtaining ethical approval (ref: 183 (6-11-E)2/073/074 and ref: 776). TEJ-CL, an 89-item parent-reported questionnaire, served as the index test, while referral status acted as the reference standard. Factor structure, internal consistency, test-retest/ cross-informant correlations and criterion validity evidence was assessed using principal component analysis, coefficient alpha, spearman's rank correlation and linear regression models, respectively.Results: Analysis was done using 179 referred and 412 non-referred individuals based on non-missing data. Principal component analysis in referred sample reduced the number of items of questionnaire to 65 from 89 and indicated six factors: externalizing behavioral issues, anxiety/worries, upset/sadness, somatic concern, miscellaneous syndrome, and severe issues with coefficient alpha ranging from .62 to .95. As criterion validity evidence, referred children showed significantly higher scores than non-referred children across composite and factor scores, except for anxiety/worries factor. Similarly, regression analyses within the referred group demonstrated significant associations between factor scores and specific diagnoses.Conclusions: Reliability and validity estimate of questionnaire is comparable to similar empirically based scales. Future research should focus on assessing the tool's generalizability and improving discriminatory indexes.

Knowledge assessment tool for pediatric parenteral nutrition: A validation study.

Parenteral nutrition (PN) is a high-risk medication, and its prescription and administration requires extensive training. Difficulties inherent with bedside teaching have made teaching these concepts challenging. Currently, no knowledge assessment tools with validity evidence exist to test the effectiveness of new PN teaching interventions. We sought to develop and provide validity evidence for a pediatric PN knowledge test to measure the effectiveness of future teaching interventions. We created a multiple-choice question knowledge assessment tool that underwent content validation by PN experts and was emailed to potential participants. We evaluated the knowledge assessment tool for factorial validity, internal consistency reliability, and discriminant validity. We enrolled 103 medical students (40%), residents and fellow trainees (55%), and attending physicians (5%) into the study between October 2021 and October 2022. Five of the 30 questions performed poorly based on their nonsignificant contribution to the primary aim of assessing pediatric PN knowledge. Following the exclusion of those questions, the knowledge assessment tool demonstrated an acceptable model fit, and the root mean squared error of approximation was <5%. The omega coefficient was 0.829, indicating acceptable levels of reliability, and using an analysis of variance test (ANOVA) demonstrated significant differences between groups, showing good discrimination between levels of experience (F[2, 80] = 39.002; P < 0.001). We have developed and provided validity evidence for a multiple-choice question knowledge test that may be used by educators and programs to evaluate knowledge of pediatric PN in physicians and trainees.

Development and Validation of a Comprehensive English Test for Sophomore Students in Liaoning, China

The English proficiency of Chinese college students varies in different regions and majors. For college students, it’s of great importance to know their strengths and weaknesses in the process of language learning and to have targeted practice . For language teachers, it is quite significant to identify the problems and needs of students in English learning, to adjust the teaching methods and contents and to improve the teaching effect. At the same time, the feedback of the test results can also promote the school’s attention to and investment i n English teaching and promote the in depth reform of English teaching. In this procedure, a comprehensive English test is an effective way. This study describes developing and validating a comprehensive English test for sophomore students at Shenyang Univ ersity of Chemistry and Technology in Liaoning, China. The test is designed to assess students' English proficiency upon completing the "New Century College English Reading" course. The research employed a mixed methods approach, combining qualitative meth ods for test development and expert review with quantitative methods for statistical validation. The test development process included content analysis of course materials, creation of test specifications aligned with course objectives, and item writi ng. T he test covers four key language domains: listening, reading, translation, and writing. Validation is conducted through expert r eview using a content, construct, and face validity checklist. Five experienced English language teaching and assessment experts evaluated the test using a 15 item checklist with a 5 point Likert scale. Results showed high overall validity, with mean scores above 4.0 for content validity, construct validity, and face validity, indicating strong agreement among experts on the test' s validation. While the study provides strong evidence for the test's validity, limitations include the sample size and geographic scope. Future research directions include longitudinal studies to assess predictive validity and cross regional validation to evaluate the test's applicability across China's educational contexts. This study contributes to English language assessment in Chines e higher education in Liaoning province. It provides a validated tool that aligns closely with curriculum objectives and a ddresses the need for comprehensive language proficiency evaluation.

Artificial intelligence for automatic and objective assessment of competencies in flexible bronchoscopy.

Bronchoscopy is a challenging technical procedure, and assessment of competence currently relies on expert raters. Human rating is time consuming and prone to rater bias. The aim of this study was to evaluate if a bronchial segment identification system based on artificial intelligence (AI) could automatically, instantly, and objectively assess competencies in flexible bronchoscopy in a valid way. Participants were recruited at the Clinical Skills Zone of the European Respiratory Society Annual Conference in Milan, 9th-13th September 2023. The participants performed one full diagnostic bronchoscopy in a simulated setting and were rated immediately by the AI according to its four outcome measures: diagnostic completeness (DC), structured progress (SP), procedure time (PT), and mean intersegmental time (MIT). The procedures were video-recorded and rated after the conference by two blinded, expert raters using a previously validated assessment tool with nine items regarding anatomy and dexterity. Fifty-two participants from six different continents were included. All four outcome measures of the AI correlated significantly with the experts' anatomy-ratings (Pearson's correlation coefficient, P value): DC (r=0.47, P<0.001), SP (r=0.57, P<0.001), PT (r=-0.32, P=0.02), and MIT (r=-0.55, P<0.001) and also with the experts' dexterity-ratings: DC (r=0.38, P=0.006), SP (r=0.53, P<0.001), PT (r=-0.34, P=0.014), and MIT (r=-0.47, P<0.001). The study provides initial validity evidence for AI-based immediate and automatic assessment of anatomical and navigational competencies in flexible bronchoscopy. SP provided stronger correlations with human experts' ratings than the traditional DC.

Validity Evidence for the Behavioral and Emotional Rating Scale—3: Factor Analysis and Measurement Invariance Across Race and Ethnicity

Validity Evidence for the Behavioral and Emotional Rating Scale—3: Factor Analysis and Measurement Invariance Across Race and Ethnicity

Campaign-Induced Interpersonal Communication: Concepts and Measurement

One way in which media campaigns produce effects is by stimulating talk about the campaigns among audience members. Scientific analysis of that talk requires a means of measuring it, something that is lacking in the current literature. To advance research on this topic, we developed a campaign-induced interpersonal communication (CIC) coding scheme. The scheme was applied to conversation data gathered from participant pairs who were exposed to health-related public service announcements (PSAs), then discussed each one. Results showed that the mutually exclusive and collectively exhaustive set of codes could be applied reliably to conversation data. Evidence of construct validity was seen in data patterns that demonstrated dyadic movement toward intersubjectivity, responsiveness to task demands, and over time changes in features of the conversations, all of which are defining features of interpersonal communication. The CIC coding scheme provides a foundation for researchers interested in studying interpersonal processes associated with media campaigns.

Navigating Ambiguity: Adapting and Validating the Personal Need for Structure Scale in Spanish.

The Personal Need for Structure (PNS) scale assesses individuals' tendency to seek out clarity and structured ways of understanding and interacting with their environment. The main aim of this study was to adapt the PNS scale to Spanish and assess its psychometric properties. There are two versions of the PNS scale being used, which vary in the number of dimensions (1 vs. 2), and in the number of items (12 vs. 11; because one version excludes Item 5). Therefore, an additional aim of this study was to compare the two existing versions of the PNS scale. This comparison aimed to address the debate regarding the inclusion of Item 5, and the number of dimensions that comprise the PNS scale. A sample of 735 individuals was collected. First, through an approach combining exploratory and confirmatory analyses, evidence was found in favor of the scale being composed of two related but distinguishable factors: Desire for Structure and Response to the Lack of Structure. Scores on these subscales showed acceptable internal consistency and test-retest reliability. Evidence supporting the invariance of the internal structure across sociodemographic variables such as gender and age was found. Validity evidence was also analyzed by examining the relationships with other relevant measures. The results indicated that Item 5 can be excluded without reducing scores validity or reliability, which supports preceding research in the literature. In conclusion, the PNS scale was satisfactorily adapted to and validated in Spanish and its use in this context is recommended.

Preliminary Reliability and Validity of the Self-Determination Inventory: Student Report French Translation.

There is a strong and growing focus on self-determination in French-speaking countries, and this pilot study reports the technical adequacy of the Self-Determination Inventory: Student Report (SDI:SR) French Translation. Data were collected with 471 French-speaking youth with and without disabilities in Canada (Quebec), Switzerland, France, and Belgium. Key findings showed it was feasible to use 20 (of 21) items to represent the self-determination construct in the French-speaking sample. The same set of items function in the same way across students with and without disabilities, and students with disabilities descriptively scored lower. Overall, this study provides promising evidence for reliability and validity of the SDI:SR French Translation and suggests ongoing development and larger-scale testing of the SDI:SR French Translation is warranted.

The Affective Neuroscience Personality Scales (ANPS) in Slovenia: Validation of the Six Primary Scales Across Three ANPS Versions

Abstract To assess primary emotions in subcortical brain regions across all mammals, the affective neuroscience personality scales (ANPS) were created for use in research involving human subjects. Later revisions of the scales have been validated across many countries and are used in diverse fields of psychology. The ANPS revisions culminated in the most recent ANPS 3.1. In this study, we evaluated the psychometric properties of the ANPS 3.1 and its two abbreviated versions (affective neuroscience personality scales – brief (BANPS) and affective neuroscience personality scales – short version (ANPS-S)) in a Slovenian community sample of 502 young adults. We simultaneously examined several kinds of validity evidence across the three versions: construct validity, internal reliability, and convergent validity. Our findings revealed acceptable construct validity of the six-factor model of the BANPS that was superior to the ANPS-S and particularly to the ANPS 3.1. The latter exhibited incremental fit issues noted in previous versions. However, we revealed sufficient internal reliability and convergent validity of the scale scores against the Big Five personality traits, and the frequency of the participants’ recently felt positive and negative emotions across all three ANPS versions. Testing the measurement invariance of BANPS across sex suggested full metric invariance and partial scalar invariance which allows direct score comparisons between males and females.

Escala de Clima de Innovación: análisis psicométrico en trabajadores peruanos

the changes caused by the COVID-19 pandemic pushed organizations to attempt to remain current and competitive. Thus, the abilityto respond to new demands and adapt to shifting conditions was an unavoidable requirement, which implies innovating in the development oftheir processes. In that sense, the growing importance of innovative work behaviors in the development of organizations has driven researchersto investigate mechanisms that foster such behaviors in workers. The objective of this study was to analyze the psychometric properties of theInnovation Climate Scale (ICS). The instrumental study involved 273 Peruvian workers (50.916 % male) ranging in age between 22 and 76 yearsold (Mean = 45.224), employed for the most part in the private sector (75.092 %). The psychometric analysis explored item clarity and relevanceby means of assessment by expert judges (content validity). Internal structure was assessed via confirmatory factorial analyses, both of obliquemodels and bi-factor. Finally, scoring reliability (alpha coefficient) and construct reliability (omega coefficient) were estimated. This study findsthe ICS items to be clear and relevant, a greater empirical support for the unidimensional model as its internal structure, and an adequate relia-bility. In conclusion there is favorable evidence of validity and reliability for the application of the ICS in Peruvian companies as a useful tool tomeasure the innovative climate perceived by its workers, providing information for decision making and planning of innovation strategies andorganizational creativity

A new neuropsychological tool for simultaneous reading and executive functions assessment: initial psychometric properties.

The development of reading and complex executive functions is fundamental for achieving social, academic, and professional success. So far, there is no single neuropsychological instrument that comprehensively assesses the domains of inhibitory control, cognitive flexibility, working memory, and reading comprehension. To assess executive functions related to reading, the "Assessment of Reading and Executive Functions" (AREF) was developed. In this study, we show initial evidence of validity and reliability for four subtests - Graphophonological-Semantic Flexibility, Inhibitory Control, Flexibility, and Working Memory. A total of 93 students from 4th to 9th grade, aged 8-14, in public (n = 61) and private (n = 32) schools were evaluated. Tasks from the AREF instrument, as well as measures of reading comprehension, inhibitory control, cognitive flexibility, working memory, and intelligence, were administered. Correlations between AREF scores and the other measures were performed to assess external construct validity. Performance differences between school groups on AREF subtests were analyzed using ANOVA, t-test, and Mann-Whitney tests, and the internal consistency of the instrument's tasks was evaluated using Cronbach's alpha coefficient. The scores of the AREF subtests demonstrated significant positive correlations with reading measures (ranging from 0.339 to 0.367) and executive functions (ranging from 0.209 to 0.396). Significant differences were found in the performance of some AREF tasks when comparing individuals from public and private schools, as well as between 4th and 5th graders compared to students in higher grades. The internal consistency of the tasks was low for Graphophonological-Semantic Flexibility (Cronbach's α = 0.566), moderate for Inhibitory Control and Flexibility (Cronbach's α = 0.768), and high for Working Memory (Cronbach's α = 0.881). The results provide initial evidence of construct validity and reliability for the AREF subtests. It is expected that this new neuropsychological test will contribute to the assessment of reading skills and executive functions, assisting in guiding clinical and educational interventions for individuals with and without neurodevelopmental disorders.

Developing and validating an L2 writing willingness to communicate scale: A sequential embedded mixed-methods approach

Current willingness to communicate (WTC) scales center on WTC in general second language (L2) learning, while L2 writing WTC is underrepresented. This study intended to close this gap by developing and validating an L2 writing WTC scale. A three-phase sequential embedded mixed-methods design was adopted to overcome the over-reliance on quantitative data and provide adequate evidence of validity. Nineteen items were generated based on our literature search and thematic analysis of the interview data ( n = 10). With quantitative data collected from 288 learners of English as a foreign language (EFL), the psychometric properties of the initial scale were examined by exploratory factor analysis. After that, the revised 17-item questionnaire was validated by confirmatory factor analysis and other validation methods with data from 224 EFL learners. The results indicated that the underlying structure involved writing task traits, English language ideology, writing teacher support, interest in English language, and self-perception of English language proficiency. The scale was further validated through factor analysis of the quantitative data ( n = 173) and thematic analysis of the immediate retrospective interview data ( n = 12) from EFL learners to test its generalizability in other L2 learning contexts and for face validity evidence. The findings showcased a promising mixed-methods design for scale development and clarified the underlying factors of L2 writing WTC. Implications for scale development and the teaching and learning of L2 writing were discussed.

Faculty verbal messages: scale development and initial validation

IntroductionThis study set out to create a scale of messages faculty use when communicating with students and provide initial validation evidence for that scale.MethodsThree studies were used to create and initially validate the scale. Study one used focus groups to have students generate a list of 65 messages. Study 2, a survey with exploratory factor analysis, yielded two factors: affirming (items) and disconfirming (items) messages. Study 3 confirmed the factor structure and provided initial construct validation based upon the relationship between messages and relational distance, motives to communicate, learner empowerment, and student motivation.ResultsWithdrawal mediates the relationship between faculty verbal messages and learner empowerment. Faculty verbal messages directly, without mediation, predict student motivation.DiscussionFaculty verbal messages matter inside and outside the classroom. Both affirming and discomfiting messages can have a positive effect on students’ learning/motivation as well as the instructor-student relationship.

Toward inclusive and equitable LGBTIQ+ measurement: Assessing gender and sexual orientation measures and scale validity in national surveys across 21 countries

AbstractDespite growing global interest in lesbian, gay, bisexual, trans, intersex, and queer/questioning (LGBTIQ+) research, variations in measurement practices across countries have remained underexplored. In this work, we focused on two fundamental aspects of measurement vital to understanding the experiences of LGBTIQ+ people. Specifically, we documented current measures of sex, gender, and sexual orientation used in national mental well‐being‐related surveys and reviewed whether the mental well‐being scales in those surveys have been validated for LGBTIQ+ people. We employed a stratified sampling strategy and evaluated national surveys from a list of randomly selected countries representing 10% of global nations (N = 21). Fewer than half of the countries measured sexual orientation and fewer than one‐third measured gender beyond the binary in their national surveys. Among the countries that measured gender or sexual orientation, the response options and question phrasing were often not inclusive. In addition, most of the mental well‐being scales lacked validity evidence for LGBTIQ+ populations. Finally, we outline recommendations for the future of reimagining LGBTIQ+ research in terms of measurement, highlighting the importance of research engagement with the global LGBTIQ+ community.

Phubbing in Adolescents: Spanish Validation of the Phubbing Scale (PS).

The Phubbing Scale (PS) is an instrument used to measure the frequency and extent of the behavior of ignoring someone you are with and giving attention to your mobile phone instead. However, there is insufficient evidence about the psychometric adequacy of the Spanish version of the instrument. The main goal of this research was to analyze the psychometric properties of PS in a representative sample of Spanish adolescents and young adults. A total of 1351 participants comprised the sample (42.78% females, age range = 12-21). Students were selected from different levels of education such as secondary school, high school, vocational training, or university. A convenient sample was used. The reliability of the scores was calculated by means of McDonald's Omega. The evidence of the internal structure of the questionnaire was analyzed by means of confirmatory factor analysis (CFA). The measurement invariance of the instrument by gender and educational level was also calculated. In addition, Pearson's correlations between phubbing and other indicators of mental health were also calculated. The goodness-of-fit indices for the two-factor model were good. The McDonald's Omega coefficient for the total score was 0.787. Measurement invariance both by gender and educational level was found. The phenomenon of phubbing was found to have statistically significant correlations with emotional well-being, other mental health indicators, and with Problematic Internet Use (PIU), with the sole exception of the hyperactivity subscale of the SDQ. This study provided validity evidence for the Spanish version of the Phubbing Scale (PS), suggesting that PS is a reliable tool for quantifying phubbing in Spanish adolescents.