Evidence Of Predictive Validity Research Articles

The cognitive structure of gross anatomy knowledge in physiotherapy students.

Cognitive structures are the mental representation of domain knowledge and its organization. A preliminary investigation of the cognitive structure of gross anatomy knowledge was conducted on physiotherapy students. The criterion-related validation study examined two data modeling strategies (multidimensional scaling and Pathfinder networks) as potential visual and quantitative representations of cognitive structure. Two criterion standards were used: expert cognitive structure (concurrent) and the student's unit grade (predictive). The raw data for both data modeling strategies were generated from an online survey of paired comparisons of 20 anatomical structures and concepts relevant to musculoskeletal clinical practice. Convenience sampling was used to recruit first-semester physiotherapy students (n = 31), gross anatomy course instructors (n = 4), and domain experts (n = 3) who completed the online survey. The results indicated moderate-to-high effect sizes, regarding the level of agreement (reliability, accuracy, and association) between student and expert cognitive structures. Multiple regression analysis was performed to examine the potential relationships with unit grades. Six predictor variables accounted for 68.9% of the variance in unit grade, indicating a large effect size. The results provide preliminary evidence of concurrent and predictive criterion-related validity for using data modeling strategies to represent cognitive structure in this knowledge domain and population. Further research is indicated to assess the potential impact of this innovative use of data modeling strategies for cognitive structure mapping on gross anatomy education, adaptive learning, and competency-based education, leading to the long-term development of expertise.

Clinical utility and psychometric properties of tools for early detection of developmental concerns and disability in young children: A scoping review.

To explore the clinical utility and psychometric properties of standardized tools for the early detection of developmental concerns or disability in young children. Systematic reviews and clinical practice guidelines containing psychometric data on tools appropriate for use with children from birth to 5 years 11 months were searched for in MEDLINE, CINAHL, Embase, and PsycINFO for the years 2000 to 2023, with no language restrictions. Eighty-six systematic reviews and six clinical practice guidelines guided identification of tools. A total of 246 tools were identified across domains of neurological, motor, cognition, communication/language, social-emotional, sensory processing, and/or specific diagnostic conditions of attention-deficit/hyperactivity disorder, autism spectrum disorder, cerebral palsy, developmental coordination disorder, and fetal alcohol spectrum disorder. After critical evaluation, 67 tools were included in the recommendations. Recommendations for screening and diagnostic assessment tools were based on best available evidence for predictive and discriminative validity, diagnostic accuracy, together with consideration of resource use and accessibility. This comprehensive scoping review provides recommendations on the best tools for primary care, medical, allied health professionals, nursing, and other health workers to detect and identify developmental concerns or disability in young children using evidence-based tools.

Validation of the Client Satisfaction Scale for Youth in Residential Psychiatric Care

ABSTRACT Youth with severe mental and behavioral health symptoms often require high levels of care, such as residential placement. Given the acuity of the residential population, the high cost of services, and the separation of children from their families, remarkably little research exists on engagement, mechanisms, and outcomes. Treatment satisfaction is a critical process and outcome indicator but is not always assessed in residential care. When satisfaction is assessed, the quality of the assessment tools varies and is rarely assessed as part of treatment progress, so little is known about how and if it might change during treatment. This study aimed to examine the psychometric properties of the newly developed Client Satisfaction Scale (CSS) for youth in residential psychiatric care. The CSS was administered to all patients in the current study (N = 2,933, M age = 15.36, SD = 1.35). Results of exploratory and confirmatory factor and item response theory analysis support evidence of structural validity, and bivariate correlation and regression supported evidence of concurrent and predictive validity, respectively. Findings suggested the CSS scale may be a viable tool for assessing youth satisfaction with treatment outcome and services at residential treatment centers.

Validating a Pragmatic Measure of Evidence-Based Practice (EBP) Delivery: Therapist Reports of EBP Strategy Delivery and Associations with Child Outcome Trajectories.

Pragmatic measures of evidence-based practice (EBP) implementation can support and evaluate implementation efforts. We examined the predictive validity of therapist reports of EBP strategy delivery for children's mental health outcomes. Data were obtained from 1,380 sessions with 248 children delivered by 76 therapists in two county systems. Children (Mage=11.8 years, SD = 3.7) presented with internalizing (52%), externalizing (27%), trauma (16%), and other (5%) concerns. Therapists reported their delivery of EBP strategies on a revised version of the EBP Concordant Care Assessment (ECCA; Brookman-Frazee, et al., Administration and Policy in Mental Health and Mental Health Services Research, 48, 155-170, 2021) that included 25 content (e.g., parenting, cognitive behavioral) and 12 technique strategies (e.g., modeling, practice/role-play). On average, 5.6 ECCA session reports (SD = 2.3) were obtained for each client, and caregivers reported symptoms on the Brief Problem Checklist (Chorpita, et al., Journal of Consulting and Clinical Psychology, 78(4), 526-536, 2010) at baseline, weekly over two months, and again at four months. Multilevel models examined whether the mean extensiveness of each EBP strategy predicted trajectories of child outcomes. More individual technique (6 of 12) than content strategies (1 of 25) were associated with outcome trajectories. For techniques, more extensive use of Performance Feedback and Live Coaching and less extensive use of Addressing Barriers were associated with greater declines in total symptoms, and more extensive use of Establishing/Reviewing Goals, Tracking/Reviewing Progress, and Assigning/Reviewing Homework was associated with declines in externalizing symptoms. For content, more extensive use of Cognitive Restructuring was associated with declines in total symptoms. In addition, higher average extensiveness ratings of the top content strategy across sessions was associated with greater declines in total and externalizing symptoms. Therapist-reported delivery of some EBP strategies showed evidence of predictive validity and may hold utility in indexing quality of care.

The PERMA+4 Positive Functioning at Work Scale: Spanish Adaptation and Validation.

Donaldson & Donaldson (2021b) developed a scale to assess Positive Functioning at Work, adding four new blocks to strengthen the original PERMA framework in the workplace. The aim of the present study was to validate this scale in a Spanish sample. 698 Spanish workers participated, 59% were women and 41% were men, 71.6% were employees and 71.2% belonged to the basic worker level. Confirmatory Factor Analysis showed a good fit of the nine-factor model (χ/df = 2.441; RMSEA = .056, 90% IC (.049, .063); SRMR =. 062; CFI = .926; TLI = .916), the bifactor model (χ/df = 1.703; RMSEA = .044, 90% CI [.037, .050]; SRMR = .050; CFI = .959; TLI = .952) and the ESEM model (χ/df = 1.736; RMSEA = .041, 90% CI [.033, .049]; SRMR = .017; CFI = .976; TLI = .942). The adapted scale also demonstrated good internal consistency, with a Cronbach's α of .90 for the scale, and evidence of discriminant, criterion, predictive, and incremental validity with other well-being and performance measures, as well as measurement invariance across educational levels. This scale provides a comprehensive measurement tool that can help in the design of workplace programs and interventions.

A Psychometric Evaluation of the Couples’ Financial Communication Scale: Findings and Implications from Two Large, Diverse Samples

AbstractCommunicating about finances is essential to develop shared meaning and goals within couple relationships. When couples struggle to discuss finances, they can experience poor couple outcomes. For researchers and clinicians to effectively study and promote healthy couple communication patterns regarding finances, a parsimonious, reliable, and valid measure of couples’ financial communication is needed. This study examined the psychometric utility of the 5-item Couples’ Financial Communication Scale (originally developed for the Flourishing Families Project; Day, R. D., Bean, R., Coyne, S., Dyer, J., Harper, J., & Walker, L. (2017). Flourishing families project: Survey of family life [codebook].) using two large, diverse samples—one of emerging adult individuals in a romantic relationship (N = 1,950) and another of dyads in a romantic relationship (N = 1,252; 69.9% beyond emerging adulthood). Similar findings emerged across both samples. Inter-item correlations, skewness, and kurtosis of the five items were within acceptable ranges. The five items loaded onto a latent construct with robust standardized factor loadings (ranging from 0.63 to 0.90) and sound model fit. Cronbach’s alpha revealed sound reliability (α = between 0.85 and 0.89). Multiple tests of measurement equivalence suggest the measure appears to be reasonably useful across theoretically meaningful groups (gender, age, income, marital status, and joint banking behaviors). Couples’ financial communication and couples’ relationship quality were positively correlated with large effect sizes—showing initial evidence of predictive validity. The parsimonious Couples’ Financial Communication Scale has sound evidence of reliability, validity, and measurement equivalence across two diverse samples, which positions it to be a useful measure in future scholarship to assess the degree to which couples engage in ongoing healthy and cooperative financial communication.

Chronic and contextual identity salience: Assessing dual-dimensional salience with the Identity Salience Questionnaire (ISQ)

ABSTRACT Identity Salience is a common construct within social identity research. However, researchers note that it is poorly defined and inconsistently operationalized. We posit that identity salience comprises two elements: chronic (perpetually thinking about the identity) and contextual (only thinking about the identity when prompted) salience. We present evidence for this claim through the development and validation of the dual-dimensional Identity Salience Questionnaire (ISQ). Studies 1-2 (Ns=414; 1,069) provide exploratory and confirmatory factor analytic evidence of the ISQ among LGBTIQA+ participants. Study 2 also provides evidence of measurement invariance, convergent, predictive, and discriminant validity, and internal reliability. Study 3 (N=318) indicates strong test-retest reliability. Study 4 (N=107 social psychologists) confirms the ISQ’s content validity. Future research for the ISQ is discussed.

Testing an online screening for autism in the COVID-19 pandemic: a psychometric study of the Q-CHAT-24 in Chilean toddlers.

The aim of this study was to examine some psychometric characteristics of the Chilean-adapted version of the Quantitative Checklist for Autism in Toddlers (Q-CHAT-24) (24) in a group of unselected children (community sample). This version was administered remotely through an online version during the pandemic period to caregivers of children, aged 18-24 months, registered in four primary care polyclinics of the Health Service Araucanía Sur, Chile. An intentional non-probabilistic sampling was used. Three hundred and thirteen toddlers were examined. Participants completed an online version of the Q-CHAT-24 which was disseminated through the REDCap platform. Evidence of reliability through internal consistency and evidence of predictive validity through ROC curve analysis were realized. The mean age of the children evaluated was 21.16 months. The Shapiro-Wilk test revealed that Q-CHAT-24 scores was normally distributed. 71 cases (23.12%) scored 38 points or more on the Q-CHAT-24, qualifying as Autistic Risk. 48 cases (15.63%) were confirmed as autistic through the ADOS-2 Module T. All items were positively correlated with Q-CHAT-24 total score. All items were positively correlated with Q-CHAT-24 total score. Internal consistency was acceptable for the Q-CHAT-24 (Cronbach ´s α=0.78). The internal consistencies were analyzed for the Q-CHAT-24 Factors, and they were good for factor 1 "Communication and Social Interaction" (Cronbach ´s α=0.85) and acceptable for factor 2 "Restrictive and Repetitive Patterns" (Cronbach ´s α=0.74). Receiver operating characteristic (ROC) curve analyses were performed. The AUC values were 0.93 with statistical significance (p<0.01). For the cut-off point of 38, the Sensitivity, Specificity and Youden index values were 0.89, 0.8 and 0.7, respectively. The Positive Predictive Value (PPV) was 86% and the Negative Predictive Value (NPV) was 85%. In accordance with the objectives of this study, evidence of reliability and predictive validity was demonstrated for the Q-CHAT-24 in this Chilean population. More importantly, this study provides Sensitivity and Specificity data for a remote application version of an autism screening tool already validated in Chile. The implications of this have to do with the possibility of establishing a remote assessment system for children at risk of autism on a population scale.

Assessing workplace coachability: a multi-study approach to design and validation

ABSTRACT Coaches’ subjective evaluation has been relied upon to assess clients’ coachability. The validation of a coachability measure could aid in more accurately evaluating the impact of coaching, creating strategies to improve the effectiveness of coaching interventions, and guiding future research. We developed and validated a workplace coachability scale using the framework proposed by Cavanaugh and colleagues, which included: awareness, openness, vulnerability, growth orientation, and external support. We took a multi-study approach in which items were generated, administered to multiple samples, refined via exploratory and confirmatory factor analyses (CFA), and investigated for validity. The CFA resulted in a 17-item scale supporting the five proposed factors. We found evidence of convergent, divergent and predictive validity. Specifically, pre-programme self-rated coachability was positively correlated with post programme engagement in leadership training and programme impact as rated by the coach (engagement r = .26, p = .01; impact r = .23, p = .03); and, pre-programme self-rated coachability was positively correlated with post programme self-rated coaching engagement processes and outcomes (r’s = .18–.22, p = .02–.05). We propose that leveraging a validated workplace coachability assessment can help coaches and clients have open discussions to prepare for and maximise the engagement.

Validation of the Youth Internalizing Problem Screener in Singapore

The use of a screening tool for school-wide screening of internalizing symptoms is an important strategy for early identification and prevention of more serious and impairing emotional and behavioral health problems in adolescents. However, threshold cut-off scores determined for screening tools may not be suitable for all populations. Using a sample of 237 Singaporean secondary school students, this study validated the Youth Internalizing Problems Screener (YIPS) for local use. Results of confirmatory factor analyses supported a one-factor solution for the construct. A threshold cut-off score of 27 was found to show good classification accuracy based on receiver operating characteristics (ROC) analyses. Correlational and path analyses provided evidence of convergent and predictive validity for using YIPS to indicate at-risk status. The YIPS status was uniquely associated with girls’ sense of school well-being over and above the nature of their interpersonal relationships and their sense of inadequacy. Overall, YIPS demonstrated comparable sensitivity and specificity rates even though a different cut-off score was used for this study sample. The use of YIPS as a screening tool in a multitier system of support and directions for future development were discussed.

The 7 Elements Communication Rating Form for Teaching and Assessing Medical Communication Competency.

Medical communication skills are a critical component of clinical medicine and patient satisfaction.Communication skills are difficult to teach and evaluate, necessitating tools that are effective and efficient. This study presents and validates the 7 Elements Communication Rating Form (7E-CRF), a streamlined, dual-purpose, evidence-based medical communication checklist that functions as a teaching and assessment tool. A 14-item teaching and assessment tool is described and validated using face, concurrent, and predictive validity indices.The study was conducted with 661 medical students from the West Virginia School of Osteopathic Medicine (WVSOM).Student performance was assessed in year 1 labs, year 2 labs, and year 2 and year 3 objective structured clinical examination (OSCE).These internal indices were compared with student performance on the Humanistic Domain of the Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 2-Performance Evaluation (PE), a licensure exam previously taken in years 3 or 4 of osteopathic medical schools. The evidence of interrater reliability and predictive validity is strong. Data from the 7E-CRF is compared to performance on the COMLEX Level 2-PE, Humanistic Domain. The 7E-CRF can identify students who are at a 10-fold increased risk of failure on the COMLEX Level 2-PE Humanistic Domain. Conclusions:The 7E-CRF integrates instruction and assessment, based on a national and international model. The simplicity, foundation in professional consensus, ease of use, and predictive efficacy make the 7E-CRF a highly valuable instrument for medical schools in teaching and evaluating competency in medical communication skills.

Personality disorder coverage, prevalence, and convergence: do the DSM-5's two models of personality disorder identify the same patients?

Research on the Alternative DSM-5 Model for Personality Disorders (AMPD) in DSM-5's Section-III has demonstrated acceptable interrater reliability, a largely consistent latent structure, substantial correlations with theoretically and clinically relevant measures, and evidence for incremental concurrent and predictive validity after controlling for DSM-5's Section II categorical personality disorders (PDs). However, the AMPD is not yet widely used clinically. One clinician concern may be caseness - that the new model will diagnose a different set of PD patients from that with which they are familiar. The primary aim of this study is to determine whether this concern is valid, by testing how well the two models converge in terms of prevalence and coverage. Participants were 305 psychiatric outpatients and 302 community residents not currently in mental-health treatment who scored above threshold on the Iowa Personality Disorder Screen (Langbehn et al., ). Participants were administered a semi-structured interview for DSM-5 PD, which was scored for both Section II and III PDs. Convergence across the two PD models was variable for specific PDs, Good when specific PDs were aggregated, and Very Good for 'any PD.' Results provide strong evidence that the AMPD yields the same overall prevalence of PD as the current model and, further, identifies largely the same overall population. It also addresses well-known problems of the current model, is more consistent with the ICD-11 PD model, and provides more complete, individualized characterizations of persons with PD, thereby offering multiple reasons for its implementation in clinical settings.

New evidence of predictive validity of SRSS-IE scores with middle and high school students

IntroductionWe report predictive validity of the newly defined Student Risk Screening Scale – Internalizing and Externalizing (SRSS-IE 9, with 9 items) when used for the first time by middle and high school teachers from 43 schools.MethodsThe sample included 11,773 middle school-aged students representing four geographic regions, and 7,244 high school-aged students representing three geographic regions.ResultsResults indicated fall SRSS-IE externalizing and internalizing latent factors as well as subscale scores (SRSS-E5, SRSS-I4, respectively) predicted year-end behavioral (office discipline referrals and in school suspensions) and academic (course failures) outcomes for middle and high school students as well as referrals to special education for middle school students. Internalizing scores also predicted referrals to special education for high school students. Externalizing and internalizing scores predicted nurse visits at the middle and high school levels with all models except for subscale models of internalizing in middle school. SRSS-IE 12 subscale scores for externalizing (SRSS-E7) and internalizing (SRSS-I5) using the original 12 items were similarly predictive of these outcomes, with few variations.DiscussionWe discuss educational implications, limitations, and directions for future inquiry.

Additional Evidence of Predictive Validity of SRSS-IE Scores With Elementary Students

In this article, we examined predictive validity of the newly defined Student Risk Screening Scale–Internalizing and Externalizing (SRSS-IE 9, with nine items) when used by teachers for the first time, with a sample of 24,535 elementary students from 74 schools in four U.S. geographic regions. Results indicated fall externalizing and internalizing latent factors as well as subscale scores (SRSS-E5 and SRSS-I4, respectively) from the SRSS-IE 9 predicted year-end behavioral outcomes (office discipline referrals, suspensions, and nurse visits), academic outcomes (reading and math scores and course failures), as well as referrals to special education for elementary students. SRSS-IE 12 subscale scores for externalizing (SRSS-E7) and internalizing (SRSS-I5) using the original 12-item version subscales were similarly predictive of these outcomes. We discussed educational implications, limitations, and directions for future inquiry.

Evidence of concurrent and predictive criterion validity of the Child Communication Screening Instrument from 0 to 36 months (IRC-36).

To determine evidence of concurrent and predictive criterion validity of the Communication Screening Instrument for children aged 0 to 36 months (IRC-36). 78 parents/guardians of children who attend the childcare service of the Family Health Centers participated in the research, in addition to 33 children aged between 0 and 36 months, invited to the second stage of the study. In its first stage, 13 health professionals were trained to apply the IRC-36 to the children's parents/guardians. In the second moment, the parents responded to a new IRC-36 application, and the children were evaluated with Denver II. IRC-36 correlated with Denver II in more than half of the cases, confirming the instrument's concurrent criterion validity. IRC-36 results in the first stage did not significantly correlate with Denver II. The instrument's cutoff value was 12, which is the reference value between children at risk and not at risk of communication disorders. The instrument had high sensitivity and an accuracy value within the recommended levels. The occurrence of risk of communication changes was higher in the second IRC-36 application. The study presented evidence of concurrent criterion validity, indicating that the instrument has evidence of accuracy and validity measures to screen communication in children aged 0 to 36 months, being able to identify the risk for communication disorders.

Validation of the Keele STarT MSK Tool for Patients With Musculoskeletal Pain in United States-based Outpatient Physical Therapy Settings

The STarT MSK tool was developed to enable risk stratification of patients with common musculoskeletal (MSK) pain conditions and help identify individuals who may require more targeted interventions or closer monitoring in primary care settings, however, its validity in U.S.-based outpatient physical therapy settings has not been investigated. The 10-item Keele STarT MSK risk stratification tool was tested for construct (convergent and discriminant) and predictive validity using a multicenter, prospective cohort study design. Participants (n = 141) receiving physical therapy for MSK pain of the back, neck, shoulder, hip, knee, or multisite regions completed intake questionnaires including the Keele STarT MSK tool, Functional Comorbidity Index (FCI), Optimal Screening for Prediction of Referral and Outcome Review-of-Systems and Optimal Screening for Prediction of Referral and Outcome Yellow Flag tools. Pain intensity, pain interference, and health-related quality of life (Medical Outcomes Study 8-item Short-Form Health Survey (SF-8) physical [PCS] and mental [MCS] component summary scores) were measured at 2- and 6-month follow-up. Participants were classified as STarT MSK tool low (44%), medium (39%), and high (17%) risk. Follow-up rates were 70.2% (2 months) and 49.6% (6 months). For convergent validity, fair relationships were observed between the STarT MSK tool and FCI and SF-8 MCS (r = .35–.37) while moderate-to-good relationships (r = .51–.72) were observed for 7 other clinical measures. For discriminant validity, STarT MSK tool risk-dependent relationships were observed for Optimal Screening for Prediction of Referral and Outcome Review-of-Systems, Optimal Screening for Prediction of Referral and Outcome Yellow Flag, pain interference, and SF-8 PCS (low < medium < high; P < .01) and FCI, pain intensity, and SF-8 MCS (low < medium-or-high; P < .01). For predictive validity, intake STarT MSK tool scores explained additional variability in pain intensity (11.2%, 20.0%), pain interference (7.5%, 14.1%), and SF-8 PCS (8.2%, 12.8%) scores at 2 and 6 months, respectively. This study contributes to the existing literature by providing additional evidence of STarT MSK tool cross-sectional construct validity and longitudinal predictive validity. PerspectiveThis study presents STarT MSK risk stratification tool validity findings from a U.S. outpatient physical therapy sample. The STarT MSK tool has the potential to help physical therapists identify individuals presenting with the most common MSK pain conditions who may require more targeted interventions or closer monitoring.

The psychometric properties of the Burnout Assessment Tool in Norway: A thorough investigation into construct-relevant multidimensionality.

The World Health Organization recognizes burnout as an occupational issue. Nevertheless, accurately identifying employee burnout remains a challenging task. To complicate matters, current measures of burnout have demonstrated limitations, prompting the development of the Burnout Assessment Tool (BAT). Given these circumstances, conducting an in-depth examination of the BAT's construct-relevant multidimensionality is crucial. This study focuses on both the original 23-item BAT and the short 12-item version, using modern factor analytic methods to investigate reliability, validity, and measurement invariance in a representative sample from Norway (n = 493; 49.54% women). Our findings revealed that the bifactor exploratory structural equation modeling solution (burnout global factor and four specific burnout component factors) best explained the data for both BAT versions. All factors demonstrated adequate omega coefficients, with the global factor showing exceptional strength. Both BAT versions correlated highly with each other and with another burnout measure, suggesting convergent validity. Furthermore, both BAT versions achieved full (strict) measurement invariance based on gender. Finally, our results showed that burnout acts as a mediator in our proposed job demands-resources model as preliminary evidence of predictive validity. The study validates the Burnout Assessment Tool in the Norwegian context. The study supports the reliability, validity, and unbiased nature of the tool across genders. The findings also reinforce the importance of job demands and resources, along with burnout as a key mediator, in understanding workplace dynamics in accordance with job demands-resources theory.

Static-99R Norms and Cross-Cultural Validity for Australian Aboriginal and Non-Aboriginal Men Convicted of Sexual Offences.

This study examined Static-99R normative data and cross-cultural validity in a sample of 811 Aboriginal and 3257 non-Aboriginal Australian men (N = 4068) serving custodial orders for sexual offences in New South Wales (NSW), Australia. Aboriginal men scored significantly higher on the Static-99R than non-Aboriginal men (M = 4.39 vs. 2.61) and were more likely to be represented in higher categories of risk. The Static-99R showed good discrimination performance for the total sample (AUC = .76; 95% CI = [.73-.80]) and acceptable calibration to expected reoffending rates for routine samples, with slight tendencies towards overestimation. Discrimination accuracy was lower for Aboriginal men (AUC = .68; 95% CI = [.60-77]) than non-Aboriginal men (AUC = .78; 95% CI = [.74-83]) although was significantly better than chance for both groups. Additional analyses indicated that cross-cultural differences in discrimination were partly associated with variance in sample composition between groups. This is the first Australian study to find evidence for significant predictive validity of the Static-99R with Aboriginal men, and while further research is needed, the results provide initial support for cross-cultural applications of the measure in local criminal justice settings.

Factorial, Construct, and Predictive Validity of the Motivation for Treatment Scale in Alcohol-Use Disorder Withdrawal Treatment

Introduction: The aim of the present study was to examine for the first time the factorial, construct, and predictive validity of the motivation for treatment (MfT) scale in a cohort of patients undergoing inpatient-qualified alcohol withdrawal treatment with the goal of referring patients to further treatment. The MfT scale has previously been evaluated in different settings of substance abuse treatment, revealing factorial ambiguity. To the best of our knowledge, the present study is the first study that conducted comprehensive factor analyses versus separate analyses of the factors conducted in prior studies in order to clarify the aforementioned factorial ambiguity. Methods: A total of 249 patients (mean age 45.2 years (SD = 10.3); 34.4% females) with alcohol dependence were assessed. Data were obtained from four inpatient clinics specialized in qualified alcohol withdrawal treatment in Germany. First, confirmatory factor analyses were carried out to examine the fit of the four models discussed in the literature. Second, an exploratory factor analysis was conducted. Correlations of the new factors with other motivational constructs and referral to a subsequent treatment were investigated as measures of construct and predictive validity. Results: None of the four models showed an acceptable fit to the data in confirmatory analyses. The exploratory analysis suggested to eliminate seven items because of inappropriate factor loadings and resulted in a shortened MfT scale, which consists of three factors based on 17 items. For the latent variables “problem recognition,” “desire for help,” and “treatment readiness,” satisfactory composite reliability was found with 0.82, 0.80, and 0.78, respectively. Evidence for predictive validity was found in the correlation between “treatment readiness” and referral to a subsequent treatment. Discussion/Conclusion: The new shortened MfT scale exhibited remarkable parsimony, which is desirable in settings such as withdrawal treatment, where patients frequently are cognitively or physically impaired. Despite its briefness, construct and predictive validity were better than in the original version of the MfT scale. The factorial validity of the suggested scale needs to be corroborated in further research.

Construct validity of a questionnaire for measuring student engagement in problem-based learning tutorials

BackgroundStudent engagement is student investment of time and energy in academic and non-academic experiences that include learning, teaching, research, governance, and community activities. Although previous studies provided some evidence of measuring student engagement in PBL tutorials, there are no existing quantitative studies in which cognitive, behavioral, and emotional engagement of students in PBL tutorials is measured. Therefore, this study aims to develop and examine the construct validity of a questionnaire for measuring cognitive, behavioral, and emotional engagement of students in PBL tutorials.MethodsA 15-item questionnaire was developed guided by a previously published conceptual framework of student engagement. Focus group discussion (n = 12) with medical education experts was then conducted and the questionnaire was piloted with medical students. The questionnaire was then distributed to year 2 and 3 medical students (n = 176) in problem-based tutorial groups at the end of an integrated course, where PBL is the main strategy of learning. The validity of the internal structure of the questionnaire was tested by confirmatory factor analysis using structural equation modeling assuming five different models. Predictive validity evidence of the questionnaire was studied by examining the correlations between students’ engagement and academic achievement.ResultsConfirmatory factor analysis indicates a good fit between the measurement and structural model of an 11-item questionnaire composed of a three-factor structure: behavioral engagement (3 items), emotional engagement (4 items), and cognitive engagement (4 items). Models in which the three latent factors were considered semi-independent provided the best fit. The construct reliabilities of behavioral, cognitive, and emotional factors were 0.82, 0.82, and 0.76, respectively. We failed however to find significant relationships between academic achievement and engagement.ConclusionsWe found a strong evidence to support the construct validity of a three-factor structure of student engagement in PBL tutorial questionnaire. Further studies are required to test the validity of this instrument in other educational settings. The predictive validity is another area needing further scrutiny.