Aspects Of Construct Validity Research Articles

MEASURING BIRTH OUTCOMES: VALIDATING THE PERINATAL OUTCOME INDEX

We undertook a study to assess the reliability and validity of a new measure of low risk maternity care. A Perinatal Outcome Index (POI), which combines intrapartum process of care and clinical outcome items into a summary index score, was originally developed and evaluated in the Netherlands. It was designed to measure the extent to which a labour and birth are "optimal", that is, one with minimal intervention resulting in a healthy mother and a healthy baby. We modified the Dutch index to make it applicable to a Canadian setting. A panel of experts who were not connected with the study reviewed the modified version for applicability, feasibility of obtaining data easily, and content validity. Data were abstracted from the health records of 324 women in one hospital and two midwifery practices to obtain Perinatal Outcome Index scores and examine aspects of construct validity. We measured the inter-rater reliability of the research assistants who abstracted information. The panel achieved consensus on all items in the modified Perinatal Outcome Index to establish content (face) validity. Labour and birth data were readily obtained from health records with high inter-rater reliability (Kappa 0.78). In a linear regression model, birth at home, multiparity, and having a midwife or family physician as a care provider were significantly associated with higher scores (having a more optimal birth) and accounted for 37% of the score variance. The Perinatal Outcome Index has satisfied our expectations for content and construct validity. Research assistants found it easy to use and data items were readily available from women's health records. Inter-rater reliability was acceptable. We believe the modified index will be useful for comparative studies among women at low or average risk, and for quality assurance programs.

Evaluating construct validity of computable acute respiratory distress syndrome definitions in adults hospitalized with COVID-19: an electronic health records based approach

BackgroundEvolving ARDS epidemiology and management during COVID-19 have prompted calls to reexamine the construct validity of Berlin criteria, which have been rarely evaluated in real-world data. We developed a Berlin ARDS definition (EHR-Berlin) computable in electronic health records (EHR) to (1) assess its construct validity, and (2) assess how expanding its criteria affected validity.MethodsWe performed a retrospective cohort study at two tertiary care hospitals with one EHR, among adults hospitalized with COVID-19 February 2020-March 2021. We assessed five candidate definitions for ARDS: the EHR-Berlin definition modeled on Berlin criteria, and four alternatives informed by recent proposals to expand criteria and include patients on high-flow oxygen (EHR-Alternative 1), relax imaging criteria (EHR-Alternatives 2–3), and extend timing windows (EHR-Alternative 4). We evaluated two aspects of construct validity for the EHR-Berlin definition: (1) criterion validity: agreement with manual ARDS classification by experts, available in 175 patients; (2) predictive validity: relationships with hospital mortality, assessed by Pearson r and by area under the receiver operating curve (AUROC). We assessed predictive validity and timing of identification of EHR-Berlin definition compared to alternative definitions.ResultsAmong 765 patients, mean (SD) age was 57 (18) years and 471 (62%) were male. The EHR-Berlin definition classified 171 (22%) patients as ARDS, which had high agreement with manual classification (kappa 0.85), and was associated with mortality (Pearson r = 0.39; AUROC 0.72, 95% CI 0.68, 0.77). In comparison, EHR-Alternative 1 classified 219 (29%) patients as ARDS, maintained similar relationships to mortality (r = 0.40; AUROC 0.74, 95% CI 0.70, 0.79, Delong test P = 0.14), and identified patients earlier in their hospitalization (median 13 vs. 15 h from admission, Wilcoxon signed-rank test P < 0.001). EHR-Alternative 3, which removed imaging criteria, had similar correlation (r = 0.41) but better discrimination for mortality (AUROC 0.76, 95% CI 0.72, 0.80; P = 0.036), and identified patients median 2 h (P < 0.001) from admission.ConclusionsThe EHR-Berlin definition can enable ARDS identification with high criterion validity, supporting large-scale study and surveillance. There are opportunities to expand the Berlin criteria that preserve predictive validity and facilitate earlier identification.

Bootstrap Exploratory Graph Analysis of the WISC-V with a Clinical Sample.

One important aspect of construct validity is structural validity. Structural validity refers to the degree to which scores of a psychological test are a reflection of the dimensionality of the construct being measured. A factor analysis, which assumes that unobserved latent variables are responsible for the covariation among observed test scores, has traditionally been employed to provide structural validity evidence. Factor analytic studies have variously suggested either four or five dimensions for the WISC-V and it is unlikely that any new factor analytic study will resolve this dimensional dilemma. Unlike a factor analysis, an exploratory graph analysis (EGA) does not assume a common latent cause of covariances between test scores. Rather, an EGA identifies dimensions by locating strongly connected sets of scores that form coherent sub-networks within the overall network. Accordingly, the present study employed a bootstrap EGA technique to investigate the structure of the 10 WISC-V primary subtests using a large clinical sample (N = 7149) with a mean age of 10.7 years and a standard deviation of 2.8 years. The resulting structure was composed of four sub-networks that paralleled the first-order factor structure reported in many studies where the fluid reasoning and visual-spatial dimensions merged into a single dimension. These results suggest that discrepant construct and scoring structures exist for the WISC-V that potentially raise serious concerns about the test interpretations of psychologists who employ the test structure preferred by the publisher.

E092 Confirmatory factor analyses of the Tampa Scale of Kinesiophobia 11-item version and Pain Catastrophising Scale: A cross-sectional study in people living with axial spondyloarthritis

Abstract Background/Aims The Fear Avoidance Model (FAM) proposes pain-related fear of movement and pain catastrophising to be prominent psychological experiences that compromise physical function and activity behaviour in people living with chronic pain. Yet, little research has tested the FAM in people living with axial spondyloarthritis (axSpA) despite the evidenced benefits of physical activity and exercise. This study examines the structural aspect of construct validity of the 11-item Tampa Scale of Kinesiophobia (TSK-11) and Pain Catastrophising Sale (PCS) as an important first step, to establish valid measurement tools to effectively test the relevance of the FAM for people with axSpA. Methods An online survey was distributed via the National Axial Spondyloarthritis Society including the TSK-11 and PCS. Ninety-eight people with axSpA (M Age = 45.62 SD = 12.16, M BASFI = 3.7 SD = 2.60, Female = 70%, NSAID use = 58.76%, bDMARDS use = 62.24%) completed the survey (December 2020- April 2021). Internal consistency was examined via Cronbach alpha values and confirmatory factor analysis tested the TSK-11's two-factor structure and PCS's three-factor structure. A model has excellent fit to the data with values &amp;lt; 3 for the CMIN/DF, and around .95 for CFI, .06 for RMSEA, .08 for SRMR. Factor loadings &amp;lt; .40 support deletion. Results Alpha values supported the internal consistency of participant responses to the somatic (.78), avoidance behaviour (.80) and overall fear of movement (.89) TSK-11 items. Fit indices also showed a good fit to the data for the TSK-11’s two-factor structure once two error terms were correlated (CMIN/DF = 1.77/42, CFI = .93, RMSEA = .09 (.05 - .12), SRMR= .06). Standardised factors loadings for the somatic and avoidance factors ranged from .43 - .80 and .60 - .72, respectively. Internal consistency for the responses to the PCS-Helplessness (.92), PCS-Magnification (.82), PCS-Rumination (.94) and overall pain catastrophising (.95) scales were also supported. A good fit to the data was shown for the three-factor structure of the PCS (CMIN/DF = 2.15/61, CFI = .94, RMSEA= .11 (.08-.14), SRMR = .04) when two errors were correlated. Standardised factors loadings ranged from .67-.90, (Helplessness), .89 - .92 (Rumination) and .73 - .85 (Magnification). Conclusion The TSK-11 and PCS can be used to measure axSpA patient’s pain related fear of movement and pain catastrophising, but data support continued examination of their construct validity in larger samples of axSpA patients. One TSK-11 item revealed a border line factor loading (.43) for the somatic factor (“pain lets me know when to stop exercising so that I don’t injure myself”). Psychometrically valid instruments will help examine the role of pain related fear of movement and pain catastrophising on important health related outcomes for axSpA patients including physical function and physical activity behaviour. Disclosure P.C. Rouse: Grants/research support; PhD studentship matched funded by Bath Institute for Rheumatic Diseases (RAG-PR-33-120318). T.A. Ingram: None. M. Standage: None. R. Sengupta: Honoraria; Received honoraria for giving talks from Abbyie, Biogen, UCB, Novartis and Pfizer. Grants/research support; Research grants have been received from UCB, Abbvie, and Novartis.

The development and validation of a new simulator for endourology

IntroductionSimulation in medicine has developed a lot in the last few decades. There is a broad range of simulators available, above all for training in surgical procedures. Endourology can benefit much from simulation because the minimally-invasive procedures of endourology frequently have long learning curves, which can be reduced by training with simulators. Materials and methodsA low-fidelity simulator was designed for practicing endourology techniques that use cystoscopy. The process of validation involved 5 experts and 19 non-experts. Experts comprised medical professionals working in a department of urology who had performed at least 100 flexible cystoscopy procedures. Non-experts were residents in internal medicine without experience in any type of endoscopy. Information about face and content validity was collected by means of Likert scales from 1 to 5. To evaluate construct validity, we measured the time to complete two tasks, for which the procedure was evaluated by means of the OSATS global evaluation scale. ResultsNew simulator was successfully built according to its design. For all evaluated aspects of construct validity, there was a significant difference (p<0.05) between the group of experts and the group of non-experts. Content validity was scored 4.66 (standard deviation ±0.56) by the experts and 4.41 (±0.71) by the non-experts. In the face validity questionnaire, the average score was 4.14 (±0.94), the question receiving the highest score: 4.6 (±0.84) concerned immersion in the procedure. ConclusionThe simulator presented is valid both for training up new urologists in endourology technique and for experts seeking to perfect their skills.

Desarrollo y validación de un nuevo simulador para endourología

IntroductionSimulation in medicine has developed a lot in the last few decades. There is a broad range of simulators available, above all for training in surgical procedures. Endourology can benefit much from simulation because the minimally-invasive procedures of endourology frequently have long learning curves, which can be reduced by training with simulators. Material and methodsA low-fidelity simulator was designed for practicing endourology techniques that use cystoscopy. The process of validation involved 5 experts and 19 non-experts. Experts comprised medical professionals working in a department of urology who had performed at least 100 flexible cystoscopy procedures. Non-experts were residents in internal medicine without experience in any type of endoscopy. Information about face and content validity was collected by means of Likert scales from 1 to 5. To evaluate construct validity, we measured the time to complete two tasks, for which the procedure was evaluated by means of the OSATS global evaluation scale. ResultsNew simulator was successfully built according to its design. For all evaluated aspects of construct validity, there was a significant difference (P<.05) between the group of experts and the group of non-experts. Content validity was scored 4.66 (standard deviation±0.56) by the experts and 4.41 (±0.71) by the non-experts. In the face validity questionnaire, the average score was 4.14 (±0.94), the question receiving the highest score: 4.6 (±0.84) concerned immersion in the procedure. ConclusionThe simulator presented is valid both for training up new urologists in endourology technique and for experts seeking to perfect their skills.

Validation of the brief instrument “Health Literacy for School-Aged Children” (HLSAC) among Norwegian adolescents

Health literacy (HL) has been identified as an important and modifiable determinant of health. To succeed in promoting HL, it is crucial to evaluate interventions addressing HL using validated instruments. However, HL measurement is an under-researched area among adolescents. The Health Literacy for School-Aged Children (HLSAC) scale is developed in Finland to measure the subjective HL levels of school-aged children. The HLSAC has been used and validated across Europe. No validated instrument for measuring HL among Norwegian adolescents has been identified. Therefore, the aim of this study was to assess the psychometric properties of the HLSAC among Norwegian adolescents in upper secondary schools. Cross-sectional data were collected in 2017 from 1 054 adolescents with a response rate of 93.5%. The participants were students aged 16–21 (mean = 17.3) years from four upper secondary schools in an urban area in Mid-Norway. Confirmatory factor analysis (CFA) was utilized to investigate the underlying dimensionality of the data together with composite reliability based on Raykov’s reliability coefficient and certain aspects of construct validity. The original ten-item one-dimensional version of the HLSAC revealed a poor fit. A one-dimensional version including six of the original ten items presented the best fit to our data, demonstrating good reliability and construct validity. This validation study suggests a one-dimensional solution of the HLSAC scale including six of the original ten items, i.e., the HLSAC-6 as a statistical valid and reliable measure for HL among adolescents in upper secondary schools. However, the modifications of the instrument indicate the need for further investigation of the HLSAC in this age group, i.e. by conducting cognitive interviews and focus-group discussions on the concept of HL among adolescents. Testing the wording of the HLSAC prior to additional psychometric evaluations of both the original HLSAC and the suggested HLSAC-6 is recommended. Finally, developing more age-appropriate items for the measurement of HL in this age-group is suggested.

Construct Validity of Inherited Retinal Disease–Specific Patient-Reported Outcome Measures

To evaluate aspects of construct validity of the Michigan Retinal Degeneration Questionnaire (MRDQ) and the Michigan Vision-related Anxiety Questionnaire (MVAQ). Subjects with a clinical diagnosis of an inherited retinal disease (IRD) were recruited prospectively and 3 tests were used to assess construct validity: the ability to distinguish different IRD phenotypes; test a priori hypothesis of an association between vision-related anxiety and vision-related disabilities; and correlate MRDQ and MVAQ with the National Eye Institute Visual Functioning Questionnaire 25 (NEI VFQ-25) and the Impact of Vision Impairment (IVI). One-way analysis of variance (ANOVA) was used to compare different phenotypes for mean domain scores for MRDQ/MVAQ. Pearson correlations were performed between; Cone-Function Anxiety and Central Vision controlling for better eye visual acuity, Rod-Function Anxiety and Scotopic Function controlling for visual field area (III4e and IV4e), and scores of MRDQ/MVAQ, NEI VFQ-25, and IVI. The study sample consisted of 146 patients evenly divided between males and females, and mean age was 50 years. The 1-way ANOVA test was significant for distinguishing IRD phenotypes in 6 domains of MRDQ/MVAQ. Cone-Function Anxiety correlated with Central Vision controlling for visual acuity, Rod-Function Anxiety correlated with Scotopic Function controlling for visual field area, and all domains in MRDQ/MVAQ had significant correlations with NEI VFQ-25 and IVI composite scores. MRDQ and MVAQ domenstrate aspects of construct-validity set forth by the US Food and Drug Administration. The study futher supports the use of both patient-reported outcome measures in IRD clinical trials and natural history studies.NOTE: Publication of this article is sponsored by the American Ophthalmological Society.

Validation of the Behavioral Anger Response Questionnaire for Children (BARQ-C) in a large community sample of Vietnamese middle adolescents in Hanoi

BackgroundDepression and other forms of psychological distress are common among Vietnamese adolescents and increase the risk of mental health problems in adulthood. As anger coping is a robust predictor of adolescent mental health difficulties, and there appear to be cultural variations in anger coping, a measure of adolescent anger coping styles that has been validated using a non-Western adolescent sample is required to inform and support early intervention to prevent or treat mental health difficulties in Vietnamese adolescents. This study examined the construct validity (structural and external) of the Behavioral Anger Response Questionnaire for Children in Vietnam (BARQC-V).MethodsBaseline data sourced from a recent randomised control trial conducted with Grade 10 Vietnamese adolescents aged 14 to 16 (N = 1084) were used to examine multiple aspects of construct validity: factorial structure (evaluated using factor analysis); internal consistency (tested using Cronbach’s alpha coefficient); and external aspect (assessed using Pearson’s correlation coefficients between the BARQC-V and Vietnamese translations of the Coping Self-Efficacy Scale, Centre for Epidemiologic Studies Depression Scale Revised, Mental Health Continuum Short Form, and the Depression Anxiety and Stress Scale).ResultsEvaluating factorial structure using confirmatory factor analysis failed to converge on a solution. Exploratory factor analysis yielded a 5-factor structure model that explained 49.32% of the BARQC-V’s total variance and was deemed to be a good fit by the final confirmatory factor analysis. Cronbach’s alpha coefficients of the 5 factors demonstrated acceptable internal reliability for the BARQC-V’s sub-scales. Concerning concurrent validity, three sub-scales predicted well-being and mental health difficulties: the maladaptive anger coping styles Rumination and Direct Anger-out were positively associated with depression and distress, and negatively associated with coping self-efficacy and mental well-being; and the adaptive anger coping style Assertion was positively associated with coping self-efficacy and mental well-being, and negatively associated with depression.ConclusionsThe BARQC-V provides a validated measure of three anger coping strategies used by adolescents in Vietnam (Rumination, Direct Anger-out, and Assertion) that can be used to improve detection and treatment of mental health difficulties in this population, and as a starting point by future research to develop a much-needed gold standard measure of anger coping for adults, adolescents and children world-wide.

Development of Evaluative-Process Learning Tools Integrated with Conceptual-Problem-Based Learning Models: Study of Its Validity and Effectiveness to Train Critical Thinking

This study aimed to develop an evaluative-process learning tool integrated with the conceptual-problem-based learning (CPBL) model to train students' critical thinking skills. The learning tools developed (lesson plans, textbooks, worksheets, and critical thinking test instruments) were evaluated for validity and Effectiveness in training students' critical thinking skills at the higher education level. Validity evaluation is carried out on content and construct validity aspects. This is done through a focus group discussion (FGD) mechanism involving four expert validators. Furthermore, the Effectiveness of the developed learning tools is evaluated by implementing them in the classroom. The experimental design (intact-group comparison) involved a sample group from the State Islamic University of Mataram. The experimental group was taught by evaluative-process learning tools integrated with the CPBL model, while lectures and discussions taught the control group. Critical thinking data were collected using a valid essay test instrument, and the results were analyzed. The validity test results show that all the elements that make up the learning tools in the aspect of content and construct validity have been declared valid. Furthermore, at the implementation stage in the classroom, evaluative-process learning tools integrated with the CPBL model have been effective in training students' critical thinking skills compared to teaching that relies on lectures and discussion. This Effectiveness is based on two aspects, (1) the conceptual framework of the CPBL model, which is constructed and arranged from a problem-based learning model with five learning steps, namely prior knowledge, organize, investigate, analyze, and evaluation; (2) the concept of evaluative-process which is integrated with the CPBL model. These two aspects support capacity in training students to think critically.

Factor Structure and Distinctiveness of the Violent Behavior Vignette Questionnaire

ABSTRACT The Violent Behavior Vignette Questionnaire (VBVQ) is a self-report measure of current violent behavior. It consists of a series of interpersonal conflicts to which participants respond by selecting from a range of violent and nonviolent options. Though there is initial evidence for the convergent validity of VBVQ scores, other important aspects of construct validity have not yet been tested. In the current study, we conducted a secondary analysis of a dataset of 471 men from the general community to explore the factor structure of the VBVQ and its distinctiveness from other measures of violence and aggression (i.e., Violent Behaviour Scale [VBS] and the Physical Aggression scale of the Aggression Questionnaire [PA-AQ]). An exploratory factor analysis (EFA) indicated that the VBVQ vignettes fit a one-factor structure. Partial correlations indicated that the VBVQ was independently correlated with the PA-AQ (controlling for the VBS), and with the VBS (controlling for the PA-AQ). Furthermore, a second EFA indicated that the VBVQ vignettes formed a distinct factor from the VBS and PA-AQ items. These results suggest that the VBVQ measures one construct and that it provides non-redundant information about violent behavior relative to the other measures of violence and aggression.

Existential Meaninglessness Scale: Scale Development and Psychometric Properties

Grounded in a tripartite existential meaninglessness model, the authors developed the 18-item Existential Meaninglessness Scale (EMS) to assess one’s concern and anxiety of existential meaninglessness. Across three samples, the EMS’s factor structure and evidence of convergent, criterion-related, and incremental validity, internal consistency, and test–retest reliability were examined. Exploratory factor analyses demonstrated three dimensions of the EMS: incomprehension, purposelessness, and insignificance. Confirmatory factor analyses revealed that a bifactor model was a better fit to the data than other models. The bifactor model provided evidence for a general factor and measurement invariance. Ancillary bifactor indices indicated EMS’s unidimensionality. Findings of bivariate correlations and hierarchical regression analyses provided evidence for different aspects of construct validity and internal consistency. Both the Concern and Anxiety measures of the EMS positively predicted depressive symptoms and suicide ideation above and beyond the effects of general existential meaninglessness, general feelings of anxiety, and presence of meaning in life. Based on the findings, the authors discuss future research directions on existential meaninglessness using the EMS.

On the problem content of tasks in commercial final and intermediate examinations – results from task analyses

One of the aims of initial vocational education and training is to enable graduates to solve professional problems. Curricula – both for vocational schools as well as for in company trainings – therefore include objectives that aim at developing and promoting (complex) problem-solving competencies. For a consistent implementation of these curricular goals, appropriate learning opportunities must be provided. And examinations must be designed in such a way that they measure problem-solving competencies. However, current examination practice has been criticised in this regard for some time now. Criticism is particularly directed at the insufficient linking of curriculum, instruction, and assessment and thus at central aspects of construct validity. This article aims to investigate the extent to which problem-solving requirements are implemented in current examination tasks. For this purpose, we analysed 1,454 tasks from previous examinations for industrial clerks and office management clerks regarding their problem-solving requirements and their proximity to professional problem-solving processes (authenticity, professional relevance). The analysis was carried out within the framework of a qualitative content analysis using deductively formed categories. The findings indicate that the examination tasks do not include many problem-solving requirements and in many cases are not authentic. Written intermediate and final examinations in their current form and in the training occupations considered here therefore only allow limited statements to be made about the extent to which trainees are able to solve professional problems.

Construct Validity of a Task-Oriented Unimanual Strength Measurement in Children with Unilateral Cerebral Palsy

Aim: Investigate aspects of construct validity of parameters “mean peak force” and “number of successful attempts” of the press button task using the Task-oriented Arm-hAnd Capacity (TAAC) in children with unilateral cerebral palsy (CP) by comparing them to outcomes of comparative measures using Consensus-based Standards for the selection of health Measurement INstruments guidelines. Methods: 118 children with unilateral CP (mean age 1 year 2 months, standard deviation 3 year 5 months) were included. Fourteen a priori hypotheses were formulated for each parameter of the TAAC. Strength and direction of the relationship between the TAAC and comparative measures were investigated by calculating Pearson correlation coefficients. Results: For the parameter “mean peak force” 8/14 (57%) hypotheses could be supported. For the parameter “number of successful attempts” 13/14 (93%) hypotheses could be supported. Conclusion: The hypothesized construct of the parameter “mean peak force” is only partially in line with our idea about the potential relationship of the compared constructs. The relationship needs to be reconsidered to some extent. The hypothesized construct of the parameter “number of successful attempts” is in line with our idea about the potential relationship of the compared constructs and can be considered to have good validity compared to the other measures. Thus, the construct of this parameter adds new and meaningful information as an outcome measure for functional strength measurements.

Evaluating the construct validity and internal consistency of the Sickness Questionnaire in a Swedish sample of adults with longstanding pain.

Low-grade inflammation is a possible contributing factor in the development and persistence of chronic primary pain syndromes. Related to inflammatory activity is sickness behavior, a set of behavioral responses including increased pain sensitivity, fatigue, malaise, fever, loss of appetite, as well as depressive behavior and anhedonia. To capture these behavioral responses and their relation to longstanding pain, psychometrically sound self-report questionnaires are needed. The Sickness Questionnaire (SicknessQ) was developed to assess self-reported sickness behavior based on studies on acute immune activation while maintaining relevance for persistent conditions. The aim of the current study was to evaluate aspects of the validity and reliability of the SicknessQ in a Swedish sample of persons with longstanding pain. Aspects of construct validity were evaluated by means of performing a confirmatory factor analysis (CFA) (testing structural validity) and by relevant hypothesis testing i.e.,that ratings of sickness behavior in combination with other related factors (e.g., depression and anxiety) would be significantly related to ratings of avoidance. Reliability was evaluated by means of analyzing the internal consistency of items. Following the CFA, a non-significant Chi-Square test (χ2 [32, N=190]=42.95, p=0.094) indicated perfect model fit. Also, the relative fit indices supported adequate model fit (CFI=0.978; TLI=0.969; RMSEA=0.0430). Sickness behavior (p<0.0001), depression (p<0.05) and pain duration (p<0.05) significantly contributed to the regression model, explaining 45% of the total variance in avoidance. Internal consistency was adequate, as indicated by a Cronbach's α value of 0.82 for the entire questionnaire. Results indicate that the SicknessQ has adequate structural validity as well as adequate internal consistency, and is significantly associated with avoidance. The SicknessQ appears to have utility as a self-report questionnaire to assess symptoms of sickness behavior for adults with longstanding pain.

Psychometric properties of an English Writing Self-Efficacy scale: aspects of construct validity

Psychometric properties of an English Writing Self-Efficacy scale: aspects of construct validity

Development of a New Measure for Assessing Mentalizing: The Multidimensional Mentalizing Questionnaire (MMQ).

This research consists of two studies which aimed to: (1) evaluate the psychometric properties of a new self-report measure for the assessment of mentalizing, the Multidimensional Mentalizing Questionnaire (MMQ); and (2) investigate the ability of the instrument to discriminate between community and clinical populations. A sample of 349 participants (19% male, 81% female; Mage = 38.6, SD = 15.3) filled in the MMQ and other self-report measures, in order to assess the factor structure, reliability and some aspects of construct validity of the measure. Then, a clinical sample (N = 46; 52% male and 48% female; Mage = 33.33, SD = 12.257) and a community one (N = 50; 42% male and 58% female; Mage = 38.86, SD = 16.008) filled in the MMQ, to assess its clinical sensitivity. The factorial analysis identified six principal dimensions of the measure: reflexivity, ego-strength, relational attunement, relational discomfort, distrust, and emotional dyscontrol. The MMQ showed satisfactory psychometric properties and a theoretically relevant factor structure. Furthermore, significantly greater impairment in mentalizing was found in the clinical sample in respect of the community one. The findings are discussed in terms of clinical implications, emphasizing the usefulness of the MMQ in both research and clinical practice.

Why Does Construct Validity Matter in Measuring Implementation Fidelity? A Methodological Case Study

In reading intervention research, implementation fidelity is assumed to be positively related to student outcomes, but the methods used to measure fidelity are often treated as an afterthought. Fidelity has been conceptualized and measured in many different ways, suggesting a lack of construct validity. One aspect of construct validity is the fidelity index of a measure. This methodological case study examined how different decisions in fidelity indices influence relative rank ordering of individuals on the construct of interest and influence our perception of the relation between the construct and intervention outcomes. Data for this study came from a large state-funded project to implement multi-tiered systems of support for early reading instruction. Analyses were conducted to determine whether the different fidelity indices are stable in relative rank ordering participants and if fidelity indices of dosage and adherence data influence researcher decisions on model building within a multilevel modeling framework. Results indicated that the fidelity indices resulted in different relations to outcomes with the most commonly used fidelity indices for both dosage and adherence being the worst performing. The choice of index to use should receive considerable thought during the design phase of an intervention study.

Development and Validation of a Measure Assessing Sustainability of Tier 2 and 3 Behavior Support Systems

To identify the most effective strategies for implementing and sustaining Tier 2 and 3 behavior support systems, a measure of general and tier-specific factors hypothesized to predict sustained implementation is needed. To address this need, we conducted two studies examining the construct validity of the Advanced Level Tier Interventions Treatment Utilization and Durability Evaluation (ALTITUDE) measure: one assessing the content aspects of construct validity (Study 1) and one assessing the structural and external aspects of construct validity (Study 2). In Study 1, participants included an expert panel of 26 members who provided iterative feedback during measure development. The results showed strong content representativeness (content validity index = .93) for assessing elements indicating sustainability. In Study 2, participants were school personnel from 646 schools who completed the measure. The results showed model fit was good for both a three-factor correlated model (CFI = .98, RMSEA = .06, SRMR = .05) and Bifactor S-1 model with correlated Tier 2 and Tier 3 specific residual factors (CFI = .98, RMSEA = .06, SRMR = .05). In addition, ALTITUDE latent factors were found to have both convergent and discriminant evidence in relation to concurrent fidelity of school-wide positive behavioral interventions and supports (SWPBIS) implementation at Tiers 1, 2, and 3. This construct validity evidence will support the use of the ALTITUDE in identifying the strongest tier-general and tier-specific predictors of sustained implementation of Tier 2 and 3 behavior support systems.

Exploring the Validity of a Comprehensive Listening Test to Identify Differences in Primary School Students’ Listening Skills

ABSTRACT Effective listening comprehension skills are an important prerequisite for the academic success of primary school students. However, the assessment of listening skills in the instructional language appears to have received only scant attention in the literature. Therefore, the goal of the present study was twofold. Firstly, a comprehensive listening test was developed and different aspects of construct validity supporting the use of the listening test were explored. The listening test was administered to 1001 sixth-grade primary school students in Flanders, the Dutch-speaking part of Belgium. Next, the test items were controlled for item difficulty and discrimination, dimensionality, model-data fit, local item independence, monotonicity, and gender differential item functioning. The final listening test consisted of 24 multiple-choice and open-ended test items. Secondly, the listening test was used to identify differences between students’ listening skills based on gender and home language. The results indicate that gender was not significantly related to listening comprehension skills, but L1 Dutch-speaking students significantly outperformed L2 Dutch-speaking students. This study also covers possible further fine-tuning of the instrument.