Construct Validation Approach Research Articles

Automatic Self Recriminations: Development and Validation of a Measure of Self-Condemnatory Internal Dialogue

This article introduces a measure of self-condemnatory internal dialogue as an element of the relationship with the self: The Automatic Self-Recrimination Scale (ASRS). Using the construct validation approach to test construction, we describe the initial development of items and report on findings from a clinical and nonclinical sample showing the ASRS is best understood as a multidimensional measure of self-critical internal dialogue composed of one higher-order factor and four lower-order facets: Not Mattering, Self as Failure, Undeserving Self, and Loathsomeness. The overall scale and four subscales demonstrated acceptable internal consistency and test-retest reliability. Moreover, there was evidence of good convergent and incremental validity of the ASRS subscales with measures of perfectionism, self-criticism, and dysfunctional attitudes. Overall, the ASRS appears to be a reliable and valid measure of an automatic self-recriminatory internal dialogue.

A critical evaluation of adolescent resilience self-report scales: A scoping review

Valid quantitative measures of adolescent resilience are important for the development of knowledge and have implications for practice with adolescents. This scoping review followed Arksey and O’Malley’s (2005) five step process and aimed to (1) identify the most used self-report scales that measure resilience of adolescents in studies published between 2000 and 2021, (2) describe the scales’ psychometric properties, (3) describe the scales’ conceptual and theoretical formulations, and (4) assess the scales’ relative strengths, weaknesses, and adequacy. A review of 118 papers revealed six commonly used scales. A construct validation approach adapted from Skinner (1981) and expanding on Pangallo et al., (2015), with evidence assessed in four stages (theoretical formulation, reliability, validity, and application) was utilised to critically evaluate the six scales. The results showed that the most adequate scale for measuring resilience in adolescent populations was the Child and Youth Resilience Measure, scoring 83% of points. The Connor-Davidson Resilience Scale (also scoring 83%) and The Resiliency Scales for Children and Adolescents (78%) were also found to be adequate. This review provides clinicians and researchers with a critical overview of common scales measuring resilience in adolescents, including their underlying theoretical basis. This is vital to ensure the measure chosen is valid and matches the theoretical aims of the research/ application. Our review also suggests that too often, researchers fail to look beyond the original validation study when selecting resilience scales, and often fail to analyse and report current psychometric data from the chosen scale.

Students’ interaction anxiety and social phobia in interprofessional education in Hong Kong: mapping a new research direction

Background Interprofessional education (IPE) has been promoted as a breakthrough in healthcare because of the impact when professionals work as a team. However, despite its inception dating back to the 1960s, its science has taken a long time to advance. There is a need to theorize IPE to cultivate creative insights for a nuanced understanding of IPE. This study aims to propose a research agenda on social interaction by understanding the measurement scales used and guiding researchers to contribute to the discussion of social processes in IPE. Method This quantitative research was undertaken in a cross-institutional IPE involving 925 healthcare students (Medicine, Nursing, Social Work, Chinese Medicine, Pharmacy, Speech Language Pathology, Clinical Psychology, Food and Nutritional Science and Physiotherapy) from two institutions in Hong Kong. Participants completed the Social Interaction Anxiety Scale (SIAS-6) and Social Phobia Scale (SPS-6). We applied a construct validation approach: within-network and between-network validation. We performed confirmatory factors analysis, t-test, analysis of variance and regression analysis. Results CFA results indicated that current data fit the a priori model providing support to within-network validity [RMSEA=.08, NFI=.959, CFI=.965, IFI=.965, TLI=.955]. The criteria for acceptable fit were met. The scales were invariant between genders, across year levels and disciplines. Results indicated that social interaction anxiety and social phobia negatively predicted behavioural engagement (F = 25.093, p<.001, R 2=.065) and positively predicted behavioural disaffection (F = 22.169, p<.001, R 2=.057) to IPE, suggesting between-network validity. Conclusions Our data provided support for the validity of the scales when used among healthcare students in Hong Kong. SIAS-6 and SPS-6 have sound psychometric properties based on students’ data in Hong Kong. We identified quantitative, qualitative and mixed methods research designs to guide researchers in getting involved in the discussion of students’ social interactions in IPE. Key Messages The Social Anxiety Scale (SIAS-6) and Social Phobia Scale (SPS-6) scales have sound psychometric properties based on the large-scale healthcare students’ data in IPE in Hong Kong. Social interaction anxiety and social phobia negatively predicted students’ behavioural engagement with IPE and positively predicted behavioural disaffection. The scales are invariant in terms of gender, year level and discipline. Quantitative, qualitative and mixed methods studies are proposed to aid researchers to contribute in healthcare education literature using the SIAS-6 and SPS-6.

Examining Schizotypal Personality Scales Within and Across Interpersonal Circumplex Surfaces.

Differing perspectives on the operationalization of schizotypal personality pathology (STPP) have led to numerous multidimensional assessment measures. The current study applied the interpersonal construct validation approach to self-report data from 856 undergraduate students to formally examine the interpersonal content, similarities, and differences in the subscales of four measures of STPP within and across two interpersonal circumplex surfaces using a bootstrapping methodology for computing confidence intervals around circumplex structural summary method parameters. Results suggested that negative-like expressions of STPP are prototypically and distinctively interpersonal constructs associated with cold and socially avoidant interpersonal problems and hypersensitivity to others' warmth and affection. Positive-like expressions of STPP as assessed by two out of four measures were prototypically and distinctively interpersonal constructs associated with vindictiveness. Across four measures, there was notable overlap in interpersonal correlates among related subscales, suggesting convergent validity. However, subscales containing social anxiety content were associated with more submissive (i.e., socially avoidant) interpersonal problems than subscales without social anxiety content.

Adaptation and validation of the vocabulary learning motivation questionnaire for Chinese learners: A construct validation approach

English as a foreign language (EFL) learners in China view learning vocabulary as one of their biggest challenges. Despite the importance of vocabulary learning for mastering English, no existing survey specifically measures Chinese students' vocabulary learning motivation. This study aimed to cross-culturally adapt the Vocabulary Learning Motivation Questionnaire (VLMQ; Tanaka, 2017) and validated it in a Chinese context. Both within- and between-network approaches were used. Results indicated the applicability of this new version of the scale (i.e., VLMQ-Chinese) in measuring the vocabulary learning motivation of Chinese high school students. Results of the within-network analyses supported the four-factor model of the scale which includes intrinsic, identified, introjected, and external motivation. Measurement invariance across gender was also supported. The between-network analyses indicated that different types of motivation were associated with students' vocabulary proficiency, self-efficacy, and self-regulating capacity in vocabulary learning. Among the different types of motivation, intrinsic motivation and identified motivation were more strongly associated with adaptive learning-related outcomes. Theoretically, this study extends the past vocabulary learning literature by focusing specifically on Chinese EFL learners' vocabulary learning motivation. Practically, this study provides researchers and educators with a tool to measure students’ motivation in vocabulary learning.

The Sleep Regularity Questionnaire: development and initial validation

Background/objectiveSleep is a critically important behavior which influences diverse aspects of health, functioning, and longevity. An increasing literature suggests the importance of sleep regularity, also referred to as sleep inconsistency, sleep variability, or intraindividual variability in sleep. Given there is no brief, subjective measure of sleep regularity, the purpose of this study was to develop the Sleep Regularity Questionnaire (SRQ) and to begin the process of examining its psychometric properties using a construct-validation approach. Participants/methodsIn an online study of sleep and health, participants (n = 3249; Mage (SD) = 42.77 (16.73); 48.5% female; 77.3% white) completed the in-development SRQ, as well as the Insomnia Severity Index and the Pittsburgh Sleep Quality Index. ResultsAn exploratory factor analysis followed by a confirmatory factor analysis revealed a two factor structure, represented by circadian regularity and sleep continuity regularity, with good model fit indices (X2 = 50.9, df = 7, p < 0.001; RMSEA = 0.06; CFI = 0.99; NFI = 0.99; IFI = 0.99; TLI = 0.98). Test-retest reliability, as well as concurrent, convergent and incremental validity were examined, with promising results. ConclusionsPreliminary psychometrics suggest that the SRQ is a valid and stable instrument for the assessment of sleep regularity in adults that is related to, but distinct from, other established sleep constructs. Future research will benefit from assessing the validity of the SRQ in various clinical samples and how it compares to measures of sleep regularity calculated from prospective daily assessments.

Factorial validity of meaning in life questionnaire in Hong Kong secondary school students: a construct validation approach

ABSTRACT The present research was designed to examine the psychometric properties of a Chinese version of the MLQ using a construct validation approach in 1,089 Hong Kong secondary school students. Confirmatory factor analyses were used to examine the structural validity of MLQ. Correlational analyses were done to assess how meaning in life dimensions relate to life satisfaction, self-reported duration of daily sleep, and connectedness to social agents. Confirmatory factor analysis showed that the modified two-factor model of meaning in life with presence of meaning and search for meaning as key dimensions, was valid and reliable in the current sample. Evidence for configural, metric, and scalar invariance was found across boys and girls. Meaning in life dimensions was differentially associated with connectedness to various social agents, life satisfaction, and average daily sleep. MLQ is a psychometrically valid tool to measure meaning in life among Hong Kong Chinese students.

The Iowa Resistance to Sleeplessness Test (iREST)

IntroductionDespite considerable individual differences in the vulnerability vs resistance to effects of sleep loss, there is no practical self-report tool to predict these differences across domains and only limited evidence whether they are general or domain-specific. To address this need, we developed the Iowa Resistance to Sleeplessness Test (iREST). MethodsA construct-validation approach was employed. During the substantive phase, self-report items were generated to capture vulnerability vs resistance to sleep loss across various psycho-behavioral domains. During the structural phase, analyses identified the underlying factor structure and examined reliability of individual scale scores. Finally, the external phase used convergent and discriminant analyses to evaluate the factors in light of related sleep and personality measures, and tested criterion validity of the scale scores in predicting neurocognitive and affective responses to experimental sleep restriction (Total N = 1018). ResultsAnalyses yielded discriminant and reliable scale scores that reflected resistance across cognitive, affective, and somatic responses, while also marking a general resistance factor. Convergent and discriminant probes revealed moderate associations of scale scores with daytime sleepiness and sleep-related distress, but small to negligible associations with other measures of sleep behavior, perceptions, and personality. Critically, criterion analyses yielded validity evidence for predicting cognitive and affective impairments in response to experimental sleep loss. ConclusionScores on the iREST show validity in capturing cognitive and affective resistance to moderate sleep loss among young adults, supporting its further exploration as a practical tool for predicting behavior due to lost sleep.

A Construct Validation Approach for Exploring Sustainability Adoption in Pakistani Construction Projects

Sustainable Project Management (SPM) is a novel theme in construction industries of developing countries and very little is known (so far) about the sustainability performance of construction projects in those settings. Accordingly, the quantitative measurement of SPM as a higher-order construct is not well established and lacks a holistic approach and homogenous taxonomy of indicators. This study explores the SPM practices of construction firms by validating SPM as a second-order construct in the Pakistani construction industry. Data were obtained from 146 construction firms, which were then analyzed using partial least square structure equation modeling. Results of the first and second level measurement model assessments showed that construction firms practice SPM, with varying degrees of attention paid to three sustainability aspects. Environmental dimension surfaced as the most important, in-terms of practice and social sustainability as least. The analysis of the second-order measurement model yielded significant results, thus, validating the higher-order structure of SPM. This study contributes to the field by presenting one of the first studies of its kind by focusing on the sustainability practices of construction firms by using a higher-order design of SPM construct within the context of developing countries.

Do overall dimension ratings from assessment centres show external construct-related validity?

ABSTRACT There have been repeated calls for an external construct validation approach to advance our understanding of the construct-related validity of assessment centre dimension ratings beyond existing internal construct-related validity findings. Following an external construct validation approach, we examined whether linking assessment centre overall dimension ratings to ratings of the same dimensions that stem from sources external to the assessment centre provides evidence for construct-related validity of assessment centre ratings. We used data from one laboratory assessment centre sample and two field samples. External ratings of the same dimensions stemmed from assessees, assessees’ supervisors, and customers. Results converged across all three samples and showed that different dimension-same source correlations within the assessment centres were larger than same dimension-different source correlations. Moreover, confirmatory factor analyses revealed source factors but no dimension factors in the latent factor structure of overall dimension ratings from the assessment centre and from external sources. Hence, consistent results across the three samples provide no support that assessment centre overall dimension ratings and ratings of the same dimensions from other sources can be attributed to dimension factors. This questions arguments that assessment centre overall dimension ratings should have construct-related validity.

Is all nonhomeostatic eating the same? Examining the latent structure of nonhomeostatic eating processes in women and men.

Nonhomeostatic eating is a complex (presumably) multidimensional construct associated with negative health outcomes. However, little research has examined the latent structure of nonhomeostatic eating processes, interrelationships among nonhomeostatic eating constructs, and differential associations between nonhomeostatic eating constructs and external correlates. This study adopted a construct validation approach to investigate these research questions in a large sample of undergraduate women and men (N = 998; 54.6% female). Exploratory and confirmatory factor analyses were conducted on items from 8 measures representing numerous nonhomeostatic eating process constructs (e.g., binge eating, loss of control [LOC] over eating, "food addiction"). The 7-factor retained solution included the following: (1) emotional eating, (2) external eating, (3) LOC over eating, (4) overeating, (5) distress over nonhomeostatic eating, (6) hedonic hunger, and (7) food addiction. LOC over eating was the nonhomeostatic eating factor most strongly related to other factors (M rs = .71 and .65 in women and men, respectively). Factor score multiple regressions conducted separately by sex indicated that distress over nonhomeostatic eating was related to body mass index, depressive symptoms, and eating-related clinical impairment in both women and men. In women, food addiction demonstrated unique associations with depressive symptoms, emotion dysregulation, and clinical impairment, whereas overeating uniquely predicted these outcomes in men. This is the first comprehensive analysis of the latent structure of nonhomeostatic eating processes and associations with external correlates, and results suggest that LOC over eating, distress over nonhomeostatic eating, food addiction (in women), and overeating (in men) exhibited the strongest relations with psychosocial impairment. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Examining the Psychometric Validity of the Five-Item Gratitude Questionnaire: An Item Response Theory Approach

The present investigation explored the psychometric validity of the five-item Gratitude Questionnaire (GQ-5) using a construct validation approach. Concerning within-network construct validity, results of item response theory (IRT) analysis via graded response model (GRM) showed that this scale could not discriminate individuals who score high in gratitude and the 7-scale response options could be modified to a 6-scale response choice. As regards to between-network construct validity, findings demonstrated that gratitude was positively correlated to both controlled and autonomous motivation. Implications are discussed to refine the assessment of gratitude in the school contexts.

Psychometric validation of team experiences questionnaire in preregistration Chinese health and social care students in Hong Kong

ABSTRACTThe complexity of patients’ healthcare needs can be addressed more adequately by professionals working as a healthcare team. In view of this, it is important to prepare students for collaborative work when they become professionals through a students’ team training at the preregistration level. As intervention programs are implemented to promote students’ collaboration, instruments like Team Experiences Questionnaire (TEQ) are needed. In this connection, this study which involved 335 Chinese students enrolled in eight health and social care programs from two universities in Hong Kong investigated the factor structures of the TEQ using construct validation approach. Using confirmatory factor analysis, we examined the fit of three competing models: unidimensional model, first-order correlated model, and second-order hierarchical model. The results demonstrated that the second-order hierarchical model obtained the best fit, with a higher-order construct called total team experiences subsuming the following first order factors: overall satisfaction with team experiences, team impact on quality of learning, satisfaction with team evaluation, team impact on clinical reasoning ability, and professional development. The “total team experience” construct explained from 21.1% to 35.7% of the variance in predicting students’ future perception of team competency and autonomy, perceived need for cooperation, and perception of actual cooperation. Results support the applicability of the TEQ second-order hierarchical model. Implications are discussed.

Validation of Academic Delay of Gratification Scale among Indian Professional Courses Students

The present study is an attempt to conduct factor analysis of the Academic Delay of Gratification Scale (ADOGS) for college students, with 10 items, prepared by Bembenutty and Karabenick (1998), on Indian professional courses students. 461 students (256 boys and 205 girls) from engineering, pharmacy, law and education professional courses of Sultan Ul Uloom Education Society, Hyderabad, voluntarily participated in the study, out of which 336 students (190 boys and 146 girls) were part of exploratory factor analysis. With the help of SPSS Statistics Ver.23, Principal Axis Factor extraction method and Varimax rotation, two factors were extracted. Monte Carlo PCA Parallel Analysis was used to settle for one factor explaining 16% variance. The reliability of the instrument using Cronbach’s Alpha was found to be 0.715. SPSS Amos Ver. 23 was used to confirm the factor structure and establish within-network construct validity of the instrument using Fit index tests like Chi test p value, DF, CMIN/DF, TLI, CFI, IFI, NFI,RMR and RMSEA from the data of 125 students (66 boys and 59 girls), followed by between network validity based on construct validation approach using Pearson’s product moment correlation the data of 136 students (100 boys and 36 girls) measuring their academic delay of gratification and emotional intelligence. There were sufficient evidences to establish that this instrument in its present form can be administered on Indian urban students for the measurement of academic delay of gratification.

Validation of the Gratitude Questionnaire in Filipino Secondary School Students.

Most studies have assessed the psychometric properties of the Gratitude Questionnaire - Six-Item Form (GQ-6) in the Western contexts while very few research has been generated to explore the applicability of this scale in non-Western settings. To address this gap, the aim of the study was to examine the factorial validity and gender invariance of the Gratitude Questionnaire in the Philippines through a construct validation approach. There were 383 Filipino high school students who participated in the research. In terms of within-network construct validity, results of confirmatory factor analyses revealed that the five-item version of the questionnaire (GQ-5) had better fit compared to the original six-item version of the gratitude questionnaire. The scores from the GQ-5 also exhibited invariance across gender. Between-network construct validation showed that gratitude was associated with higher levels of academic achievement (β = .46, p <.001), autonomous motivation (β = .73, p <.001), and controlled motivation (β = .28, p <.01). Conversely, gratitude was linked to lower degree of amotivation (β = -.51, p <.001). Theoretical and practical implications are discussed.

Psychometric Validity and Gender Invariance of the Academic Buoyancy Scale in the Philippines: A Construct Validation Approach

The current study examined the psychometric validity and gender invariance of the Academic Buoyancy Scale in the Philippines through a construct validation approach. In terms of within-network construct validity, our results demonstrated that the unidimensional model of academic buoyancy significantly fit the current sample and was invariant across gender. Male students scored significantly higher than female students on academic buoyancy. Regarding between-network construct validity, our results revealed that academic buoyancy was positively associated with behavioral and emotional engagement. Implications of the findings of the study are discussed.

The Physical Self Description Questionnaire: Furthering Research Linking Physical Self-Concept, Physical Activity and Physical Education

Physical self-concept (PSC) is an important mediator of outcomes in physical education. Recent research has demonstrated that PSC contributes to improving levels of physical activity and skill development associated with physical education classes beyond what can be explained by prior measures of skill and activity. PSC represents a discreet domain within Shavelson, Hubner and Stanton’s (1976) multidimensional, hierarchical model of self-concept. The Physical Self-Description Questionnaire (PSDQ; Marsh, 1996) was developed to measure PSC through scales measuring nine specific and two global components of PSC. This instrument has proven to be psychometrically robust. Applying advanced methodology in the development of short forms of an established instrument, we demonstrate a new short form of the PSDQ (PSDQ-S), which retains good reliability and a well defined, replicable factor structure. This new instrument, tested with the Australian general community, utilizing a construct validation approach, would be an excellent measure for use in physical education programs.

Toward a parsimonious model of arbitrator acceptability

PurposeThe aim of this study is to take a construct validation approach toward developing and testing a parsimonious, yet comprehensive, higher‐order factor model of arbitrator acceptability that helps to identify what matters most in the arbitrator selection process.Design/methodology/approachThe paper uses structural equation modeling (SEM) techniques to examine data from a sample of union advocates attending an arbitration conference to test a hypothesized model of arbitrator acceptability that includes procedural justice, experience, and education as first‐order indicators of arbitrator acceptability. Three competing alternative models that add age, race/gender, and distributive justice, respectively, to the hypothesized model as an additional indicator of arbitrator acceptability were also examined.FindingsResults suggest that the hypothesized model is the best fitting of the four models tested. In addition, the path coefficients of the added paths in each of the alternative models were non‐significant.Research limitations/implicationsKnowledge regarding the relative importance of the various factors involved in arbitrator acceptability will be of interest to future researchers in determining which variables to study in arbitrator selection research as well as to practitioners seeking to better understand and manage the complex arbitrator selection process.Originality/valueThis study makes an important contribution to the literature by being among the first to examine both arbitrator characteristics and organizational justice concepts simultaneously in a single model. This study also takes an important step toward more clearly defining and validating the arbitrator acceptability construct. Finally, this study helps to provide an answer to the question of what matters most in the arbitrator selection process.

Making Intangibles “Tangible” in Tests of Resource-Based Theory

The authors identify theoretical disconnects among resource-based theory (RBT), conceptualizations of intangible resources (intangibles), and measures of intangibles. Specifically, the authors’ survey of 186 recent empirical tests of RBT suggests that measures of intangibles are often assigned with little theoretical conceptualization of the intangible construct or connection with RBT. This lack of theoretical justification undermines confidence in measures and empirical findings, leaves central RBT questions unaddressed, and constricts the usefulness of RBT. Furthermore, their survey suggests that the dominant approach to measuring intangibles is mechanical and unidisciplinary (i.e., rooted in either economics or psychology). The authors argue that the challenge of measuring intangibles is primarily theoretical. Thus, mechanical and unidisciplinary approaches are problematic for they sustain theoretical disconnects among RBT and intangible conceptualizations and measures. Moreover, a multidisciplinary construct validation approach is required to remedy the respective deficiencies of economic and psychological approaches and integrate their complementary strengths. The authors present such an approach, which they call the multidisciplinary assessment process (MAP). The MAP guides scholars in creating theories of intangibles that are specific to their RBT study. These theories are the focal point of the MAP and clarify what, specifically, the intangible is and how firms use the intangible to create and appropriate economic value. In turn, these theories inform the measurement and validation steps of the MAP, so that intangibles are assessed with greater credibility.

Development of a System for the Evaluation of the Teaching Qualities of Anesthesiology Faculty

GRADUATE medical education is one of the core missions of academic medical centers, wherein medical specialists are responsible for teaching and supervising their future colleagues. However, being a medical specialist is no longer a sufficient qualification or proxy for competence in medical education aimed at training residents. This is particularly true given the modernization requirements for competency-based teaching and training promoted by accreditation institutions in some countries (such as the Accreditation for Council for Graduate Medical Education in the United States). These modernization efforts accelerate faculty development of clinicianeducators needed to achieve and maintain the highest standard of postgraduate medical education. An effective faculty development track should include measuring medical teaching effectiveness. This requires valid and reliable instruments, as well as providing the findings in a clear and concise format to faculty. Several studies have found that systematic and constructive feedback can result in improved teaching. There are few published and validated evaluation systems or even instruments aimed at supporting the graduate medical education qualities of clinical faculty. In anesthesiology, there are few published instruments and systems, and the existing ones tend to focus on faculty evaluation by residents only without any self-evaluations by faculty. To ensure actual behavioral change, individuals must usually undergo a stepwise change process. Evaluation insights obtained from feedback should be followed by creating positive intentions to change, trying out new behaviors and integrating them into practice. Supporting this change process has been shown to be effective. To support the specialty-specific evaluation of teaching qualities of anesthesiology faculty in an academic medical center, we developed the System for Evaluation of Teaching Qualities (SETQ) comprising (1) a Webbased self-evaluation by faculty, (2) a Web-based residents’ evaluation of faculty, (3) individualized faculty feedback, and (4) individualized faculty follow-up support. This paper has three main objectives: (1) to investigate the psychometric properties of the two instruments underlying the SETQ system, (2) to explore the relationship between residents’ evaluation and faculty self-evaluation, and (3) to gauge the feasibility of reliably using residents’ evaluation of faculty by estimating the number of such evaluations needed per faculty. We also place these objectives in context by describing SETQ. SETQ was initially developed in the anesthesiology department of a large academic medical center that has over 7,000 staff (including about 500 faculty and 400 residents) in the Netherlands. It was later expanded to include specialty-specific modules for internal medicine, surgery, and obstetrics and gynecology. At the time of writing, most of the remaining specialties have signed up for SETQ, resulting in more than 90% faculty coverage in 2009. SETQ is receiving nationwide attention.