Construct Validity Study Research Articles

Evaluation of the psychometrics properties of the alarm distress baby scale among 122 Brazilian children.

The Alarm Distress Baby Scale (ADBB) aims at assessing infant withdrawal behavior. A previous validation study revealed acceptable reliability and validity indices. The present study investigated the psychometric properties of the scale in a larger sample from a culturally different population. Pediatricians evaluated the behavior of 122 infants, 2 and 19 months old, using the ADBB during routine physical consultation. Four investigators (two pediatricians and two nurses not specialized in pediatric care) examined video recordings of the evaluations. Results showed good interrater reliability coefficients among pediatricians and poor correlation when all professionals were grouped together. Test-retest reliability revealed good intraexaminer agreement (r = 0.91). The pediatricians' evaluation using the ADBB was compared with a psychiatric examination to investigate the scale's criterion validity. The cutoff point of 5 provided the best clinical validity (sensitivity of 79%, specificity of 81%). Results from construct validity showed that the scale had three dimensions. Comparison of the factor solution with other construct validity studies of the same instrument revealed similarities and differences. Results suggest that the ADBB may be a useful screening instrument to detect signs of psychiatric alterations related to withdrawal behavior in primary care services, and it is likely to provide consistent information.

Test of the Construct and Criteria Validity of a German Measure of Organizational Justice

Organizational justice is a well-known and increasingly often measured construct in work-related psychological research, for which, however, different kinds of measures are used. Colquitt (2001 ) developed a four-dimensional measure of organizational justice to allow for the comparison of different studies. Two studies provide evidence for the construct (Study 1) and criteria validity (Study 2) of the German version of the Colquitt measure with regard to perceived supervisor behavior as well as employees’ attitudes and intentions. In Study 1, the measure demonstrated construct validity using a German sample of employees (N = 227) from different companies: Each dimension correlated differently with work-related dimension of supervisor behavior. To demonstrate criteria validity in Study 2 (N = 315 employees), the justice measure predicted different, theoretically linked work-related employee behaviors or attitudes (i.e., intentions to leave, job satisfaction, job stress, individual organizational citizenship behavior). Results are discussed with regard to theoretical and practical implications, intercultural differences, and future research.

Mental Activity and Referential Activity of Beginning Therapists: A Construct Validity Study of the Countertransference Rating System (CRS)

This study investigates the construct validity of a revised version of the Countertransference Rating System (CRS) by means of convergence with Referential Activity (RA). The CRS operationalizes three mental activity dimensions (rational-objective, reactive, and reflective) as processes of transformation of countertransferential contents elicited in a therapist by a patient's object-relations units. The participants were 36 novice psychotherapists who shared their spontaneous reactions toward parental descriptions provided by conduct-disordered male adolescent patients. Globally, the reflective dimension was positively correlated with RA, whereas the other two dimensions--rational-objective and reactive--showed no association. Dominant categories within each dimension displayed patterns of correlation with RA that are consistent with the constructs. These results are discussed with reference to potential impact on intervention and training.

Can we differentiate sensory over-responsivity from anxiety symptoms in toddlers? perspectives of occupational therapists and psychologists.

Suppose a toddler becomes distressed after hearing a loud noise or when getting messy; are these indicators of sensory over-responsiveness or indicators of anxiety? There is little evidence regarding the distinction between sensory over-responsivity and anxiety disorders in toddlers. This construct validity study examined differences between occupational therapists' and psychologists' judgments of behaviors as representing sensory processing disorders (SPD) versus anxiety disorders. Twenty-four occupational therapists and 25 psychologists completed a mailed survey, rating items from sensory and anxiety scales as representing sensory and/or anxiety disorders in toddlers, and analyzing cases of toddlers with these disorders. Occupational therapists were more likely to rate items as representing SPD than psychologists, and occupational therapists were more certain of the distinction of the sensory scales from anxiety. For the case designed to present a general anxiety disorder and the one designed to present sensory over-responsivity, more occupational therapists diagnosed as sensory over-responsive, while more psychologists diagnosed with a general anxiety disorder. The overlap in judgments of sensory over-responsivity and anxiety supports the notion that these constructs in part reflect different professionals' perspectives upon behaviors as well as the difficulty in distinguishing these constructs in toddlers.

Case Study

Three major reviews of the assessment center (AC) construct-validity literature have disagreed as to the most appropriate analytic model for AC postexercise dimension ratings. We report a Monte Carlo study addressing the following questions: (a) To what extent does the “true” model (i.e., the model that generated the data) actually appear to fit the data well? (b) To what extent can a model appear to fit the data well even though it is the wrong model? and (c) Is model fit actually a useful empirical criterion for judging which model is most likely the population model? Results suggest that “true” models may not always appear as the best fitting models, whereas “false” models sometimes appear to offer better fit than the true models. Implications for the study of AC construct validity are discussed.

Construct validation of analytic rating scales in a speaking assessment: Reporting a score profile and a composite

This is a construct validation study of a second language speaking assessment that reported a language profile based on analytic rating scales and a composite score. The study addressed three key issues: score dependability, convergent/discriminant validity of analytic rating scales and the weighting of analytic ratings in the composite score. Confirmatory factor analysis (CFA) and multivariate generalizability theory (G theory) were combined to analyze the responses of 214 admits to a study-abroad program to two role-play speaking tasks in a Spanish speaking assessment designed for student placement and diagnosis. The CFA and G theory approaches provided complementary information, which generally confirmed the key features of the assessment design: (1) the multicomponential and yet highly correlated nature of the five analytic rating scales: Pronunciation, Vocabulary, Cohesion, Organization and Grammar, (2) the high dependability of the ratings and the resulting placement decisions appropriate for the high-stakes decision-making context based on these analytic rating scales and the composite score, and (3) the largest contribution of Grammar to the composite score variance, which was consistent with the intention of program faculty members to reflect in the test design the relative importance of knowledge of grammar for students' academic success in the study-abroad program.

Assessment of an in vitro model for laparoscopic pelvic lymphadenectomy

An in vitro model was developed for laparoscopic pelvic lymphadenectomy. A construct validity study was performed comparing experienced laparoscopic surgeons with inexperienced trainees. Outcome measures included global and task-orientated scoring and data from electromagnetic motion analysis of hands. Marking was performed by three independent surgeons using video playback. A significant difference was found in the time taken, number of movements made and total distance travelled between the groups. Both scoring systems discriminated between the expert and inexperienced group, demonstrating the model's construct validity.

Brazilian–Portuguese version of defensive style questionnaire-40 for the assessment of defense mechanisms: construct validity study

The aim of this study is to analyze the reliability and validity of the Brazilian–Portuguese version of the Defensive Style Questionnaire (DSQ). Participants were evaluated by clinical interview and the Mini-International Neuropsychiatric Interview. Reliability was assessed by factor analysis and Cronbach's alpha. The 3-factor model's replication was tested using confirmatory factor analysis (CFA). Controls were reevaluated after 4 months to assess test–retest reliability. Subsamples were also evaluated using the Minnesota Multiphasic Personality Inventory and Defensive Functioning Scale (DFS). Factor analysis of the various defense mechanisms produced a distribution of mechanisms among the factors that was largely similar to previous analyses based on the original instrument. The 3-factor model was supported by CFA. The test–retest evaluation showed sufficient intraclass correlation for all factors. Patients used more immature and neurotic defenses than controls. There was a correlation between DFS and immature defenses. Individuals with pathological personality traits scored higher on the immature and neurotic factors than those without. The Brazilian–Portuguese version of the DSQ-40 has psychometric features that allow the use of this instrument in our culture.

The Kinesthetic and Visual Imagery Questionnaire (KVIQ) for Assessing Motor Imagery in Persons with Physical Disabilities: A Reliability and Construct Validity Study

To benefit from mental practice training after stroke, one must be able to engage in motor imagery, and thus reliable motor imagery assessment tools tailored to persons with sensorimotor impairments are needed. The aims of this study were to (1) examine the test-retest reliability of the Kinesthetic and Visual Imagery Questionnaire (KVIQ-20) and its short version (the KVIQ-10) in healthy subjects and subjects with stroke, (2) investigate the internal consistency of both KVIQ versions, and (3) explore the factorial structure of the two KVIQ versions. The KVIQ assesses on a five-point ordinal scale the clarity of the image (visual: V subscale) and the intensity of the sensations (kinesthetic: K subscale) that the subjects are able to imagine from the first-person perspective. Nineteen persons who had sustained a stroke (CVA group) and 46 healthy persons (CTL group) including an age-matched (aCTL: n = 19) control group were assessed twice by the same examiner 10 to 14 days apart. The test-retest reliability was assessed using intraclass correlation coef- ficients (ICCs). The internal consistency (Cronbach alpha) and the factorial structure of both KVIQ versions were studied in a sample of 131 subjects. In the CVA group, the ICCs ranged from 0.81 to 0.90, from 0.73 to 0.86 in the aCTL group, and from 0.72 to 0.81 in the CTL group. When imagining movements of the affected and unaffected limbs (upper and lower limbs combined) ICCs in the CVA group ranged, respectively, from 0.71 to.87 and from 0.86 to 0.94. Likewise, when imagining movement of the dominant and nondominant limbs, ICCs in the aCTL group ranged, respectively, from 0.75 to 0.89 and from 0.81 to.92. Cronbach a values were, respectively, 0.94 (V) and 0.92 (K) for the KVIQ-20 and 0.89 (V) and 0.87(K) for the KVIQ-10. The factorial analyses indicated that two factors explained 63.4% and 67.7% of total variance, respectively. Both versions of the KVIQ present similar psychometric properties that support their use in healthy individuals and in persons post-stroke. Because the KVIQ-10 can be administered in half the time, however, it is a good choice when assessing persons with physical disabilities.

Construct Validity and Reliability of Hartman's Color Code Personality Profile1

The three reliability and two construct validity studies presented here represent the first psychometric research on the commercially successful and popular Hartman Color Code Personality Profile. The reliability studies found high reliability after 3 and 6 weeks. The construct validity studies substantiated that the Color Code assesses some personality traits. The magnitude of the error variance suggests caution, however, in using the Color Code classifications at the individual level.

A clinical construct validity study of a novel computerized battery for the diagnosis of ADHD in young adults

The cognitive profile of adult attention deficit hyperactivity disorder (ADHD) remains understudied despite difficulty in diagnosis. Further, no battery of neuropsychological tests has been shown valid in adult ADHD. Continuous performance tests are widely used for ADHD but provide limited information on cognitive functioning in general. The present study evaluated the construct and discriminant validity of Mindstreams® (NeuroTrax Corp., NY), a computerized battery assessing multiple cognitive domains. Twenty-nine young male adults with ADHD diagnosis completed a Mindstreams battery, including a multi-stage continuous performance (‘Expanded Go-NoGo’) test, and the Conners' CPT-II (Multi-Health Systems Inc., NY). Discriminant validity was assessed by comparisons with cognitively healthy controls of comparable age and education. Expanded Go-NoGo and corresponding CPT-II outcomes were significantly correlated in ADHD participants, and the Expanded Go-NoGo test exhibited excellent discriminant validity, with ADHD participants performing more poorly than controls. ADHD participants also performed more poorly on Stroop and Staged Information Processing Speed tests.

A Construct Validity Study of Bullying

The construct validity of the Bully/Victim Questionnaire (Olweus, 1996a) as a measure of bullying was examined. Although researchers have defined bullying and reactive aggression as two distinct types of aggression, this study examined how closely related they are using empirical data. Reports from students and teachers on measures of bullying and reactive aggression were compared for evidence of convergent validity (degree of similarity) and divergent validity (degree of difference). Comparisons of correlation coefficients indicate that convergent validity coefficients (.26 to .85) are not higher than divergent validity coefficients (.21 to .74). In addition, correlations between teachers and students differ. It is concluded that measuring bullying is problematic and that bullying and reactive aggression are highly correlated whereas teacher and student reports are not.

The International Physical Activity Questionnaire (IPAQ): a study of concurrent and construct validity

The International Physical Activity Questionnaire (IPAQ) was developed to measure health-related physical activity (PA) in populations. The short version of the IPAQ has been tested extensively and is now used in many international studies. The present study aimed to explore the validity characteristics of the long-version IPAQ. Forty-six voluntary healthy male and female subjects (age, mean +/- standard deviation: 40.7 +/- 10.3 years) participated in the study. PA indicators derived from the long, self-administered IPAQ were compared with data from an activity monitor and a PA log book for concurrent validity, and with aerobic fitness, body mass index (BMI) and percentage body fat for construct validity. Strong positive relationships were observed between the activity monitor data and the IPAQ data for total PA (rho = 0.55, P < 0.001) and vigorous PA (rho = 0.71, P < 0.001), but a weaker relationship for moderate PA (rho = 0.21, P = 0.051). Calculated MET-h day(-1) from the PA log book was significantly correlated with MET-h day(-1) from the IPAQ (rho = 0.67, P < 0.001). A weak correlation was observed between IPAQ data for total PA and both aerobic fitness (rho = 0.21, P = 0.051) and BMI (rho = 0.25, P = 0.009). No significant correlation was observed between percentage body fat and IPAQ variables. Bland-Altman analysis suggested that the inability of activity monitors to detect certain types of activities might introduce a source of error in criterion validation studies. The long, self-administered IPAQ questionnaire has acceptable validity when assessing levels and patterns of PA in healthy adults.

Validity of the new Backache Index (BAI) in patients with low back pain

Background context The Backache Index (BAI) is applied to patients with low back pain (LBP) in order to help therapists, doctors, and surgeons perform physical examinations easily. It is carried out within a short space of time (<2 min) without using inclinometric instruments. Purpose To explore the reliability, validity, and responsiveness of this new Backache Index in patients with LBP, which can fulfil the existing need for a reliable routine examination in the clinical environment. Study design/setting Patients with LBP filled in disability questionnaires; pain rating scales and physical impairment tests were completed in function of construct validity and correlation studies. A subgroup was evaluated for interobserver and test-retest reliability, and a second group was reassessed after two active treatment sessions to verify the responsiveness compared with other examined variables. Patient sample In total, 75 patients with subacute LBP (3–12 weeks) participated in a randomized controlled study. Outcome measures The validity of the BAI was explored through a correlation with the standard Oswestry LBP Disability Index (ODI), the McGill LBP Questionnaire Index (MPQ), and the Visual Analogue Scale (VAS). Methods The BAI consisted of a scoring system that includes pain factors and stiffness estimation at the end of a series of five different lumbar movements of a patient standing in an erect position. Results The correlations between the separate outcomes and the BAI ranged from 0.61 to 0.76 (p<.001). The interobserver reliability between two experienced observers for the five outcome scores was good (intraclass correlation coefficient [ICC]>0.86) and even perfect for the BAI (ICC=0.96). A BAI change of one unit is able to exclude a measurement error. A significantly good correlation (p<.001) was found between the BAI at baseline, the ODI (R=0.62), and the total degree of pain rating index (MPQ-PRI-T) (R=0.57), a moderate correlation with the total number of chosen adjectives from the whole list of adjectives (MPQ-NWC-T) (R=0.48), and the VAS (R=0.47), but a lower correlation was found with the MPQ-Quality of life index (R=0.43). The effect size and discriminative ability of the measures were explored after two treatment sessions of deep transverse friction myotherapy by means of study of the receiver operating characteristics curve (ROC) and the greatest area under the curve (AUC). The greatest level of distinction was found for the MPQ-PRI-T and the BAI (AUC>0.93), followed by the ODI (AUC=0.92). A lower level of distinction was found for the MPQ-NWC-T and the VAS (AUC>0.82). Conclusions The BAI appears to be a reliable and valid assessment of overall restricted spinal movements in case of LBP and discriminates between successful and unsuccessful treatment outcome.

A Cross-Country Construct Validation of Cognitive Age

This study tests the universality of the Western-based concept known as cognitive age within Japan. It assesses both the internal and external validity of cognitive age among Japanese seniors and compares the findings to the same measures of cognitive age with a sample of senior respondents from the United States. The study finds that the semantic differential scale has the largest trait variance among all aging concepts studied (average cognitive age, average ideal age, and average least-desired age), while the Likert scale possesses the largest trait variance for average cognitive age. The ratio scale was found to have the lowest trait variance of the three scaling formats evaluated. External construct validation studies revealed a remarkable similarity between Japanese females and males, and contrasts between Japanese and American seniors revealed reasonably good generalizability between countries. Average cognitive age appears to be universal within two culturally disjoint countries (the United States and Japan), yet the efficacy of individual measurement scales varies between them.

Computer-assisted assessment of one-handed knot tying skills performed within various contexts: a construct validity study

Background The widespread use of computer-assisted assessment of technical proficiency in surgical residents shows the need for further investigations of the construct validity of these devices (eg, hand motion analysis) before implementation into competency testing. Methods Thirty general surgery residents performed 1-handed knot tying in 2 contexts: superficial and deep. The Imperial College Surgical Assessment Device (ICSAD) was used to evaluate performance. It was hypothesized that senior residents (postgraduate year [PGY] 4-5) would perform better then junior residents (PGY 1-3) and that the superficial version of the skill would be easier than the deep. Results Technical efficiency scores were better for seniors than for juniors ( P < .001) and on the superficial versus the deep model ( P < .001). Both groups were equally affected by the contextual changes to the skill, suggesting a consistent impact on the skill-specific movement patterns. Conclusions Additional evidence for the validity of ICSAD as a competency assessment tool has been shown. First, it distinguished senior and junior residents. Second, it discerned differences on the same skill performed in 2 different contexts.

Assessing Multidimensional Physical Activity Motivation: A Construct Validity Study of High School Students

This study presents a new, multidimensional approach to physical activity motivation that is operationalized through four primary factors: adaptive cognitive dimensions, adaptive behavioral dimensions, impeding cognitive dimensions, and maladaptive behavioral dimensions. Among 171 Australian high school students, the study assessed the structure of this four-factor framework (a within-network construct validity approach) and also examined the relationships between motivation and three key correlates: flow in physical activity, physical self-concept, and physical activity level (a between-network construct validity approach). The four-factor framework demonstrated within-network validity in the form of reliable subscales and a sound factor structure. In terms of between-network validity, relationships between the adaptive behavioral and cognitive aspects of motivation and physical self-concept, flow, and activity levels were found to be positive and significant, whereas significant inverse relationships were found between impeding and maladaptive motivation dimensions and flow and physical self-concept. Additional analysis utilizing multiple-indicator multiple-cause (MIMIC) modeling showed that during earlier adolescence girls are more motivated than boys to engage in physical activity, but by later adolescence boys are more motivated to do so. Results are interpreted in terms of future directions for possible physical activity interventions aimed at increasing both the uptake and continuation of activity.

Validación del Cuestionario de Variables de Interacción Psicosocial (VIP): hacia un modelo de tratamiento de las conductas adictivas «guiado por la personalidad»

Objective The personality can be studied at three levels: clinical (disorders), traits and the level of interactive variables. There are valid instruments for the measurement of both first, but we do not have any that explores in a combined way the interactional level. We propose a model of «personality guided treatment» for the addictive behaviours in the direction of the one proposed by Millon for axis II disorders, based on function of drug in relation to its personality pattern. The aim of the study is to design and validation of a questionnaire that explores in a suitable and simple way a certain number of psycho-social interaction variables in the individuals that are in treatment by abuse/dependency of substances and gambling. We choose as variables: self-esteem, general self-efficacy, optimism, locus of control, social abilities, self-control and coping styles. Material and methods It is described the creation process of the Questionnaire of Variables of Psycho-social Interaction (VIP), until reaching its definitive form, with 84 items. We validate on a sample of 776 subjects, 569 in treatment by abuse/ dependency of heroin, cocaine, alcohol, cannabis, benzodiazepines and compulsive gambling; and 197 subjects of general population. Results All participants compliment the VIP and diverse complementary questionnaires for the validity study. The global internal consistency is 0.93 and the one of the scales that compose it locates between 0.82 and 0.91, except for those of coping oriented to search of social support (5 items), 0.78; and the one of locus of control (3 items), 0.65. The temporary consistency of the scales (between 4 and 6 weeks) is 0.92 for general population and 0.80 for subjects in treatment. The apparent, content, convergent and discriminative and construct validity studies, are suitables. The scales of the VIP predict an average of 45% of the variance of the clinical syndromes of axis I, and 33% of pathological personality patterns (reaching more of 50% in 5 of them) considered by means of the MCMI-II. Conclusions We considered that the VIP is an instrument of utility for the core personality variables study, that allows the understanding of functional value of the addictive behaviours in relation to the persons general behaviour pattern, and can facilitate the design and evaluation of the specific treatments, although can be applied in other scopes.

Recovery From Schizophrenia: A Concept in Search of Research

Recovery From Schizophrenia: A Concept in Search of Research

Stages of Change for Osteoporosis Preventive Behaviors

This article presents a construct validation study for osteoporosis preventive behaviors based on a survey completed by 113 older adults. Participants were classified using stages of change for osteoporosis preventive behaviors according to an algorithm adapted from the Transtheoretical Model of health behavior change. One-way analyses of variance and a hierarchical regression model indicate that stage of change location is significantly related to one's level of knowledge about osteoporosis as well as age, gender, and educational attainment. These findings bring support to the validity of the instrument. Findings from this study indicate that elders' attitudes and knowledge about osteoporosis reflect public health messages heralding osteoporosis as the disease of the older White women. Significant health care expenditures in the nontargeted groups call for a revision in these messages. Further applications with more heterogeneous samples are necessary for thorough validation and reliability testing of the proposed instrument.