Evidence Of External Validity Research Articles

Evaluation of the Monell Forced-Choice, Paired-Comparison Tracking Procedure for Determining Sweet Taste Preferences across the Lifespan

Lack of methodology to assess taste in children limits its measurement in research studies that include pediatric populations. We used the Monell 2-series, forced-choice tracking method to measure sucrose preferences of a racially/ethnically diverse sample (n = 949) of children, adolescents, and adults. Reliability was assessed by comparing the results of the first series with the second series. Validity was assessed by relating participants' sucrose preferences to their preferences for foods varying in sweetness. The task required, on average, 7 presentations of aqueous sucrose solution pairs. Children and adolescents preferred more concentrated sweetness than adults (P < 0.001). Black children/adolescents preferred a more concentrated sucrose solution than did White children/adolescents even when gender, parental education level, and family income were used as covariates. Data from a single series were sufficient to detect age-related differences but insufficient to detect racial/ethnic differences in sweet preferences. Level of sweetness preferred significantly correlated with the sugar content of favorite cereals (P < 0.001) and beverages (P < 0.02). This method is brief and has evidence of reliability and external validity. Although a single series will yield useful information about age-related differences in taste preferences, the 2-series version should be considered when differences in race/ethnicity are of interest.

Neurocognitive Disorders in DSM-V: Forensic Perspective

Although determination of diagnosis of cognitive impairments constitutes one of the core objectives of forensic neuropsychological assessment, the diagnostic criteria for cognitive disorders provided by the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR, American Psychiatric Association 2000) have been of low internal and external validity and limited utility. This paper reviews the evolution of diagnosis of cognitive disorders from current DSM-IV-TR to the proposed DSM-V (American Psychiatric Association 2010). The conceptual framework is tracked, along with the body of literature associated with the DSM-V proposal on Minor and Major Neurocognitive Disorders. The quality of evidence for internal and external validity is reviewed, as well as the fairness and utility of newly proposed diagnoses from both clinical and forensic perspectives. Significant conceptual, evidentiary, psychometric, assessment, and practical questions are raised related to the new classification, and recommendations are offered for improvement to the proposal of the DSM-V Work Group on Neurocognitive Disorders.

Validity Evidence for the Child Occupational Self Assessment

To ensure that researchers and practitioners make valid interpretations from children's self-reports, evidence must demonstrate that the self-report has appropriate psychometric properties. The Child Occupational Self Assessment (COSA) is a self-report of occupational competence and value for everyday activities designed to involve children in identifying goals and assessing outcomes. Five hundred two children with disabilities, ages 6-17, completed the COSA. We used a Rasch Partial Credit model and parametric and nonparametric statistics to obtain validity evidence. Evidence suggests that the COSA has good content, structural, and substantive validity as given by item and child fit statistics and unidimensionality evaluation. Evidence for external validity was mixed because child fit status and measures varied with some demographic and assessment administration variables. Evidence suggests that most children's responses to the COSA can be validly interpreted as indicators of occupational competence and value for everyday activities.

Establishing validity evidence for the Sport Multidimensional Perfectionism Scale-2 in intercollegiate sport

Objectives The purpose of the two studies presented in this paper is to establish validity evidence for the Sport Multidimensional Perfectionism Scale-2 (Sport-MPS-2: Gotwals & Dunn, 2009). Studies 1 and 2 are respectively focused on the production of structurally-related and external validity evidence. Methods In Study 1, 263 male and female intercollegiate volleyball players ( M age = 19.98 years) completed the Sport-MPS-2. In Study 2, 181 male intercollegiate ice hockey players ( M age = 22.16 years) responded to the Sport-MPS-2, the Multidimensional Perfectionism Scale ( Frost, Marten, Lahart, & Rosenblate, 1990), and a modified Sport Anxiety Scale ( Smith, Smoll, & Schutz, 1990). Results Exploratory factor analysis results in Study 1 revealed supportive structurally-related validity evidence for the six factors of the Sport-MPS-2. Study 2 generated supportive external validity evidence by producing anticipated moderate to large positive correlations between conceptually analogous Sport-MPS-2 and Frost-MPS subscales (all ps < .001), using canonical correlation to demonstrate that the Sport-MPS-2 was able to represent theory-based relationships between perfectionism and competitive trait anxiety (CTA), and indicating that the Sport-MPS-2—but not the Frost-MPS—was capable of identifying an anticipated link between healthy perfectionism and CTA. Conclusions Results provide support for the continued use of the Sport-MPS-2 as a measure of multidimensional perfectionism in sport.

Measuring basic needs satisfaction: Evaluating previous research and conducting new psychometric evaluations of the Basic Needs Satisfaction in General Scale

Self-Determination Theory specifies the existence of three basic psychological needs: autonomy, competence, and relatedness. The current set of studies (a) provides a narrative review of past research on the Basic Needs Satisfaction in General Scale, (b) examines its dimensionality which has been assumed but not empirically studied, and (c) gathers external validity evidence. Confirmatory factor analysis was used to test the existence of a one- and a three-factor solution; neither model fit the data. After patterns of misfit were examined across three independent samples, a reduced, 16-item three-factor model with a negative-worded method effect was championed. External validity evidence, collected by examining the differential relationships between the three needs and measures of well-being and worry, supported the distinctiveness of the three needs. Although the results are promising, future research is needed to examine the generalizability of the psychometric properties of the modified scale.

Consumer preferences for additional ethical attributes of organic food

This paper examines the relevance of additional ethical attributes of organic food for consumers’ purchase decisions. By means of an Information-Display-Matrix (IDM) and an accompanying consumer survey, the information acquisition behaviour of consumers regarding seven additional ethical attributes and the product price of organic food was investigated in five European countries. The ethical attributes, ‘animal welfare’, ‘regional production’ and ‘fair prices to farmers’ turned out to be the most important. The results also show that the vast majority of consumers of organic food was willing to pay higher prices for organic products with additional ethical attributes. Contrasting the results with those from direct inquiry by means of a ‘single source approach’ revealed considerable differences between both methods, particularly regarding the preferences for the product price relative to some of the ethical attributes. This may serve as evidence for higher external validity of the IDM with respect to social desirability bias.

Adaptation of the ESPA29 Parental Socialization Styles Scale to the Basque Language: Evidence of Validity

The aim of this study is to adapt the ESPA29 scale of parental socialization styles in adolescence to the Basque language. The study of its psychometric properties is based on the search for evidence of internal and external validity. The first focuses on the assessment of the dimensionality of the scale by means of exploratory factor analysis. The relationship between the dimensions of parental socialization styles and gender and age guarantee the external validity of the scale. The study of the equivalence of the adapted and original versions is based on the comparisons of the reliability coefficients and on factor congruence. The results allow us to conclude the equivalence of the two scales.

Gathering Evidence of External Validity for the Foundations of Medicine Examination: A Collaboration Between the National Board of Medical Examiners and the University of Minho

To gather evidence of external validity for the Foundations of Medicine (FOM) examination by assessing the relationship between its subscores and local grades for a sample of Portuguese medical students. Correlations were computed between six FOM subscores and nine Minho University grades for a sample of 90 medical students. A canonical correlation analysis was run between FOM and Minho measures. Moderate correlations were noted between FOM subscores and Minho grades, ranging from -0.02 to 0.53. One canonical correlation was statistically significant. The FOM variate accounted for 44% of variance in FOM subscores and 22% of variance in Minho end-of-year grades. The Minho canonical variate accounted for 34% of variance in Minho grades and 17% of the FOM subscore variances. The FOM examination seems to supplement local assessments by targeting constructs not currently measured. Therefore, it may contribute to a more comprehensive assessment of basic and clinical sciences knowledge.

Validation of the Seating and Mobility Script Concordance Test

ABSTRACT The purpose of this study was to develop the scoring system for the Seating and Mobility Script Concordance Test (SMSCT), obtain and appraise internal and external structure evidence, and assess the validity of the SMSCT. The SMSCT purpose is to provide a method for testing knowledge of seating and mobility prescription. A sample of 106 therapists and 15 spinal cord injury experts contributed to the development of the scoring system. Validity evidence was obtained using 15 seating and mobility experts, 10 orthopedic experts, and 66 therapists with varying levels of seating and mobility expertise. Proxy measures of clinical expertise were used for external validity evidence since no criterion measures exist. The SMSCT was found to differentiate between seating and mobility experts' and orthopedic experts' intervention subtest scores (p = 0.04). The proxy measure of clinical expertise, seating and mobility hours/week, was found to predict SMSCT intervention scores (p = 0.002). The internal structure of the SMSCT may include evidence of reduced item performance but satisfactory convergent and discriminate evidence by construct definition. Although the SMSCT may be a promising approach for measuring seating and mobility expertise, limitations exist in the corrected content. Future application of the SMSCT should only be used after further development of the tool occurs.

The psychometric property and validation of a fatalism scale

In this article, we conceptualised fatalism as a set of health beliefs that encompass the dimensions of predetermination, luck and pessimism. A 20-item scale was developed as a measurement instrument. Confirmatory factor analyses were performed to test the dimensionality of the scale. Three external variables (i.e. genetic determinism, perceived benefits of lifestyle change and intention to engage in healthy behaviour) were used as reference variables to test the construct validity of the scale. Data from a web-based national survey (N = 1218) showed that the scale was unidimensional on the second order, and with good reliability (α = 0.88). The relationships between the external variables and the first- and second-order factors provided evidence of the scale's external consistency and construct validity.

A Multi-Method Multi-Analytic Approach to Establishing Internal Construct Validity Evidence: The Sport Multidimensional Perfectionism Scale 2

This article presents a chronology of three empirical studies that outline the measurement process by which two new subscales (Doubts about Actions and Organization) were developed and integrated into a revised version of Dunn, Causgrove Dunn, and Syrotuik's (2002)Sport Multidimensional Perfectionism Scale (i.e., the Sport-MPS-2). All studies were designed to provide internal validity evidence that supported the inclusion of these new subscales into the instrument. Phase 1 determined that expert judges regarded the new Doubts about Actions and Organization items (n = 12) as possessing adequate levels of content-relevance and content-representativeness. Phases 2 and 3 examined the latent dimensionality of the Sport-MPS-2 using multidimensional scaling and factor analytic techniques. At a structural level, results indicated that the new Doubts about Actions and Organization subscales were suitable for inclusion in the Sport-MPS-2. Initial external validity evidence is presented via correlational and regression analyses surrounding the relationships between Sport-MPS-2 subscales and global self-esteem. The benefits of utilizing a multi-method multi-analytic approach in the initial stages of the construct validation process are discussed.

Shoulder Ultrasound Abnormalities, Physical Examination Findings, and Pain in Manual Wheelchair Users With Spinal Cord Injury

Brose SW, Boninger ML, Fullerton B, McCann T, Collinger JL, Impink BG, Dyson-Hudson TA. Shoulder ultrasound abnormalities, physical examination findings, and pain in manual wheelchair users with spinal cord injury. Objectives To investigate the presence of ultrasound (US) abnormalities in manual wheelchair users with spinal cord injury (SCI) using a quantitative Ultrasound Shoulder Pathology Rating Scale (USPRS). To investigate physical examination (PE) findings using a quantitative Physical Examination of the Shoulder Scale (PESS), and to obtain data about pain and other subject characteristics such as age, years with SCI, and weight. Design Case series. Setting National Veterans' Wheelchair Games 2005 and 2006. Participants Volunteer sample of manual wheelchair users with SCI participating in the National Veterans' Wheelchair Games. Interventions Not applicable. Main Outcome Measures Presence of relationships between US findings, PE findings, pain, and subject characteristics. Results The USPRS correlated with age, duration of SCI, and weight (all P<.01), and showed a positive trend with the total Wheelchair User's Shoulder Pain Index (WUSPI) score ( r=.258, P=.073). Several US findings related to presence of PE findings for specific structures. The PESS score correlated with the WUSPI ( r=.679, P<.001) and duration of SCI ( P<.05). The presence of untreated shoulder pain that curtailed activity was noted in 24.5% of subjects, and this was related to increased WUSPI scores ( P=.002). Conclusions PE and US abnormalities are common in manual wheelchair users with SCI. The USPRS and PESS demonstrated evidence for external validity and hold promise as research tools. Untreated shoulder pain is common in manual wheelchair users with SCI, and further investigation of this pain is indicated.

Development and Validation of the Goal Content for Exercise Questionnaire

Self-determination theory (SDT; Deci & Ryan, 2000) proposes that intrinsic, relative to extrinsic, goal content is a critical predictor of the quality of an individual's behavior and psychological well-being. Through three studies, we developed and psychometrically tested a measure of intrinsic and extrinsic goal content in the exercise context: the Goal Content for Exercise Questionnaire (GCEQ). In adults, exploratory (N = 354; Study 1) and confirmatory factor analyses (N = 312; Study 2) supported a 20-item solution consisting of 5 lower order factors (i.e., social affiliation, health management, skill development, image and social recognition) that could be subsumed within a 2-factor higher order structure (i.e., intrinsic and extrinsic). Evidence for external validity, temporal stability, gender invariance, and internal consistency of the GCEQ was found. An independent sample (N = 475; Study 3) provided further support for the lower order structure of the GCEQ and some support for the higher order structure. The GCEQ was supported as a measure of exercise-based goal content, which may help understand how intrinsic and extrinsic goals can motivate exercise behavior.

Promoting Regular Mammography Screening I. A Systematic Assessment of Validity in a Randomized Trial

Most health promotion trials in cancer screening offer limited evidence of external validity. We assessed internal and external validity in a nationwide, population-based trial of an intervention to promote regular mammography screening. Beginning in September 2000, study candidates age 52 years and older (n = 23,000) were randomly selected from the National Registry of Women Veterans and sent an eligibility survey. Consistent with intention-to-treat principles for effectiveness trials, we randomly assigned eligible respondents and nonrespondents to one of five groups. We mailed baseline surveys to groups 1-3 followed by intervention materials of varying personalization to groups 1 and 2. We delayed mailing baseline surveys to two additional control groups to coincide with the mailing of postintervention follow-up surveys to groups 1-3 at year 1 (group 4) and year 2 (group 5). Mammography rates were determined from self-report and Veterans Health Administration records. To assess internal validity, we compared groups on participation and factors associated with mammography screening at each stage. To assess external validity, we compared groups 3, 4, and 5 on mammography rates at the most recent follow-up to detect any cueing effects of prior surveys and at the respective baselines to uncover any secular trends. We also compared nonparticipants with participants on factors associated with mammography screening at the trial's end. We established study eligibility for 21,340 (92.8%) of the study candidates. Groups 1-3 were similar throughout the trial in participation and correlates of mammography screening. No statistically significant survey cueing effects or differences between nonparticipants and participants across groups were observed. Mammography screening rates over the 30 months preceding the respective baselines were lower in group 5 (82.3% by self-report) than in groups 1-4 (85.1%, P = .024, group 5 vs groups 1-4 combined), suggesting a decline over time similar to that reported for US women in general. This systematic assessment provides evidence of the trial's internal and external validity and illustrates an approach to evaluating validity that is readily adaptable to future trials of behavioral interventions.

Construct Validation and Psychometric Evaluation of the Self-Efficacy Scale for Driver Competence

The aim of the present study was to examine the construct validity of the Self-Efficacy Scale for Driver Competence (SSDC). The SSDC was based on a theoretical model for perceived driver competence, based on the self-efficacy construct and five aspects of driver competence. Two samples of driving license examinees (n = 805, n = 721) completed two parallel versions of the SSDC. Participants’ mean age was 21.5 years and of them 44% were women. Both versions of the SSDC had sound psychometric properties. The results provided support for substantial and structural aspects of construct validity. Some evidence of external validity of the test scores was obtained, although the relationship between perceived and actual driver competence was weaker than expected. The consequences of the use and interpretation of SSDC are discussed.

Investigating the Youth Physical Activity Promotion Model: Internal Structure and External Validity Evidence for a Potential Measurement Model

The purpose of the study was to investigate the measurement properties of questionnaires associated with the Youth Physical Activity Promotion (YPAP) model. Data were collected from 296 children in Grades 5-8 using several existing questionnaires corresponding to YPAP model components, a physical activity questionnaire, and 6 consecutive days of pedometer data. Internal validity of the questionnaires was tested using confirmatory factor analyses, and external validity was investigated via correlations with physical activity and body composition. Initial model fit of the questionnaires ranged from poor to very good. After item removal, all scales demonstrated good fit. Correlations with percentage body fat and objectively measured physical activity were low but in the theoretically predicted direction. The current study provides good internal validity evidence and acceptable external validity evidence for a brief set of questionnaire items to investigate the theoretical basis for the YPAP model.

Vineland Survey: Self-administered Checklist Format for Teachers of Young Children in Rehabilitation

We examined the feasibility of administering the Vineland Adaptive Behavior Survey Edition in checklist format to teachers as a means of obtaining more useful teacher ratings of low functioning young children. Teachers completed the Vineland Classroom and Survey Editions as checklists to rate 36 preschoolers (ages 4–1 to 5–11 years; 22 male, 14 female) in a rehabilitation day treatment setting. The Survey Edition was administered to parents using the standard semi-structured interview. The Classroom Edition yielded higher skill estimates than did the two Survey Editions (p < 0.001), which did not differ significantly from each other. The individually administered Diagnostic Inventory for Screening Children (n = 27) provided evidence of external validity of the self-administered Survey Edition format for teachers. The results support use of the Survey Edition in checklist format for teachers who are rating skills of young children with developmental disabilities.

Moving beyond academic achievement goal measures: A study of social achievement goals

The current research explored the theory of social goal orientation. More specifically, we conducted three studies utilizing six-independent university student samples to evaluate the construct validity of the Social Achievement Goal Orientation Scale (SAGOS; Ryan & Hopkins, 2003), a measure representing the construct of social goal orientation. The purpose of Study 1 was to: (1) compare the three-dimensional (mastery, performance-approach, and performance-avoidance) model of social goal orientation to three theoretically based competing models, (2) examine item functioning, and (3) assess generalizability of the factor structure. The fit of the proposed three-factor model was promising; however, areas of misfit and problematic items were identified. Stronger support for the three-factor structure of goal orientation was found using scores from an abbreviated 13-item SAGOS. In Study 2, item wording was altered slightly to evaluate a revised Social Achievement Goal Scale (SAGS), yet resulted in similar findings. Study 3 examined external validity evidence for the SAGS, garnering some support for the meaning of the scores. Although continued refinement of the SAGOS and SAGS is recommended, the findings help contribute to our general understanding and conceptualization of social goal theory and the role that social goals may play in academic contexts.

Investigating Validity Evidence for the Experiences in Close Relationships-Revised Questionnaire

The current study gathered internal structural validity and external criterion validity evidence for the Experiences in Close Relationships-Revised Questionnaire (ECR-R) scores. Specifically, confirmatory factor analysis of the data provided general support for the hypothesized two-factor model, and hypothesized relationships with external criteria were substantiated. However, minor model misfit and low communalities ( R2) suggested that some items may represent extraneous constructs. Further avenues of study regarding the functioning of the instrument are provided.

Online measurement of motivational processes: Introducing the Continuous Delay Aversion Test (ConDAT)

The Continuous Delay Aversion Test (ConDAT), a new computer task for online monitoring and continuously measuring delay aversion (DA), is introduced. DA is a motivational style related to a shortened delay gradient which is proposed as a major endophenotype of attention deficit hyperactivity disorder (ADHD). It is characterised by avoiding or escaping from delay-rich situations despite the prospects of a reward. In each ConDAT trial the rapidly diminishing reward/delay ratio, which tends asymptotically towards zero, is visually presented on the computer screen. The test subject is permanently confronted with the question whether to quit or to continue the trial in the face of the deteriorating reward/time ratio. An elaborated control of stimuli and responses, including the sending of trigger codes to external recording devices, makes the task useful for neurophysiological or brain imaging experiments. Compared to existing tasks, the ConDAT is more flexible and sensitive due to its asymptotic open-ended trials and the interval-scaled output measure. Pilot data give evidence for satisfactory reliability and external validity of the task.