Content Validity Studies Research Articles

Transatlantic peer-to-peer learning: an initial feasibility analysis.

Peer-to-peer learning is a well-established learning modality, which has been shown to improve learning outcomes, with positive implications for clinical practice. The purpose of this pilot study was to explore the feasibility of linking students from North America and Europe with a peer-to-peer learning approach. Face and content validity studies were completed on the previously designed and validated online repository http://www.pilgrimshospital.com. Four medical students from the University of Toronto, Canada, were paired with four students from University College Cork, Ireland. Each student was invited to upload two pieces of information learned from a senior colleague that day. Each student was asked to review the information uploaded by their partner, editing with references if needed. Quantitative and qualitative evaluations of the e-peer system were conducted. Over the study period, the system recorded a total of 10 079 individual page views. Questionnaires completed by participants demonstrated that 6/8 found the system either "very easy" or "easy" to use, whereas all found that the system promoted evidenced-based and self-directed learning. Structured interviews revealed 3 main themes: The Peer Connection, Trust in Data Veracity, and Aid to Clinical Learning. This pilot study demonstrates it is feasible to link students from separate continents in a community of peer-to-peer learning. This is viewed positively by students and enhances evidenced-based learning, and the aspect of peer connectivity was important to participating students. Such an approach encourages peer cooperation and has the potential to disseminate key clinical learning experiences widely.

Adaptation and Validation of the Brazilian Version of the Hope Index

The objective of this study was to adapt and gather validity evidence for a Brazilian sample version of the Hope Index and to verify if cultural differences would produce different results than those found in the United States. In this study, we present a set of analyses that together comprise a comprehensive validity argument for the use of a test in a new language or culture. Participants were 844 undergraduate Brazilian students (mean age 21.3 years). A content validity study suggested that new items could be added to the scale. Factor analyses extracted two factors replicating the original instrument structure. The internal consistency was adequate with and without the added items. The results illustrate how different validation procedures can be used to support the use of an adapted version of a test, suggesting that in Brazil, researchers may choose the original adapted version (16 items) or the more emic version (21 items) depending on the objectives of their study. The indication that the Hope Index is valid in a Brazilian sample strengthens the idea that this construct may be universal. The results also stress the importance of conducting content validity studies as part of a cross-cultural adaptation of a test.

Evaluation of the Orcamp Hybrid High-fidelity Environment for Simulation Training

e l l a “ e o a c c e c In the accompanying article examining operator appraisal of a high-fidelity procedure suite, Lonn et al (1) contend that simulation-based learning is enhanced by performing that training in a room that is made as close as possible to a real procedure suite or operating room by the use of a simulated procedural environment that includes a physical mockup of a C-arm and operating table. The authors describe face and content validity studies on an Orcamp room (Orzone, Gothenburg, Sweden), which housed a full-physics Vascular Instructional System Trainer virtual reality simulator (Mentice, Gothenburg, Sweden). (Full disclosure: I was the medical leader of the team that developed the Vascular Instructional System Trainer, but I have no conflicts of interest related to its invention.) The quantitative evaluation studies in this study (1) are performed on the space surrounding the simulator, not on the simulator itself, which has undergone numerous rigorous validity studies. Similar reports about the Orcamp system, but in the context of an operating room, have been reported at the Vascular and Endovascular Issues, Techniques and Horizons Symposium (2). The authors show high user ratings for the functionality and realism of the C-arm and “operating table” among a subject group of radiologists and cardiologists. Separately, they also show statistically significantly positive assessment ratings for the realism of the catheter, guide wire, fluoroscopy image, and physiologic readouts displayed by the simulator. However, these are two different categories of variable. The image display, fluoroscopic realism, physiologic monitoring, and catheter and guide wire realism are all components of the physics built into the simulator. Indeed, interactive real-time synthetic fluoroscopy was first created for this machine (3). Establishing face and content validity for the device that looks and acts like a C-arm and an operating table is the most base level of assessment of those objects. The authors conclude by stating that the hybrid operating room experience provides a higher fidelity level of training in a realistic context or the procedure suite (1). Yet,

Content Validity of the Female Sexual Function Index (FSFI) in Pre‐ and Postmenopausal Women with Hypoactive Sexual Desire Disorder

The Female Sexual Function Index (FSFI) has consistently been shown to have discriminant validity, test-retest reliability, and internal consistency as a measure of female sexual function. However, the content validity (relevance, clarity, comprehensiveness) of the instrument in women with hypoactive sexual desire disorder (HSDD) must also be established. The aim of this study were to assess the content validity of the FSFI, specifically the FSFI desire domain, in pre- and postmenopausal women with HSDD. Two single-visit content validation studies were conducted in the United States. Eligible premenopausal (both studies) and postmenopausal (second study only) women with HSDD completed the FSFI followed by one-on-one, face-to-face cognitive debriefing interviews including open-ended questions to capture information on their perceptions of the instrument. Information on women's experiences of decreased sexual desire was also captured. The main outcome measures of this study were the women's ratings of the clarity, ease of understanding, comprehensiveness, and relevance of the 19 items of the FSFI. Interviews with 15 premenopausal women (first study), and 30 pre- and 31 postmenopausal women (second study), were analyzed. Across the whole sample, most women (80-100%) found every item of the FSFI clear and easy to understand. The majority (53-70%) felt that the FSFI captured all their feelings about decreased sexual desire and other sexual problems, and most (84-90%) indicated that additional questions were unnecessary. Most women in both studies (93-100%) reported that the two items comprising the FSFI desire domain were clear, easy to understand, and were relevant to them. The majority of women thought that a recall period of ≥7 days is most relevant for recall of their sexual desire. These studies establish the content validity of the FSFI in pre- and postmenopausal women with HSDD, supporting the use of this instrument as a measure of sexual function in women with this condition.

The Examination for Professional Practice in Psychology (EPPP) in the era of evidence-based practice.

Professional psychology has increasingly moved toward evidence-based practice. However, instrumentsused to assess psychologists seeking licensure, such as the Examination for Professional Practice inPsychology (EPPP), have received relatively little empirical scrutiny. Therefore, the authors evaluatedthe available evidence in support of the EPPP’s validity and current use as a core component ofprofessional licensure. Although the EPPP has in many ways been extensively evaluated, there is apaucity of criterion, predictive, and incremental validity evidence available. Further, several aspects ofthe content validation studies were examined, and the authors question whether the EPPP, as currentlyconstructed, can meet its stated goals. Given that the EPPP is a high-stakes examination and given theauthors’ best estimate (based on a sample of 16 states) that 35% of applicants fail the examination, it isrecommended that the EPPP be more extensively evaluated. An outline of major decision points in thisproposed evaluation process is provided, several suggestions for further research are proposed, and thefield is encouraged to discuss these issues further.Keywords: ExaminationforProfessionalPracticeinPsychology,EPPP,licensure,educationandtraining,competence

Three Models of Corporate Social Responsibility: Interrelationships between Theory, Research, and Practice

Three Models of Corporate Social Responsibility: Interrelationships between Theory, Research, and Practice

The Expert Nurse Profile and Diagnostic Content Validity of Sedentary Lifestyle: The Spanish Validation

To identify the diagnostic content validity of Sedentary Lifestyle and to identify the expert nurse profile in validating this nursing diagnosis in the Spanish cultural context. Fehring's Diagnostic Content Validity (DCV), the factorial validity of the defining characteristics, the analysis of convergent validity, and the expert profile were assessed. The DCV index for experts was .70. The factorial validity showed two different factors: the expression of laziness and the performance of activities of daily living. On the expert profile related factors analysis, two factors, experience and education, were identified. The DCV for Sedentary Lifestyle was high among the expert nurses. A nurse was considered to be an expert who was able to accurately answer the four labels identified as nursing diagnoses, had read at least one nursing process article in the past year, and was able to list three nursing process reference books. Clarifying the manifestations of sedentary lifestyle will assist clinical nurses in determining this diagnosis, and the expert profiles will assist in the selection of participants for content validity studies.

The development of a tool to audit the safety policies and practices of community sports clubs

Despite increased national effort directed at sports injury prevention in Australia since the mid 1990s, there is a lack of information available about the sports safety policies and practices of community sports clubs. The aim of this study was to develop a valid and reliable sports safety audit tool (SSAT) to identify these safety policies and practices. A literature review identified issues to be covered by the SSAT. Consultation with "experts" and piloting the SSAT with 19 community sports clubs in metropolitan Sydney established face and content validity. Test-retest reliability was assessed in six clubs. Inter-rater reliability was assessed using twenty-four independent representatives from eight clubs. Face and content validity studies identified issues to include in the SSAT and improvements to language and layout. Test-retest reliability was 91% (range 68-100%). Inter-rater reliability ranged from 40-65% when missing data and 'don't know' answers were included, and from 62-75% when only 'definitive' answers were included. Club presidents and secretaries provided more definitive information than other informants. A preliminary list of safety issues that clubs addressed well or poorly was identified. The SSAT is a useful tool for gathering baseline data, benchmarking and targeting sports safety interventions with community sports clubs. Club presidents and secretaries are the preferred contact point and a face-to-face interview is the best administration mode. A tool to identify safety policies and practices is now available for use by anyone supporting community sports clubs to improve safety.

Assessment of quality of life in a multicultural cancer patient population.

Quality of life (QOL) is increasingly assessed in cancer patients. In this article, the authors examined the psychometric performance of a commonly used QOL questionnaire, the Quality of Life Questionnaire--Cancer 30 (QLQ-C30; N. K. Aaronson et al., 1993), in multiethnic cancer patients. Content validation studies in patients and clinicians identified possible new items. Multiple-group confirmatory factor analysis supported equivalent structure across ethnic groups (Caucasians and Asian/Pacific Islanders [APIs]). A higher order QOL factor appeared to directly affect functioning scales and symptom count. Exploratory factor analysis examined effects of new items. Ten factors were extracted, 6 consistent with the original instrument and 4 reflecting potentially new aspects of QOL: Positive Social Support, Coping, Existential Well-Being, and Sexuality/Intimacy. The QLQ-C30 appears appropriate for use in API cancer patients. Further work needs to ensure that it includes all important domains.

Gathering and Analyzing Content Validity Data

If the scores from educational assessments are to be useful and defensible, the content validity of the tests must be established This article briefly describes content validity theory and illustrates new and traditional approaches for conducting content validity studies. Newer approaches are based on multidimensional scaling analysis of item-similarity ratings Traditional approaches are based on ratings of item-objective congruence and relevance Drawing on previous research in this area, guidelines for gathering and analyzing content validity data are provided.

Preemployment Physical Evaluation

There is a growing trend toward using preemployment tests to select employees for physically demanding jobs. Women are, in increasing numbers, entering physically demanding occupations that were traditionally dominated by men. Under current Federal employment law, it is illegal to disqualify an employee for a job because of race, color, religion, sex, national origin, and with the recent passage of the American Disabilities Act (ADA), handicap. Because of gender differences in strength, body composition, and VO2max, preemployment tests for physically demanding jobs tend to screen out more females than males. Employers are using preemployment tests not only to enhance worker productivity, but also to minimize the threat of litigation for discriminatory hiring practices and to reduce the risk of musculoskeletal injuries. The primary ergonomic methods used in industry to reduce the risk of back injuries are preemployment testing and job redesign. When a test results in adverse impact, the validity of the test must be established. Validity in this context means that the test represents or predicts the applicant's capacity to perform the job. Criterion-related, content, and construct validation studies are the means used to establish validity. The validity of preemployment hiring practices for physically demanding jobs has been decided in the courts. The most common reason for ruling an employment practice invalid is the failure to show that the test measured important job behaviors. Much of this litigation has involved height and weight requirements for public safety jobs. The courts have generally ruled that using height and weight standards as a criteria for employment is illegal because they were not job related. If fitness tests comprise part or all of the preemployment test, it is essential to demonstrate that the fitness component is related to job performance. Although there are many factors to consider when establishing a cut score, there is a growing trend toward establishing the cut score on the basis of the job's physical demands, defined by VO2max and strength. This literature is limited because most validation studies are not published. They more typically take the form of a technical report to the governmental agency or company that funded the project. There are published preemployment validation studies for outdoor telephone craft jobs involving pole-climbing tasks; firefighters; highway patrol officers; steel workers; underground coal miners; chemical plant workers; electrical transmission lineworkers; and various military jobs.

Assessment of school students' needs for therapy services

The development of a screening instrument to assess school students' needs for physiotherapy, occupational therapy and speech pathology is reported in this paper. Items for primary and secondary level students were generated by teachers and therapists to form a draft referral instrument for classroom teachers. Ninety six teachers in 18 metropolitan and country schools reviewed the four forms of the draft instrument with a total of 636 students. The results indicate that the draft Screening Instrument for Teacher Referral to Therapy Services in Schools (SRT) has face validity and utility in the educational setting. Plans to further refine the SRT items, and to conduct reliability and content validity studies are discussed.

Development of a Quality-of-Movement Measure for Children with Cerebral Palsy

Development of a suitable measure of quality of movement, or gross motor performance, for children with cerebral palsy is a complex undertaking. A variety of conceptual, methodological, and practical issues inherent in such a project are discussed in this article. We report on the methodology used in the planning and construction of the Gross Motor Performance Measure. The measure has been developed by a multicenter, interdisciplinary group of therapists, methodologists, research staff, and international experts. Five attributes of gross motor performance have been defined, scaled, and operationalized. Results of content validity studies demonstrate that the measure has adequate completeness, clarity, and potential for evaluating change in quality of movement in children who have cerebral palsy. The measure is currently undergoing extensive testing to determine the reliability, validity, and responsiveness of the obtained scores. [Boyce WF, Gowland C, Hardy S, et al. Development of a quality-of-movement measure for children with cerebral palsy.

Content Validity Studies of Licensing Examinations

Can measurement specialists’current ideas about content validation be implemented with licensure examinations? Does pressure of litigation facilitate or inhibit conducting validity studies?

Content validation studies of the AP Chemistry examination

In 1985-1986, the development committee for AP Chemistry undertook an ambitious two-part study aimed at a critical evaluation of both the appropriateness of content and the level of difficulty of the AP Chemistry examination.

On the Statistical Determination of Content Validity

Several criteria related to individual exposure-nonexposure responses are suggested for consideration in content validity studies. Current validation procedures which concentrate on an item analysis are shown to be insensitive to the suggested procedures. Examples of this insensitivity are given.

Interpreting the Leiter IQ

Language-disordered children with normal range performance IQ may nevertheless evidence cognitive deficits. This suggests the need for content validity studies of performance tests of intelligence. In the current investigation, items from the Leiter International Performance Scale (LIPS) were analyzed for their conceptual versus perceptual character; these item groupings were then used to create intratest LIPS profiles for 16 language-disordered children and 16 children with normal language, aged 4:4-5:5, who were matched for CA, sex, and Leiter IQ. Interjudge (N = 4) reliability for determination of item type was 96%, and the results of the item analysis revealed that 65% of LIPS items in the 2-8 year range were perceptual in nature. A repeated-measures ANOVA indicated that subjects performed relatively better on the perceptual items regardless of group (p less than .01) and that there were no significant group differences. Analysis of individual subject profiles, however, revealed accentuated intratest profiles for two of the language-disordered subjects and reversal of the dominant perceptual greater than conceptual trend for four of the language-disordered subjects. These results indicate that LIPS scores for children under age 8 primarily represent their ability to assess the physical characteristics of visual stimuli and may not adequately reflect the transforming, interpretive aspects of nonverbal cognition.