Examine Content Validity Research Articles

Development of the Gross Motor Function Classification System (1997)

To address the need for a standardized system to classify the gross motor function of children with cerebral palsy, the authors developed a five-level classification system analogous to the staging and grading systems used in medicine. Nominal group process and Delphi survey consensus methods were used to examine content validity and revise the classification system until consensus among 48 experts (physical therapists, occupational therapists, and developmental pediatricians with expertize in cerebral palsy) was achieved. Interrater reliability (kappa) was 0.55 for children less than 2 years of age and 0.75 for children 2 to 12 years of age. The classification system has application for clinical practice, research, teaching, and administration.

Examining the content validity of the WHOQOL-BREF from respondents’ perspective by quantitative methods

Content validity, the extent to which a measurement reflects the specific intended domain of content, is a basic type of validity for a valid measurement. It was usually examined qualitatively and relied on experts’ subjective judgments, not on respondents’ responses. Therefore, the purpose of this study was to introduce and demonstrate how to use quantitative methods to examine the content validity for a certain measurement from respondents’ viewpoint with quantitative methods. In the current study, content validity of the WHOQOL-BREF was examined with quantitative methods among 102 undergraduate students and 128 community adults. They were asked to rate the appropriateness of each item with the four domains of the WHOQOL-BREF respectively and also asked to sort the items into the four domains. Then, three quantitative methods on examining content validity were applied to analyze the rating and sorting data, including (1) proportion of substantive agreement, (2) substantive validity, and (3) ANOVA approach. These results were used to compare with the original content structure of the WHOQOL-BREF, to see if the original structure is consistent with the structure of subjects’ judgments. The results showed that the content structure gained from appropriateness rating and item sorting was not totally consistent with the original content structure of the WHOQOL-BREF. Among 24 items, 12 items did not have adequate content validity. More discussion on these items and the issue of content validity were further provided.

TEACHER SELF-EFFICACY BELIEF

This study presents development and validation of a new measurement instrument to explore student-teachers' self-efficacy beliefs toward teaching. We developed and administered a survey to 346 student teachers at Hacettepe University, Ankara, Turkey in the Department of Elementary Education in the spring term of the 2003–2004 academic year. The nature of the study is descriptive. The aim is to describe what the real situation is. Standard scale development methods and factor analysis were used. The results of the study were strongly supported by the validity and reliability of the survey. Cronbach's alpha was calculated and the reliability coefficient was 0.92. For purposes of examining content validity, the opinions of experts on 5 subject matters were taken. Factor analysis was made to elicit the fundamental dimensions of the survey. As a result of the Kalse-Meyer-Olkin and Bartlett test (0.93), factor analysis was administered to the survey. A single-factor model was specified for the structure of the survey as anticipated.

Functional Capacity Evaluation as a Performance Measure

To report the research and development of a new approach to Functional Capacity Evaluation, the Gibson Approach to Functional Capacity Evaluation (GAPP FCE) for chronic back pain clients. Four studies, including pilot and feasibility testing, expert review, and preliminary interrater reliability examination, are described here. Participants included 7 healthy young adults and 19 rehabilitation clients with back pain who underwent assessment using the GAPP FCE. Thirteen therapists were trained in the approach and were silently observed administering the Functional Capacity Evaluations by at least 1 other trained therapists or the first investigator or both. An expert review using 5 expert occupational therapists was also conducted. Study 1, the pilot with healthy individuals, indicated that the GAPP FCE was a feasible approach with good utility. Study 2, a pilot using 2 trained therapists assessing 5 back pain clients, supported the clinical feasibility of the approach. The expert review in Study 3 found support for GAPP FCE. Study 4, a trial of the approach with 14 rehabilitation clients, found support for the interrater reliability of recommendations for return to work based on performance in the GAPP FCE. The evidence thus far available supports the GAPP FCE as an approach that provides a sound method for evaluating the performance of the physical demands of work with clients with chronic back pain. The tool has been shown to have good face and content validity, to meet acceptable test standards, and to have reasonable interrater reliability. Further research is occurring to look at a larger interrater reliability study, to further examine content validity, and to examine predictive validity.

Psycometrics properties of subjetive indicator in children

To evaluate the psychometric properties of the Oral Heath Impact Profile (OHIP-14) to measure oral health in children. The OHIP-14 questionnaire was applied to 312 schoolchildren aged 12 years residing in Sabará, Brazil, in 2001. OHIP-14's reliability was evaluated using Cronbach's alpha and correlation intraclass coefficient. For examining content validity, Pearson's correlation and logistic regression analysis were applied. For construct validity, Student's t-test and Tukey-Kramer test were used. Tooth loss among the subjects was low in this study (between 85% and 100%). Caries prevalence was very low and not detectable in 59.0% of the sample. The logistic regression model showed high predictive values for reporting treatment need and intermediate values for dental caries perception. The comparison between different OHIP average scores, according to the DMF-S index's three categories of decayed, missing, filled surfaces, indicated significant difference only for the categories social disability and deficit. The study results indicated that the adolescents did not report high prevalence of oral conditions. The OHIP was associated with perceived treatment need, reporting of dental caries and increased DMF-S index. These results suggest that the OHIP has good psychometric properties when applied to children and could be a promising instrument for screening priority care groups.

The development of a client health status outcome evaluation instrument in home care

This study was to develop a client health status outcome evaluation instrument, and examine content validity, reliability, construct validity, and the acceptability of this instrument. A preliminary list was made of such key information as standards, criteria, indicators and measures, by means of a broad review of literature within the field. After determining the preliminary instruments, the study sought to obtain examination, consensus, and modification of two groups of experts in the home-care field. Finally, the instrument examined content validity, reliability, construct validity, and the acceptability of this instrument. The tool was considered of 13 criteria, 48 indicators, and 167 detail measures. The content validity index of the tool was above 0.8 according to the expert group. Regarding the reliability of the evaluators of standards 1 and 2, the degree of agreement between evaluators was high(96.4% through 98.2%). Construct validity in this study, the difference in the mean score between the baseline point and the follow up point of each of standards 1 and 2 was significant, and the mean score of the follow up point was more than that of the baseline point. After examining the acceptability of this instrument with practice managers and home care nurses in home care institutions, a positive opinion was given of this instrument, and it was indicated that to be useful and applicable in home care practice. The results of evaluating client outcome will contribute to overall outcome-based quality improvement and service marketing in home care by providing a constant gauge of home care effectiveness.

High-risk pregnancy: Preterm labor

Fatigue is among the most prevalent and distressing symptoms in pregnant and postpartum women. Estimating fatigue severity with a psychometrically sound instrument provides the most fundamental information for understanding women's fatigue experience and assessing the need for intervention to improve maternal and infant health outcomes.To evaluate the psychometric properties of a Chinese version of the 7-item Lee Fatigue Scale-Short Form (C-LFS-SF) in pregnant and postpartum women.The study was composed of two phases: translation of the scale into Chinese and examination of content validity, and testing to establish the reliability and validity. A convenience sample of 124 women completed health-related questionnaires, kept a fatigue diary and wore a wrist actigraph for 7 days during the third trimester of pregnancy and within three months postpartum.The C-LFS-SF showed satisfactory internal consistency (Cronbach's alpha ≥ .97) and stability over the 7 days at each time point (intraclass correlation coefficient > .87). Exploratory factor analysis showed that 88–94% of the total variance was explained by the one-factor fatigue model. Significant associations among the C-LFS-SF and actigraphic sleep quantity and quality variables supported adequate construct validity.The C-LFS-SF has satisfactory psychometric properties and is an easy and promising tool for assessing maternal fatigue during routine prenatal and postpartum care. This scale needs to be further tested in a more diverse population of pregnant and postpartum women, like women with high risk pregnancies or with medical conditions.

Construction of scales to measure thematic tendencies of paranoid ideation

In order to measure thematic tendencies of paranoid ideation in non-clinical samples, Delusional Ideation Checklist (DICL) was developed. A wide range of items indicating themes of delusion were collected. From the results of factor analysis on the data of 308 college students, eight scales were constructed. They were: alienation, belittlement, persecution, other-harming, guardedness, self-conceit, favoredness, and other-manipulation. These scales could be classified in terms of emotion: positive versus negative, and direction: self versus others, and could deal with themes of paranoid ideation in a comprehensive and systematic way. Alpha coefficients of the scales were between .66 and .87, and test-retest reliability between .49 and .71. To examine content validity, ten psychiatrists were asked to rate diagnostic importance for each scale item. Six scales were rated as significantly more important in the diagnosis of paranoid schizophrenia than that of anxiety neurosis. Current data revealed that ordinary students experienced delusional ideation more frequently than psychiatrists would expect.

Validation of the Arizona Governor's Quality Award criteria: a test of the Baldrige criteria

AbstractTens of thousands of firms have used the Malcolm Baldrige National Quality Award (MBNQA) criteria to improve the quality of their products and processes. Because the MBNQA data are confidential, there has been no published assessment of the validity and reliability of these criteria for evaluating the quality management practices of an organization. In this research, we analyze data from the Arizona Governor's Quality Award (AGQA) in order to check the validity of the model used to evaluate organizational quality. The criteria for the AGQA closely parallel the MBNQA criteria. Hence, the conclusions drawn from this study provide a basis for generalization to the MBNQA criteria. In the literature review, the MBNQA criteria are compared to two other instruments that have been developed and tested for measuring quality management. The MBNQA criteria are found to be more comprehensive and less prescriptive than the other two. In our research, we examine content validity, construct validity, and predictive validity of the AGQA. Finally, we present our conclusions and implications for researchers and managers.

Development and reliability of a system to classify gross motor function in children with cerebral palsy.

To address the need for a standardized system to classify the gross motor function of children with cerebral palsy, the authors developed a five-level classification system analogous to the staging and grading systems used in medicine. Nominal group process and Delphi survey consensus methods were used to examine content validity and revise the classification system until consensus among 48 experts (physical therapists, occupational therapists, and developmental pediatricians with expertise in cerebral palsy) was achieved. Interrater reliability (kappa) was 0.55 for children less than 2 years of age and 0.75 for children 2 to 12 years of age. The classification system has application for clinical practice, research, teaching, and administration.

Service Quality: A Measure of Information Systems Effectiveness

The IS function now includes a significant service component. However, commonly used measures of IS effectiveness focus on the products, rather than the services, of the IS function. Thus, there is the danger that IS researchers will mismeasure IS effectiveness if they do not include in their assessment package a measure of IS service quality. SERVQUAL, an instrument developed by marketing researchers, is offered as a possible measure of IS service quality. SERVQUAL measures service dimensions of tangibles, reliability, responsiveness, assurance, and empathy. The suitability of SERVQUAL was assessed in three different types of organizations in three countries. After examination of content validity, reliability, convergent validity, nomological validity, and discriminant validity, the study concludes that SERVQUAL is an appropriate instrument for researchers seeking a measure of IS service quality.

Evaluating peer reviews. Pilot testing of a grading instrument

To measure the reliability and preliminary validity of a grading instrument for editors to evaluate the quality of peer reviews. The consecutive sample design included 53 reviews of 23 manuscripts. Reviews were systematically assigned to interrater reliability (n = 41; power greater than 0.90 to detect a difference of greater than one point) and preliminary criterion-related validity (n = 12) subsamples. Content validity was closely examined. Nonclinical. Three graders evaluated reliability. One individual examined content validity and two editors tested preliminary criterion-related validity. INTERVENTION (INSTRUMENT)--Attributes reflecting two basic dimensions, review content and format, were identified and scored (values are possible points/percent contribution): timeliness, 3/21%; grade sheet, 1/7%; etiquette, 1/7%; sectional narratives, 3/21%; citations, 2/14%; narrative summary, 2/14%; and insights, 2/14%. A scoring guide was provided. Statistical analyses used to test the interrater reliability of the total score included the intraclass correlation coefficient and analysis of variance with the expectation to uphold the null hypothesis. Kendall's coefficient of concordance was used to test preliminary criterion-related validity. The intraclass correlation coefficient was .84 (P < .001) and a lack of difference between mean scores was demonstrated by analysis of variance (P = .46). Content validity was confirmed and preliminary criterion-related validity was indicated (Kendall's coefficient of concordance = .94, P = .038). The instrument is reliable. Content validation has been completed, and further criterion-related validation is warranted.

Evaluating Peer Reviews

<h3>Objective.</h3> —To measure the reliability and preliminary validity of a grading instrument for editors to evaluate the quality of peer reviews. <h3>Design.</h3> —The consecutive sample design included 53 reviews of 23 manuscripts. Reviews were systematically assigned to interrater reliability (n=41; power greater than 0.90 to detect a difference of greater than one point) and preliminary criterion-related validity (n=12) subsamples. Content validity was closely examined. <h3>Setting.</h3> —Nonclinical. <h3>Participants.</h3> —Three graders evaluated reliability. One individual examined content validity and two editors tested preliminary criterion-related validity. <h3>Intervention (Instrument).</h3> —Attributes reflecting two basic dimensions, review content and format, were identified and scored (values are possible points/percent contribution): timeliness, 3/21%; grade sheet, 1/7%; etiquette, 1/7%; sectional narratives, 3/21%; citations, 2/14%; narrative summary, 2/14%; and insights, 2/14%. A scoring guide was provided. <h3>Main Outcome Measures.</h3> —Statistical analyses used to test the interrater reliability of the total score included the intraclass correlation coefficient and analysis of variance with the expectation to uphold the null hypothesis. Kendall's coefficient of concordance was used to test preliminary criterion-related validity. <h3>Results.</h3> —The intraclass correlation coefficient was.84 (<i>P</i>&lt;.001) and a lack of difference between mean scores was demonstrated by analysis of variance (<i>P</i>=.46). Content validity was confirmed and preliminary criterion-related validity was indicated (Kendall's coefficient of concordance=.94,<i>P</i>=.038). Conclusions.—The instrument is reliable. Content validation has been completed, and further criterion-related validation is warranted. (<i>JAMA</i>. 1994;272:98-100)

Content validation of impaired skin integrity and urinary incontinence in the home health setting.

The purpose of this study was to examine content validity of two nursing diagnoses in the home health setting: impaired skin integrity and altered urinary elimination: total incontinence. Eighty-two home health nurses rated the importance of each of the defining characteristics for the two nursing diagnoses. Diagnostic content validity scores (DCV) were calculated for each defining characteristic for both diagnoses using the method described by Fehring (1986). Four characteristics for each diagnosis had DCV scores greater than .75, indicating characteristics were critical indicators for the diagnosis. Erythema, denuded skin, disruptions of dermal and epidermal tissue, and lesions were critical indicators for impaired skin integrity. Lack of awareness of incontinence, constant flow of urine, lack of awareness of bladder cues to void, and incontinence refractory to other treatments were critical indicators for alteration in urinary elimination: total incontinence.

Using the Delphi technique to examine the content validity of nursing diagnoses.

Official labeling of phenomena that nurses diagnose and treat requires that researchers validate the existence of proposed diagnoses. One part of the validation process is the examination of content validity. The Delphi technique is a useful methodologic technique for such an examination. This article describes the characteristics, background, advantages, and disadvantages of the Delphi technique, as well as suggested strategies for using it to validate nursing diagnoses. In discussing strategies for using the Delphi technique to validate nursing diagnoses, the format, essential components, and steps of each round of the traditional and modified Delphi procedure are addressed. Round I of the traditional Delphi technique involves selection of a panel of content experts, developing and mailing questionnaires and content analysis, and generating a list of defining characteristics and operational definitions for a nursing diagnosis. Round II involves developing and mailing questionnaires, determining percentage of agreement, and providing statistical analysis. Round III includes developing and mailing questionnaires and providing statistical analysis of data. Round IV consists of developing and mailing the final report, which includes a statistical summary of individual and grouped responses in round III, as well as a minority report. The format and statistical analysis of the modified Delphi procedure is similar to rounds II, III, and IV of the traditional Delphi technique, except the modified technique usually will require only three rounds.

A Content Validity Study of the Test of Orientation for Rehabilitation Patients

The purpose of this study was to examine the content validity of the Test of Orientation for Rehabilitation Patients (TORP). Content validity was examined at both the item and domain levels using an interdisciplinary group of 11 content experts. At the item level, percent agreement with the intended domain and item-objective congruence were evaluated in tandem. At the domain level, Lu's coefficient of agreement was calculated. Results of the analyses provided moderate to strong support for the retention of 46 of the TORP's 49 items and for grouping the items by domain of orientation. This study exemplified a quantitative approach to the examination of content validity using content experts.

Examining Content Validity through the Use of Content Experts

The most often-used and practical approach to the content validation of occupational therapy assessments involves the use of content experts. Few procedures exist, however, to quantify these judgments. This article briefly reviews the literature on the examination of content validity through the use of expert judgment and describes a method for data collection and analysis with the index of item-objective congruence. This index permits the examination of content validity at the item, subtest, and test levels. Its strengths, as well as concerns regarding its use, are discussed.

Psychometric Properties of the LPC Scale: An Extension of Rice's Review

This article aims to extend Rice's review of psychometric properties of the Least Preferred Co-worker (LPC) scale by providing additional critical evaluation and new directions for LPC research. The American Psychological Association, in its revised Standards for Educational and Psychological Tests indicates that construct validity can be demonstrated only when a theoretical definition of the construct exists. In addition, a construct can only be understood or inferred from its network of interrelationships based on the construct definition. The authors note that Rice's review also does not attempt to directly examine content validity. Instead, he discusses the fact that different versions of the LPC scale exist, all with different item content.