Sufficient Evidence Of Validity Research Articles

Validation of the Revised Patient Perception of Migraine Questionnaire (PPMQ-R): Measuring Satisfaction with Acute Migraine Treatment in Clinical Trials

This study was aimed to evaluate in clinical trial settings the psychometric properties of the revised Patient Perception of Migraine Questionnaire (PPMQ-R), a satisfaction measure for acute migraine treatment. The PPMQ-R was administered 24 h post dosing in 1304 migraineurs randomized to two identical Phase 3, single-attack trials. Reliability, concurrent and construct validity and known-groups validity were evaluated using Cronbach's alpha, Pearson correlations and analysis of variance, respectively. PPMQ-R scale and Total scores (Efficacy, Functionality and Ease of use) showed very good internal consistency reliability (alpha 0.84-0.99). Efficacy, Functionality and Total PPMQ-R scores showed large, inverse relationships with migraine pain severity, number of migraine symptoms and work ability (r = -0.62 to -0.75; all P < 0.0001). All scales discriminated among migraine pain severity levels (all P < 0.001). The PPMQ-R has sufficient evidence of validity and reliability for measuring patient satisfaction, an important benchmark of quality and effective care.

Sources of validity evidence for an internal medicine student evaluation system: an evaluative study of assessment methods

Medical students' final clinical grades in internal medicine are based on the results of multiple assessments that reflect not only the students' knowledge, but also their skills and attitudes. To examine the sources of validity evidence for internal medicine final assessment results comprising scores from 3 evaluations and 2 examinations. The final assessment scores of 8 cohorts of Year 4 medical students in a 6-year undergraduate programme were analysed. The final assessment scores consisted of scores in ward evaluations (WEs), preceptor evaluations (PREs), outpatient clinic evaluations (OPCs), general knowledge and problem-solving multiple-choice questions (MCQs), and objective structured clinical examinations (OSCEs). Sources of validity evidence examined were content, response process, internal structure, relationship to other variables, and consequences. The median generalisability coefficient of the OSCEs was 0.62. The internal consistency reliability of the MCQs was 0.84. Scores for OSCEs correlated well with WE, PRE and MCQ scores with observed (disattenuated) correlation of 0.36 (0.77), 0.33 (0.71) and 0.48 (0.69), respectively. Scores for WEs and PREs correlated better with OSCE than MCQ scores. Sources of validity evidence including content, response process, internal structure and relationship to other variables were shown for most components. There is sufficient validity evidence to support the utilisation of various types of assessment scores for final clinical grades at the end of an internal medicine rotation. Validity evidence should be examined for any final student evaluation system in order to establish the meaningfulness of the student assessment scores.

Assessment of diagnostic tests to inform policy decisions--visual electrodiagnosis.

To conduct a systematic review of the evidence for the effectiveness of five visual electrodiagnostic tests to inform the Medical Services Advisory Committee (MSAC) of the Department of Health and Ageing (Australia) in its decision in allocating public funding for new technologies. We searched the biomedical literature to identify English-language articles published from 1966 to September 2000. We assessed validity of methodology of included studies against the following criteria: investigators (i) compared test with an appropriate reference test; (ii) tested an appropriate spectrum of patients; (iii) masked assessment of study and reference tests; (iv) measured the study test independently of clinical information; and (v) measured the reference test before any interventions. Sixty-one articles met inclusion criteria for critical appraisal: nineteen were cross-sectional studies that compared a study test with another test, thirty-four were case-control studies that compared a test in a group of patients with an eye disease to a group of subjects without eye disease, and eight studies were case series. None of the included studies met all of the validity criteria. Only four studies provided enough information to calculate diagnostic characteristics but were flawed due to inclusion of patients already diagnosed with disease or lack of an appropriate reference test and, thus, overestimated test accuracy. Identified studies did not provide sufficient valid evidence of the clinical value of the five visual tests in diagnosing diseases of the retina or optic nerve. Thus, MSAC recommended that the tests not be supported by public funding.

Objectivity, Reliability, and Validity for a Revised Push-Up Test Protocol

When executing a push-up an individual lowers the body to a down position and then raises it to an up position. A down position often used in recent years is a 90° angle at the elbows (90° push-up) as in FITNESSGRAM. Several researchers have found the interscorer objectivity and stability reliability for 90° push-up scores to be low. The purpose of this study was to (a) estimate the interscorer objectivity and stability reliability for scores from a revised push-up test protocol and (b) obtain evidence concerning the validity of interpretations based on revised push-up test scores. Interscorer objectivity was estimated for a score of one scorer and stability reliability was estimated for a score obtained on 1 day. Four studies were conducted. In the first study, pilot study, the revised push-up test protocol was developed and refined. In the second study, objectivity study, the push-up test was administered once to 49 female and 31 male college-aged students. Two scorers independently scored each student. Interscorer objectivity coefficients of·75 for women and ·88 for men were obtained. In the third study, objectivity and reliability study, the push-up test was administered on each of 2 days to 89 female and 63 male college-aged students. Two scorers independently scored each student on each day. Interscorer objectivity coefficients of ·97 and ·95 for women and ·98 and ·99 for men were obtained. Stability reliability coefficients of ·90 and ·93 for women and ·95 and ·95 for men were obtained. In the fourth study, validity study, validity was estimated using a logical approach, group difference approach, and criterion approach. There were 58 male and 48 female college students in the validity study. The revised push-up test protocol is very similar to protocols presently used and as expected the men scored significantly (p < .01) better than the women on the revised push-up test. The correlation between revised push-up scores and number of bench press executions with a percentage of the body weight was·80 for women and ·87 for men. The interscorer objectivity and stability reliability coefficients are very acceptable. Sufficient validity evidence was provided that the revised push-up scores relate to the amount of arm and shoulder girdle strength and endurance a person has to move the body weight.

Management and Organizational Indicators of Process Safety

This study is part of a Nuclear Regulatory Commission research element on organizational factors in plant safety under the Human Factors research program. The study investigated the chemical industry, to find leading management or organizational tools which could be useful for the NRC. After collecting information from a variety of information sources, we concentrated our study on two types of indicators currently in use: the first is audit- or review-based, and concentrates on programmatic factors; the second, based on frequent behavioral observations, concentrates on the management of individual worker behaviors. We analyzed data on the relationships between the leading indicators and direct indicators such as accident and injury rates in three case studies, to determine whether sufficient evidence of validity and utility exists to justify consideration of these indicators as public safety indicators. On the basis of statistical associations and other evidence, we concluded that the two indicator types have promise for use as plant safety performance indicators, and that further development and testing of the candidate indicators should be performed.

Problems in Examining the Validity of the ACTFL Oral Proficiency Interview

The primary problems in measuring speaking ability through an oral interview procedure are not those related to efficiency or reliability, but rather those associated with examining the validity of the interview ratings as measures of ability in speaking and of the uses that are made of such ratings. In order to examine all aspects of validity, the abilities measured must be clearly distinguished from the elicitation procedures, in both the design of the interview and in the interpretation of ratings.Research from applied linguistics and language testing is consistent with the position that language proficiency consists of several distinct but related abilities. Research from language testing also indicates that the methods used to measure language ability have an important effect on test performance. Two frameworks—one of communicative language ability and the other of test method facets—are proposed as a basis for distinguishing abilities from elicitation procedures and for informing a program of empirical research and development.The validity of the ACTFL Oral Proficiency Interview (OPI) as it is currently designed and used cannot be adequately examined, much less demonstrated, because it confounds abilities with elicitation procedures in its design, and it provides only a single rating, which has no basis in either theory or research. As test developer, ACTFL has yet to fully discharge its responsibility for providing sufficient evidence of validity to support uses that are made of OPI ratings.