Measurement Properties Of Instruments Research Articles

Avaliação do comportamento de crianças em situações de dor: revisão da literatura

OBJECTIVE: To analyze the scientific production published between 2001 and 2006 describing studies assessing children in painful situations, with a focus on preschool children. SOURCES: Searches were run for empirical articles indexed in electronic databases using the keywords pain assessment and behavior and their equivalents in Portuguese. SUMMARY OF THE FINDINGS: A total of 33 articles were identified, 18 of which were studies of the psychometric properties of instruments for the measurement of pain, while 11 dealt with the application of such instruments and four were studies of intervention procedures for the relief of pain. CONCLUSIONS: The results indicate that more recent literature on the assessment of preschool children's behavior in painful situations is focused on testing the psychometric properties of measurement instruments. Acute pain, especially during surgical procedures, was most often assessed using one-dimensional instruments. Pain in children was most often assessed using facial activity as indicator and intensity as dimension. With relation to assessment of pain in special populations, advances were observed related to assessment of pain in preverbal children or preschool children with special needs.

Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework.

Comparative public health research makes wide use of self-report instruments. For example, research identifying and explaining health disparities across demographic strata may seek to understand the health effects of patient attitudes or private behaviors. Such personal attributes are difficult or impossible to observe directly and are often best measured by self-reports. Defensible use of self-reports in quantitative comparative research requires not only that the measured constructs have the same meaning across groups, but also that group comparisons of sample estimates (eg, means and variances) reflect true group differences and are not contaminated by group-specific attributes that are unrelated to the construct of interest. Evidence for these desirable properties of measurement instruments can be established within the confirmatory factor analysis (CFA) framework; a nested hierarchy of hypotheses is tested that addresses the cross-group invariance of the instrument's psychometric properties. By name, these hypotheses include configural, metric (or pattern), strong (or scalar), and strict factorial invariance. The CFA model and each of these hypotheses are described in nontechnical language. A worked example and technical appendices are included.

Commentary on “Aggressive course of disease in dementia”

Gauthier and colleagues (this issue) provide a valuable service to Alzheimer’s disease (AD) clinicians and investigators by reviewing the sizable literature on rapidly progressive dementia and the variables that contribute to it. The authors are entirely correct in declaring that there is great variability in rates of cognitive and functional decline in AD, but that there are no uniform methods for defining or measuring such decline. They are also correct in identifying several risk factors for rapid decline that emerge relatively consistently from the methodologically diverse studies performed to date: younger age at onset, early presence of psychotic features (hallucinations and delusions), early Parkinsonism, and specific genetic factors. This is an area of investigation fraught with conceptual, methodologic, and data analytic problems [1], many, but not all, of which are acknowledged by the authors. First, diagnostic criteria need to be sufficiently operational and precise to allow easy replication, and research samples should be as diagnostically homogeneous as possible. To discuss factors predictive of rapid course of disease in dementia, rather than in a specific dementia-causing disease, is to invite trouble. In fact, many of the studies described by Gauthier et al. are of patients with Parkinson’s disease (PD) or dementia with Lewy bodies (DLB), rather than AD. Although these disorders often coexist in the same patient, they undoubtedly have unique pathophysiologies and should therefore be kept distinct, especially when discussing genetic and other biological influences on rate of decline and response to pharmacologic treatment. Second, it is not clear whether patients who have an “aggressive course” or are rapid decliners constitute a distinct subtype. This is implied to be the case when one attempts to describe the prevalence of “the condition.” But are patients with an aggressive course categorically different from patients with a more gradual decline? Or is it instead the case that rate of progression is a continuously distributed variable, and that patients described as having an aggressive course are simply at one end of the distribution? There appears to be greater support for this latter position. There is a certain circularity to much of the research in this area. Many of the so-called predictors of rapid decline (eg, low neuropsychological test scores, poor nutrition, patient apathy) may, in fact, be consequences of rapid decline. To conduct a study and conclude that patients who have deteriorated quickly before first being assessed are likely to continue to progress rapidly, or that those who decline rapidly have a poorer prognosis (ie, will reach clinical milestones sooner), seems to be discovering the obvious. The authors point out that there is no uniformity among studies in the choice of outcome variables. Some investigators examine rate of change of scores (typically points per year) on 1 or more cognitive tests or rating scales. The Mini-Mental State Exam (MMSE)— either the original, 30point form or any of a number of expanded modifications— has been used often for this purpose. Other investigators use survival approaches, modeling time to reach critical endpoints (eg, a particular MMSE score, requiring nursing home care, death). Neither of these approaches is inherently superior to the other, and both have limitations. It is impossible to study the natural history of a disease like AD, including its rate of progression, apart from the instruments and metrics used to assess it. Therefore, attention to the psychometric properties of measurement instruments is critical in this area of dementia research. When carefully examined, the MMSE may be a poor choice for the detection of patients with an aggressive course, primarily because of its insensitivity to change relative to its test– retest reliability. The average annual decline in MMSE score among AD patients followed up prospectively in the CERAD study was 3.4 points, only marginally greater than the standard deviation of the measurement (2.8 points) [2]. As a result, even after 4 years of follow-up, approximately 16% of the CERAD AD patients had no clinically meaningful decline (defined as 3 points). Multiple research groups, studying very diverse samples, have concluded that

Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments

BackgroundChoosing an adequate measurement instrument depends on the proposed use of the instrument, the concept to be measured, the measurement properties (e.g. internal consistency, reproducibility, content and construct validity, responsiveness, and interpretability), the requirements, the burden for subjects, and costs of the available instruments. As far as measurement properties are concerned, there are no sufficiently specific standards for the evaluation of measurement properties of instruments to measure health status, and also no explicit criteria for what constitutes good measurement properties. In this paper we describe the protocol for the COSMIN study, the objective of which is to develop a checklist that contains COnsensus-based Standards for the selection of health Measurement INstruments, including explicit criteria for satisfying these standards. We will focus on evaluative health related patient-reported outcomes (HR-PROs), i.e. patient-reported health measurement instruments used in a longitudinal design as an outcome measure, excluding health care related PROs, such as satisfaction with care or adherence. The COSMIN standards will be made available in the form of an easily applicable checklist.MethodAn international Delphi study will be performed to reach consensus on which and how measurement properties should be assessed, and on criteria for good measurement properties. Two sources of input will be used for the Delphi study: (1) a systematic review of properties, standards and criteria of measurement properties found in systematic reviews of measurement instruments, and (2) an additional literature search of methodological articles presenting a comprehensive checklist of standards and criteria. The Delphi study will consist of four (written) Delphi rounds, with approximately 30 expert panel members with different backgrounds in clinical medicine, biostatistics, psychology, and epidemiology. The final checklist will subsequently be field-tested by assessing the inter-rater reproducibility of the checklist.DiscussionSince the study will mainly be anonymous, problems that are commonly encountered in face-to-face group meetings, such as the dominance of certain persons in the communication process, will be avoided. By performing a Delphi study and involving many experts, the likelihood that the checklist will have sufficient credibility to be accepted and implemented will increase.

Measuring Professionalism: A Review of Studies with Instruments Reported in the Literature between 1982 and 2002

To describe the measurement properties of instruments reported in the literature that faculty might use to measure professionalism in medical students and residents. The authors reviewed studies published between 1982 and 2002 that had been located using Medline and four other databases. A national panel of 12 experts in measurement and research in medical education extracted data from research reports using a structured critique form. A total of 134 empirical studies related to the concept of professionalism were identified. The content of 114 involved specific elements of professionalism, such as ethics, humanism, and multiculturalism, or associated phenomena in the educational environment such as abuse and cheating. Few studies addressed professionalism as a comprehensive construct (11 studies) or as a distinct facet of clinical competence (nine studies). The purpose of 109 studies was research or program evaluation, rather than summative or formative assessment. Sixty five used self-administered instruments with no independent observation of the participants' professional behavior. Evidence of reliability was reported in 62 studies. Although content validity was reported in 86 studies, only 34 provided strong evidence. Evidence of concurrent or predictive validity was provided in 43 and 16 studies, respectively. There are few well-documented studies of instruments that can be used to measure professionalism in formative or summative evaluation. When evaluating the tools described in published research it is essential for faculty to look critically for evidence related to the three fundamental measurement properties of content validity, reliability, and practicality.

Événements stressants de la vie : méthodologie et résultats

The literature on scientific research into stressful life events and their relations to disease made its first real debut in 1957 with the studies by Holmes and Rahe. In parallel with changes in the conceptualization of events, methodologies became increasingly refined over time, going from lists of events with standard weightings independent from the perceptions of the subjects (1957) to clinical interview guidelines (1978) aimed at achieving a contextual evaluation of the importance of each event in each individual. The psychometric properties of measurement instruments (validity, reliability) improved in parallel with these changes in methodology. The choices of research strategies (retrospective, prospective) slowly became more discerning, and the need for a control group was finally accepted. Despite the wide variations in methodologies used over time and even across current studies, most studies suggested a chronological relationship between life events and disease. This positive association supports a progression based on the nature of the disease: somatic, psychosomatic, or mental. The credibility of published results depends on the methodologies used to obtain them. In the area of mental diseases, most notably depression, suicide, and schizophrenia, studies have consistently found a relation between the onset of the disease or occurrence of relapses and the presence of stressful life events during the previous months. This is the area in which the effect of life events is easiest to demonstrate, as the differences between patients and controls are large. However, high-quality study designs should provide further refinements in the results. In the areas of somatic and psychosomatic diseases, differences between patients and controls are small, so that complex study designs are needed to detect associations between events and diseases and to unravel the underlying mechanisms.

Measurement Data on Commonly Used Scales to Measure Oral Communication and Writing Apprehensions

Curriculum changes and training advances in business communication have provided students and practitioners with an opportunity to develop and improve communication skills. Despite such changes, research continues to demonstrate that communication apprehension can injuriously impede skills attainment. Yet, the measurement properties of instruments used to measure oral and writing apprehension have received limited attention. In particular, research has not fully explored the impact of question order on the measurement properties of these instruments. This article presents the results of an investigation about the effect of question order randomization on the psychometric properties of two frequently used oral and written apprehension instruments. Results showed that the measurement properties of these instruments were significantly altered when the question order was randomized. ********** It is commonly accepted that business students need effective communication skills to be successful professionals. Consequently, many universities have instituted curriculum and pedagogy changes to provide students with the opportunity to develop those skills. Common changes include opportunities for group work, oral presentations, writing assignments, as well as critical thinking and unstructured problem solving. However, research in communication and psychology demonstrates that apprehension affects skills attainment and performance (Bennett & Rhodes, 1988; Bourhis & Allen, 1992; Daly, 1978; Freimuth, 1976; McCroskey, 1984) In a comprehensive review of communication research in accounting education, Stout and Rebele (1996) stressed the need to evaluate the effectiveness of teaching methods in developing student skills. Since the presence of apprehension may limit skills development, empirical research on communication skills attainment cannot be thoroughly investigated without understanding the effect of communication apprehension. As a result, the instruments used to measure communication apprehension are important to business communication professionals and researchers. The level of oral and written communication apprehension has been investigated extensively in many disciplines. However, investigations into the measurement (psychometric) properties of these instruments are limited. The versions of the instruments used in these studies have generally mirrored the instruments originally developed by McCroskey (1984) and Daly and Miller (1975). Analysis of different versions of survey instruments is important because it allows researchers to assess the degree to which the measurement properties are an artifact of survey design. This is particularly true for the oral communication apprehension scale where the original survey items are grouped by theoretical construct. It has been observed that the lack of survey item randomization can result in spurious relations among constructs (Budd, 1987). The purpose of this article is to provide measurement data on versions of McCroskey's (1984) oral communication apprehension scale and Daly and Miller's (1975) writing apprehension scale where the order of the items within each scale has been randomized. The primary means of analysis, factor analysis and internal consistency estimates, are based on correlations among individual data items. If the correlations among individual data items do not significantly change after reordering, the results of the analysis will be consistent with previous studies regarding the number of factors and the level of Cronbach's alpha. Greater knowledge of each scale's measurement properties will enable business communication professionals and researchers to determine if these instruments need to be further modified before using. Such an understanding will also enhance researchers' confidence in the results of studies that use these scales. The next section provides relevant background about research related to apprehension. …

Measuring trauma and health status in refugees: a critical review.

Refugees experience multiple traumatic events and have significant associated health problems, but data about refugee trauma and health status are often conflicting and difficult to interpret. To assess the characteristics of the literature on refugee trauma and health, to identify and evaluate instruments used to measure refugee trauma and health status, and to recommend improvements. MEDLINE, PsychInfo, Health and PsychoSocial Instruments, CINAHL, and Cochrane Systematic Reviews (searched through OVID from the inception of each database to October 2001), and the New Mexico Refugee Project database. Key terms and combination operators were applied to identify English-language publications evaluating measurement of refugee trauma and/or health status. Information extracted for each article included author; year of publication; primary focus; type (empirical, review, or descriptive); and type/name and properties of instrument(s) included. Articles were excluded from further analyses if they were review or descriptive, were not primarily about refugee health status or trauma, or were only about infectious diseases. Instruments were then evaluated according to 5 criteria (purpose, construct definition, design, developmental process, reliability and validity) as described in the published literature. Of 394 publications identified, 183 were included for further analyses of their characteristics; 91 (49.7%) included quantitative data but did not evaluate measurement properties of instruments used in refugee research, 78 (42.6%) reported on statistical relationships between measures (presuming validity), and 14 (7.7%) were only about statistical properties of instruments. In these 183 publications, 125 different instruments were used; of these, 12 were developed in refugee research. None of these instruments fully met all 5 evaluation criteria, 3 met 4 criteria, and 5 met only 1 of the criteria. Another 8 standard instruments were designed and developed in nonrefugee populations but adapted for use in refugee research; of these, 2 met all 5 criteria and 6 met 4 criteria. The majority of articles about refugee trauma or health are either descriptive or include quantitative data from instruments that have limited or untested validity and reliability in refugees. Primary limitations to accurate measurement in refugee research are the lack of theoretical bases to instruments and inattention to using and reporting sound measurement principles.

Exposure measurement in cohort studies: the challenges of prospective data collection.

Cohort study designs have several advantages over case-control studies in terms of exposure measurement. If exposure measurement occurs before disease occurrence, cohort studies are much less prone to differential measurement error. Prospective data collection should also reduce measurement error due to poor recall of past exposures. The primary drawback of cohort studies is the large sample size leading to high data collection costs. Several approaches to reduce such costs have been discussed in this presentation, such as selection of lower cost measurement methods and fully measuring the exposure only on a subsample of the cohort (e.g., nested case-control design). However, other innovative approaches to reduce costs are needed. In addition, study reviewers should also consider that the higher costs are justified in relation to the several benefits of this study design, which include not only less measurement error, but also less susceptibility to selection bias and often the ability to study multiple disease outcomes. Improving the accuracy of exposure measurement is increasingly important for cohort studies as we move on to the study of exposures that are difficult to measure or to those with lower relative risks of disease. In such studies, attenuation of the relative risk by the effects of measurement error can lead to failure to detect an association between exposure and disease. The validity of exposure measurements could be improved by a better understanding of the biologically active agent and etiologically important time period of the exposure-disease relation, and by incorporating these into the measure. Long-term cohort studies which cover the etiologically relevant time period could improve the accuracy of measures of exposures by use of repeated biologic measures or repeated updates of self-reported exposures. Measurement error also can be reduced by judicious choice of a cohort to study and by careful attention to quality control procedures. Continued emphasis on the evaluation and improvement of the measurement properties of instruments used in epidemiologic studies will improve the validity of the results of cohort studies.