Issues Of Measurement Error Research Articles

Affect and the Functional Bases of Behavior: On the Dimensional Structure of Affective Experience

Discussions of the dimensional structure of affect usually are based on results of factor analyses. Disagreements focus largely on issues of measurement and measurement error. I argue that the structure of affect is not discussed meaningfully without considering functional implications of affects. A functional analysis is outlined in which approach and incentive-related affects (both positive and negative) are managed by 1 self-regulatory system, and avoidance and threat-related affects (positive and negative) are managed by another self-regulatory system. In both cases, positive and negative affects are posited to convey information about whether the behavior being engaged in is going well or poorly. This view argues for the existence of 2 bipolar affective dimensions, the properties of which are informed by an understanding of both behavior and feelings.

RESPONSE: HOW BIG IS BIG? ONLY TIME WILL TELL

Wood and Welsch (7) purport that the 59.9 kcal·d-1 mean difference we observed between days of continuous and intermittent walking was not meaningful because of unaccounted errors of measurement with the TRITRAC-R3D accelerometer. We agree that one should consider measurement errors when interpreting mean differences. We do not agree with their interpretation of our findings. Two empirical examples render moot the issue of measurement error in interpreting our findings. Using a sample similar to that studied by Fulton et al. (2), Mâsse et al. (5) estimated the inter-accelerometer agreement of the TRITRAC-R3D to be 0.99. We used this estimate to compute the standard error of measurement (SEM) as follows (1) : MATHwhere rxx’ = reliability of the accelerometer, 0.99, and ςx = standard deviation of the difference, 123.9 kcal·d-1 (not provided in the original paper). Using the SEM, we found that the true value of the observed mean difference lies between 35.6 and 84.2 kcal·d-1 with 95% confidence. Wood and Welsch (7) suggest using an estimate of accelerometer reliability of 0.95 from Nichols et al. (6). In doing so, the SEM was 27.7 kcal·d-1, and the true value lies between 5.6 and 115.2 kcal·d-1 with 95% confidence. Hence, either reliability estimate shows that the true value for the mean difference is not zero. Wood and Welsch suggest that the lower average daily MET value for the intermittent condition (4.05) relative to the continuous condition (4.13) accounts for our findings, estimating a difference of 2.6% (0.08 METs). We calculated a statistically nonsignificant difference (P = 0.57, p. 165) of 2.0% (0.08/4.05 = 1.98) as was noted also for the difference in duration (Fig. 2). We agree that activity intensity is an important aspect of energy expenditure to consider, as are frequency and duration, and we concede that “marginally lower” was generous to describe the smaller MET value. One must gain perspective to establish how meaningful is the difference we observed. Our study had ample statistical power to detect a 10% difference between the two walking conditions, amounting to 15 kcal·d-1 (e.g., 1.5 miles·100 kcal·mile-1·10%). Our finding was four times greater relative to this prescribed amount. However, prescribed and nonprescribed activity contributed to total daily energy expenditure (23% and 77%, respectively), revealing the need to compare the mean difference relative to the daily total. Wood and Welsch (7) calculate this difference to be 2.7% (presumably for the 2180 kcal·d-1 continuous walking condition). They question whether such a small percentage difference is meaningful even though we suggested it may thwart a weight gain of 4.5 pound·yr-1. Others have suggested that even smaller amounts of daily energy expenditure could also be very important (3,4,9). Thus, our findings have relevance for the worldwide obesity epidemic (8). Contrary to Wood and Welsch, we do not find that the difference of 59.9 kcal·d-1 between the intermittent and continuous conditions arises from either measurement error or differences in reported intensity. Whether the difference in estimated daily energy expenditure is reproducible and biologically important for public health must be resolved in future empirical investigations. Only time will tell. Janet E. Fulton, Ph.D. Louise C. Mâsse, Ph.D. Carl J. Caspersen, Ph.D., M.P.H.

A Developmental Trajectory in Implicit Memory is Revealed by Picture Fragment Completion

Dissociations between performance on implicit and explicit tasks have often been taken as evidence that different neural structures subserve the two types of memory. One such dissociation involves developmental differences that emerge in explicit tasks, but which appear to be absent in implicit tasks. Such findings are consistent with the idea that implicit memory is subserved by a more primitive system that evolves earlier at both phylogenetic and ontogenetic levels. The present paper reviews previous studies that claimed to find evidence that implicit memory is fully developed in very young children. Issues of measurement error, ceiling effects, and insufficient power brought up questions about those studies with respect to the developmental issue. The present study compares performance on implicit (picture fragment completion) and explicit (free recall and recognition) memory tasks with groups ranging in age from 5–28 years. We find a reliable developmental trend in both implicit and explicit performance in which the former cannot be attributed to the operation ofexplicit memory processes. Thus, we conclude that implicit memory, like explicit memory, develops with age.

Unified BHPS Work-Life Histories: Combining Multiple Sources into a User-Friendly Format

Longitudinal data is often difficult to use, and continuous histories collected in a panel are a particularly unfriendly case. This paper reports an exercise to re-organise the British Household Panel Study's work-life history data into a format more convenient for analysis. The British Household Panel Study collects extensive labour market history information from its respondents, both during the panel period and retrospectively from labour market entry. That this information is of necessity stored in multiple locations, and of varying levels of detail, has made use somewhat inconvenient. This paper describes an exercise to bring the labour market information together in a more convenient format. It also considers some of the problems of retrospective and panel longitudinal data, and discusses issues of recall error and measurement error. The data files described are available through the UK Data Archive. Longitudinal Data, Work-Life Histories, Recall Bias, Measurement Error.

The Estimation of Reliability in Longitudinal Models

Despite the increasing attention devoted to the study and analysis of longitudinal data, relatively little consideration has been directed toward understanding the issues of reliability and measurement error. Perhaps one reason for this neglect has been that traditional methods of estimation (e.g. generalisability theory) require assumptions that are often not tenable in longitudinal designs. This paper first examines applications of generalisability theory to the estimation of m easurement error and reliability in longitudinal research, and notes how factors such as missing data, correlated errors, and true score instability prohibit traditional variance com ponent estimation. Next, we discuss how estimation methods using restricted maximum likelihood can account for these factors, thereby providing m any advantages over traditional estimation methods. Finally, we provide a substantive exam ple illustrating these advantages, and include brief discussions of programming and software considerations.

Measuring outcomes of behavioural interventions for STD/HIV prevention.

Sexually transmitted disease (STD)/HIV specialists present a structured overview of the major challenges facing the behavioral intervention field today especially as they apply to STD/HIV prevention. Outcome measures are essential for intervention research program evaluation and surveillance. The implications of using a particular outcome measure for the conclusions and interpretation to be deduced from the findings may differ depending on whether it is used in the context of intervention research program evaluation or surveillance. Outcomes may be measured at the individual or population level. The analysis of the various aspects revolving around the selection definition and empirical specification of outcome measures associated with STD/HIV prevention-related behavioral interventions has resulted in three conclusions concerning the choice of outcome measures in STD/HIV prevention. The highest priority is to identify and define conceptually and operationally the outcome of interest. The researchers needs to consider carefully the implications of context (surveillance program evaluation and intervention research) and level of measurement (individual vs. population). The proper choice of the outcome of interest is more important than issues of measurement error. The next conclusion is that researchers should focus carefully on the issue of the use of surrogate measures (e.g. behavioral outcomes as surrogates for biomedical outcomes of interest). They should clearly state all assumptions made in choosing a particular surrogate outcome. The last conclusion is that consideration of context is very important when choosing outcome measures. Each context (surveillance vs. program evaluation vs. intervention research) implies a distinct set of strengths and weaknesses for each possible outcome of interest.

Dynamic magnetic resonance imaging in the study of vocal tract configuration.

Studies of vocal tract configuration using magnetic resonance imaging (MRI) techniques have relied on static images. These images fail to identify transient movements and are subject to distortion from motion artifact limiting research application to stable motor events. This paper describes a dynamic MRI technique that permits study of transient movements within the vocal tract during speech and nonspeech tasks. Following description of the technique, results of two preliminary studies are presented. The initial study evaluated issues of measurement error and reliability. Results indicated that distance and area measurements obtained from this technique are accurate in reference to a calibration referent and reliable both within and among judges. The second study compared two aspects of vocal tract configuration in patients with adductor spasmodic dysphonia before and following treatment with Botox injection. Changes in vocal tract configuration are discussed in reference to prior observations of patients with spasmodic dysphonia. These preliminary investigations suggest that dynamic MRI has promise as a useful technique in the study of vocal tract configuration.

Measurement error issues in substance abuse services research: Lessons from structural equation modeling and psychometric theory

Evaluating the effectiveness of treatment for long-term chronic conditions requires more sensitive measures of quality of life and daily functioning than those treatments for acute care. Unfortunately, most of these measures require some degree of subjective reports and are affected by a number of systematic and unsystematic measurement errors that typically increase the probability of incorrectly declaring the treatment ineffective or indicative of a Type II error. These include method effects, such as social desirability or acquiescence, and structural errors that occur when the items reliably measure something other than what they were intended to measure. This paper explores some potential methods for removing or otherwise controlling random measurement error, assessment artifacts, irrelevant variation in outcome measures, and confounding sources of covariation in a structural equations paradigm. Using examples with some popular measures of quality of life and functioning, the paper also looks at the appropriateness of their use in observational, quasi-, and randomized field experiments.

Necessary and sufficient conditions for a Mueller matrix to be derivable from a Jones matrix

Necessary and sufficient conditions for an optical system characterized by a Mueller matrix to be characterized by a Mueller–Jones, and thence by a Jones, matrix are considered, and the issue of measurement error is examined. It is shown that a Mueller matrix can be expressed as a linear combination of at most four trace orthonormal Mueller–Jones matrices, and an algorithm for the construction is given.

Cross Validation of the Hypnotic Induction Profile Factor Structure: Simultaneous Confirmatory Factor Analysis of Two Published Samples

This study cross-validated factor structure and psychometric characteristics of the Hypnotic Induction Profile (HIP). Earlier studies identified two constructs of the HIP, but their methodology did not allow empirical confirmation of the number of factors. I have used a new methodology, confirmatory factor analysis, and submitted the data from the previous studies to a simultaneous, multisample, factor analysis. The results showed that the HIP indeed consists of two factors. Although the HIP two-factor model is statistically adequate, 7 of the 10 scales have very low item reliability. Issues of measurement error and clinical implications of the study are also discussed in relation to the current findings.

Testing for measurement errors in expectations from survey data: An instrumental variables approach

This paper examines the issue of measurement errors in expectations from survey data in the testing of models of expectations formation. We demonstrate that tests for measurement errors can be conducted through a generalized method of moments specification test and apply our estimation strategy to the SRC expected inflation series.

Positional accuracy and measurement error in digital databases of land use: an empirical study

This paper discusses the issues of positional accuracy and measurement error in the context of a large empirical study of landscape change in England and Wales. The epsilon band model of digitizing accuracy is used to make estimates of the levels of positional uncertainty and measurement error that is due to digitizing polygon outlines. The degree of error expected had the same polygons been captured in raster format is then determined. These results prompt a general discussion of the nature of error in spatial databases.

Formation of Inflation Expectations in British Manufacturing Industries

Over the past decade increasing use has been made of direct information on price expectations in empirical studies of the expectations formation process. In the United States, Livingston's inflation expectation series have been extensively analysed by Turnovsky (I970), Turnovsky and Wachter (I972), Pesando (I975), McGuire (I976), Carlson (I977), Mullineaux (I978, I980), Jacobs and Jones (I980), Figlewski and Wachter (I98I) and others. In the United Kingdom, the general public expectations of consumer prices made available in the Gallup polls have been studied by Carlson and Parkin (I975), Holden and Peel (I977), Smith (I982), Severn (1983) and Evans and Gulamani (I984). Similar studies have also been carried out with respect to surveys conducted in Germany, France and most other OECD countries. However, despite the apparent importance of businessmen's expectations for the understanding of the inflationary process, the empirical research on inflation expectations in the United Kingdom has been mainly concerned with the consumers' price expectations.' The present paper is an attempt to rectify this shortcoming by extending the empirical analysis of the inflation expectation process to British manufacturing industries using the results of the industrial trend surveys conducted by the Confederation of British Industries (CBI). The plan of the paper is as follows: Section I deals with the methodological problems of deriving quantitative inflation expectations series from qualitative responses such as the ones reported in the CBI surveys. The subjective probability method due to Theil, Carlson and Parkin, and the regression method recently proposed by the author (Pesaran, I984) for the conversion of categorical responses to quantitative measures are briefly discussed. Section II sets out the alternative models of inflation expectations formation. Section III describes the data on inflation expectations in British manufacturing industries, and discusses the forecasting value of alternative measures of inflation expectations. Section IV focuses on the issue of measurement errors in expectations and the econometric problems involved. Section V reports the results of estimation of alternative models of inflation expectations, and investigates the quantitative impact of government policy and general economic conditions on inflation

Aggregation, ratio variables, and measurement problems in criminological research

Issues of measurement error, level of aggregation, and ratio variables have been considered serious problems in criminological research. Although there have been many recent discussions of these issues in sociology and criminology, studies designed to assess the impact of these problems on the results of empirical research have, for the most part, been absent. After reviewing what is known theoretically and conceptually about these issues, an investigation which compares empirical analyses of a particular type of crime, homicide, that use different measurement strategies, different levels of aggregation, and ratio versus nonratio variables is presented. Utilizing homicide data from the mid-1970s and selected independent variables, the results of this investigation indicate that these three problems can interact in an empirical setting such that potential solutions to these problems do not always apply in the manner suggested in previous studies. The results also indicate that there is great risk in ignoring one or more of these problems in empirical research, in that different substantive conclusions can be reached from analyses that ignore these issues compared with analyses that deal directly with them.

Media public affairs exposure: Issues and alternative strategies

This study examined the reliability and stability of newspaper and television public affairs exposure; the relationship of social structural variables and media exposure, when measurement error is taken into account; and the effect of these social structural variables on change in exposure. A secondary analysis was performed on a two wave national study composed of 7201 respondents. Using a LISREL model, the results indicated that: (1) the indicators of newspaper public affairs exposure were more reliable than the indicators for television public affairs exposure; conversely, the television public affairs construct was more stable than the newspaper construct, although both yielded fairly high stability estimates; (2) various indicators of location in the social structure (age, education, income, perceived social class, and sex) had effects on exposure to newspaper and television public affairs information, and effects on the change in this type of exposure over time. It was concluded that more attention should be directed toward measurement error issues and toward operationally and theoretically defining media exposure.