WHEN STUDY DESIGNS CONTROL FOR REGRESSION TO THE MEAN

Abstract

Dear Editor-in-Chief: In a recent article, we demonstrated that baseline level of function is an important variable to consider when studying the relationship between exercise therapy and subsequent improvement in physical function (3). Dr. Shephard argues that the results remain suspect due to the possible confounding influence of regression to the mean (RM). He stated that the apparent effect of initial physical function should have been evaluated relative to either a zero-treatment control group or that potential effects for RM should have been estimated using standard statistical techniques. We appreciate the opportunity to clarify this point for Dr. Shephard and others who may have had a similar concern. First, simply having a zero-order control group would not control for RM unless participants had been randomized to all treatment conditions (2). There are numerous examples in the literature of quasi-experimental designs that include a zero-treatment control condition and falsely conclude that the results are not confounded by threats to internal validity such as RM (1). The key to controlling for threats to internal validity such as RM is the existence of multiple treatment groups where participants are randomized to conditions. As stated by Campbell and Kenny (1) (p. 51): “Randomized experiments have much to recommend them because they eliminate RM as a plausible rival hypothesis.” In our study, we employed a four-group randomized controlled clinical trial that was stratified by gender. Thus, we were able to make sound conclusions about the effect of baseline physical function on changes in function because the interpretations were made relative to other randomized treatment conditions where this effect was not observed. Parenthetically, the use of a zero-order control comparison in our design as opposed to a standard of care that is usually positive would have been careless and misleading on our part. With respect to Dr. Shephard’s second option—statistical adjustment—we are uncertain as to what he had in mind. First, we would point out that the common approach to controlling for baseline physical function scores on change in physical function over time in a statistical model is to covary baseline values. A careful read of our statistical analysis will reveal that, in fact, this is what we did. It was the significant baseline by treatment interaction that led us to our interpretation. Thus, we employed very rigorous methods both in design and analysis that support the integrity of our original conclusion. As for estimating RM in a nonexperimental design, we would point the reader to Campbell and Kenny’s primer on regression artifacts (1). A discussion of this topic is far beyond the scope of a letter to the editor. However, it is important to remember that the potential bias caused by RM increases as reliability of measurement decreases, where true-score estimation is computed as follows (one of the most important formulas in psychometrics for understanding biases due to RM): X'T = rX(X - MX) + MX, where X'T is the predicted true score, rX is the reliability of X, and MX is the mean of X (1). W. Jack Rejeski, Ph.D. Lawrence R. Brawley, Ph.D. James L. Norris, Ph.D.