Some Comments on the Relation Between Reliability and Statistical Power

Abstract

Several articles have discussed the curious fact that a difference score with zero reliability can nonetheless allow a powerful test of change. This statistical legerde main should not be overemphasized for three reasons. First, although the reliability of the difference score may be unrelated to power, the reliabilities of the var iables used to create the difference scores are directly related to the power of the test. Second, with what some will regard as additional legerdemain, it is possi ble to define reliability in the context of a difference score in such a way that power is a direct function of reliability. The third and most serious objection to the conclusion that the reliability of a difference score is unimportant is that the underlying statistical model used in its derivation is rarely appropriate for psycho logical data. Index terms: control of individual dif ferences, difference scores, reliability, reliability of the marginal distribution, statistical power, within- group reliabilities.