Valid Paired Data Designs: Make Full Use of the Data Without “Double-Dipping”

Abstract

“Analysis of Paired Data in Physical Therapy Research: Time to Stop Double-Dipping?” was a guest editorial that appeared in the August 2005 issue of the JOSPT. This editorial, written by Dr Hylton Menz, raised an important statistical issue about how to properly manage study designs that involve paired data. “Double-dipping” refers to using an inappropriate statistical analysis for paired data. For example, if a study used 12 subjects and measured both feet of each subject, it would be inappropriate to calculate a standard error that is based on an n of 24 (2 × 12) independent feet. Double-dipping is a problem because the paired limbs within a subject are not likely to be statistically independent. This practice can result in a standard error that is smaller than it should be. Based on a spuriously small standard error and an inflated n, a statistical test may appear to be more "significant" than it actually is. Dr Menz concluded his editorial by recommending several strategies to select only 1 of a paired limb per patient for measurement to maintain the statistical independence of the data. For example, we could randomly select and measure only 1 foot per subject, and then use an n of 12 (1 × 12) independent measurements in the statistical analysis. However, would we lose valuable information in doing so? In his efforts to persuade readers to avoid the error of double-dipping, Dr Menz may have inadvertently discouraged the use of valid study designs and appropriate analysis methods for paired data. My purpose in writing this guest editorial is to discuss these designs and encourage investigators to consider all of the options available to them in planning their studies. This editorial discusses 3 situations where an appropriate design and analysis for paired data may be advantageous. I will use feet and ankles in my illustrations of paired data, but these comments also extend to other pairs of limbs and organs that may be the focus of research in physical therapy. J Orthop Sports Phys Ther 2006; 36(2):42–44. doi:10.2519/jospt.2006.0102