The papers by Subar et al. (1) and Kipnis et al. (2) in this issue of the Journal provide a unique set of data and a useful contribution to nutritional epidemiology. The Observing Protein and Energy Nutrition (OPEN) Study they report on was based on a reasonable sample of participants, had a remarkably high completion rate, and is the largest known study to use doubly labeled water as the standard for comparison. In addition, the measurements of doubly labeled water, which can be highly error prone (3), appear to have been made with a high degree of technical precision. The study also has several limitations inherent to using this approach, including that it enabled evaluation of only total energy and one specific nutrient, protein. Furthermore, the results cannot be readily generalized to other food frequency questionnaires because the values for total energy intake estimated by use of their questionnaire were substantially lower than typically provided by other full-length questionnaires in current use. In addition, although great expense was devoted to biomarker measurements, only 2 days of 24-hour diet recalls were used, many fewer days of assessment than are included in usual questionnaire validation studies. A more serious flaw in the design of the OPEN Study (1, 2) is failure to include a realistic measure of the withinperson variability of their standard, doubly labeled water plus urinary nitrogen. This measure should have been obtained for a sample of participants by collecting a replicate measure a number of months apart, say roughly 6 to 12, which would be consistent with the 1-year time frame of their questionnaire. In their study, the only measure of reproducibility was for 25 participants, without any interval between the replicates; the authors report a coefficient of variation of 5.1 percent total energy expenditure, suggesting little within-person variation. However, this is likely to underestimate biologic variability substantially in relation to the time frame of the dietary questionnaire. Increasing variability given a greater time interval would be expected, for example, because persons are likely to change their physical activity with time because of differences in seasons and other reasons (the fact that the authors note no average seasonal variation in total energy expenditure is not relevant to individual variability). This phenomenon is well documented in a quantitative review of the reproducibility of total energy expenditure measured by using doubly labeled water in 25 studies with repeated measurements (4). In this report, Black and Cole found that, overall, the within-person coefficient of variation was of similar magnitude to the betweenperson variation whether or not the studies involved an intervention, and it clearly increased with increasing interval between repeated measurements. They estimated that the within-person coefficient of variation was approximately twice as great at an interval of 1 year compared with having no interval between replicates. The lack of a realistic measure of variability in their standard method (1, 2), which is surprising for a study focusing on error, seriously limits the value of the findings and is likely to have led to misleading conclusions. As one example, Subar et al. (1) emphasize the importance of “intake-related bias” associated with their dietary questionnaire or 24-hour recalls, meaning that the ratio of intake measures divided by biomarker measures is inversely correlated with the biomarker measures. Because any unaccounted-for variation (random within-person error) in the biomarker would appear in the denominator of the ratio and the numerator of the biomarker variable, the inverse correlation could just be all or part artifact.
Read full abstract