Beware of Big Gifts in Small Studies

Abstract

In the current issue of JPEN, Tiengou and colleagues present a small randomized study comparing a standard isotonic tube-feeding formula vs a predigested formula for jejunal feeding of acute pancreatitis. Accepting a current mantra that many patients with acute pancreatitis should be fed jejunally, the authors sought to answer the question, what type of formula should be infused into the jejunal tube, a predigested (semi-elemental) formula or an intact formula requiring digestion (polymeric)? Reporting on 30 patients who successfully completed the study, the authors suggest that, while both formulas were well and equally tolerated, the predigested formula ensured a more favorable clinical course, as evidenced by a shorter length of hospital stay and less weight loss. How robust are these conclusions, and should we now feed every pancreatitic patient a jejunal predigested formula? I would like to approach these questions with a 3-pronged focus. How does consideration of study methodology contribute to our understanding of the strength of the conclusions? What are some questions that arise from the study? Finally, is the question asked even appropriate; where are we with respect to certainty that jejunal feeding is appropriate for acute pancreatitis? Why does study quality matter? Is not a randomized controlled trial sufficient to draw conclusions? Quality matters because it has been demonstrated by those who ponder study quality that lower-quality studies tend to demonstrate more positive outcomes; differences between approaches are less likely to be pronounced in high-quality studies. What makes a study “high quality?” A variety of measures have been proposed. A “Jadad score” was developed to assess bias in clinical trials and is often used to assess study quality. This system asks 3 questions: (1) was the study described as randomized; (2) was the study described as double blinded; and (3) are all participants in the trial included in the final analysis? Additional points are awarded in questions 1 and 2 with respect to appropriateness of randomization and blinding. High-quality studies have a maximum score of 5, and low-quality studies score 1 or 2. How would the present study score? Randomization appears to have occurred and to have used appropriate methodology (stratification for illness severity, centralization by an outside source, using sealed envelopes). Score 2 points. The study was said to be single blinded, but this presumably was the patient who did not know which formula he or she received. Although the authors suggest that the “clinician who decided resumption of oral feeding and discharge was not the doctor responsible for the nutrition prescription,” this would imply that the feeding formula was known to clinic staff and that unintended bias could have been introduced into the evaluation. No points given. Finally, of 36 patients entered into the protocol, full data are available on only 30. The authors give data on length of stay, including all 36 as an afterthought, but full information is not available. Give 0 points for full follow-up, resulting in a final score of 2, low quality. Over the years, I have developed my own quality checklist which I teach our gastroenterology fellows and medical residents, the list being a compilation of reading points acquired or lifted from various colleagues and the literature. Starting with a randomized controlled clinical trial, the first question that I ask of the study is what is the question that the study asks? In order to answer a question, a question, or more specifically, a hypothesis needs to be formulated. Even more specifically, the hypothesis needs to be measurable and defined. The hypothesis cannot be a fishing expedition: what might we find if we administer a predigested formula vs a standard diet; the hypothesis must state what single result do we expect: length of stay will be shortened by 3 days, death rate will decrease from 30%–10%, infectious complications (defined) will decrease from 20%–10%, etc. Why do I look for this? Because the study’s primary statistics depend on it. Because with this estimate of an effect size, a power calculation can be performed to determine a sample size: how many subjects are needed in each group to “prove” to a reasonable extent the hypothesis. So effect size, power calculation, sample size determination—all important for good study quality. If these are present, a statistical difference is most likely meaningful. If not, the difference could have occurred by Received for publication September 28, 2005. Accepted for publication October 11, 2005. Correspondence: Timothy O. Lipman, MD, Chief, GI-HepatologyNutrition Section, Department of Veterans Affairs Medical Center, 50 Irving St NW, Washington, DC 20422. Electronic mail may be sent to timothy.lipman@med.va.gov. 0148-6071/06/3001-0066$03.00/0 Vol. 30, No. 1 JOURNAL OF PARENTERAL AND ENTERAL NUTRITION Printed in U.S.A. Copyright © 2006 by the American Society for Parenteral and Enteral Nutrition