If You Don’t Know Where You’re Starting from, You Can’t Get Directions

Abstract

Commentary I can feel the excitement of these authors when they started designing their study in the early 1990s: We had all been told that randomized controlled trials (RCTs) were the answer, and they were going to contribute with a well-designed study. Roll on twenty years and I can feel their disappointment. Fortunately, their hard work has been rewarded with publication in JBJS. However, the results they report are no more illuminating than those in previous papers—with words like “underpowered” being used to diminish the importance and impact of the study. This is unfair to the authors and unfair to the orthopaedic community. These RCT models arose from drug trials and may not be the best model in orthopaedics. The concept of a clinically significant difference might make some sense in a placebo-controlled study on a blood pressure medication, although I would contend that blood pressure is not the true outcome in that case—the true outcome is mortality reduction. Fortunately, orthopaedic outcomes research has been geared toward true outcomes and not been accepting of surrogate measures such as radiographic measurements. So, if power analysis is of little use for drug studies and no use for clinical studies, the slavish application of power analysis is wrong. An effects size table may be a better way to determine the proper sample size. I believe that the authors consequently followed the correct approach by not expanding the study. The study was further limited because the validity and responsiveness of some of the outcome instruments have not been documented. Loss of patients due to exclusion criteria further frustrates enrollment; patients are not clones, and many trauma patients have other diseases or life factors that make them hard to recruit. Finally, trauma studies lack sensitivity to measure change, as a preoperative score cannot be obtained. Outcomes in orthopaedics also require a long time to obtain. In the drug studies that formed the model for orthopaedic statistical methods, the outcomes were measured over days or weeks, making patients easy to trace. Outcomes in orthopaedics require years to measure and obtain, and these studies are at real risk of investigator and participant fatigue. The investigators may retire or die before the study is completed. The need to publish in order to attain promotion may be long past by the end of this time period. The dedication and perseverance of authors such as those in this group must be recognized and applauded. What struck me as I read through this paper was that so many of the outcome measures outlined pointed in favor of surgical treatment. At the one-year follow-up, three of four primary outcomes and three of four secondary outcomes favored surgery. At the latest follow-up, all four primary outcomes and three of four secondary outcomes favored surgery, with the last measure being a draw. Furthermore, surgical procedures and the resulting outcome scores have improved since the study was started. I believe that the results therefore support operative care of calcaneal fractures. I will continue to offer surgical treatment to patients, provided that my assessment of the risks and benefits for that patient favors surgery. I would encourage readers of JBJS to do the same in the best interests of patient care, and to listen to their patients after surgical and nonsurgical treatment to add the art of medicine to the science offered in these articles until the science improves. The authors also observed a higher complication rate with surgical treatment. For the future, we need to improve the results of surgery and reduce the complication rate. Reducing the invasiveness of surgery and reducing wound complications would go a long way toward this goal. Improvement of the quality of reduction would also help in improving outcomes. We also need some really smart PhD statisticians in outcomes research to redefine orthopaedic outcomes research from the ground up so that future authors do not dedicate the time and effort to this type of outcomes research and end up with equivocal results. Authors considering embarking on similar studies should seek lots of high-quality statistical advice. This will ensure that study design and outcome measurements are optimal to allow the conclusions to be reached correctly twenty years from now and avoid a type-II error—concluding that there is no difference between the groups when there really is.