The beta error and sample size determination in clinical trials in emergency medicine

Abstract

In the analysis of a clinical trial an investigator may fail to discern a statistically significant difference in outcome between control and experimental groups, when in fact one exists. Failure to demonstrate such a difference when it actually exists is known as "type II" error, and its probability of occurring is termed "beta." The purpose of our study was to determine the distribution of beta errors in negative trials in the Journal of the American College of Emergency Physicians (JACEP) (1972-1979) and Annals of Emergency Medicine (1980-1984). All negative comparative clinical trials appearing in JACEP and Annals from volume 1 (1972) to volume 13 (1984) were surveyed and were eligible for inclusion in the study. A trial was defined as negative if the investigator specifically stated that there was no significant difference in outcome between the experimental and control groups. For each negative trial the following parameters were calculated: beta error, based on the sample size used and the difference determined to be important to detect clinically; sample size required to detect a clinically meaningful difference as determined by the authors of this study; and minimum true difference that had to be detected in the trial at a beta equal to 0.20, to discern a statistically significant result. For the 13 years surveyed, we found 21 endpoints in 14 negative trials that were analyzable. Only one of the trials (7.1%) addressed the issues of beta errors and sample size determination. In the remaining 13 negative trials, the calculated beta error ranged from .60 to .97. For the endpoints analyzed, a sample size of up to 450 times larger than that used would have been required to detect a clinically important difference.(ABSTRACT TRUNCATED AT 250 WORDS)