When can we trust responders? Serious concerns when using 50% response rate to assess clinical trials.

Abstract

Individual seizure rates are highly volatile, with large fluctuations from month-to-month. Nevertheless, changes in individual mean seizure rates are used to measure whether or not trial participants successfully respond to treatment. This study aims to quantify the challenges in identifying individual treatment responders in epilepsy. A power calculation was performed to determine the trial duration required to detect a significant 50% decrease in seizure rates (P<.05) for individuals. Seizure rate simulations were also performed to determine the number of people who would appear to be 50% responders by chance. Seizure rate statistics were derived from long-term seizure counts recorded during a previous clinical trial for an implantable seizure monitoring device. We showed that individual variance in monthly seizure rates can lead to an unacceptably high false-positive rate in the detection of individual treatment responders. This error rate cannot be reduced by increasing the trial population; however, it can be reduced by increasing the duration of clinical trials. This finding suggests that some drugs may be incorrectly evaluated as effective; or, conversely, that helpful drugs could be rejected based on 50% response rates. It is important to pursue more nuanced approaches to measuring individual treatment response, which consider the patient-specific distributions of seizure rates.