The cumulative sum (CUSUM) control chart is a method for detecting whether the mean of a time series process has shifted beyond some tolerance (ie, is out of control). Originally developed in an industrial process control setting, the CUSUM statistic is typically reset to zero once a process is discovered to be out of control since the industrial process is then recalibrated to be in control. The CUSUM method is also used to detect disease outbreaks in prospective disease surveillance, with a disease outbreak coinciding with an out-of-control process. In a disease surveillance setting, resetting the CUSUM statistic is unrealistic, and a nonrestarting CUSUM chart is used instead. In practice, the nonrestarting CUSUM provides more information but suffers from a high false alarm rate following the end of an outbreak. In this paper, we propose a modified hypothesis test for use with the nonrestarting CUSUM when testing whether a process is out of control. By simulating statistics conditional on the presence of an out-of-control process in recent time periods, we are able to retain the CUSUM's power to detect an out-of-control process while controlling the post-out-of-control false alarm rate at the desired level. We demonstrate this method using data on a Salmonella Newport outbreak that occurred in Germany in 2011. We find that in 7 out of 8 states where the outbreak was detected, the outbreak was detected at the same speed as an unmodified nonrestarting CUSUM while controlling the postoutbreak rate of false alarms at the desired level.