Automatic Recovery System in the Belle II Operation

Abstract

The Belle II experiment is designed to search for physics beyond the standard model of particle physics exploiting the large number of B meson decays produced by the SuperKEKB accelerator. We maintain a good data-taking efficiency since the beginning of our operation on March 2019. Nevertheless, we encounter various problems and errors during our operation. The instantaneous luminosity of SuperKEKB is gradually increasing; in this phase, we occasionally suffer from huge background events induced by the beam operation. In order to minimize the time without data-taking, it is essential to establish a system which diagnoses errors and performs proper recovery actions quickly and automatically. We feed all the required operational data to the Elastic Stack in order to analyze the data effectively; this stack consists of a distributed analysis engine, data-feeding applications, and a web interface to visualize the data in the analysis engine. The stack is interfaced to the applications, which analyze the operational data, diagnose errors in real time, and perform the appropriate recovery actions. Using the new system, our data-taking efficiency is improved by close to 90%. We present the current status of the system together with the difficulties experienced during the operation and the future plan.