An effective sequential statistical test for probabilistic monitoring

Abstract

Context A monitor checks if a system behaves according to a specified property at runtime. This is required for quality assurance purposes. Currently several approaches exist to monitor standard and real-time properties. However, a current challenge is to provide a comprehensive approach for monitoring probabilistic properties, as they are used to formulate quality of service requirements like performance, reliability, safety, and availability. The main problem of these probabilistic properties is that there is no binary acceptance condition. Objective To overcome this problem, this article presents an improved and generic statistical decision procedure based on acceptance sampling and sequential hypothesis testing. Method The developed decision procedure is validated using several experiments that determine the operating characteristic, runtime overhead as well as the expected sample sizes. Results and conclusion The experimental validation provides evidence that the developed testing procedure reduces the runtime overhead and improves the accuracy of classification. Thus, the statistical decision procedure is superior to the existing statistical tests currently used in probabilistic monitoring.