Compositional Probabilistic Analysis of Temporal Properties Over Stochastic Detectors

Abstract

Runtime monitoring is a vital part of safety-critical systems. However, early stage assurance of monitoring quality is currently limited: it relies either on complex models that might be inaccurate in unknown ways or on data that would only be available once the system has been built. To address this issue, we propose a compositional framework for modeling and analysis of noisy monitoring systems. Our novel 3-value detector model uses probability spaces to represent atomic (noncomposite) detectors, and it composes them into a temporal logic-based monitor. The error rates of these monitors are estimated by our analysis engine, which combines symbolic probability algebra, independence inference, and estimation from labeled detection data. Our evaluation on an autonomous underwater vehicle found that our framework produces accurate estimates of error rates while using only detector traces, without any monitor traces. Furthermore, when data are scarce, our approach shows higher accuracy than noncompositional data-driven estimates from monitor traces. Thus, this article enables accurate evaluation of logical monitors in early design stages before deploying them.