Methodology for alerting-system performance evaluation

Abstract

A probabilistic-analysis methodology is described that provides quantitative measures of alerting-system performance, including the probabilities of a false alarm and missed detection. As part of the approach, the alerting decision is recast as a signal-detection problem, and system operating-characteristic curves are introduced to describe the tradeoffs between alerting-threshold placement and system performance. The methodology fills the need for a means to determine appropriate alerting thresholds and to quantify the potential benefits that are possible through changes in the design of the system. Because the methodology is developed in a generalized manner, it can be used in a variety of vehicle, transportation-system, and process-control applications. The methodology is demonstrated through an example application to the Traffic Alert and Collision Avoidance System (TCAS). Recent changes in TCAS alerting thresholds are shown to reduce the probability of a false alarm in situations known to produce frequent nuisance alerts in actual operations. Nomenclature A, N, T = probabilistic-state trajectories E = event of encountering a hazard fx (x) = probability density function for the random variable x h = estimated relative altitude h = estimated relative-altitude rate / = incident event ^maif - probability of alerting-system malfunction P(x) = probability of event ;c PT(X) = probability of event x evaluated along trajectory T r = estimated relative range r = estimated relative-range rate