A Reliability Assessment Paradigm for Automated Video Tracking Systems

Abstract

Most existing performance evaluation methods concentrate on defining separate metrics over a wide range of conditions and generating standard benchmarking video sequences for examining the effectiveness of video tracking systems. In other words, these methods attempt to design a robustness margin or factor for the system. These methods are deterministic in which a robustness factor, for example, 2 or 3 times the expected number of subjects to track or the strength of illumination would be required in the design. This often results in over design, thus increasing costs, or under design causing failure by unanticipated factors. In order to overcome these limitations, we propose in this paper an alternative framework to analyze the physics of the failure process and, through the concept of reliability, determine the time to failure in automated video tracking systems. The benefit of our proposed framework is that we can provide a unified and statistical index to evaluate the performance of automated video tracking system for a task to be performed. At the same time, the uncertainty problem about a failure process, which may be caused by the system's complexity, imprecise measurements of the relevant physical constants and variables, or the indeterminate nature of future events, can be addressed accordingly based on our proposed framework.