General quantitative approach to performance evaluation of automatic target recognition (ATR) systems

Abstract

This paper presents a general quantitative approach to performance evaluation of ATR systems, especially for FLIR imagery. The general quantitative evaluation approach we proposed consists of three main steps. First, multiple sets of testing image sequences with significantly varying difficulty for specific environment, which can be precisely measured, are transmitted into an ATR system through a general protocol interface. Secondly, the computerized results of each set of testing image sequence are gotten through the interface after the ATR system works on each received image. Finally, data analysis will be prosecuted on the former experimental data sets. For convenience of arrangement, the different sets of testing image sequences with measured parameters and their homologous results are managed as the form of testing groups. When evaluating an ATR system, we design numerous groups of testing data which grade from easy to hard. We have developed the General Evaluation Software for Performance of ATR Systems (GESPATRS), which running under Windows in PC. As an example, an ATR system for FLIR imagery is evaluated by the GESPATRS. More details about it are given in the paper to demonstrate the feasibility of the proposed approach.