Novel image fusion quality metrics based on sensor models and image statistics

Abstract

This paper presents progress in image fusion modeling. One fusion quality metric based on the Targeting Task performance (TTP) metric and another based on entropy are presented. A human perception test was performed with fused imagery to determine effectiveness of the metrics in predicting image fusion quality. Both fusion metrics first establish which of two source images is ideal in a particular spatial frequency pass band. The fused output of a given algorithm is then measured against this ideal in each pass band. The entropy based fusion quality metric (E-FQM) uses statistical information (entropy) from the images while the Targeting Task Performance fusion quality metric (TTPFQM) utilizes the TTP metric value in each spatial frequency band. This TTP metric value is the measure of available excess contrast determined by the Contrast Threshold Function (CTF) of the source system and the target contrast. The paper also proposes an image fusion algorithm that chooses source image contributions using a quality measure similar to the TTP-FQM. To test the effectiveness of TTP-FQM and E-FQM in predicting human image quality preferences, SWIR and LWIR imagery of tanks were fused using four different algorithms. A paired comparison test was performed with both source and fused imagery as stimuli. Eleven observers were asked to select which image enabled them to better identify the target. Over the ensemble of test images, the experiment showed that both TTP-FQM and E-FQM were capable of identifying the fusion algorithms most and least preferred by human observers. Analysis also showed that the performance of the TTP-FQM and E-FQM in identifying human image preferences are better than existing fusion quality metrics such as the Weighted Fusion Quality Index and Mutual Information.