Thermal imaging spatial noise removal via deep image prior and step-variable total variation regularization

Abstract

The imaging quality of an uncooled long wave infrared camera suffers severely from time-drifted spatial noises if not regulated by a thermoelectric cooler (TEC). These noises can be categorized within the frequency domain as either fixed pattern noise (FPN) or low-frequency noise (LPN). The time-drifted FPN is primarily governed by the temperature fluctuations of the focal plane array (FPA), whereas the LPN predominantly originates from non-uniform background radiations. Most existing methods have been proposed to address the FPN or LPN independently rather than from a unified perspective. In this paper, a self-supervised convolution neural network (CNN) is proposed to remove these two types of noise simultaneously. Because of the combination of the deep image prior (DIP) architecture and the total variation (TV) regularizer, this method is named TV-DIP. With the assumption that the spatial noises are constant in a short period of time, the spatial noise pattern is extracted directly from multiple continuous raw frames without ground-truth noise-free images. In this way, the noise pattern can be updated periodically during running time, without requiring the use of a mechanical shutter. Besides, the original TV regularizer is extended with a variable step, which shows a significant enhancement on the LPN denoising task. Furthermore, both simulation and experimental results have demonstrated that this approach achieves better performance in terms of the removal of spatial noise, the preservation of detail, and the suppression of artifacts. The source code and dataset are available on the website: https://github.com/brunolinux/TV-DIP