Scene-based Nonuniformity Correction Research Articles

Scene-Based Nonuniformity Correction Method Using Principal Component Analysis for Infrared Focal Plane Arrays

In this paper, principal component analysis is introduced to form a scene-based nonuniformity correction method for infrared focal plane arrays. The estimation of the gain and offset of the infrared detector and the correction of nonuniformity based on the neural network method with a novel estimation of desired target value are achieved concurrently. The current frame and several adjacent registered frames are decomposed onto a set of principal components, and then the first principal component is extracted to construct the desired target value. It is practical, forms fewer ghosting artifacts, and considerably promotes correction precision. Numerical experiments demonstrate that the proposed method presents excellent performance when dealing with clean infrared data with synthetic pattern noise as well as the real infrared video sequence.

Strong non-uniformity correction algorithm based on spectral shaping statistics and LMS.

The existence of non-uniformity in infrared detector output images is a widespread problem that significantly degrades image quality. Existing scene-based non-uniformity correction algorithms typically struggle to balance strong non-uniformity correction with scene adaptability. To address this issue, we propose a novel scene-based algorithm that leverages the frequency characteristics of the non-uniformity, combine and improve single-frame stripe removal, multi-scale statistics, and least mean square (LMS) methods. Following the "coarse-to-fine" correction process, the coarse correction stage introduces an adaptive progressive correction strategy based on Laplacian pyramids. By improving 1-D guided filtering and high-pass filtering to shape high-frequency sub-bands, non-uniformity can be well separated from the scene, effectively suppressing ghosting. In the fine correction stage, we optimize the expected image estimation and spatio-temporal adaptive learning rates based on guided filtering LMS method. To validate the efficacy of our algorithm, we conduct extensive simulation and real experiments, demonstrating its adaptability to various scene conditions and its effectiveness in correcting strong non-uniformity.

Infrared scene-based non-uniformity correction based on deep learning model

In this paper, an infrared scene-based non-uniformity correction method based on deep learning technology has been proposed. This method combines the scene-based infrared non-uniformity correction with the state-of-the-art deep learning technology. The traditional scene-based non-uniformity correction technologies generally face the problem that when the radiation of the scene is changed, the correction parameters may not converge anymore, and the correction process will rise again. Multiple times of correction will potentially increase the risk of the correction failure. The deep learning can help setting up a systematic correction parameter which self-adaptive to the thermal imager, which means that once the parameters are computed, the non-uniformity will be corrected according to the deep learning network by itself. Although the pre-calculation period of setting up the parameters take much time, the upcoming correction process is much easier than the traditional technologies. We use the feed-forward denoising convolutional network as the fundamental structure, and deploy the modified residual learning process as well as the batch normalization process onto it. Figures and charts show the priority of our method.

A Median-Ratio Scene-Based Non-Uniformity Correction Method for Airborne Infrared Point Target Detection System.

Infrared detectors suffer from severe non-uniform noise which highly reduces image resolution and point target signal-to-noise ratio. This is the restriction for airborne point target detection systems in reaching the background limit. The existing methods are either not accurate enough, or too complex to be applied to engineering. To improve the precision and reduce the algorithm complexity of scene-based Non-Uniformity Correction (NUC) for an airborne point target detection system, a Median-Ratio Scene-based NUC (MRSBNUC) method is proposed. The method is based on the assumption that the median value of neighboring pixels is approximately constant. The NUC coefficients are calculated recursively by selecting the median ratio of adjacent pixels. Several experiments were designed and conducted. For both the clear sky scene and scene with clouds, the non-uniformity is effectively reduced. Furthermore, targets were detected in outfield experiments. For Target 1 48.36 km away and Target 2 50.53 km away, employing MRSBNUC the SNR of the target increased 2.09 and 1.73 times respectively compared to Two-Point NUC. It was concluded that the MRSBNUC method can reduce the non-uniformity of the detector effectively which leads to a longer detection distance and fewer false alarms of the airborne point target detection system.

Statistical Scene-Based Non-Uniformity Correction Method with Interframe Registration.

The non-uniform response in infrared focal plane array (IRFPA) detectors inevitably produces corrupted images with a fixed-pattern noise. In this paper, we present a novel and adaptive scene-based non-uniformity correction (NUC) method called Correction method with Statistical scene-based and Interframe Registration (CSIR), which realizes low delay calculation of correction coefficient for infrared image. This method combines the statistical method and registration method to achieve a better NUC performance. Specifically, CSIR estimates the gain coefficient with statistical method to give registration method an appropriate initial value. This combination method not only reduces the need of interactive pictures, which means lower time delay, but also achieves better performance compared to the statistical method and other single registration methods. To verify this, real non-uniformity infrared image sequences collected by ourselves were used, and the advantage of CSIR was compared thoroughly on frame number (corresponding to delay time) and accuracy. The results show that the proposed method could achieve a significantly fast and reliable fixed-pattern noise reduction with the effective gain and offset.

Scene-based Nonuniformity Correction by Deep Neural Network with Image Roughness-like and Spatial Noise Cost Functions

Scene-based Nonuniformity Correction by Deep Neural Network with Image Roughness-like and Spatial Noise Cost Functions

A Novel Infrared Focal Plane Non-Uniformity Correction Method Based on Co-Occurrence Filter and Adaptive Learning Rate

Non-uniformity commonly exists in the infrared focal plane, which behaves as the fixed pattern noise (FPN) and seriously affects the image quality of the infrared imaging system. This paper proposed a novel scene-based non-uniformity correction method with a new edge-preserve filter and adaptive learning rate. First, using co-occurrence filter as the desired image estimation, the proposed method removed the FPN while preserving the image details. Then, an adaptive learning rate connected with both temporal motion and spatial correlation factor is utilized to decrease the effect of ghosting artifacts. In this way, the proposed method overcomes the shortcomings of the traditional scene-based non-uniformity. Several real infrared image sequences collected in different conditions are used to verify the performance of the proposed method. The experimental results demonstrate that the proposed method has a much better visual effect, making a great balance between the non-uniformity correction and details preservation. Compared with other good NUC methods, this method also has better performance in the aspects of applicability and robustness, which has great application value.

FPN estimation based nonuniformity correction for infrared imaging system

In this paper, to compensate for the fixed pattern noise (FPN) of infrared imaging system, a novel scene-based nonuniformity correction method is proposed based on FPN estimation. The FPN is supposed to be equal to the offset of detectors in IRFPA and is estimated based on the temporal filtering results of adjacent pixel difference. The temporal filters we used are the one order lag filter and mean filter. At the same time, the threshold constraint is introduced to select the candidate pixels of the original images for the estimation of FPN. This leads to the less interference of the details in scenes and the temporarily still scenes, thus our proposed method performs well on the actual scene. Besides this, the low computational complexity, simplicity of implementation and good real-time performance make our proposed method more suitable for hardware implementation. Experiments are conducted with three infrared video sequences to evaluate the performance of our proposed method and other two related methods, the correction results and quality indexes show that the performance of our proposed method is the best among these three methods.

Interior Radiation Noise Reduction Method Based on Multiframe Processing in Infrared Focal Plane Arrays Imaging System

Infrared focal plane arrays imaging system tend to be affected by the interior thermal radiation that drifts slowly over the time, resulting in the continuous spatial low-frequency nonuniformity noise in the image. In this paper, a scene-based nonuniformity correction method for interior thermal radiation is proposed. The gradient domain least mean square (LMS) algorithm is used to estimate the gradient caused by the interior radiation noise according to the fixed spatial low-frequency information in multiframe images. After that, the calibration parameters are obtained by fitting the derivatives of the polynomial-based correction model with the calculated gradient image. The performance of the proposed method is thoroughly studied with infrared image sequences with simulated nonuniformity and real nonuniformity. The results show that the proposed algorithm can remove interior radiation noise more completely without ghosting artifacts than the state-of-art algorithms and is more robust to different scenarios.

Scene-based Nonuniformity Correction for Neural Network Complemented by Reducing Lense Vignetting Effect and Adaptive Learning rate

Scene-based Nonuniformity Correction for Neural Network Complemented by Reducing Lense Vignetting Effect and Adaptive Learning rate

The On-Orbit Non-Uniformity Correction Method with Modulated Internal Calibration Sources for Infrared Remote Sensing Systems

The scanning infrared focal plane array (IRFPA) suffers from stripe-like non-uniformity due to the usage of many detectors, especially when working with a large time scale. Typical calibration systems tend to block the sensor aperture and expose the detectors to an on-board blackbody calibration source. They may also point at deep space. Full aperture calibration sources of this type tend to be large and expensive. To address these problems, a dynamic non-uniformity correction (NUC) method is proposed based on a modulated internal calibration device. By employing the on-board calibration device to generate a dynamic scene and fully integrating the system characteristics of the scanning IRFPA into the scene-based non-uniformity correction (SBNUC) algorithm, on-orbit high dynamic range NUC is achieved without blocking the field of view. Here we simulate an internal calibration system alternative, where a dynamic calibration signal is superimposed on the normal imagery, thus requiring no mechanisms and a smaller size. This method using this type of calibrator shows that when the sensor is pointing at deep space for calibration, it provides an effective non-uniformity correction of the imagery. After performing the proposed method, the NU of the two evaluation images was reduced from the initial 12.99% and 8.72% to less than 2%. Compared to other on-board NUC methods that require an extended reference blackbody source, this proposed approach has the advantages of miniaturization, a short calibration time, and strong adaptability.

Nonuniformity Correction Based on Adaptive Sparse Representation Using Joint Local and Global Constraints Based Learning Rate

The infrared focal plane array (IRFPA) suffers from the problem of non-uniform photoelectric responsivity of infrared detectors, namely nonuniformity. This paper aims at removing the fixed pattern noise (FPN) brought by nonuniformity in IRFPA through scene-based nonuniformity correction (SBNUC). Different from traditional neural network-based methods where spatial filters are adopted, a new approach of acquiring the desired image (the target of optimization task) is proposed, which is based on the sparse representation theory. The sparse representation of the image corrected by the current correction coefficients is used as the desired image. During sparse coding, the error tolerance is computed from an adaptive criterion, which is negatively correlated to the root-mean-square error between the input image and the corrected image. The resulted desired image is more accurate than its counterparts in traditional filter-based methods, as the edges can be preserved without heavily blurring image details, while the elements of FPN are omitted as redundant when coding the representation coefficients. To suit this desired image, a new adaptive learning rate based on joint local and global constraints is proposed. Different from traditional local spatial standard deviation-based learning rate, the constraints are calculated based on the variations among the input image, the corrected image, and the desired image. A new motion detection rule is also incorporated to this learning rate, which could be maximized to accelerate convergence only if the scene motion is sufficient both globally and locally. Experiments are conducted on both public infrared image sets (the Video Verification of Identity program data set and the Automonous Systems Lab data set) and a private infrared sequence. Experimental results demonstrate that, the proposed method achieves better performance of FPN removal on infrared sequences with both simulated and real nonuniformity, compared with SBNUC methods based on spatial filter (including Gaussian filter, guided filter, and bilateral filter).

Background Registration-Based Adaptive Noise Filtering of LWIR/MWIR Imaging Sensors for UAV Applications.

Unmanned aerial vehicles (UAVs) are equipped with optical systems including an infrared (IR) camera such as electro-optical IR (EO/IR), target acquisition and designation sights (TADS), or forward looking IR (FLIR). However, images obtained from IR cameras are subject to noise such as dead pixels, lines, and fixed pattern noise. Nonuniformity correction (NUC) is a widely employed method to reduce noise in IR images, but it has limitations in removing noise that occurs during operation. Methods have been proposed to overcome the limitations of the NUC method, such as two-point correction (TPC) and scene-based NUC (SBNUC). However, these methods still suffer from unfixed pattern noise. In this paper, a background registration-based adaptive noise filtering (BRANF) method is proposed to overcome the limitations of conventional methods. The proposed BRANF method utilizes background registration processing and robust principle component analysis (RPCA). In addition, image quality verification methods are proposed that can measure the noise filtering performance quantitatively without ground truth images. Experiments were performed for performance verification with middle wave infrared (MWIR) and long wave infrared (LWIR) images obtained from practical military optical systems. As a result, it is found that the image quality improvement rate of BRANF is 30% higher than that of conventional NUC.

A novel scene-based non-uniformity correction method for SWIR push-broom hyperspectral sensors

A novel scene-based non-uniformity correction (NUC) method for short-wavelength infrared (SWIR) push-broom hyperspectral sensors is proposed and evaluated. This method relies on the assumption that for each band there will be ground objects with similar reflectance to form uniform regions when a sufficient number of scanning lines are acquired. The uniform regions are extracted automatically through a sorting algorithm, and are used to compute the corresponding NUC coefficients. SWIR hyperspectral data from airborne experiment are used to verify and evaluate the proposed method, and results show that stripes in the scenes have been well corrected without any significant information loss, and the non-uniformity is less than 0.5%. In addition, the proposed method is compared to two other regular methods, and they are evaluated based on their adaptability to the various scenes, non-uniformity, roughness and spectral fidelity. It turns out that the proposed method shows strong adaptability, high accuracy and efficiency.

An improved non-uniformity correction algorithm and its hardware implementation on FPGA

The Non-uniformity of Infrared Focal Plane Arrays (IRFPA) severely degrades the infrared image quality. An effective non-uniformity correction (NUC) algorithm is necessary for an IRFPA imaging and application system. However traditional scene-based NUC algorithm suffers the image blurring and artificial ghosting. In addition, few effective hardware platforms have been proposed to implement corresponding NUC algorithms. Thus, this paper proposed an improved neural-network based NUC algorithm by the guided image filter and the projection-based motion detection algorithm. First, the guided image filter is utilized to achieve the accurate desired image to decrease the artificial ghosting. Then a projection-based moving detection algorithm is utilized to determine whether the correction coefficients should be updated or not. In this way the problem of image blurring can be overcome. At last, an FPGA-based hardware design is introduced to realize the proposed NUC algorithm. A real and a simulated infrared image sequences are utilized to verify the performance of the proposed algorithm. Experimental results indicated that the proposed NUC algorithm can effectively eliminate the fix pattern noise with less image blurring and artificial ghosting. The proposed hardware design takes less logic elements in FPGA and spends less clock cycles to process one frame of image.

Scene-based nonuniformity correction for airborne point target detection systems.

Images acquired by airborne infrared search and track (IRST) systems are often characterized by nonuniform noise. In this paper, a scene-based nonuniformity correction method for infrared focal-plane arrays (FPAs) is proposed based on the constant statistics of the received radiation ratios of adjacent pixels. The gain of each pixel is computed recursively based on the ratios between adjacent pixels, which are estimated through a median operation. Then, an elaborate mathematical model describing the error propagation, derived from random noise and the recursive calculation procedure, is established. The proposed method maintains the characteristics of traditional methods in calibrating the whole electro-optics chain, in compensating for temporal drifts, and in not preserving the radiometric accuracy of the system. Moreover, the proposed method is robust since the frame number is the only variant, and is suitable for real-time applications owing to its low computational complexity and simplicity of implementation. The experimental results, on different scenes from a proof-of-concept point target detection system with a long-wave Sofradir FPA, demonstrate the compelling performance of the proposed method.

Robust Approach for Nonuniformity Correction in Infrared Focal Plane Array.

In this paper, we propose a new scene-based nonuniformity correction technique for infrared focal plane arrays. Our work is based on the use of two well-known scene-based methods, namely, adaptive and interframe registration-based exploiting pure translation motion model between frames. The two approaches have their benefits and drawbacks, which make them extremely effective in certain conditions and not adapted for others. Following on that, we developed a method robust to various conditions, which may slow or affect the correction process by elaborating a decision criterion that adapts the process to the most effective technique to ensure fast and reliable correction. In addition to that, problems such as bad pixels and ghosting artifacts are also dealt with to enhance the overall quality of the correction. The performance of the proposed technique is investigated and compared to the two state-of-the-art techniques cited above.

Scene-based nonuniformity correction based on bilateral filter with reduced ghosting

In order to correct the nonuniformity noise of the IR imaging system, a nonuniformity correction deghosting algorithm based on high-brightness region detection was proposed. Firstly, the bilateral filter was used to filter the image. The high-brightness regions in the result image were detected, and the correction factors corresponding to the regions were reduced. And then, the positions were detected which are high-brightness pixels in former frame but are not in current frame, and the correction factors corresponding to the regions were enlarged. Thus, the wrong update of the correction factor caused by high-brightness region of both the current frame and the previous frame was reduced. The correction method was validated using a real image sequence. The experimental results showed that the suppression effect of the proposed algorithm on the ‘ghost’ effect caused by the high-brightness region is better than the other two algorithms named SLTH-NUC and BFTH-NUC, respectively. Moreover, the proposed algorithm can also suppress the ‘ghost’ effect caused by the sharp edge of the scene.

An evaluation method based on absolute difference to validate the performance of SBNUC algorithms

Scene-based non-uniformity correction (SBNUC) algorithms are an important part of infrared image processing; however, SBNUC algorithms usually cause two defects: (1) ghosting artifacts and (2) over-correction. In this paper, we use the absolute difference based on guided image filter (AD-GF) method to validate the performance of SBNUC algorithms. We obtain a self-separation source using the improved guided image filter to process the input image, and use the self-separation source to obtain the space-high-frequency parts of the input image and the corrected image. Finally, we use the absolute difference between the two space-high-frequency parts as the evaluation result. Based on experimental results, the AD-GF method has better robustness and can validate the performance of SBNUC algorithms even if ghosting artifacts or over-correction occur. Also the AD-GF method can measure how SBNUC algorithms perform in the time domain, it’s an effective evaluation method for SBNUC algorithm.

Guided filter and adaptive learning rate based non-uniformity correction algorithm for infrared focal plane array

Imaging non-uniformity of infrared focal plane array (IRFPA) behaves as fixed-pattern noise superimposed on the image, which affects the imaging quality of infrared system seriously. In scene-based non-uniformity correction methods, the drawbacks of ghosting artifacts and image blurring affect the sensitivity of the IRFPA imaging system seriously and decrease the image quality visibly. This paper proposes an improved neural network non-uniformity correction method with adaptive learning rate. On the one hand, using guided filter, the proposed algorithm decreases the effect of ghosting artifacts. On the other hand, due to the inappropriate learning rate is the main reason of image blurring, the proposed algorithm utilizes an adaptive learning rate with a temporal domain factor to eliminate the effect of image blurring. In short, the proposed algorithm combines the merits of the guided filter and the adaptive learning rate. Several real and simulated infrared image sequences are utilized to verify the performance of the proposed algorithm. The experiment results indicate that the proposed algorithm can not only reduce the non-uniformity with less ghosting artifacts but also overcome the problems of image blurring in static areas.