Real Infrared Image Sequences Research Articles

High-Speed Spatial–Temporal Saliency Model: A Novel Detection Method for Infrared Small Moving Targets Based on a Vectorized Guided Filter

Infrared (IR) imaging-based detection systems are of vital significance in the domains of early warning and security, necessitating a high level of precision and efficiency in infrared small moving target detection. IR targets often appear dim and small relative to the background and are easily buried by noise and difficult to detect. A novel high-speed spatial–temporal saliency model (HS-STSM) based on a guided filter (GF) is proposed, which innovatively introduces GF into IR target detection to extract the local anisotropy saliency in the spatial domain, and substantially suppresses the background region as well as the bright clutter false alarms present in the background. Moreover, the proposed model extracts the motion saliency of the target in the temporal domain through vectorization of IR image sequences. Additionally, the proposed model significantly improves the detection efficiency through a vectorized filtering process and effectively suppresses edge components in the background by integrating a prior weight. Experiments conducted on five real infrared image sequences demonstrate the superior performance of the model compared to existing algorithms in terms of the detection rate, noise suppression, real-time processing, and robustness to the background.

Facet Derivative-Based Multidirectional Edge Awareness and Spatial–Temporal Tensor Model for Infrared Small Target Detection

Infrared (IR) small target detection in the complex background is an important but challenging research hotspot in the field of target detection. The existing methods usually cause high false alarms in the complex background and fail to make full use of the complete information of the image. In this article, a novel IR small target detection model that combines facet derivative-based multidirectional edge awareness with spatial–temporal tensor (FDMDEA-STT) is presented. First, we construct an STT model (STTM) to transform the target detection problem into a low-rank and sparse tensor optimization problem based on the prior information of the target and background in the spatial–temporal domain. Then, based on the facet derivative, we define a multidirectional edge awareness mapping and fuse it into the STTM as sparse prior information. Finally, an effective algorithm based on the alternating direction method of multipliers (ADMM) is designed to solve the above model. The effectiveness of the proposed method is verified on eight real IR image sequences. Experimental results demonstrate that the proposed method has better detection performance than the existing state-of-the-art methods.

Fusion of interpolated frames superresolution in the presence of atmospheric optical turbulence

An extension of the fusion of interpolated frames superresolution (FIF SR) method to perform SR in the presence of atmospheric optical turbulence is presented. The goal of such processing is to improve the performance of imaging systems impacted by turbulence. We provide an optical transfer function analysis that illustrates regimes where significant degradation from both aliasing and turbulence may be present in imaging systems. This analysis demonstrates the potential need for simultaneous SR and turbulence mitigation (TM). While the FIF SR method was not originally proposed to address this joint restoration problem, we believe it is well suited for this task. We propose a variation of the FIF SR method that has a fusion parameter that allows it to transition from traditional diffraction-limited SR to pure TM with no SR as well as a continuum in between. This fusion parameter balances subpixel resolution, needed for SR, with the amount of temporal averaging, needed for TM and noise reduction. In addition, we develop a model of the interpolation blurring that results from the fusion process, as a function of this tuning parameter. The blurring model is then incorporated into the overall degradation model that is addressed in the restoration step of the FIF SR method. This innovation benefits the FIF SR method in all applications. We present a number of experimental results to demonstrate the efficacy of the FIF SR method in different levels of turbulence. Simulated imagery with known ground truth is used for a detailed quantitative analysis. Three real infrared image sequences are also used. Two of these include bar targets that allow for a quantitative resolution enhancement assessment.

Anti-Interference Aircraft-Tracking Method in Infrared Imagery.

In this paper, we focus on developing an algorithm for infrared-imaging guidance that enables the aircraft to be reliably tracked in the event of interference. The key challenge is to track the aircraft with occlusion caused by decoys and drastic appearance changes resulting from a diversity of attacking angles. To address this challenge, an aircraft-tracking algorithm was proposed, which provides robustness in tracking the aircraft against the decoys. We reveal the inherent structure and infrared signature of the aircraft, which are used as discriminative features to track the aircraft. The anti-interference method was developed based on simulated images but validate the effectiveness on both real infrared image sequences without decoys and simulated infrared imagery. For frequent occlusion caused by the decoys, the mechanism of occlusion detection is exploited according to the variation of the model distance in tracking process. To have a comprehensive evaluation of tracking performance, infrared-image sequences with different attack angles were simulated, and experiments on benchmark trackers were performed to quantitatively evaluate tracking performance. The experiment results demonstrate that our aircraft-tracking method performs favorably against state-of-the-art trackers.

A Novel Infrared Focal Plane Non-Uniformity Correction Method Based on Co-Occurrence Filter and Adaptive Learning Rate

Non-uniformity commonly exists in the infrared focal plane, which behaves as the fixed pattern noise (FPN) and seriously affects the image quality of the infrared imaging system. This paper proposed a novel scene-based non-uniformity correction method with a new edge-preserve filter and adaptive learning rate. First, using co-occurrence filter as the desired image estimation, the proposed method removed the FPN while preserving the image details. Then, an adaptive learning rate connected with both temporal motion and spatial correlation factor is utilized to decrease the effect of ghosting artifacts. In this way, the proposed method overcomes the shortcomings of the traditional scene-based non-uniformity. Several real infrared image sequences collected in different conditions are used to verify the performance of the proposed method. The experimental results demonstrate that the proposed method has a much better visual effect, making a great balance between the non-uniformity correction and details preservation. Compared with other good NUC methods, this method also has better performance in the aspects of applicability and robustness, which has great application value.

Spatial-temporal local contrast for moving point target detection in space-based infrared imaging system

Infrared (IR) moving point target detection is an important technique in many onboard applications such as remote sensing, IR searching and tracking (IRST) and early warning system. However, it has been facing great challenges due to the complicated background and the limited processing resources in the onboard system. In this paper, a novel spatial-temporal local contrast method is proposed for moving point target detection in space-based IR imaging system. Firstly, a simple but effective spatial filter based on multi-direction filtering fusion is designed to obtain the spatial local contrast map (SLCM). An enhanced time domain difference method is also proposed to obtain the temporal local contrast map (TLCM). Then, the spatial–temporal local contrast map (STLCM) is obtained by multiplying the newly defined SLCM and TLCM. Finally, an enhanced threshold segmentation method is proposed for target detection and false alarm suppression. To verify the performance of our detection algorithm, we conduct several groups of experiments on four different real IR image sequences with simulated targets. The final experimental results show that our algorithm significantly outperforms other methods in terms of background suppression and target detection.

Total Variation Based Neural Network Regression for Nonuniformity Correction of Infrared Images

Many existing scene-adaptive nonuniformity correction (NUC) methods suffer from slow convergence rate together with ghosting effects. In this paper, an improved NUC algorithm based on total variation penalized neural network regression is presented. Our work mainly focuses on solving the overfitting problem in least mean square (LMS) regression of traditional neural network NUC methods, which is realized by employing a total variation penalty in the cost function and redesigning the processing architecture. Moreover, an adaptive gated learning rate is presented to further reduce the ghosting artifacts and guarantee fast convergence. The performance of the proposed algorithm is comprehensively investigated with artificially corrupted test sequences and real infrared image sequences, respectively. Experimental results show that the proposed algorithm can effectively accelerate the convergence speed, suppress ghosting artifacts, and promote correction precision.

Infrared Small Target Detection via Nonnegativity-Constrained Variational Mode Decomposition

Infrared small target detection is one of the key techniques in the infrared search and track system. Frequency differences among target, background, and noise are often important information for target detection. In this letter, a nonnegativity-constrained variational mode decomposition (NVMD) method is proposed. Unlike the traditional frequency-domain methods, the proposed method can adaptively decompose the input signal into several separated band-limited subsignals, with the nonnegativity constraint. First, a bandpass filter is used as a preprocessing step. Second, by exploring the frequency and nonnegativity properties of the small target, the NVMD model is constructed. The potential target subsignal can be obtained by solving the NVMD model. By performing threshold segmentation on the potential target subsignal, we can obtain the detection result of the infrared small target. Experiments on six real infrared image sequences demonstrate that the proposed method has a good performance in target enhancement and background suppression. Additionally, the proposed method shows strong robustness under various backgrounds.

Particle filter-based relative rolling estimation algorithm for non-cooperative infrared spacecraft

The issue of feature point mismatching among infrared image sequence would bring big challenge to estimating the relative motion of non-cooperative spacecraft for it couldn’t provide the prior knowledge about its geometric structure and motion pattern. The paper introduces particle filter to precisely match the feature points within a desired region predicted by a kinetic equation, and presents a least square estimation-based algorithm to measure the relative rolling motion of non-cooperative spacecraft. The state transition equation and the measurement update equation of non-cooperative spacecraft are represented by establishing its kinetic equations, and then the relative pose measurement is converted to the maximum posterioriprobability estimation via assuming the uncertainties about geometric structure and motion pattern as random and time-varying variables. These uncertainties would be interpreted and even solved through continuously measuring the image feature points of the rotating non-cooperative infrared spacecraft. Subsequently, the feature point is matched within a predicted region among sequence infrared image using particle filter algorithm to overcome the position estimation noise caused by the uncertainties of geometric structure and motion pattern. Finally, the position parameters including rotation motion are estimated by means of solving the minimum error of feature point mismatching using least square estimate theory. Both simulated and real infrared image sequences are induced in the experiment to evaluate the performance of the relative rolling estimation, and the experimental data show that the rolling motion estimated by the proposed algorithm is more robust to the feature extraction noise and various rotation speed. Meanwhile, the relative rolling estimation error would increase dramatically with distance and rotation speed increasing.

Nonuniformity correction for an infrared focal plane array based on diamond search block matching.

In scene-based nonuniformity correction algorithms, artificial ghosting and image blurring degrade the correction quality severely. In this paper, an improved algorithm based on the diamond search block matching algorithm and the adaptive learning rate is proposed. First, accurate transform pairs between two adjacent frames are estimated by the diamond search block matching algorithm. Then, based on the error between the corresponding transform pairs, the gradient descent algorithm is applied to update correction parameters. During the process of gradient descent, the local standard deviation and a threshold are utilized to control the learning rate to avoid the accumulation of matching error. Finally, the nonuniformity correction would be realized by a linear model with updated correction parameters. The performance of the proposed algorithm is thoroughly studied with four real infrared image sequences. Experimental results indicate that the proposed algorithm can reduce the nonuniformity with less ghosting artifacts in moving areas and can also overcome the problem of image blurring in static areas.

Multiscale random projection based background suppression of infrared small target image

The heavy clutters background in a single infrared image containing small targets is difficult to be efficiently suppressed using the traditional methods. To overcome this difficulty, a novel infrared image background suppression method based on multiscale random projection is proposed in this paper. On the one hand, using nonsubsampled pyramid transform that decomposes the original infrared image into a low-frequency subband and a series of high-frequency subbands, the proposed method separates the background and targets from the original infrared image. On the other hand, due to a subband image cube is formed by concatenating together all the high-frequency subbands and a random projection is applied to the image cube, the proposed method can well preserve target information and reduces the spatial redundancy of the target and background information. Experimental results on three real infrared image sequences under different clutters background demonstrate that the proposed method is efficient and can obtain better performance than other methods for background suppression of infrared small target images.

배경 적응적 문턱치 맵(Threshold Map)을 이용한 클러터 제거 기법

In this paper, we propose a robust clutter pre-thresholding method using background adaptive Threshold Map for the clutter rejection in the complex coastal environment. The proposed algorithm is composed of the use of Threshold Map's and method of its calculation. Additionally we also suggest an automatic decision method of Thresold Map's update. Experimental results on some sets of real infrared image sequence show that the proposed method could remove clutters effectively without any loss of detection rate for the aim target and reduce processing time dramatically.

배경제거 가산투영 방법을 이용한 근거리 해상 표적 탐지

In this paper, we propose a robust background removal integral projection(BRIP) method which can detect close surface targets in coastal environment for IRST. Row pixels of background region from coastal infrared image show similar response. Thus, the proposed BRIP is calculated after horizontal and vertical background estimations and removals are performed sequentially. Finally, surface large targets can be detected using the results of the BRIP. Experimental results on a set of real infrared image sequence show that the proposed method could fully detect ships in every frame.

클러터 환경에 강인한 고속/소형의 접근 표적 탐지/추적

In this paper, we propose a robust method which can detect and track a high-speed small approaching target in a cluttered environment for Korean Active Protection System. The proposed method uses a temporal and spatial filter, tracking filter to detect and track a single target in consecutive order. And it is comprised of a candidate target detection step, a prior target selection step and a target tracking. Field tests on real infrared image sequences show that the proposed method could stably track a high speed and small target in complex background and target occlusion.

Improved level-set framework-based algorithm for small infrared target detection

An improved algorithm integrating wavelet decomposition, multilevel filtering, and an additive operator splitting (AOS)-based level-set framework for infrared small target detection is proposed. This model has two components: a filtering operation, and level-set evolution. In the filtering step, the original image is first decomposed using a wavelet transform. After determining the location of sea-sky line, we construct a subimage based on the sea-sky-line position, and then execute multilevel filtering on this subimage. This filtering framework provides the input image for the level-set evolution. Using the level-set formulation, complex curves can be detected while naturally handling topological changes of the evolving curves. To reduce the computational cost required by an explicit implementation of the level-set formulation, a new solver named AOS is proposed. Additionally, the quantitative analyses for our algorithm are also given. Experiments on real infrared image sequences indicate that our method is efficient and robust.

Nonuniformity correction algorithm with nonlinear model for infrared focal plane arrays

For the detector in infrared focal plane arrays (IRFPA) with a large dynamic range of response, a nonlinear model of response curve of the detector in IRFPA is introduced in this paper. With the model, the Kalman-filter nonuniformity correction (NUC) algorithm with linear model, developed by Torres and Hayat, is extended. In the extended algorithm, the raw image is translated into a linearized one firstly by directly employing a logarithm-based transformation. Then the linearized image is corrected by the Kalman-filter NUC algorithm with linear model, and the corrected linearized image is obtained. Finally the uniformity image of the original one is achieved by fulfilling an exponent transformation to the corrected linearized image. The presented algorithm not only inherits the advantage of the original algorithm that resolves the problem of the temporal drift in the gain and the bias in each detector by updating NUC parameters with information of the current scene, but also reduce the influence of the detector nonlinear response to the NUC performance, so it is suitable for IRFPA under large response-range. The NUC ability of the presented algorithm is demonstrated by experiments with real infrared image sequences.

Scene-Based Nonuniformity Correction with Reduced Ghosting Using a Gated LMS Algorithm

In this paper, we present a scene-based nouniformity correction (NUC) method using a modified adaptive least mean square (LMS) algorithm with a novel gating operation on the updates. The gating is designed to significantly reduce ghosting artifacts produced by many scene-based NUC algorithms by halting updates when temporal variation is lacking. We define the algorithm and present a number of experimental results to demonstrate the efficacy of the proposed method in comparison to several previously published methods including other LMS and constant statistics based methods. The experimental results include simulated imagery and a real infrared image sequence. We show that the proposed method significantly reduces ghosting artifacts, but has a slightly longer convergence time.

Scene-based nonuniformity correction technique for infrared focal-plane arrays

A scene-based nonuniformity correction algorithm is presented to compensate for the gain and bias nonuniformity in infrared focal-plane array sensors, which can be separated into three parts. First, an interframe-prediction method is used to estimate the true scene, since nonuniformity correction is a typical blind-estimation problem and both scene values and detector parameters are unavailable. Second, the estimated scene, along with its corresponding observed data obtained by detectors, is employed to update the gain and the bias by means of a line-fitting technique. Finally, with these nonuniformity parameters, the compensated output of each detector is obtained by computing a very simple formula. The advantages of the proposed algorithm lie in its low computational complexity and storage requirements and ability to capture temporal drifts in the nonuniformity parameters. The performance of every module is demonstrated with simulated and real infrared image sequences. Experimental results indicate that the proposed algorithm exhibits a superior correction effect.

基于粒子滤波的红外目标跟踪

To address two challenging problems in infrared target tracking, target appearance changes and unpredictable abrupt motions, a novel particle filtering based tracking algorithm is introduced. In this method, a novel saliency model is proposed to distinguish the salient target from background, and the eigenspace model is invoked to adapt target appearance changes. To account for the abrupt motions efficiently, a two-step sampling method is proposed to combine the two observation models. The proposed tracking method is demonstrated through two real infrared image sequences, which include the changes of luminance and size, and the drastic abrupt motions of the target.

Noise-cancellation-based nonuniformity correction algorithm for infrared focal-plane arrays

The spatial fixed-pattern noise (FPN) inherently generated in infrared (IR) imaging systems compromises severely the quality of the acquired imagery, even making such images inappropriate for some applications. The FPN refers to the inability of the photodetectors in the focal-plane array to render a uniform output image when a uniform-intensity scene is being imaged. We present a noise-cancellation-based algorithm that compensates for the additive component of the FPN. The proposed method relies on the assumption that a source of noise correlated to the additive FPN is available to the IR camera. An important feature of the algorithm is that all the calculations are reduced to a simple equation, which allows for the bias compensation of the raw imagery. The algorithm performance is tested using real IR image sequences and is compared to some classical methodologies.