Dim Small Target Research Articles

Global induced local network for infrared: dim small target detection

Abstract It is challenging to detect infrared dim targets submerged in complicated backgrounds due to their small size and faint intensity. The previous attention-based detection networks frequently require global long-range dependence. Significant calculations are required to determine the target's sparse but meaningful position. To prevent wasting calculations on the background, this paper offers a detection network guided by global context for local feature learning, named GILNet (Global Induced Local Network). It designs a global location module (GLM) and a local feature interaction module (LFIM) to capture the global position and features of targets, respectively. More specifically, using global context interaction, the GLM finds the region that might contain dim small targets, that is, the coarse location. In the coarsely located regions, the LFIM further acquires feature information about targets. Next, we also design an eight-directional attention operation to obtain the contour information of targets in the low feature map. It is fused with the high feature map in the multi-directional feature fusion module (MFFM), which retains more semantic and spatial information about targets. Finally, quantitative and qualitative analysis show that the GILNet performs better than eight comparison methods on two public datasets.

TMP: Temporal Motion Perception with spatial auxiliary enhancement for moving Infrared dim-small target detection

Infrared dim and small target detection has significant research value and application prospects in both military affairs and civil safety scenes. This topic presents substantial challenges due to the small size of the targets and their low contrast against the background in images. In the past ten years, most detection methodologies have primarily focused on single-frame images, mainly relying on spatial-domain image features to identify dim and small targets, with limited consideration of temporal-domain motion features. However, unlike general objects, the image features of dim-small targets are often easy to be interfered by background, and even seriously lost in capturing stages. These deficiencies can be the primary factors to impact the detection performance of single-frame methods. To promote infrared dim-small target detection, this paper proposes a novel multi-frame pipeline called Temporal Motion Perception with spatial auxiliary enhancement (TMP), which consists of two parallel feature extraction branches. One auxiliary branch is designed to capture traditional image features from spatial domain, and the other is proposed to extract the motion features of targets from temporal domain. In addition, a new Complementary Symmetry Weighting module is specially designed to fulfill the cross-domain fusion of spatio-temporal features, further enhancing feature representation. Experimental results on three public datasets demonstrate the efficiency of our proposed scheme. Compared with existing methods, it could often refresh the state-of-the-art (SOTA) performance on most metrics. Our codes are available at https://github.com/UESTC-nnLab/TMP.

Infrared Dim Small Target Detection Networks: A Review.

In recent years, with the rapid development of deep learning and its outstanding capabilities in target detection, innovative methods have been introduced for infrared dim small target detection. This review comprehensively summarizes public datasets, the latest networks, and evaluation metrics for infrared dim small target detection. This review mainly focuses on deep learning methods from the past three years and categorizes them based on the six key issues in this field: (1) enhancing the representation capability of small targets; (2) improving the accuracy of bounding box regression; (3) resolving the issue of target information loss in the deep network; (4) balancing missed detections and false alarms; (5) adapting for complex backgrounds; (6) lightweight design and deployment issues of the network. Additionally, this review summarizes twelve public datasets for infrared dim small targets and evaluation metrics used for detection and quantitatively compares the performance of the latest networks. Finally, this review provides insights into the future directions of this field. In conclusion, this review aims to assist researchers in gaining a comprehensive understanding of the latest developments in infrared dim small target detection networks.

Infrared moving small-target detection using strengthened spatial-temporal tri-layer local contrast method

Infrared Search and Tracking (IRST) is challenged by detecting dim and small targets in complex backgrounds. In the context of moving small target detection, a perturbed, highly illuminated background is prone to engendering a high rate of false alarms. Furthermore, the variance in movement speed and scale of the targets can easily undermine the robustness of detection methods when extracting inter-frame information. In order to overcome these inadequacies, an effective method that leverages spatial and temporal profile information is proposed. In the spatial domain, targets are enhanced by computing the ratio difference as local contrast, and layered gradient kernel preprocessing along with gray difference calculations are applied to mitigate the impact of highly illuminated background. In the time domain, a tri-layer window for temporal profile of target pixels is utilized as an enhancement. By combining detections from both domains, target extraction is achieved through simple adaptive thresholding segmentation. The experimental results demonstrate that the proposed method is capable of effectively extracting slowly moving infrared dim small targets in complex backgrounds. Compared to existing spatiotemporal joint detection methods, the robustness is enhanced, false alarm rates are reduced, and higher computational efficiency is achieved.

Dim Small Target Detection and Tracking: A Novel Method Based on Temporal Energy Selective Scaling and Trajectory Association

Dim Small Target Detection and Tracking: A Novel Method Based on Temporal Energy Selective Scaling and Trajectory Association

Guided Attention and Joint Loss for Infrared Dim Small Target Detection

Guided Attention and Joint Loss for Infrared Dim Small Target Detection

SSTNet: Sliced Spatio-Temporal Network With Cross-Slice ConvLSTM for Moving Infrared Dim-Small Target Detection

SSTNet: Sliced Spatio-Temporal Network With Cross-Slice ConvLSTM for Moving Infrared Dim-Small Target Detection

Diffusion-Based Continuous Feature Representation for Infrared Small-Dim Target Detection

Diffusion-Based Continuous Feature Representation for Infrared Small-Dim Target Detection

Infrared Small Dim Target Detection Using Group Regularized Principle Component Pursuit

The detection of an infrared small target faces the problems of background interference and non-obvious target features, which have yet to be efficiently solved. By employing the non-local self-correlation characteristic of the infrared images, the principle component pursuit (PCP)-based methods are demonstrated to be applicable to infrared small target detection in a complex scene. However, existing PCP-based methods heavily depend on the uniform distribution of the background pixels and are prone to generating a high number of false alarms under strong clutter situations. In this paper, we propose a group low-rank regularized principle component pursuit model (GPCP) to solve this problem. First, the local image patches are clustered into several groups that correspond to different grayscale distributions. These patch groups are regularized with a group low-rank constraint, enabling an independent recovery of different background regions. Then, GPCP model integrates the group low-rank components with a global sparse component to extract small targets from the background. Different singular value thresholds can be exploited for image groups corresponding to different brightness and grayscale variance, boosting the recovery of background clutters and also enhancing the detection of small targets. Finally, a customized optimization approach based on alternating direction method of multipliers is proposed to solve this model. We set three representative detection scenes, including the ground background, sea background and sky background for experiment analysis and model comparison. The evaluation results show the proposed model has superiority in background suppression and achieves better adaptability for different scenes compared with various state-of-the-art methods.

MGAF‐net: Gaussian saliency features guided infrared small target detection network

AbstractRobust infrared dim small target detection under sparse features is a difficult problem in the field of small target detection. Therefore, in order to solve this problem, this paper proposes an infrared dim small target detection network based on multi‐scale Gaussian significance and attention feature fusion (MGAF)‐net. In MGAF‐net, MGAF module is proposed to extract multi‐scale Gaussian saliency features of small targets. At the same time, in order to better fuse the multi‐scale Gaussian saliency features of the target, the attention mechanism is introduced in the MGAF module to enrich the feature expression of the target. The experimental results show that the proposed detection method has higher detection accuracy and higher detection speed than the existing advanced infrared dim small target detection methods.

SFD-SR: strengthened feature description and stabilized re-detection-based infrared small–dim target tracking algorithm

SFD-SR: strengthened feature description and stabilized re-detection-based infrared small–dim target tracking algorithm

SiamIST: Infrared small target tracking based on an improved SiamRPN

The continuous tracking of infrared dim-small targets is significant, due to limited spatial resolution and low thermal features. Tracking algorithms based correlation filter may perform not well referring to infrared information. Therefore, a Deep Learning (DL) model is proposed for the tracking task with public data sets of small targets. To be specific, the Siamese Region Proposal Network (SiamRPN) is improved by the style recalibration module, which can obtain the perception of image styles. Furthermore, the proposed algorithm takes advantage of transfer learning technology referring to labeled target images, obtaining good features. To distinguish the small target from the background edges, the side window filtering is combined with the improved SiamRPN model. The experimental results show the good performance of the proposed small target tracking, namely SiamIST, in public near-infrared videos, compared to several related algorithms. Importantly, the designed algorithm uses the DL model to track small infrared targets for the first time, achieving an overall precision of 78.8%.

Eye detection and coarse localization of pupil for video-based eye tracking systems

A video-based eye tracking system generally captures NIR images, each of which contains one or two eyes of a subject. The subject’s point of gaze is then determined using 3D eye model and pupil centre corneal reflection technique. Eye detection and pupil localization play significant roles in video-based eye tracking systems. However, face rotation, wearing glasses, eye-shape variation and illumination variation make it difficult to detect an eye and localize a pupil accurately in the images captured by video-based eye tracking systems. In this paper, we proposed an eye detector that adopts a coarse-to-fine strategy. The eye detector consists of three classifiers: an ATLBP-THACs feature-based cascade classifier, a branch CNN and a multi-task CNN. We also proposed a method for coarse pupil localization. Coarse localization is an important step for pupil localization since it can provide initial pupil coordinates for a fine pupil localization method. Given a downscaled eye image, a shallow CNN is used to estimate the location of seven landmarks. On this basis, pupil center and radius are estimated. A method for small dim target enhancement is used to increase the contrast between pupil and background. The main goal of pupil enhancement is to make it easier to binarize an eye image. At last, component filtering is made by utilizing the estimated pupil center and radius. We collected a dataset named neepuEYE dataset that consists of 5500 NIR eye images from 109 people. The images can be used to generate augmented samples for training an eye detector since they contain eyes with different shape, orientation and pupil localization. Experimental results show that our eye detector is a fast and robust detector. Furthermore, our method for coarse pupil localization can obtain not only high detection rate but also high localization speed.

Infrared Dim Small Target Detection Based on Nonconvex Constraint with L1–L2 Norm and Total Variation

Infrared dim small target detection has received a lot of attention, because it is a crucial component of the IR search and track systems (IRST). The robust principal component analysis (RPCA) is a common detection framework, which works with poor performance with complex background edges and sparse clutters due to the inappropriate approximation of sparse items. A nonconvex constraint detection method based on the difference between the L1 and L2 (L1–L2) norm and total variation (TV) is presented. The L1–L2 norm is a more accurate sparse item approximation of L0 norm, which can achieve a better description of the sparse item to separate the target from the complex backgrounds. Then, the total variation norm is conducted on the target image to suppress the sparse clutters. The new model is solved using the alternating direction method of multipliers (ADMM) method. Then, the subproblems in the model are tackled by the difference of convex algorithm (DCA) and the Newton conjugate gradient (Newton-CG) solving L1–L2 norm and TV norm, respectively. In the experiment, we conducted experiments on multiple and single target datasets, and the proposed model outperforms the state-of-the-art (SOTA) methods in terms of background suppression and robustness to accurately detect the target. It can achieve a higher true position rate (TPR) with a low false position rate (FPR).

An improved dynamic programming tracking-before-detection algorithm based on LSTM network value function

The detection and tracking of small and weak maneuvering radar targets in complex electromagnetic environments is still a difficult problem to effectively solve. To address this problem, this paper proposes a dynamic programming tracking-before-detection method based on long short-term memory (LSTM) network value function(VL-DP-TBD). With the help of the estimated posterior probability provided by the designed LSTM network, the calculation of the posterior value function of the traditional DP-TBD algorithm can be more accurate, and the detection and tracking effect achieved for maneuvering small and weak targets is improved. Utilizing the LSTM network to model the posterior probability estimation of the target motion state, the posterior probability moving features of the maneuvering target can be learned from the noisy input data. By incorporating these posterior probability estimation values into the traditional DP-TBD algorithm, the accuracy and robustness of the calculation of the posterior value function can be enhanced, so that the improved architecture is capable of effectively recursively accumulating the movement trend of the target. Simulation results show that the improved architecture is able to effectively reduce the aggregation effect of a posterior value function and improve the detection and tracking ability for non-cooperative nonlinear maneuvering dim small target. Abbreviations LSTM: Long short-term memory; DP-TBD: Dynamic programming-based tracking before detection; DBT: Detection before tracking; TBD: Tracking before detection; HT-TBD: Tracking-before-detection algorithm based on the Hough transform; PF-TBD: Tracking-before-detection algorithm based on particle filtering; RFS-TBD: Tracking-before-detection algorithm based on random finite sets; SNR: Signal-to-noise ratio; DP: Dynamic programming; EVT: Extreme value theory; EVT: Generalized extreme value theory; GLRT: Generalized likelihood ratio detection; KT: Keystone transformation; PGA: Phase gradient autofocusing; CFAR: Constant false-alarm rate; J-CA-CFAR: Joint intensity-spatial CFAR; MF: Merit function; CP-DP-TBD: Candidate plot-based DP-TBD; CIT: Coherent integration time; RNN: Recurrent neural network; CS: Current statistical; Pd: Detection probability; Pt: Tracking probability.

Visual Detection and Association Tracking of Dim Small Ship Targets from Optical Image Sequences of Geostationary Satellite Using Multispectral Radiation Characteristics

By virtue of the merits of wide swath, persistent observation, and rapid operational response, geostationary remote sensing satellites (e.g., GF-4) show tremendous potential for sea target system surveillance and situational awareness. However, ships in such images appear as dim small targets and may be affected by clutter, reef islands, clouds, and other interferences, which makes the task of ship detection and tracking intractable. Considering the differences in visual saliency characteristics across multispectral bands between ships and jamming targets, a novel approach to visual detecting and association tracking of dense ships based on the GF-4 image sequences is proposed in this paper. First, candidate ship blobs are segmented in each single-spectral image of each frame through a multi-vision salient features fusion strategy, to obtain the centroid position, size, and corresponding spectral grayscale information of suspected ships. Due to the displacement of moving ships across multispectral images of each frame, multispectral association with regard to the positions of ship blobs is then performed to determine the final ship detections. Afterwards, precise position correction of detected ships is implemented for each frame in image sequences via multimodal data association between GF-4 detections and automatic identification system data. Last, an improved multiple hypotheses tracking algorithm with multispectral radiation and size characteristics is put forward to track ships across multi-frame corrected detections and estimate ships’ motion states. Experiment results demonstrate that our method can effectively detect and track ships in GF-4 remote sensing image sequences with high precision and recall rate, yielding state-of-the-art performance.

YOLO-FR: A YOLOv5 Infrared Small Target Detection Algorithm Based on Feature Reassembly Sampling Method.

The loss of infrared dim-small target features in the network sampling process is a major factor affecting its detection accuracy. In order to reduce this loss, this paper proposes YOLO-FR, a YOLOv5 infrared dim-small target detection model, based on feature reassembly sampling, which refers to scaling the feature map size without increasing or decreasing the current amount of feature information. In this algorithm, an STD Block is designed to reduce the loss of features during down-sampling by saving spatial information to the channel dimension, and the CARAFE operator, which increases the feature map size without changing the feature mapping mean, is adopted to ensure that features are not distorted by relational scaling. In addition, in order to make full use of the detailed features extracted by the backbone network, the neck network is improved in this study so that the feature extracted after one down-sampling of the backbone network is fused with the top-level semantic information by the neck network to obtain the target detection head with a small receptive field. The experimental results show that the YOLO-FR model proposed in this paper achieved 97.4% on mAP50, which is a 7.4% improvement compared to the original network, and it also outperformed J-MSF and YOLO-SASE.

ANLPT: Self-Adaptive and Non-Local Patch-Tensor Model for Infrared Small Target Detection

Infrared small target detection is widely used for early warning, aircraft monitoring, ship monitoring, and so on, which requires the small target and its background to be represented and modeled effectively to achieve their complete separation. Low-rank sparse decomposition based on the structural features of infrared images has attracted much attention among many algorithms because of its good interpretability. Based on our study, we found some shortcomings in existing baseline methods, such as redundancy of constructing tensors and fixed compromising factors. A self-adaptive low-rank sparse tensor decomposition model for infrared dim small target detection is proposed in this paper. In this model, the entropy of image block is used for fast matching of non-local similar blocks to construct a better sparse tensor for small targets. An adaptive strategy of low-rank sparse tensor decomposition is proposed for different background environments, which adaptively determines the weight coefficient to achieve effective separation of background and small targets in different background environments. Tensor robust principal component analysis (TRPCA) was applied to achieve low-rank sparse tensor decomposition to reconstruct small targets and their backgrounds separately. Sufficient experiments on the various types data sets show that the proposed method is competitive.

YOLOSR-IST: A deep learning method for small target detection in infrared remote sensing images based on super-resolution and YOLO

Infrared remote sensing imaging has a wide range of military and civilian applications. The detection of dim small targets is one of the most valuable research topics in this field. However, model-driven methods are not robust enough to noise, target size and contrast in images, and the currently proposed deep learning methods have insufficient ability to process and fuse important features, resulting in more missed detections and false alarms in these methods. To solve these problems, in this paper, a detection method based on super-resolution and deep learning is proposed. First, we use super-resolution preprocessing and multiple data augmentation on the input images. Secondly, based on the characteristics of infrared small target, we propose a new deep learning network named YOLOSR-IST. This network is based on a series of improvements on YOLOv5, including adding Coordinate Attention to backbone, introducing a high-resolution feature map P2 in the feature fusion, and replacing bottleneck layer of the C3 module in the head of the network with Swin Transformer Blocks. The proposed method achieves mAP@0.5 of 99.2% and 94.6% on two public datasets respectively, and solves the problem of missed detections and false alarms more effectively compared with current advanced data-driven detection methods.

Infrared Small Target Detection via Center-Surround Gray Difference Measure With Local Image Block Analysis

Exristing algorithms may suffer from high false alarm rate and low detection probability when detecting dim small target under intricate background clutters and heavy noise. To address this problem, a target detection method based on local image block analysis and center-surround gray difference measure (CGDM) is proposed in this article. First, the infrared image is decomposed into a group of local blocks. The gray information of each block are sorted in ascending order and the last element is deleted, then the maximum gray jump point (MGJP) is extracted by using differential operation. Second, a protection zone is formed with MGJP as the center, and the maximal intensity of protection zone (MIPZ) is extracted. Next, by comparing the gray intensity of MGJP and MIPZ values, the potential target blocks can be reliably selected, while most background blocks and noise blocks will be discarded. After that, the CGDM is presented to enhance the target and suppress background clutters, and then, the center-surround local contrast measure (CLCM) is designed to further suppress the high-intensity clutter residues by searching the target center and revising the protection zone. Finally, the weighted center-surround gray difference measure (WCGDM) is defined by CLCM WCGDM map to recognize real targets. Extensive experiments show that the proposed method outperforms several existing algorithms in small target detection under complex background clutters, and it is robust to various target shapes, target sizes, and noise types.