Improving Automatic Target Recognition With Infrared Imagery Using Vision Transformers and Focused Data Augmentation

Automatic target recognition (ATR) is a challenging task for several computer vision applications. It requires efficient, accurate, and robust methods for target detection and target identification. Deep learning has shown great success in many computer vision applications involving color RGB images. However, the performance of these networks in ATR with infrared sensor data needs further investigation. In this paper, we propose a multistage automatic target detection and recognition (ATDR) system that performs both target detection and target classification on infrared (IR) imagery using deep learning. Our system processes large IR image frames where targets take <1 % of the total number of pixels. First, we train a state-of-the-art object detector you only look once (YOLO) to localize all potential targets in the input image frame. Then, we train a convolutional neural network (CNN) to identify these detections as targets or false alarms. In this second phase, we adapt and analyze the performance of three CNN architectures: a compact and fully connected CNN, VGG16 with batch normalization, and a wide residual neural network (WRN). We also explore the use of a loss function that optimizes directly the area under the receiver operating characteristic (ROC) curve (AUC), and adapt it to our ATR application. To enhance the robustness of the proposed ATR to perturbation and variations introduced during the detection stage, we train our CNN classifiers on automatically detected targets using YOLO, in addition to ground truth bounding boxes and apply selected data augmentation techniques. To simulate real testing environments, where the spatial location of the targets within the image frame is unknown, only YOLO-detected boxes are used during validation. We evaluate our ATDR on a real benchmark dataset that includes different vehicles captured at different resolutions. Our experiments have shown that YOLO can detect most of the targets at the expense of generating a high number of false alarms. We show that the VGG-16 network with batch normalization, which is the best performing model, can correctly identify the classes of the targets, as well as classify the majority of YOLO’s false detections into an additional nontarget class. We also show that the proposed training modification to optimize an AUC-based loss function for ATR proved to be advantageous mainly in identifying difficult targets.

The spatial resolution is increasingly high as development of optical remote sensing, and more and more optical sensors can achieve the detection ability of sub meter, which lays down the data foundation for automatic recognition of ship targets. However, mature technology is lacked to identify the ship models automatically with remote sensing images. In this study, an automatic recognition method for ship targets is proposed based on the local invariant feature extraction algorithm SIFT (Scale Invariant Feature Transform), which is consist of feature extraction and description, feature matching and target recognition. The model of unknown target is identified based on the target library using the matching difference of targets with the same model and different models. The experiment results show that this automatic recognition flow is effective to identify the ship targets of interest based on the target library, and the total correct recognition rate is 92%. This method provides a new flow for automatic model recognition of ship targets, and has considerable potential for wide applications.

Synthetic aperture radar (SAR) is an advanced microwave imaging system of great importance. The recognition of real-world targets from SAR images, i.e., automatic target recognition (ATR), is an attractive but challenging issue. The majority of existing SAR ATR methods are designed for single-view SAR images. However, multiview SAR images contain more abundant classification information than single-view SAR images, which benefits automatic target classification and recognition. This paper proposes an end-to-end deep feature extraction and fusion network (FEF-Net) that can effectively exploit recognition information from multiview SAR images and can boost the target recognition performance. The proposed FEF-Net is based on a multiple-input network structure with some distinct and useful learning modules, such as deformable convolution and squeeze-and-excitation (SE). Multiview recognition information can be effectively extracted and fused with these modules. Therefore, excellent multiview SAR target recognition performance can be achieved by the proposed FEF-Net. The superiority of the proposed FEF-Net was validated based on experiments with the moving and stationary target acquisition and recognition (MSTAR) dataset.

Image complexity metric is an important part of automatic target recognition(ATR) performance evaluation, the relationship between infrared image complexity metric and target recognition is studied, which is important for infrared imaging system performance prediction and evaluation and the performance comparison of target recognition algorithms. Aiming at this problem, an automatic target recognition infrared image complexity metric method is proposed. Firstly, the infrared imaging mechanism is analyzed to find the main factors affecting target recognition. The image complexity is defined from the similarity degree of target and clutter and the submergence degree of target and clutter, which clarify for the influence of target recognition. To increase the universality of image complexity, the concept of feature space was introduced. Finally, the weighted processing and statistical formula F1-Score is used to combine the three indexes, the complexity of the frame image is established. The experimental results show that the proposed metric is more valid than traditional metrics, such as SV and SCR, has a strong correlation with automatic target recognition algorithm, while the values are in better agreement with the actual situation.

Underwater acoustics has been implemented mostly in the field of sound navigation and ranging (SONAR) procedures for submarine communication, the examination of maritime assets and environment surveying, target and object recognition, and measurement and study of acoustic sources in the underwater atmosphere. With the rapid development in science and technology, the advancement in sonar systems has increased, resulting in a decrement in underwater casualties. The sonar signal processing and automatic target recognition using sonar signals or imagery is itself a challenging process. Meanwhile, highly advanced data-driven machine-learning and deep learning-based methods are being implemented for acquiring several types of information from underwater sound data. This paper reviews the recent sonar automatic target recognition, tracking, or detection works using deep learning algorithms. A thorough study of the available works is done, and the operating procedure, results, and other necessary details regarding the data acquisition process, the dataset used, and the information regarding hyper-parameters is presented in this article. This paper will be of great assistance for upcoming scholars to start their work on sonar automatic target recognition.

JPL is developing a comprehensive Automatic Target Recognition (ATR) system that consists of an innovative anomaly detection preprocessing module and an automatic training target recognition module. The anomaly detection module is trained with an imaging data feature retrieved from an imaging sensor suite that represents the states of the normalcy model. The normalcy model is then trained from a self-organizing learning system over a period of time and fed into the anomaly detection module for scene anomaly monitoring and detection. The abnormal event detection will be sent to a human operator for further investigation responses. The target recognition will be continuously updated with the normal' input sensor data. The combination of the anomaly detection preprocessing module to the re-trainable target recognition processor will result in a dynamic ATR system that is capable of automatic detection of anomaly event and provide an early warning to a human operator for in-time warning and response.

Neural networks for automatic target recognition

This paper presents a technique for automatic airborne target recognition and tracking in forward-looking infrared (FLIR) images with a complex background. An image splitting and merging method is applied for detecting target signals. The presence of a complex background due to clouds and sun glint generates clutter in the image with the resulting possibility of false alarms. A Bayesian classifier trained using the NMI (normalized moment of inertia) feature is proposed for efficient clutter rejection. After classification, target candidates are entered into a tracking filter. As an efficient and robust multi-target tracking filter in cluttered environments, the JDC-JIHPDAF is proposed. Experimental results using a wide range of real FLIR images ensure reliable classification and automatic target recognition performance.

Many applications reported in artificial neural networks are associated with military problems. This paper reviews concepts associated with the processing of military data to find and recognize targets -- automatic target recognition (ATR). A general-purpose automatic target recognition system does not exist. The work presented here is demonstrated on military data, but it can only be considered proof of principle until systems are fielded and proven `under- fire.' ATR data can be in the form of nonimaging one-dimensional sensor returns, such as ultra-high range-resolution radar returns (UHRR) for air-to-air automatic target recognition and vibration signatures from a laser radar for recognition of ground targets. The ATR data can be two-dimensional images. The most common ATR images are infrared, but current systems must also deal with synthetic aperture radar (SAR) images. Finally, the data can be three-dimensional, such as sequences of multiple exposures taken over time from a nonstationary world. Targets move, as do sensors, and that movement can be exploited by the ATR. Hyperspectral data, which are views of the same piece of the world looking at different spectral bands, is another example of multiple image data; the third dimension is now wavelength and not time. ATR system design usually consists of four stages. The first stage is to select the sensor or sensors to produce the target measurements. The next stage is the preprocessing of the data and the location of regions of interest within the data (segmentation). The human retina is a ruthless preprocessor. Physiologically motivated preprocessing and segmentation is demonstrated along with supervised and unsupervised artificial neural segmentation techniques. The third design step is feature extraction and selection: the extraction of a set of numbers which characterize regions of the data. The last step is the processing of the features for decision making (classification). The area of classification is where most ATR related neural network research has been accomplished. The relation of neural classifiers to Bayesian techniques is emphasized along with the more recent use of feature sequences to enhance classification. © (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

A new 1-D template-based automatic target recognition (ATR) algorithm is developed and tested on high range resolution (HRR) profiles formed from synthetic aperture radar (SAR) images of targets taken from the Moving and Stationary Target Acquisition and Recognition (MSTAR) data set. In this work, a superresolution technique known as High Definition Vector Imaging (HDVI) is applied to the HRR profiles before the profiles are passed through ATR classification. The new I-D ATR system using HDVI demonstrates significantly improved target recognition compared with previous I-D ATR systems that use conventional image processing techniques. This improvement in target recognition is quantified by improvement in probability of correct classification (PCC). More importantly, the application of HDVI to HRR profiles helps to maintain the same ATR performance with reduced radar resource requirements.

Mass production of high-quality synthetic SAR training imagery is essential for boosting the performance of deep-learning (DL)-based SAR automatic target recognition (ATR) algorithms in an open-world environment. To address this problem, we exploit both the widely used Moving and Stationary Target Acquisition and Recognition (MSTAR) SAR dataset and the Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset, which consists of selected samples from the MSTAR dataset and their computer-generated synthetic counterparts. A series of data augmentation experiments are carried out. First, the sparsity of the scattering centers of the targets is exploited for new target pose synthesis. Additionally, training data with various clutter backgrounds are synthesized via clutter transfer, so that the neural networks are better prepared to cope with background changes in the test samples. To effectively augment the synthetic SAR imagery in the SAMPLE dataset, a novel contrast-based data augmentation technique is proposed. To improve the robustness of neural networks against out-of-distribution (OOD) samples, the SAR images of ground military vehicles collected by the self-developed MiniSAR system are used as the training data for the adversarial outlier exposure procedure. Simulation results show that the proposed data augmentation methods are effective in improving both the target classification accuracy and the OOD detection performance. The purpose of this work is to establish the foundation for large-scale, open-field implementation of DL-based SAR-ATR systems, which is not only of great value in the sense of theoretical research, but is also potentially meaningful in the aspect of military application.

The paper presents a second part of the paper: Automatic SAR Target Recognition and Pose Estimation, in which we analyze ATR, automatic target recognition. The first part deals with pose angle determination and target data base search space reduction using a variety of geometrical methods. Both papers use US Government MSTAR, public target data base released for academic research and development. SAR target images are separated by a small pose angle (between 1° and 2°). They are obtained form an air surveillance moving plane platform at a certain depression angle and at a certain distance which all produce SAR images of 1 × 1 foot pixel resolution. The objects (targets) data base consists of 100s of commercial and military vehicles, as well as wall and building structures. We focus on three typical targets which have symmetric geometry, each of different size and shape. In addition to these three real targets we also generate several synthetic targets of various symmetric shapes to serve as the ideal test cases. The target recognition analysis is based on simple first and second order statistics including correlation and stochastic processes independence analysis. This analysis is done both in spatial as well as corresponding frequency domain. The overall methodology aims at significantly reducing computational time which is of order of fractions of a second. The end result is very accurate target type determination algorithm of the order of 97–99% precision. Once the pose angle (Part 1) and ATR (Part 2) are determined, this information can be used for target tracking. The methodologies developed in this work can be also applied to other objects, such as facial recognition. Other applications are in analysis and recognition of sound effects which may be useful to police and home land security applications.

Automatic target recognition (ATR) in hyperspectral imagery is a challenging problem due to recent advances of remote sensing instruments which have significantly improved sensor's spectral resolution. As a result, small and subtle targets can be uncovered and extracted from image scenes, which may not be identified by prior knowledge. In particular, when target size is smaller than pixel resolution, target recognition must be carried out at subpixel level. Under such circumstance, traditional spatial-based image processing techniques are generally not applicable and may not perform well if they are applied. The work presented here investigates this issue and develops spectral-based algorithms for automatic spectral target recognition (ASTR) in hyperspectral imagery with no required a priori knowledge, specifically, in reconnaissance and surveillance applications. The proposed ASTR consists of two stage processes, automatic target generation process (ATGP) followed by target classification process (TCP). The ATGP generates a set of targets from image data in an unsupervised manner which will subsequently be classified by the TCP. Depending upon how an initial target is selected in ATGP, two versions of the ASTR can be implemented, referred to as desired target detection and classification algorithm (DTDCA) and automatic target detection and classification algorithm (ATDCA). The former can be used to search for a specific target in unknown scenes while the latter can be used to detect anomalies in blind environments. In order to evaluate their performance, a comparative and quantitative study using real hyperspectral images is conducted for analysis.

While Machine Learning (ML) Automatic Target Recognition (ATR) represents the state-of-the art in target recognition, model-based ATR plays a valuable role. Model-based ATR complements machine learning ATR approaches by filling a near-term niche. While explainable Artificial Intelligence (AI) is not yet fully realized, model-based ATR serves to validate machine learning recognition decisions, and thus instills confidence in ML target calls. Alternatively, model-based ATR can act as a stand-alone ATR component, particularly in scenarios in which a small number of targets are of interest, e.g., “target-of-the-day” engagements. Model-based ATR approaches need no training data, and thus provide an alternative to machine learning approaches in the absence of sufficient quantities of real, or sufficiently high-fidelity synthetic, training data. In this paper, we present an approach to model-based ATR, called Shape-Based ATR (SB-ATR), which captures salient target shape information for recognizing targets in wide-area satellite imagery. SB-ATR finds the right blend of coarse 3-D target shape abstraction and target realism to provide robustness against target variations and environmental operating conditions, while simultaneously providing high-performance target recognition. The approach uses newer, robust forms of image correlation for matching a predicted target shape against the image. Shape prediction searches over target pose, and uses satellite metadata and solar geometry to generate realistic target shape and shadow predictions. The correlation matchers provide tolerance to illumination variations, moderate occlusions, image distortions and noise, and geometric differences between models and real targets. We present technical details of the shape-based approach, and provide numerical target recognition results on real-world satellite imagery demonstrating performance.

PurposeThe purpose of this paper is to propose a novel target automatic recognition method for unmanned aerial vehicle (UAV), which is based on backpropagation – artificial neural network (BP-ANN) algorithm, with the objective of optimizing the structure of backpropagation network, to increase the efficiency and decrease the recognition time. A hardware-in-the-loop system for UAV target automatic recognition is also developed.Design/methodology/approachThe hybrid model of BP-ANN structure is established for aircraft automatic target recognition. This proposed method identifies controller parameters and reduces the computational complexity. Approaching speed of the network is faster and recognition accuracy is higher. This kind of network combines or better fuses the advantages of backpropagation artificial neural algorithm and Hu moment. with advantages of two networks and improves the speed and accuracy of identification. Finally, a hardware-in-the-loop system for UAV target automatic recognition is also developed.FindingsThe double hidden level backpropagation artificial neural can easily increase the speed of recognition process and get a good performance for recognition accuracy.Research limitations/implicationsThe proposed backpropagation artificial neural algorithm can be ANN easily applied to practice and can help the design of the aircraft automatic target recognition system. The standard backpropagation algorithm has some obvious drawbacks, namely, converging slowly and falling into the local minimum point easily. In this paper, an improved algorithm based on the standard backpropagation algorithm is constructed to make the aircraft target recognition more practicable.Originality/valueA double hidden levels backpropagation artificial neural algorithm is presented for automatic target recognition system of UAV.