Automatic Target Recognition Algorithms Research Articles

AI Explainability and Acceptance: A Case Study for Underwater Mine Hunting

In critical operational context such as Mine Warfare, Automatic Target Recognition (ATR) algorithms are still hardly accepted. The complexity of their decision-making hampers understanding of predictions despite performances approaching human expert ones. Much research has been done in Explainability Artificial Intelligence (XAI) field to avoid this “black box” effect. This field of research attempts to provide explanations for the decision-making of complex networks to promote their acceptability. Most of the explanation methods applied on image classifier networks provide heat maps. These maps highlight pixels according to their importance in decision-making. In this work, we first implement different XAI methods for the automatic classification of Synthetic Aperture Sonar (SAS) images by convolutional neural networks (CNN). These different methods are based on a post hoc approach. We study and compare the different heat maps obtained. Second, we evaluate the benefits and the usefulness of explainability in an operational framework for collaboration. To do this, different user tests are carried out with different levels of assistance, ranging from classification for an unaided operator to classification with explained ATR. These tests allow us to study whether heat maps are useful in this context. The results obtained show that the heat maps explanation has a disputed utility according to the operators. Heat map presence does not increase the quality of the classifications. On the contrary, it even increases the response time. Nevertheless, half of operators see a certain usefulness in heat maps explanation.

Automatic Classification of Unexploded Ordnance (UXO) Based on Deep Learning Neural Networks (DLNNS)

Abstract This article discusses the use of a deep learning neural network (DLNN) as a tool to improve maritime safety by classifying the potential threat to shipping posed by unexploded ordnance (UXO) objects. Unexploded ordnance poses a huge threat to maritime users, which is why navies and non-governmental organisations (NGOs) around the world are using dedicated advanced technologies to counter this threat. The measures taken by navies include mine countermeasure units (MCMVs) and mine-hunting technology, which relies on the use of sonar imagery to detect and classify dangerous objects. The modern mine-hunting technique is generally divided into three stages: detection and classification, identification, and neutralisation/disposal. The detection and classification stage is usually carried out using sonar mounted on the hull of a ship or on an underwater vehicle. There is now a strong trend to intensify the use of more advanced technologies, such as synthetic aperture sonar (SAS) for high-resolution data collection. Once the sonar data has been collected, military personnel examine the images of the seabed to detect targets and classify them as mine-like objects (MILCO) or non mine-like objects (NON-MILCO). Computer-aided detection (CAD), computer-aided classification (CAC) and automatic target recognition (ATR) algorithms have been introduced to reduce the burden on the technical operator and reduce post-mission analysis time. This article describes a target classification solution using a DLNN-based approach that can significantly reduce the time required for post-mission data analysis during underwater reconnaissance operations.

A blending method for forward-looking sonar mosaicing handling intra- and inter-frame artifacts

Forward-looking sonar (FLS) has been gaining attractions in the realm of near-bottom, close-range underwater inspections, owing to its high-resolution and rapid framerate capabilities. Although automatic target recognition (ATR) algorithms have been tentatively employed for object detection tasks, the necessity for human oversight remains vital, particularly in sensitive areas. A comprehensive FLS mosaic encapsulating all relevant information is highly desired to aid experts in managing an extensive array of perception data. Yet, prior research has often presumed that FLS operates under an ideal system configuration, assuming optimal sonar imaging setups and the availability of precise positioning data. Without these assumptions, intra-frame and inter-frame artifacts would emerge, deteriorating the quality of the resultant mosaic by obscuring essential information. In this study, we propose an innovative blending method specifically for FLS mosaicing aiming preserve important information. We first develop a long–short time sliding window (LST-SW) to rectify the local statistics of raw sonar images. These statistics then serve to create a global variance map (GVM). This step helps to emphasize the useful information contained in images in the blending phase by classifying the informative and featureless pixels, thereby enhancing the quality of final mosaic. This approach is substantiated through data collected in real-world scenarios. The results show that our method can preserve more details in FLS mosaics for human inspection purposes in practice.

A 3D U-Net Based on a Vision Transformer for Radar Semantic Segmentation.

Radar data can be presented in various forms, unlike visible data. In the field of radar target recognition, most current work involves point cloud data due to computing limitations, but this form of data lacks useful information. This paper proposes a semantic segmentation network to process high-dimensional data and enable automatic radar target recognition. Rather than relying on point cloud data, which is common in current radar automatic target recognition algorithms, the paper suggests using a radar heat map of high-dimensional data to increase the efficiency of radar data use. The radar heat map provides more complete information than point cloud data, leading to more accurate classification results. Additionally, this paper proposes a dimension collapse module based on a vision transformer for feature extraction between two modules with dimension differences during dimension changes in high-dimensional data. This module is easily extendable to other networks with high-dimensional data collapse requirements. The network's performance is verified using a real radar dataset, showing that the radar semantic segmentation network based on a vision transformer has better performance and fewer parameters compared to segmentation networks that use other dimensional collapse methods.

CycleGAN-Based SAR-Optical Image Fusion for Target Recognition

The efficiency and accuracy of target recognition in synthetic aperture radar (SAR) imagery have seen significant progress lately, stemming from the encouraging advancements of automatic target recognition (ATR) technology based on deep learning. However, the development of a deep learning-based SAR ATR algorithm still faces two critical challenges: the difficulty of feature extraction caused by the unique nature of SAR imagery and the scarcity of datasets caused by the high acquisition cost. Due to its desirable image nature and extremely low acquisition cost, the simulated optical target imagery obtained through computer simulation is considered a valuable complement to SAR imagery. In this study, a CycleGAN-based SAR and simulated optical image fusion network (SOIF-CycleGAN) is designed and demonstrated to mitigate the adverse effects of both challenges simultaneously through SAR-optical image bidirectional translation. SAR-to-optical (S2O) image translation produces artificial optical images that are high-quality and rich in details, which are used as supplementary information for SAR images to assist ATR. Conversely, optical-to-SAR (O2S) image translation generates pattern-rich artificial SAR images and provides additional training data for SAR ATR algorithms. Meanwhile, a new dataset of SAR-optical image pairs containing eight different types of aircraft has been created for training and testing SOIF-CycleGAN. By combining image-quality assessment (IQA) methods and human vision, the evaluation verified that the proposed network possesses exceptional bidirectional translation capability. Finally, the results of the S2O and O2S image translations are simultaneously integrated into a SAR ATR network, resulting in an overall accuracy improvement of 6.33%. This demonstrates the effectiveness of SAR-optical image fusion in enhancing the performance of SAR ATR.

Sparse Signal Models for Data Augmentation in Deep Learning ATR

Automatic target recognition (ATR) algorithms are used to classify a given synthetic aperture radar (SAR) image into one of the known target classes by using the information gleaned from a set of training images that are available for each class. Recently, deep learning methods have been shown to achieve state-of-the-art classification accuracy if abundant training data are available, especially if they are sampled uniformly over the classes and in their poses. In this paper, we consider the ATR problem when a limited set of training images are available. We propose a data-augmentation approach to incorporate SAR domain knowledge and improve the generalization power of a data-intensive learning algorithm, such as a convolutional neural network (CNN). The proposed data-augmentation method employs a physics-inspired limited-persistence sparse modeling approach, which capitalizes on the commonly observed characteristics of wide-angle synthetic aperture radar (SAR) imagery. Specifically, we fit over-parametrized models of scattering to limited training data, and use the estimated models to synthesize new images at poses and sub-pixel translations that are not available in the given data in order to augment the limited training data. We exploit the sparsity of the scattering centers in the spatial domain and the smoothly varying structure of the scattering coefficients in the azimuthal domain to solve the ill-posed problem of the over-parametrized model fitting. The experimental results show that, for the training on the data-starved regions, the proposed method provides significant gains in the resulting ATR algorithm’s generalization performance.

Improving SAR wind retrieval through automatic anomalous pixel detection

Satellite observations are increasingly in demand and valuable for monitoring the ocean and coastal winds in near-real-time and analyzing the data historically. However, soared up human activities, man-made constructions, and marine traffics negatively interfere and affect the reliability of observations and raise concerns about the trustworthiness of using satellite winds for many applications, such as offshore wind energy, even though Sentinel synthetic aperture radar (SAR) data is freely and easily accessible.In this paper, we presented a novel methodological framework for localization and suppression of anomalous pixels (APs), which deteriorate the confidence and accuracy of the retrieved SAR wind speeds. The backbone of the implementation relied on the constant false alarm rate (CFAR) algorithm, one of the automatic target recognition algorithms. We used a large dataset of SAR subscenes to model different sea clutter conditions after removing the potential targets. We selected the suitable distribution model that maintains a constant probability of false alarm (pfa) for the entire SAR scene. The proper pfa was optimized in various SAR scenes ranging from scenes with a considerable number of APs, such as ships and wind park clusters, to scenes with no visible APs under different wind speed conditions. The pfa optimization process prevented real wind pixels from being misidentified as APs.Applying CFAR to the logarithmic scale of raw digital numbers (log-DNs) domain with a pfa of about 3.17 × 10-5, we extracted SAR wind speeds around each ocean buoy before and after CFAR process, comparing them with their corresponding in situ wind measurements. This led to significant improvement in the quality and accuracy of SAR wind data retrieval, especially in regions with numerous man-made constructions and high marine traffics. We tested and validated our approach in the North Sea and two regions along the eastern coast of the U.S. The enclosed region, serving as a marine route for harbors in Delaware Bay, showed a substantial improvement in RMSE and BIAS after applying our approach, reducing RMSE from 2.4 m/s to 1.6 m/s and the BIAS from −1.5 m/s to −0.3 m/s. The open ocean region demonstrated stable RMSE and BIAS, indicating preservation of real wind data, as only APs affecting SAR wind speed were considered during the process.

Automatic recognition of cephalometric landmarks via multi-scale sampling strategy

The identification of head landmarks in cephalometric analysis significantly contributes in the anatomical localization of maxillofacial tissues for orthodontic and orthognathic surgery. However, the existing methods face the limitations of low accuracy and cumbersome identification process. In this pursuit, the present study proposed an automatic target recognition algorithm called Multi-Scale YOLOV3 (MS-YOLOV3) for the detection of cephalometric landmarks. It was characterized by multi-scale sampling strategies for shallow and deep features at varied resolutions, and especially contained the module of spatial pyramid pooling (SPP) for highest resolution. The proposed method was quantitatively and qualitatively compared with the classical YOLOV3 algorithm on the two data sets of public lateral cephalograms, undisclosed anterior-posterior (AP) cephalograms, respectively, for evaluating the performance. The proposed MS-YOLOV3 algorithm showed better robustness with successful detection rates (SDR) of 80.84% within 2 mm, 93.75% within 3 mm, and 98.14% within 4 mm for lateral cephalograms, and 85.75% within 2 mm, 92.87% within 3 mm, and 96.66% within 4 mm for AP cephalograms, respectively. It was concluded that the proposed model could be robustly used to label the cephalometric landmarks on both lateral and AP cephalograms for the clinical application in orthodontic and orthognathic surgery.

Target Recognition in SAR Images by Deep Learning with Training Data Augmentation

Mass production of high-quality synthetic SAR training imagery is essential for boosting the performance of deep-learning (DL)-based SAR automatic target recognition (ATR) algorithms in an open-world environment. To address this problem, we exploit both the widely used Moving and Stationary Target Acquisition and Recognition (MSTAR) SAR dataset and the Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset, which consists of selected samples from the MSTAR dataset and their computer-generated synthetic counterparts. A series of data augmentation experiments are carried out. First, the sparsity of the scattering centers of the targets is exploited for new target pose synthesis. Additionally, training data with various clutter backgrounds are synthesized via clutter transfer, so that the neural networks are better prepared to cope with background changes in the test samples. To effectively augment the synthetic SAR imagery in the SAMPLE dataset, a novel contrast-based data augmentation technique is proposed. To improve the robustness of neural networks against out-of-distribution (OOD) samples, the SAR images of ground military vehicles collected by the self-developed MiniSAR system are used as the training data for the adversarial outlier exposure procedure. Simulation results show that the proposed data augmentation methods are effective in improving both the target classification accuracy and the OOD detection performance. The purpose of this work is to establish the foundation for large-scale, open-field implementation of DL-based SAR-ATR systems, which is not only of great value in the sense of theoretical research, but is also potentially meaningful in the aspect of military application.

Transductive Prototypical Attention Reasoning Network for Few-Shot SAR Target Recognition

Deep learning-based synthetic aperture radar (SAR) automatic target recognition (ATR) algorithms have achieved outstanding performance under the condition of hundreds or thousands of training samples in recent years. Nevertheless, it is often rare to acquire great quantities of target samples in real SAR application scenarios. This article proposes a novel ATR method called transductive prototypical attention reasoning network (TPARN) to solve the problem of SAR target recognition with only a few training samples. To be specific, a region awareness-based feature extraction model is first developed, which can effectively focus on the target region of interest and suppress the background clutter by embedding direction-aware and position-sensitive information to extract more transferable knowledge. To heighten the discrimination of the sample features, a cross-feature spatial attention module is then proposed following the feature embedding model. Finally, a transductive prototype reasoning method is presented to realize the identity reasoning of the target, which can continuously update each class prototype with training samples and test samples together, thereby improving the classification accuracy. In addition, a marginal adaptive hybrid loss is proposed to obtain a discriminative feature embedding space with intra-class compactness and inter-class divergence, aiming to facilitate subsequent target identity reasoning. Extensive experiments on the moving and stationary target acquisition and recognition (MSTAR) benchmark dataset reveal that the proposed method outperforms some state-of-the-arts under different few-shot SAR ATR tasks.

Contrastive Feature Disentangling for Partial Aspect Angles SAR Noncooperative Target Recognition

Deep learning algorithms have achieved state-of-the-art progress in synthetic aperture radar (SAR) automatic target recognition (ATR) tasks. They theoretically assume that training and test samples are independent and identically distributed (i.i.d.) for generalization, but it is intractable for practical ATR scenarios. In this paper, we propose a novel contrastive feature disentangling framework termed ConFeDent to learn features with improved generalization performance under a condition of a weaker distribution consistency. More specifically, ConFeDent aims to describe the semantic interactions between two arbitrary SAR training samples instead of treating them independently. It can implicitly disentangle features encoding the pose and identity knowledge from the whole samples with a semi-parametric geometric transformation model and a second-order energy model. In particular, except for the identity label, we use deductive-based geometry knowledge as supervision to teach the model to learn the concept of aspect angle variation. A progressively amortized inference scheme is constructed for efficient feature learning and recognition in an end-to-end manner. Finally, we further release a strengthened version, called ConFeDent+, which can explicitly utilize and learn more information from cross-category samples. Experimental results on the moving and stationary target acquisition and recognition (MSTAR) benchmark demonstrate the effectiveness of our proposed models in the SAR ATR. In particular, we validate the algorithms in a more challenging scenario where the range of aspect angles for training and testing samples is permitted to be disparate. Our model can achieve much higher recognition accuracy than other SAR ATR algorithms.

Improving the automatic target recognition algorithm’s accuracy through an examination of the different time-frequency representations and data augmentation

Abstract This research focuses on an improved automatic target recognition algorithm for solving the classification challenge of ground-moving targets from pulsed-Doppler radar. First, it was studied how decision-making intervals affect the proposed algorithm. Second, the altering of the data augmentation process was investigated. Third, a consideration of the three time-frequency signal representations and finally the use of different deep learning models for the classification issues were examined. It is proven that the proposed algorithm can efficiently recognize all targets enclosed in the publicly available RadEch dataset, with 4 s of radar echoes. When the decision-making time is only 1 s, a classification probability of 99.9% was obtained, which is an improvement related to the other research studies in this area. Furthermore, when the decision-making time is reduced 16 times the classification accuracy is reduced by only 1.3%. Moreover, the proposed algorithm was successful on another dataset enclosing ground-moving targets from comparable pulsed-Doppler radar.

Imaging sonar performance estimation

Imaging sonar systems including forward looking sonars, synthetic aperture sonars, and real aperture sonars produce large quantities of data and are often connected to automated target recognition (ATR) algorithms. Performance of these algorithms degrades, sometimes significantly, with changes in environmental reverberation, seafloor sediment characteristics, and target characteristics. However, most ATR algorithms have no capability to assess their own performance, flag data that is invalid, or convey when their results should not be trusted or utilized by humans or autonomous algorithms. Performance estimation algorithms can be used in conjunction with ATR to supplement these shortcomings. Estimates of signal-to-noise ratio, image contrast, and resolution can be used to assess deviation from nominal performance capabilities, while regressing ATR performance against imagery characteristics or other through-the-sensor metrics can be used to estimate false alarm rate.

Using Classify-While-Scan (CWS) Technology to Enhance Unmanned Air Traffic Management (UTM)

Drone detection radar systems have been verified for supporting unmanned air traffic management (UTM). Here, we propose the concept of classify while scan (CWS) technology to improve the detection performance of drone detection radar systems and then to enhance UTM application. The CWS recognizes the radar data of each radar cell in the radar beam using advanced automatic target recognition (ATR) algorithm and then integrates the recognized results into the tracking unit to obtain the real-time situational awareness results of the whole surveillance area. Real X-band radar data collected in a coastal environment demonstrate significant advancement in a powerful situational awareness scenario in which birds were chasing a ship to feed on fish. CWS technology turns a drone detection radar into a sense-and-alert planform that revolutionizes UTM systems by reducing the Detection Response Time (DRT) in the detection unit.

Non-Cooperative SAR Automatic Target Recognition Based on Scattering Centers Models.

This article proposes an Automatic Target Recognition (ATR) algorithm to classify non-cooperative targets in Synthetic Aperture Radar (SAR) images. The scarcity or nonexistence of measured SAR data demands that classification algorithms rely only on synthetic data for training purposes. Based on a model represented by the set of scattering centers extracted from purely synthetic data, the proposed algorithm generates hypotheses for the set of scattering centers extracted from the target under test belonging to each class. A Goodness of Fit test is considered to verify each hypothesis, where the Likelihood Ratio Test is modified by a scattering center-weighting function common to both the model and target. Some algorithm variations are assessed for scattering center extraction and hypothesis generation and verification. The proposed solution is the first model-based classification algorithm to address the recently released Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset on a synthetic training data basis. As a result, an accuracy of in a 10-target test within a class experiment under Standard Operating Conditions (SOCs) was obtained. The algorithm was also pioneered in testing the SAMPLE dataset in Extend Operating Conditions (EOCs), assuming noise contamination and different target configurations. The proposed algorithm was shown to be robust for SNRs greater than dB.

Multiview Deep Feature Learning Network for SAR Automatic Target Recognition

Multiview synthetic aperture radar (SAR) images contain much richer information for automatic target recognition (ATR) than a single-view one. It is desirable to establish a reasonable multiview ATR scheme and design effective ATR algorithm to thoroughly learn and extract that classification information, so that superior SAR ATR performance can be achieved. Hence, a general processing framework applicable for a multiview SAR ATR pattern is first given in this paper, which can provide an effective approach to ATR system design. Then, a new ATR method using a multiview deep feature learning network is designed based on the proposed multiview ATR framework. The proposed neural network is with a multiple input parallel topology and some distinct deep feature learning modules, with which significant classification features, the intra-view and inter-view features existing in the input multiview SAR images, will be learned simultaneously and thoroughly. Therefore, the proposed multiview deep feature learning network can achieve an excellent SAR ATR performance. Experimental results have shown the superiorities of the proposed multiview SAR ATR method under various operating conditions.

Few-shot SAR automatic target recognition based on Conv-BiLSTM prototypical network

In recent studies, synthetic aperture radar (SAR) automatic target recognition (ATR) algorithms have achieved high recognition accuracy in the moving and stationary target acquisition and recognition (MSTAR) data set. However, these algorithms usually require hundreds or more training samples of each target type. In order to extract azimuth-insensitive features in a SAR ATR task with only a few training samples, a convolutional bidirectional long short-term memory (Conv-BiLSTM) network is designed as an embedding network to map the SAR images into a new feature space where the classification problem becomes easier. Based on the embedding network, a novel few-shot SAR ATR framework called Conv-BiLSTM Prototypical Network (CBLPN) is proposed. Experimental results on the MSTAR benchmark data set have illustrated that the proposed method performs well in SAR image classification with only a few training samples.

Automatic Target Recognition on Synthetic Aperture Radar Imagery: A Survey

Automatic target recognition (ATR) for military applications is one of the core processes toward enhancing intelligence and autonomously operating military platforms. Spurred by this and given that Synthetic Aperture Radar (SAR) presents several advantages over its counterpart data domains, this article surveys and assesses current SAR ATR algorithms that employ the most popular dataset for the SAR domain, namely the moving and stationary target acquisition and recognition (MSTAR) dataset. Specifically, we perform a direct comparison between current SAR ATR methods and highlight the strengths and weaknesses of each technique under both standard and extended operational conditions. Additionally, despite MSTAR being the standard SAR ATR benchmarking dataset, we also highlight its weaknesses and suggest future research directions.

Extended convolutional capsule network with application on SAR automatic target recognition

Deep learning-based algorithms have a wide range of applications in synthetic aperture radar (SAR) automatic target recognition (ATR). However, most deep learning-based ATR algorithms perform poorly under extended operation conditions (EOCs), such as large depression variation, noise corruption, limited training samples, partial occlusion, and etc. This paper presents a novel deep neural network named extended convolutional capsule network to achieve robust SAR target recognition under EOCs. The proposed method is composed of an encoder network and a decoder network. Specifically, to mitigate the impact of noise on recognition, we first adopt multiple dilated convolutions to extract multi-scale features in the encoder network. Secondly, a feature refinement module is embedded in the multi-scale channels, which aims to extract discriminative and robust features by adaptively highlighting informative features and suppressing useless ones. Finally, we deploy capsule unit-based feature pose preserving layers to retain the spatial relationship among different features in the top-level network layer of the encoder network, which is very useful for improving the recognition performance under limited training samples. The decoder network is formed by stacking transposed convolutional layers to encourage the encoder network to learn discriminative image features. Experiments on the moving and stationary target acquisition and recognition (MSTAR) data demonstrate the effectiveness of the proposed method.

Hybrid Inference Network for Few-Shot SAR Automatic Target Recognition

Synthetic aperture radar (SAR) automatic target recognition (ATR) plays an important role in SAR image interpretation. However, at least hundreds of training samples are usually required for each target type in the existing SAR ATR algorithms. In this article, a novel few-shot learning framework named hybrid inference network (HIN) is proposed to tackle the problem of SAR target recognition with only a few training samples. The recognition procedure of HIN consists of two main stages. In the first stage, an embedding network is utilized to map the SAR images into an embedding space. In the second stage, a hybrid inference strategy that combines the inductive inference and the transductive inference is adopted to classify the samples in the embedding space. In the inductive inference section, each sample is recognized independently according to a metric based on Euclidean distance. In the transductive inference section, all samples are recognized as a whole according to their manifold structures by label propagation. Finally, in the hybrid inference section, the classification result is obtained by combining the above two inference methods. To train the framework more effectively, a novel loss function named enhanced hybrid loss is proposed to constrain samples to gain better interclass separability in the embedding space. Experimental results on the moving and stationary target acquisition and recognition (MSTAR) benchmark data set illustrate that HIN performs well in few-shot SAR image classification.