Mine-like Objects Research Articles

Automatic Classification of Unexploded Ordnance (UXO) Based on Deep Learning Neural Networks (DLNNS)

Abstract This article discusses the use of a deep learning neural network (DLNN) as a tool to improve maritime safety by classifying the potential threat to shipping posed by unexploded ordnance (UXO) objects. Unexploded ordnance poses a huge threat to maritime users, which is why navies and non-governmental organisations (NGOs) around the world are using dedicated advanced technologies to counter this threat. The measures taken by navies include mine countermeasure units (MCMVs) and mine-hunting technology, which relies on the use of sonar imagery to detect and classify dangerous objects. The modern mine-hunting technique is generally divided into three stages: detection and classification, identification, and neutralisation/disposal. The detection and classification stage is usually carried out using sonar mounted on the hull of a ship or on an underwater vehicle. There is now a strong trend to intensify the use of more advanced technologies, such as synthetic aperture sonar (SAS) for high-resolution data collection. Once the sonar data has been collected, military personnel examine the images of the seabed to detect targets and classify them as mine-like objects (MILCO) or non mine-like objects (NON-MILCO). Computer-aided detection (CAD), computer-aided classification (CAC) and automatic target recognition (ATR) algorithms have been introduced to reduce the burden on the technical operator and reduce post-mission analysis time. This article describes a target classification solution using a DLNN-based approach that can significantly reduce the time required for post-mission data analysis during underwater reconnaissance operations.

A Survey of Underwater Acoustic Data Classification Methods Using Deep Learning for Shoreline Surveillance.

This paper presents a comprehensive overview of current deep-learning methods for automatic object classification of underwater sonar data for shoreline surveillance, concentrating mostly on the classification of vessels from passive sonar data and the identification of objects of interest from active sonar (such as minelike objects, human figures or debris of wrecked ships). Not only is the contribution of this work to provide a systematic description of the state of the art of this field, but also to identify five main ingredients in its current development: the application of deep-learning methods using convolutional layers alone; deep-learning methods that apply biologically inspired feature-extraction filters as a preprocessing step; classification of data from frequency and time–frequency analysis; methods using machine learning to extract features from original signals; and transfer learning methods. This paper also describes some of the most important datasets cited in the literature and discusses data-augmentation techniques. The latter are used for coping with the scarcity of annotated sonar datasets from real maritime missions.

Automatic Target Recognition for Mine Countermeasure Missions Using Forward-Looking Sonar Data

The detection of objects on the seafloor is a complex task. The domain of the detection and classification of naval mines is additionally complicated by the high risk nature of the task. Autonomous underwater vehicles (AUVs) have been used in naval mine countermeasures (MCM) operations to search large areas using sensors such as sidescan or synthetic aperture sonars. These sensors generally have a high coverage rate, while sacrificing spatial resolution. Conversely, sensors with higher resolution but lower coverage (such as forward-looking sonars and electro–optical cameras) are employed for the later classification and identification stages of the MCM mission. However, to autonomously execute a target reacquisition mission, it is important to be able to collect and process data automatically and, in near real time, onboard an AUV. For this purpose, an automatic target recognition (ATR) system is required. This article proposes an ATR, which can be used onboard an autonomous vehicle, capable of detecting mine-like objects in forward-looking sonar data. The ATR combines a detector and a classifier, based on convolutional neural network models, with a probabilistic grid map that filters out false positives and combines reported detections at nearby locations. A strategy, combining a survey pattern with target-mapping maneuvers automatically activated by the ATR, has been designed to maximize the performance of this ATR. The whole system has been tested in simulation as well as using data from previous MCM exercises, the results of which are presented here.

A Review of Underwater Mine Detection and Classification in Sonar Imagery

Underwater mines pose extreme danger for ships and submarines. Therefore, navies around the world use mine countermeasure (MCM) units to protect against them. One of the measures used by MCM units is mine hunting, which requires searching for all the mines in a suspicious area. It is generally divided into four stages: detection, classification, identification and disposal. The detection and classification steps are usually performed using a sonar mounted on a ship’s hull or on an underwater vehicle. After retrieving the sonar data, military personnel scan the seabed images to detect targets and classify them as mine-like objects (MLOs) or benign objects. To reduce the technical operator’s workload and decrease post-mission analysis time, computer-aided detection (CAD), computer-aided classification (CAC) and automated target recognition (ATR) algorithms have been introduced. This paper reviews mine detection and classification techniques used in the aforementioned systems. The author considered current and previous generation methods starting with classical image processing, and then machine learning followed by deep learning. This review can facilitate future research to introduce improved mine detection and classification algorithms.

An analytical and probabilistic model with concordance for detecting mine-like objects with mirror symmetry

PurposeThe purpose is to develop search and detection strategies that maximize the probability of detection of mine-like objects.Design/methodology/approachThe author have developed a methodology that incorporates variational calculus, number theory and algebra to derive a globally optimal strategy that maximizes the expected probability of detection.FindingsThe author found a set of look angles that globally maximize the probability of detection for a general class of mirror symmetric targets.Research limitations/implicationsThe optimal strategies only maximize the probability of detection and not the probability of identification.Practical implicationsIn the context of a search and detection operation, there is only a limited time to find the target before life is lost; hence, improving the chance of detection will in real terms be translated into the difference between success or failure, life or death. This rich field of study can be applied to mine countermeasure operations to make sure that the areas of operations are free of mines so that naval operations can be conducted safely.Originality/valueThere are two novel elements in this paper. First, the author determine the set of globally optimal look angles that maximize the probability of detection. Second, the author introduce the phenomenon of concordance between sensor images.

Environmentally adaptive automated recognition of underwater mines with synthetic aperture sonar imagery.

This work demonstrates that automated mine countermeasure (MCM) tasks are greatly facilitated by characterizing the seafloor environment in which the sensors operate as a first step within a comprehensive strategy for how to exploit information from available sensors, multiple detector types, measured features, and target classifiers, depending on the specific seabed characteristics present within the high-frequency synthetic aperture sonar (SAS) imagery used to perform MCM tasks. This approach is able to adapt as environmental characteristics change and includes the ability to recognize novel seabed types. Classifiers are then adaptively retrained through active learning in these unfamiliar seabed types, resulting in improved mitigation of challenging environmental clutter as it is encountered. Further, a segmentation constrained network algorithm is introduced to enable enhanced generalization abilities for recognizing mine-like objects from underrepresented environments within the training data. Additionally, a fusion approach is presented that allows the combination of multiple detectors, feature types spanning both measured expert features and deep learning, and an ensemble of classifiers for the particular seabed mixture proportions measured around each detected target. The environmentally adaptive approach is demonstrated to provide the best overall performance for automated mine-like object recognition.

АЛГОРИТМ МАНЕВРИРОВАНИЯ АВТОНОМНОГО НЕОБИТАЕМОГО ПОДВОДНОГО АППАРАТА ПРИ ПРОВОДКЕ СУДНА ЧЕРЕЗ ЗАМИНИРОВАННЫЙ РАЙОН

One of the tasks assigned to the autonomous underwater vehicles (AUV) is the fightagainst mine danger, which includes: 1) search for and destroy mines in the mined area;2) ensure their own safety when passing through the mined area or when working in this area;3) ensure the navigation of ships (including submarines) through the mined area. AUV can beconsidered as a further development of mine-fighting equipment, since they have a number ofadvantages over mine-fighting ships: 1) exclude the loss of life in the event of a mine explosion;2) have a lower level of physical fields to which the fuses of sea mines react; 3) are able to maneuverat the optimal depth for searching and classifying mines. In view of this, the creation ofspecialized AUV to combat the mine danger is very important. The search for mines with thehelp of AUV can be carried out either in the interests of their destruction (clearing the area) orfor the purpose of guiding ships through the mined area. This article deals with the secondproblem. The description of the maneuvering algorithm of the AUV when ensuring the passageof a vessel through a mined area is given. The task is solved by detecting different types ofmines using hydroacoustic and magnetometric means of searching and classifying underwaterobjects and bypassing of the mines at a safe distance. To reduce the time required to solve thetask, the detected underwater objects are classified into the "mine-like object" and "other objects"classes. The use of the "mine-like object" class, which includes both the actual mines andobjects that are indistinguishable from mines at the distances of their detection by means ofmine search, allows you to avoid approaching detected underwater objects at a distance of theirconfident classification using high-frequency sonar and television equipment, and thereby increasethe safety of the AUV and reduce the time of passage through the mined area. The algorithmtakes into account the fact that the minimum safe distance between vessel and mine greatlyexceeds the range of detection of mines of different types that does not allow AUV to find asafe vessel passage through the mined area by a single crossing and requires complex maneuveringof the AUV. Algorithm is intended for implementation in the AUV control system.

Visual information processing for deep-sea visual monitoring system

Due to the rising demand for minerals and metals, various deep-sea mining systems have been developed for the detection of mines and mine-like objects on the seabed. However, many of them contain some issues due to the diffusion of dangerous substances and radioactive substances in water. Therefore, efficient and accurate visual monitoring is expected by introducing artificial intelligence. Most recent deep-sea mining machines have little intelligence in visual monitoring systems. Intelligent robotics, e.g., deep learning-based edge computing for deep-sea visual monitoring systems, have not yet been established. In this paper, we propose the concept of a learning-based deep-sea visual monitoring system and use testbeds to show the efficiency of the proposed system. For example, to sense the underwater environment in real time, a large quantity of observation data, including captured images, must be transmitted from the seafloor to the ship, but large-capacity underwater communication is difficult. In this paper, we propose using deep compressed learning for real-time communication. In addition, we propose the gradient generation adversarial network (GGAN) to recover the heavily destroyed underwater images. In the application layer, wavelet-aware superresolution is used to show high-resolution images. Therefore, the development of an intelligent remote control deep-sea mining system with good convenience using deep learning applicable to deep-sea mining is expected.

Enhanced automated recognition of underwater mine-like objects through environmentally adaptive fusion of detectors, features, and classifiers

The complexity of the natural underwater environment creates a challenging arena in which to find underwater mines. In this work, we demonstrate that automated mine-like object detection tasks are greatly facilitated by a comprehensive fusion process. Our approach begins with characterization of the seafloor based on textures within synthetic aperture sonar (SAS) imagery and uses this to exploit information from the available sensors, multiple detector types, measured features, and target classifiers, to facilitate mine-like object recognition. Our approach is able to adapt as environmental characteristics change, including the ability to recognize novel seabed types. We then adaptively retrain classifiers through active learning in these novel seabed types resulting in improved mitigation of challenging environmental clutter as it is encountered, and develop a segmentation constrained network (SCN) algorithm which enables increased generalization abilities for recognizing mine-like objects in both under-represented and novel, unseen environments in available training data. Additionally, we present a fusion approach that allows us to combine multiple detectors, feature types spanning both measured expert features and deep learning, and an ensemble of classifiers, for the particular seabed mixture proportions measured around each detected target. [Work supported by the Office of Naval Research.]

The Orbiting Dubins Traveling Salesman Problem: planning inspection tours for a minehunting AUV

The Orbiting Dubins Traveling Salesman Problem (ODTSP) is to plan a minimum-time tour for a Dubins vehicle model to inspect a set of targets in the plane by orbiting each target along a circular arc. This problem arises in underwater minehunting, where targets are mine-like objects on the sea bottom that are inspected by a sonar-equipped underwater vehicle. Each orbit subtends a prescribed angle so that the target’s acoustic response is measured from a variety of target-sensor relative geometries to aid in classifying it. ODTSP tours consist of circular-arc orbits joined by Dubins paths, and the optimization problem is to partition the set of targets into orbits and determine the position, radius, direction, and vehicle entry angle of each. Algorithms are presented for the restricted case, where each orbit inspects a single target (only), and the general case, where orbits inspect multiple targets. The approach is facilitated by analytical conditions that identify admissible clusters of targets as cliques of a disk graph. The ODTSP is extended to consider path planning in the presence of a steady uniform current.

Robust A*-Search Image Segmentation Algorithm for Mine-like Objects Segmentation in SONAR Images

This paper addresses a sonar image segmentation method employing a Robust A*-Search Image Segmentation (RASIS) algorithm. RASIS is applied on Mine-Like Objects (MLO) in sonar images, where an object is defined by highlight and shadow regions, i.e. regions of high and low pixel intensities in a side-scan sonar image. RASIS uses a modified A*-Search method, which is usually used in mobile robotics for finding the shortest path where the environment map is predefined, and the start/goal locations are known. RASIS algorithm represents the image segmentation problem as a path-finding problem. Main modification concerning the original A*-Search is in the cost function that takes pixel intensities and contour curvature in order to navigate the 2D segmentation contour. The proposed method is implemented in Matlab and tested on real MLO images. MLO image dataset consist of 70 MLO images with manta mine present, and 70 MLO images with cylinder mine present. Segmentation success rate is obtained by comparing the ground truth data given by the human technician who is detecting MLOs. Measured overall success rate (highlight and shadow regions) is 91% for manta mines and 81% for cylinder mines.

MiNet: Efficient Deep Learning Automatic Target Recognition for Small Autonomous Vehicles

On-the-fly automatic target recognition (ATR) is a challenge for small autonomous vehicles performing remote sensing. Advances in deep learning have made object detection practicable on data from a variety of sensor types, and neural network-based object detector models trained on big data sets of natural images are commonly adapted to the remote sensor (RS) domain via transfer learning. However, constraints of small vehicle hardware, such as computational performance and battery power, limit capacity for running deep learning models onboard. Standard pretrained object detection models, such as YOLO and R-CNN, contain large convolutional neural networks requiring tens to hundreds of billions of floating-point operations to distinguish between many natural image object classes. Such large models may be overly complex for ATR tasks in RS data. This letter describes an efficient deep learning model, MiNet, developed to detect mine-like objects in sonar data. It was built in Keras and TensorFlow and trained entirely on real and synthetically generated sonar data using an incremental training procedure. MiNet was successfully deployed onboard small OceanServer Iver3 autonomous underwater vehicles during the REBOOT sea trial and predicted the latitude, longitude, and class of objects detected in sonar images within minutes of the completion of each mission leg.

Deep Gabor Neural Network for Automatic Detection of Mine-Like Objects in Sonar Imagery

With the advances in sonar imaging technology, sonar imagery has increasingly been used for oceanographic studies in civilian and military applications. High-resolution imaging sonars can be mounted on various survey platforms, typically autonomous underwater vehicles, which provide enhanced speed and improved data quality with long-range support. This paper addresses the automatic detection of mine-like objects using sonar images. The proposed Gabor-based detector is designed as a feature pyramid network with a small number of trainable weights. Our approach combines both semantically weak and strong features to handle mine-like objects at multiple scales effectively. For feature extraction, we introduce a parameterized Gabor layer which improves the generalization capability and computational efficiency. The steerable Gabor filtering modules are embedded within the cascaded layers to enhance the scale and orientation decomposition of images. The entire deep Gabor neural network is trained in an end-to-end manner from input sonar images with annotated mine-like objects. An extensive experimental evaluation on a real sonar dataset shows that the proposed method achieves competitive performance compared to the existing approaches.

Underwater Object Classification in Sidescan Sonar Images Using Deep Transfer Learning and Semisynthetic Training Data

Sidescan sonars are increasingly used in underwater search and rescue for drowning victims, wrecks and airplanes. Automatic object classification or detection methods can help a lot in case of long searches, where sonar operators may feel exhausted and therefore miss the possible object. However, most of the existing underwater object detection methods for sidescan sonar images are aimed at detecting mine-like objects, ignoring the classification of civilian objects, mainly due to lack of dataset. So, in this study, we focus on the multi-class classification of drowning victim, wreck, airplane, mine and seafloor in sonar images. Firstly, through a long-term accumulation, we built a real sidescan sonar image dataset named SeabedObjects-KLSG, which currently contains 385 wreck, 36 drowning victim, 62 airplane, 129 mine and 578 seafloor images. Secondly, considering the real dataset is imbalanced, we proposed a semisynthetic data generation method for producing sonar images of airplanes and drowning victims, which uses optical images as input, and combines image segmentation with intensity distribution simulation of different regions. Finally, we demonstrate that by transferring a pre-trained deep convolutional neural network (CNN), e.g. VGG19, and fine-tuning the deep CNN using 70% of the real dataset and the semisynthetic data for training, the overall accuracy on the remaining 30% of the real dataset can be eventually improved to 97.76%, which is the highest among all the methods. Our work indicates that the combination of semisynthetic data generation and deep transfer learning is an effective way to improve the accuracy of underwater object classification.

Transmission of images by unmanned underwater vehicles

As an acoustic communications medium, water is characterized by frequency dependent attenuation, short range, very low bandwidth, scattering, and multi-path. It is generally difficult to acoustically communicate even terse messages underwater much less images. For the naval mine counter-measures mission, there is value in transmitting images of mine-like objects, acquired by side-scan sonar on-board unmanned underwater vehicles, to the above-water operator for review. The contribution of this paper is a methodology and implementation, based on vector quantization, to compress and transmit snippets of side-scan sonar images from underway unmanned underwater vehicles to an operator. The work has been validated through controlled indoor tank tests and several at-sea trials. The fidelity of the received images is such that trained operators can recognize targets in the received images as well as they would have in the original images. Future work investigates machine learning to improve the compression basis and psycho-visual studies for the specialized skill of feature recognition in sonar images.

Underwater Object Recognition Using Transformable Template Matching Based on Prior Knowledge

Underwater object recognition in sonar images, such as mine detection and wreckage detection of a submerged airplane, is a very challenging task. The main difficulties include but are not limited to object rotation, confusion from false targets and complex backgrounds, and extensibility of recognition ability on diverse types of objects. In this paper, we propose an underwater object detection and recognition method using a transformable template matching approach based on prior knowledge. Specifically, we first extract features and construct a template from sonar video sequences based on the analysis of acoustic shadows and highlight regions. Then, we identify the target region in the objective image by fast saliency detection techniques based on FFT, which can significantly improve efficiency by avoiding an exhaustive global search. After affine transformation of the template according to the orientation of the target, we extract normalized gradient features and calculate the similarity between the template and the target region, which can solve various difficulties mentioned above using only one template. Experimental results demonstrate that the proposed method can well recognize different underwater objects, such as mine‐like objects and triangle‐like objects and can satisfy the demands of real‐time application.

Classification performance assessment in side scan sonar image while underwater target object recognition using random forest classifier and support vector machine

Ocean mine have been a major threat to the safety of vessels and human lives for many years. Identification of mine-like objects is a pressing need for military, and other ocean meets. In mine, countermeasures operations, sonar equipment are utilised to detect and classify mine- like objects if their sonar signatures are similar to known signatures of mines. The classification of underwater mines is an important task, for the safety of ports, harbors and the open sea. Mine detection is needed in military applications because it has been a threat to many lives and vessels. Although the task of finding mine like objects has received recent attention, very little has been published on the problem of discriminating mine-like (target) objects (MLO) and non-mine like (target) objects of similar size and shape. This paper deals with the recognition of mine like and non mine like objects. The recognition is done through robust Random Forest technique.

Detection of Underwater Targets Using a Subspace-Based Method With Learning

This paper introduces a new subspace-based detection method for multichannel (high frequency and broadband) synthetic aperture sonar (SAS) imagery. An image-dependent dictionary learning method is applied to form the appropriate dictionary matrices for representing target and nontarget image snippets. The hypothesis testing is done by forming a test statistic that relies on the residual error power ratio in representing an unknown image snippet using the target and nontarget dictionary matrices. To avoid the computational bottleneck in most dictionary learning methods, a new recursive method is introduced which does not require any matrix inversion or singular value decomposition (SVD). The proposed detection method was then implemented and benchmarked against a matched subspace detection method for detecting mine-like objects. Results are then presented on two sonar imagery data sets collected in two geographically disparate locations.

Transmission of side-scan sonar snippets from an underway unmanned underwater vehicle

An unique vector quantization compression methodology was applied to compress and encode side-scan sonar snippets of mine-like objects generated by automated target recognition tools on-board underway unmanned underwater vehicles (UUV). These compressed and encoded images were then further formed into acoustic packets. The objective was to transmit these acoustic packets, underwater, as representations of the sonar snippets (mugshots). The ability to transmit sonar snippets underwater while the UUV is underway is important as it allows the above-water operator to examine an image of mine-like objects, without recovering the UUV, for a timely decision on whether the object is actually a mine. This vector quantization method was used because of its terseness and thus it could be transmitted by WHOI underwater micromodems integrated on IVER3 UUVs. This presentation describes the algorithm, its implementation, and its initial in-water validation in local waters. This capability was also validated and demonstr...

A Brain–Computer Interface (BCI) for the Detection of Mine-Like Objects in Sidescan Sonar Imagery

Detection of mine-like objects (MLOs) in sidescan sonar imagery is a problem that affects our military in terms of safety and cost. The current process involves large amounts of time for subject matter experts to analyze sonar images searching for MLOs. The automation of the detection process has been heavily researched over the years and some of these computer vision approaches have improved dramatically, providing substantial processing speed benefits. However, the human visual system has an unmatched ability to recognize objects of interest. This paper posits a brain–computer interface (BCI) approach, that combines the complementary benefits of computer vision and human vision. The first stage of the BCI, a Haar-like feature classifier, is cascaded in to the second stage, rapid serial visual presentation (RSVP) of images chips. The RSVP paradigm maximizes throughput while allowing an electroencephalography (EEG) interest classifier to determine the human subjects' recognition of objects. In an additional proposed BCI system we add a third stage that uses a trained support vector machine (SVM) based on the Haar-like features of stage one and the EEG interest scores of stage two. We characterize and show performance improvements for subsets of these BCI systems over the computer vision and human vision capabilities alone.