Inshore Ship Research Articles

A method for estimating ship berthing angle based on 2D bird's eye view point cloud

The accurate estimation of berthing angle is essential for the parallel berthing of inshore ships. Herein, a new method for calculating the ship berthing angle using a shore-based LiDAR system is proposed. First, the LiDAR-scanned point cloud data is preprocessed, including coordinate conversion, noise filtering, and data downsampling. Then, the processed point cloud data is projected onto a two-dimensional (2D) plane, and the height value is used as the pixel value to generate bird's eye view (BEV) images. Finally, an improved Hough transform is proposed to detect the straight line representing the ship's nearshore profile in the BEV images and replace the ship midline with the detected line for calculating the berthing angle. The performance of the proposed method is verified by applying it on simulated and real ship data. Simulation experiment results show that the maximum absolute error of our method is only 0.151°. Real ship experiments were conducted at Dalian Port (China) to verify the reliability and stability of the method compared with other berthing angle estimation methods and the compass equipment. Overall, the proposed method can effectively obtain the berthing angle of ships, assist in parallel berthing, and improve berthing efficiency.

Closely arranged inshore ship detection using a bi-directional attention feature pyramid network

ABSTRACTThe detection of inshore ships in Synthetic Aperture Radar (SAR) images is seriously disturbed by shore buildings, especially for closely arranged inshore ships whose appearance is similar when compared with detection of deep-sea ships. There are many interference factors such as speckle noise, cross sidelobes, and defocusing in SAR images. These factors can seriously interfere with feature extraction, and the traditional Fully Convolutional One-Stage (FCOS) network often cannot effectively distinguish small-scale ships from backgrounds. Additionally, for closely arranged inshore ships, missed detections and inaccurate positioning often occur. In this paper, a method of inshore ship detection based on Bi-directional Attention Feature Pyramid Network (BAFPN) is proposed. In order to improve the detection ability of small-scale ships, the BAFPN is based on the FCOS network, which connects a Convolutional Block Attention Module (CBAM) to each feature map of the pyramid and can extract rich semantic features. Then, the idea from Path-Aggregation Network (PANet) is adopted to splice a bottom-up pyramid structure behind the original pyramid structure, further highlighting the features of different scales and improving the ability of the network to accurately locate ships under complex backgrounds, thereby avoiding missed detections in closely arranged inshore ship detection. Finally, a weighted feature fusion method is proposed, which makes the feature information extracted from the feature map have different focuses and can improve the accuracy of ship detection. Experiments on SAR image ship datasets show that the mAP for the SSDD and HRSID reached 0.902 and 0.839 respectively. The proposed method can effectively improve the ship positioning accuracy while maintaining a fast detection speed, and achieves better results for ship detection under complex background.

Inshore Ship Detection Based on Multi-Modality Saliency for Synthetic Aperture Radar Images

Synthetic aperture radar (SAR) ship detection is of significant importance in military and commercial applications. However, a high similarity in intensity and spatial distribution of scattering characteristics between the ship target and harbor facilities, along with a fuzzy sea-land boundary due to the strong speckle noise, result in a low detection accuracy and high false alarm rate for SAR ship detection with complex inshore scenes. In this paper, a new inshore ship detection method based on multi-modality saliency is proposed to overcome these challenges. Four saliency maps are established from different perspectives: an ocean-buffer saliency map (OBSM) outlining more accurate coastline under speckle noises; a local stability saliency map (LSSM) addressing pixel spatial distribution; a super-pixel saliency map (SPSM) extracting critical region-based features for inshore ship detection; and an intensity saliency map (ISM) to highlight target pixels with intensity distribution. By combining these saliency maps, ship targets in complex inshore scenes can be successfully detected. The method provides a novel interdisciplinary perspective (surface metrology) for SAR image segmentation, discovers the difference in spatial characteristics of SAR image elements, and proposes a novel robust CFAR procedure for background clutter fitting. Experiments on a public SAR ship detection dataset (SSDD) shows that our method achieves excellent detection performance, with a low false alarm rate, in offshore scenes, inshore scenes, inshore scenes with confusing metallic port facilities, and large-scale scenes. The results outperform several widely used methods, such as CFAR-based methods and super-pixel methods.

AARN: Anchor‐guided attention refinement network for inshore ship detection

AbstractInshore ship detection is an important task in several fields, for example, maritime transportation, maritime supervision, and port management. However, due to the diversified categories and locations of different ships and interference of complex surroundings, capturing discriminative characteristics of multi‐scale inshore ships for accurate detection is still challenging. Here, an anchor‐guided attention refinement network (AARN) is proposed to alleviate the problems by prominently designing an attention feature filter module (AFFM) and an anchor‐guided alignment detection module (AADM). In AFFM, of which the attention supervision generated from high‐level semantic features, is used to highlight informative target cues and suppress background interference when establishing a four‐layer feature pyramid. In AADM, the anchor‐aligned features are adopted to eventually identify potential inshore ships, which both alleviates misalignment between refined anchors and pyramidal features and improves the performance further. Extensive experiments conducted on the public Seaships7000 dataset verify the contributions of the proposed modules and the effectiveness of our method for detecting multi‐scale inshore ships in comparison to both domain‐specific and general CNN‐based methods.

Preliminary study for motion pose of inshore ships based on point cloud: Estimation of ship berthing angle

This paper presents a method to obtain inshore-ship angle based on LiDAR 3D point cloud data. When the inshore ship is berthing now, the compass equipment is usually used to get the angle information of the ship. This paper puts forward an idea of obtaining ship angle information through point cloud data by studying the characteristics of the ship hull and treating LiDAR signals when the ship is berthing. The point cloud data in the range of the ship hull was determined, and the angle information between ship and wharf was finally estimated through these point cloud data. In this paper, by establishing a 3D scene of the port, the point cloud data of ships berthing from multiple angles are simulated, and the algorithm is verified on these point cloud data. The results show that the maximum absolute error of the algorithm in this paper is 0.49°, the root-mean-square error (RMSE) is 0.27°, the mean absolute error (MAE) is 0.17°, and our method is competitive with other existing inshore ship angle estimation methods. Therefore, the method in this paper can effectively obtain the angle information of the ship when it is berthing and help the ship keep the parallel berthing as much as possible so that the berthing operation of the ship is safer.data is at: https://drive.google.com/drive/folders/1msiQdONOXWWgtX2pS_4fAe6mBPlZ4g7k?usp=sharingcode is at: https://github.com/serendipitylxd/Estimation-of-berthing-angle

Inshore Dense Ship Detection in SAR Images Based on Edge Semantic Decoupling and Transformer

Synthetic aperture radar ship detection has recently received significant attention from scholars. However, accurately distinguishing between ships is challenging due to the significant overlap between inshore ship labels. In addition, some labeled boxes contain interference information, such as land areas, which can cause false alarms and confusion in ship feature learning. To address these challenges, this article creates an edge semantic decoupling (ESD) module, adds semantic segmentation branches, and introduces the edge semantic information of ships into the training process. As a result, the model can accurately distinguish between ship targets even when significant overlap exists between inshore labeled boxes. In addition, considering that transformer has the benefit of capturing global and contextual information, this article introduces it into the detection layer to construct a transformer detection layer (TDL) to limit the interference of land and other regions within the labeled box. Experimental results from the public SAR ship detection dataset show that the proposed ESD module and TDL detection layer effectively distinguish different ship targets in the inshore dense ship area, which is less affected by interference areas, such as land in the labeled box. The average precision improves to 96.72%, and both false alarms and miss detections inshore are reduced.

A Semi-Soft Label-Guided Network With Self-Distillation for SAR Inshore Ship Detection

A Semi-Soft Label-Guided Network With Self-Distillation for SAR Inshore Ship Detection

Motion pose estimation of inshore ships based on point cloud

This paper proposes a new method to obtain the pose information estimation of the ship at multiple points based on point cloud data. Compared with the traditional pose estimation method, which regards the ship as a point to estimate the pose information of the ship, the proposed algorithm can effectively obtain the pose information of the ship from multiple points. This paper uses the spatial location of numerous key points from point cloud data by studying the spatial structure information of the ship. By tracking the spatial trajectory changes of the key points of the ship, the real-time angle, distance, velocity, and other information of the ship’s multiple points are finally estimated. The algorithm is verified by simulating the point cloud data when the inshore is berthing. The results show that the error of the proposed method is much smaller than that of other existing methods, and the proposed method can provide more accurate and abundant ship pose information. The method presented in this paper is competitive with other existing nearshore attitude estimation methods.data is at: https://drive.google.com/file/d/1LYN_tMrqtncnENzC2E5Fer3DTwL8tXH9/view?usp=sharingcode is at: https://github.com/serendipitylxd/Estimation-of-ship-pose

MEA-Net: A Lightweight SAR Ship Detection Model for Imbalanced Datasets

The existing synthetic aperture radar (SAR) ship datasets have an imbalanced number of inshore and offshore ship targets, and the number of small, medium and large ship targets differs greatly. At the same time, the existing SAR ship detection models in the application have a huge structure and require high computing resources. To solve these problems, we propose a SAR ship detection model named mask efficient adaptive network (MEA-Net), which is lightweight and high-accuracy for imbalanced datasets. Specifically, we propose the following three innovative modules. Firstly, we propose a mask data balance augmentation (MDBA) method, which solves the imbalance of sample data between inshore and offshore ship targets by combining mathematical morphological processing and ship label data to greatly improve the ability of the model to detect inshore ship targets. Secondly, we propose an efficient attention mechanism (EAM), which effectively integrates channel features and spatial features through one-dimensional convolution and two-dimensional convolution, to improve the feature extraction ability of the model for SAR ship targets. Thirdly, we propose an adaptive receptive field block (ARFB), which can achieve more effective multi-scale detection by establishing the mapping relationship between the size of the convolution kernel and the channel of feature map, to improve the detection ability of the model for ship targets of different sizes. Finally, MEA-Net is deployed on the Jeston Nano edge computing device of the 2 GB version. We conducted experimental validation on the SSDD and HRSID datasets. Compared with the baseline, the AP of MEA-Net increased by 2.18% on the SSDD dataset and 3.64% on the HRSID dataset. The FLOPs and model parameters of MEA-Net were only 2.80 G and 0.96 M, respectively. In addition, the FPS reached 6.31 on the Jeston Nano, which has broad application prospects.

Inshore Ship Detection in Large-Scale SAR Images Based on Saliency Enhancement and Bhattacharyya-like Distance

While the detection of offshore ships in synthetic aperture radar (SAR) images has been widely studied, inshore ship detection remains a challenging task. Due to the influence of speckle noise and the high similarity between onshore buildings and inshore ships, the traditional methods are unable to achieve effective detection for inshore ships. To improve the detection performance of inshore ships, we propose a novel saliency enhancement algorithm based on the difference of anisotropic pyramid (DoAP). Considering the limitations of IoU in small-target detection, we design a detection framework based on the proposed Bhattacharyya-like distance (BLD). First, the anisotropic pyramid of the SAR image is constructed by a bilateral filter (BF). Then, the differences between the finest two scales and the coarsest two scales are used to generate the saliency map, which can be used to enhance ship pixels and suppress background clutter. Finally, the BLD is used to replace IoU in label assignment and non-maximum suppression to overcome the limitations of IoU for small-target detection. We embed the DoAP into the BLD-based detection framework to detect inshore ships in large-scale SAR images. The experimental results on the LS-SSDD-v1.0 dataset indicate that the proposed method outperforms the basic state-of-the-art detection methods.

Oriented Ship Detection Based on Strong Scattering Points Network in Large-Scale SAR Images

Ship detection has broad applications in many areas, including fishery management, maritime rescue, and maritime monitoring. Recently, numerous detectors based on deep learning have been carried in ship detection in synthetic aperture radar (SAR) images. However, detecting the inshore ships faces enormous challenges because of the strong scattering interference of the inland area. In order to address such issues, a novel method named strong scattering points network for ship detection is proposed in this article. First, according to the SAR imaging mechanism, the ships usually appear strong scattering phenomenon in the SAR images. Therefore, the proposed method detects the strong scattering points on the ship and then aggregates their positions to obtain the ship’s arbitrary orientation box. Second, our method designs an embedding vector to cluster these points as an individual object to regress the oriented bounding box. Third, in order to distinguish the strong scattering points on land, a ship attention module is employed to extract the image texture features and representations of local features. It can suppress the false alarm caused by land interference in the detection process. Furthermore, to demonstrate the effectiveness of the proposed algorithm, this article introduces a new ship dataset for oriented ship detection named large-scale dataset for ship detection in SAR images (LDSD). Moreover, the public SAR ship detection dataset (SSDD) is utilized to verify the robustness and generalization ability of the detector. The experimental results on two datasets show that our method has a strong anti-interference ability in the inshore background and achieves state-of-the-art detection performance.

DSDet: A Lightweight Densely Connected Sparsely Activated Detector for Ship Target Detection in High-Resolution SAR Images

Traditional constant false alarm rate (CFAR) based ship target detection methods do not work well in complex conditions, such as multi-scale situations or inshore ship detection. With the development of deep learning techniques, methods based on convolutional neural networks (CNN) have been applied to solve such issues and have demonstrated good performance. However, compared with optical datasets, the number of samples in SAR datasets is much smaller, thus limiting the detection performance. Moreover, most state-of-the-art CNN-based ship target detectors that focus on the detection performance ignore the computation complexity. To solve these issues, this paper proposes a lightweight densely connected sparsely activated detector (DSDet) for ship target detection. First, a style embedded ship sample data augmentation network (SEA) is constructed to augment the dataset. Then, a lightweight backbone utilizing a densely connected sparsely activated network (DSNet) is constructed, which achieves a balance between the performance and the computation complexity. Furthermore, based on the proposed backbone, a low-cost one-stage anchor-free detector is presented. Extensive experiments demonstrate that the proposed data augmentation approach can create hard SAR samples artificially. Moreover, utilizing the proposed data augmentation approach is shown to effectively improves the detection accuracy. Furthermore, the conducted experiments show that the proposed detector outperforms the state-of-the-art methods with the least parameters (0.7 M) and lowest computation complexity (3.7 GFLOPs).

N-YOLO: A SAR Ship Detection Using Noise-Classifying and Complete-Target Extraction

High-resolution images provided by synthetic aperture radar (SAR) play an increasingly important role in the field of ship detection. Numerous algorithms have been so far proposed and relative competitive results have been achieved in detecting different targets. However, ship detection using SAR images is still challenging because these images are still affected by different degrees of noise while inshore ships are affected by shore image contrasts. To solve these problems, this paper introduces a ship detection method called N-YOLO, which based on You Only Look Once (YOLO). The N-YOLO includes a noise level classifier (NLC), a SAR target potential area extraction module (STPAE) and a YOLOv5-based detection module. First, NLC derives and classifies the noise level of SAR images. Secondly, the STPAE module is composed by a CA-CFAR and expansion operation, which is used to extract the complete region of potential targets. Thirdly, the YOLOv5-based detection module combines the potential target area with the original image to get a new image. To evaluate the effectiveness of the N-YOLO, experiments are conducted using a reference GaoFen-3 dataset. The detection results show that competitive performance has been achieved by N-YOLO in comparison with several CNN-based algorithms.

SAR inshore ship detection algorithm in complex background

SAR inshore ship detection algorithm in complex background

Multi-Scale Inshore Ship Detection Based on Feature Re-Focusing Network

Multi-Scale Inshore Ship Detection Based on Feature Re-Focusing Network

Balance Scene Learning Mechanism for Offshore and Inshore Ship Detection in SAR Images

Huge imbalance of different scenes’ sample numbers seriously reduces synthetic aperture radar (SAR) ship detection accuracy. Thus, to solve this problem, this letter proposes a balance scene learning mechanism (BSLM) for offshore and inshore ship detection in SAR images. BSLM involves three steps: 1) based on unsupervised representation learning, a generative adversarial network (GAN) is used to extract the scene features of SAR images; 2) using these features, a scene binary cluster (offshore/inshore) is conducted by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${K}$ </tex-math></inline-formula> -means; and 3) finally, the small cluster’s samples (inshore) are augmented via replication, rotation transformation or noise addition to balance another big cluster (offshore), so as to eliminate scene learning bias and obtain balanced learning representation ability that can enhance learning benefits and improve detection accuracy. This letter applies BSLM to four widely used and open-sourced deep learning detectors, i.e., faster regions-convolutional neural network (Faster R-CNN), Cascade R-CNN, single shot multibox detector (SSD), and RetinaNet, to verify its effectiveness. Experimental results on the open SAR ship detection data set (SSDD) reveal that BSLM can greatly improve detection accuracy, especially for more complex inshore scenes.

Ship detection based on fused features and rebuilt YOLOv3 networks in optical remote-sensing images

ABSTRACT Automatic ship detection in optical remote-sensing (ORS) images has wide applications in civil and military fields. Research on ship detection in ORS images started late compared to synthetic aperture radar (SAR) images, and it is difficult for traditional image-processing algorithms to achieve high accuracy. Therefore, we propose a ship-detection method based on a deep convolutional neural network that is modified from YOLOv3. We call it fused features and rebuilt (FFR) YOLOv3. We tried some improvements to enhance its performance in ship-detection regions. We added a squeeze-and-excitation (SE) structure to the backbone network to strengthen the ability to extract features. Through a large number of experiments, we optimized the backbone network to improve the speed. We improved the multi-scale detection of YOLOv3 by fusing multi-scale feature maps and regenerating them with a high-resolution network, which can improve the accuracy of detection and location. We used the public HRSC2016 ship-detection dataset and remote-sensing images collected from Google Earth to train, test, and verify our network, which reached a detection speed of about 27 frames per second (fps) on an NVIDIA RTX2080ti, with recall (R) = 95.32% and precision (P) = 95.62%. Experiments show that our network has better accuracy and speed than other methods. In addition, it has strong robustness and can adapt to complex environments like inshore ship detection.

Arbitrary-Oriented Inshore Ship Detection based on Multi-Scale Feature Fusion and Contextual Pooling on Rotation Region Proposals

Inshore ship detection plays an important role in many civilian and military applications. The complex land environment and the diversity of target sizes and distributions make it still challenging for us to obtain accurate detection results. In order to achieve precise localization and suppress false alarms, in this paper, we propose a framework which integrates a multi-scale feature fusion network, rotation region proposal network and contextual pooling together. Specifically, in order to describe ships of various sizes, different convolutional layers are fused to obtain multi-scale features based on the baseline feature extraction network. Then, for the purpose of accurate target localization and arbitrary-oriented ship detection, a rotation region proposal network and skew non-maximum suppression are employed. Finally, on account of the disadvantages that the employment of a rotation bounding box usually causes more false alarms, we implement inclined context feature pooling on rotation region proposals. A dataset including port images collected from Google Earth and a public ship dataset HRSC2016 are employed in our experiments to test the proposed method. Experimental results of model analysis validate the contribution of each module mentioned above, and contrast results show that our proposed pipeline is able to achieve state-of-the-art performance of arbitrary-oriented inshore ship detection.

Small Sample Set Inshore Ship Detection From VHR Optical Remote Sensing Images Based on Structured Sparse Representation

Inshore ship detection from very high resolution (VHR) optical remote sensing images has been playing a critical role in various civil and military applications. However, it brings up an important challenge, which is difficult to complete effective and robust feature extraction when valid inshore ship training sample acquired is limited, and the severe imbalance problem exists of positive and negative samples. In order to tackle the abovementioned difficulties, the structured sparse representation model (SSRM) is proposed to achieve inshore ship detection in more effectively and robustly way by circumstances of the small sample set. Here, SSRM has two steps that include inshore ship region proposal (RP) and orientation prediction (OP). Related to the RP process, the error matrix embedded in SSRM not only prevents to build the high-dimension background subdictionary and imbalance problem of positive and negative samples, but also achieves an effective intraclass robustness description of inshore ships and background. For the OP stage, the low-rank constraint of common sharing atoms in SSRM can make inshore ship direction be extracted by their sparse coding. In addition, based on RP and OP guidance, the proposed comprehensive structure voting can achieve an accurate contour detection of inshore ships. Finally, several experimental results employ that Google Earth service, HRSC 2016, and DOTA datasets proved the effectiveness of the proposed method. The results show that proposed inshore ship detection method can provide approximately 83.7% Recall and 72.3% Precision by using only over 100 positive training samples, which outperforms the state of the art methods.

Contour Refinement and EG-GHT-Based Inshore Ship Detection in Optical Remote Sensing Image

Inshore ship detection becomes challenging in high-resolution optical remote sensing image (RSI) because inshore ships are often incomplete and deformed due to the poor imaging condition and shadow of ship superstructure, and there are various interferences in harbor. A contour refinement and the improved generalized Hough transform (GHT)-based inshore ship detection scheme is proposed for RSI with complex harbor scenes. First, the suspected region of ships (SRS) is located in the entire RSI according to the line segments of ship body and docks. The contours in each SRS are then refined to repair the damaged ship head contour (SHC) using the convex set characteristics of ship head and subsequently reduce non-SHC by curvature filtering. In each refined SRS, equal frequency quantification instead of equal width quantification for R-Table construction and Gini coefficient-based decision criterion combining the number and distribution of votes are proposed to improve GHT (i.e., EG-GHT) and to extract SHCs as candidate targets. The false candidates are removed according to pixel proportion described by the structured binarization feature. Applying the border scoring strategy, the best candidates with the largest score among all the overlapped bounding boxes are selected as the final detection targets. Using the public RSIs with various cases, including turbid water, cloud occlusion, ships moored together, and ships with the different sizes, experimental results demonstrate the proposed scheme outperforms state-of-the-art contour-based methods and deep learning-based methods in terms of precision–recall rate and average precision, respectively.