Detection Subnetwork Research Articles

Context-Aware DGCN-Based Ship Formation Recognition in Remote Sensing Images

Ship detection and formation recognition in remote sensing have increasingly garnered attention. However, research remains challenging due to arbitrary orientation, dense arrangement, and the complex background of ships. To enhance the analysis of ship situations in channels, we model the ships as the key points and propose a context-aware DGCN-based ship formation recognition method. First, we develop a center point-based ship detection subnetwork, which employs depth-separable convolution to reduce parameter redundancy and combines coordinate attention with an oriented response network to generate direction-invariant feature maps. The center point of each ship is predicted by regression of the offset, target scale, and angle to realize the ship detection. Then, we adopt the spatial similarity of the ship center points to cluster the ship group, utilizing the Delaunay triangulation method to establish the topological graph structure of the ship group. Finally, we design a context-aware Dense Graph Convolutional Network (DGCN) with graph structure to achieve formation recognition. Experimental results on HRSD2016 and SGF datasets demonstrate that the proposed method can detect arbitrarily oriented ships and identify formations, attaining state-of-the-art performance.

STSN-Net: Simultaneous Tooth Segmentation and Numbering Method in Crowded Environments with Deep Learning.

Accurate tooth segmentation and numbering are the cornerstones of efficient automatic dental diagnosis and treatment. In this paper, a multitask learning architecture has been proposed for accurate tooth segmentation and numbering in panoramic X-ray images. A graph convolution network was applied for the automatic annotation of the target region, a modified convolutional neural network-based detection subnetwork (DSN) was used for tooth recognition and boundary regression, and an effective region segmentation subnetwork (RSSN) was used for region segmentation. The features extracted using RSSN and DSN were fused to optimize the quality of boundary regression, which provided impressive results for multiple evaluation metrics. Specifically, the proposed framework achieved a top F1 score of 0.9849, a top Dice metric score of 0.9629, and an mAP (IOU = 0.5) score of 0.9810. This framework holds great promise for enhancing the clinical efficiency of dentists in tooth segmentation and numbering tasks.

Adaptive clustering object detection method for UAV images under long-tailed distributions

UAV images are characterized by small targets, difficult to identify in the background image, clustering and sparse distribution of targets, etc. Many researchers have proposed the clustering target detection method (ClusDet) for UAV images. However, due to the large differences in target scales and uneven distribution of targets in UAV images, showing long-tailed distribution, the traditional ClusDet algorithm tends to truncate large and medium targets in the process of clustering; in the detection process, the fixed-threshold NMS method in the ClusDet algorithm is difficult to adaptively detect targets of different sizes, clustering and mutual occlusion. To address the above problems, this paper proposes an adaptive clustered target detection algorithm based on UAV images under long-tail distribution. The method is divided into three sub-networks: the adaptive clustering sub-network, which outputs several segmented images of small target clustering regions by extracting potential small target clustering regions in UAV aerial images; the segmentation and filling sub-network, which fills the images with disproportionate aspect ratio for the output of the adaptive clustering network to keep the size of the images within the reasonable range required by the detection network; and the detection sub-network, which detects the targets within the reasonable range required by the detection network by introducing attention mechanism, using variable threshold NMS, and training using sample balancing strategy effectively improve the detection accuracy of targets in the clustered region. Trained in VisDrone 2019 dataset, the simulation results show that the UAV image adaptive clustering target detection method based on long-tailed distribution has a large improvement in the detection accuracy of small targets, and can effectively improve the detection accuracy of the model for targets in the aggregation region, while the model has good generalization ability.

Region-Aware Deep Feature-Fused Network for Robust Facial Landmark Localization

In facial landmark localization, facial region initialization usually plays an important role in guiding the model to learn critical face features. Most facial landmark detectors assume a well-cropped face as input and may underperform in real applications if the input is unexpected. To alleviate this problem, we present a region-aware deep feature-fused network (RDFN). The RDFN consists of a region detection subnetwork and a region-wise landmark localization subnetwork to explicitly solve the input initialization problem and derive the landmark score maps, respectively. To exploit the association between tasks, we develop a cross-task feature fusion scheme to extract multi-semantic region features while trading off their importance in different dimensions via global channel attention and global spatial attention. Furthermore, we design a within-task feature fusion scheme to capture the multi-scale context and improve the gradient flow for the landmark localization subnetwork. At the inference stage, a location reweighting strategy is employed to transform the score maps into 2D landmark coordinates. Extensive experimental results demonstrate that our method has competitive performance compared to recent state-of-the-art methods, achieving NMEs of 3.28%, 1.48%, and 3.43% on the 300W, AFLW, and COFW datasets, respectively.

SFA-Net: A Selective Features Absorption Network for Object Detection in Rainy Weather Conditions.

In recent years, object detection approaches using deep convolutional neural networks (CNNs) have derived major advances in normal images. However, such success is hardly achieved with rainy images due to lack of visibility. Aiming to bridge this gap, in this article, we present a novel selective features absorption network (SFA-Net) to improve the performance of object detection not only in rainy weather conditions but also in favorable weather conditions. SFA-Net accomplishes this objective by utilizing three subnetworks, where the feature selection subnetwork is concatenated with the object detection subnetwork through the feature absorption subnetwork to form a unified model. To promote further advancement in object detection impaired by rain, we propose a large-scale rainy image dataset, named srRain, which contains both synthetic rainy images and real-world rainy images for training and testing purposes. srRain is comprised of 25 900 rainy images depicting diverse driving scenarios in the presence of rain with a total of 181 164 instances interpreting five common object categories. Experimental results display that our SFA-Net reaches the highest mean average precision (mAP) of 77.53% on a normal image set, 62.52% on a synthetic rainy image set, 37.34% on a collected natural rainy image set, and 32.86% on a published real rainy image set, surpassing current state-of-the-art object detectors and the combination of image deraining and object detection models while retaining a high speed.

Dualistic cascade convolutional neural network dedicated to fully PolSAR image ship detection

Influenced by the imaging mechanism, the occurrence of interference clutter in synthetic aperture radar (SAR) renders the identification of false alarms using detectors challenging. Polarimetric SAR has the potential to improve ship detection performance owing to its distinctive polarization characteristics. The present study proposes a dualistic cascade convolutional neural network (DCCNN) algorithm driven by polarization characteristics for ship detection with fully PolSAR data. First, the new characterizations of fully PolSAR data—Optimized SPAN (OSPAN) and 6-D polarization vector (P6), were mined and defined based on the polarization coherence matrix to introduce more intact information of targets. Then, a backbone feature extraction network with parallel dualistic cascade architecture, basic geometric feature extraction network (BGFENet), and polarization feature enhancement network (PFENet) was specifically constructed, which provided the comprehensive feature representation of targets via feature fusion. Finally, the classification and regression tasks were accomplished in the fully convolutional detection subnetwork relying on extracted multi-scale fusion feature maps, while focusing on target location regression, leading to a decrease in the cost of detection efficiency caused by redundant PFENet. In addition, the corresponding training strategy according to the special architecture of DCCNN was designed to overcome the problem of labeled fully PolSAR data insufficiency and migrate refined geometric knowledge of targets in the single-polarization SAR amplitude image. Experimental results on the established fully polarized SAR dataset show that DCCNN outperforms the competitive CNN-based target detection methods by at least 6.79% in terms of average precision. Moreover, experimental results on the typical large scenes show that DCCNN is at least 0.88% higher than the well-known conventional methods in terms of F1 score.

LaserNet: a method of laser stripe center extraction under non-ideal conditions.

The extraction of the center of a laser stripe is a key step in line-structure measurement, where noise interference and changes in the surface color of an object are the main factors affecting extraction accuracy. To obtain sub-pixel level center coordinates under such non-ideal conditions, we propose LaserNet, a novel deep learning-based algorithm, to the best of our knowledge, which consists of a laser region detection sub-network and a laser position optimization sub-network. The laser region detection sub-network is used to determine potential stripe regions, and the laser position optimization sub-network uses the local image of these regions to obtain the accurate center position of the laser stripe. The experimental results show that LaserNet can eliminate noise interference, handle color changes, and give accurate results under non-ideal conditions. The three-dimensional reconstruction experiments further demonstrate the effectiveness of the proposed method.

3FL-Net: An Efficient Approach for Improving Performance of Lightweight Detectors in Rainy Weather Conditions

Numerous lightweight detection models have been presented in recent years, yet these detectors are inclined to develop for operating under normal weather conditions without adequate studies for rainy conditions. This is one of the causes leads drastically performance degradation of object detectors due to the decrease in visibility. To address above insufficiency, we propose a new and effective approach, named 3FL-Net, to elevate the performance of lightweight object detectors in the presence of rain. Our approach fulfills the goal by closely incorporating four subnetworks, namely feature enhancement subnetwork, feature extraction subnetwork, feature adaptation subnetwork, and lightweight detection subnetwork. The lightweight detection subnetwork achieved the accuracy improvement by learning diverse features from the feature enhancement subnetwork and feature extraction subnetwork via the feature adaptation subnetwork. To further drive the development in object detection induced by rain, we introduce a large-scale driving dataset, called iRain. The full iRain consists of 17,950 real-world rain images, which covers most of the driving scenarios and 85,081 instances explaining five prevalent object classes. Experiment results on divergent rain datasets expose that our 3FL-Net considerably improves the performance of lightweight detectors and surpasses that of the combination models between rain removal and object detection methods.

Anchor-free Proposal Generation Network for Efficient Object Detection

Deep learning object detection methods are usually based on anchor-free or anchor-based scheme for extracting object proposals and one-stage or two-stage structure for producing final predictions. As each scheme or structure has its own strength and weakness, combining their strength in a unified framework is an interesting research topic. However, this topic has not attracted much attention in recent years. This paper presents a two-stage object detection method that utilizes an anchor-free scheme for generating object proposals in the initial stage. For proposal generation, this paper employs an efficient anchor-free network for predicting object corners and assigns object proposals based on detected corners. For object prediction, an efficient detection network is designed to enhance both detection accuracy and speed. The detection network includes a lightweight binary classification subnetwork for removing most false positive object candidates and a light-head detection subnetwork for generating final predictions. Experimental results on the MS-COCO dataset demonstrate that the proposed method outperforms both anchor-free and two-stage object detection baselines in terms of detection performance.

A Pseudoinverse Siamese Convolutional Neural Network of Transformation Invariance Feature Detection and Description for a SLAM System

Simultaneous localization and mapping (SLAM) systems play an important role in the field of automated robotics and artificial intelligence. Feature detection and matching are crucial aspects affecting the overall accuracy of the SLAM system. However, the accuracy of the position and matching cannot be guaranteed when confronted with a cross-view angle, illumination, texture, etc. Moreover, deep learning methods are very sensitive to perspective change and do not have the invariance of geometric transformation. Therefore, a novel pseudo-Siamese convolutional network of a transformation invariance feature detection and a description for the SLAM system is proposed in this paper. The proposed method, by learning transformation invariance features and descriptors, simultaneously improves the front-end landmark detection and tracking module of the SLAM system. We converted the input image to the transform field; the backbone network was designed to extract feature maps. Then, the feature detection subnetwork and feature description subnetwork were decomposed and designed; finally, we constructed a convolutional network of transformation invariance feature detections and a description for the visual SLAM system. We implemented many experiments in datasets, and the results of the experiments demonstrated that our method has a state-of-the-art performance in global tracking when compared to that of the traditional visual SLAM systems.

Automated detection and quantification of contact behaviour in pigs using deep learning

Change in the frequency of contact between pigs within a group may be indicative of a change in the physiological or health status of one or more pigs within a group, or indicative of the occurrence of abnormal behaviour, e.g. tail-biting. Here, we developed a novel framework that detects and quantifies the frequency of interaction, i.e., a pig head to another pig rear, between pigs in groups. The method does not require individual pig tracking/identification and uses only inexpensive camera-based data capturing infrastructure. We modified the architecture of well-established deep learning models and further developed a lightweight processing stage that scans over pigs to score said interactions. This included the addition of a detection subnetwork to a selected layer of the base residual network. We first validated the automated system to score the interactions between individual pigs within a group, and determined an average accuracy of 92.65% ± 3.74%, under a variety of settings, e.g., management set-ups and data capturing. We then applied the method to a significant welfare challenge in pigs, that of the detection of tail-biting outbreaks in pigs and quantified the changes that happen in contact behaviour during such an outbreak. Our study shows that the system is able to accurately monitor pig interactions under challenging farming conditions, without the need for additional sensors or a pig tracking stage. The method has a number of potential applications to the field of precision livestock farming of pigs that may transform the industry. • An automated quantification method for pig social interactions was developed. • The automated method is based on convolutional deep neural networks. • Detection of pig interactions, such as head-to-rear is highly accurate (92.6%). • Robust method performance was achieved under different management settings. • The method detected changes in the interactions during a tail biting outbreak.

QDL-CMFD: A Quality-independent and deep Learning-based Copy-Move image forgery detection method

One of the prevalent methods of image forgery is copy-move, where one or more regions of an image are duplicated and moved elsewhere in the image. It is usually difficult to detect this type of forgery due to the similarity of the copied and forged areas. Also, forgers perform pre-processing and/or post-processing operations on the manipulated regions to make it even more difficult to detect. In this study, an image quality-independent method based on deep learning approach, termed QDL-CMFD, is presented for detecting this type of forgery. QDL-CMFD utilizes generative adversarial networks for image quality enhancement, and convolutional neural networks (CNN) for forgery detection. A tailored dual-branch CNN architecture is introduced consisting of two subnetworks, namely a manipulation detection subnetwork and a similarity detection subnetwork. Accordingly, unlike most existing methods, QDL-CMFD is able to simultaneous detection of several forged areas, as well as determining the source and target of the forgery. Also, QDL-CMFD is robust against various pre-processing/post-processing attacks. It shows excellent performance for detecting low-quality forged images and small areas. Experiments conducted on the CASIA and CoMoFoD benchmark datasets confirm that QDL-CMFD performs significantly better than the competitors. All the implementation source codes of QDL-CMFD are available at https://github.com/MehradAria/QDL-CMFD.

Salient object detection network with center pooling and distance-weighted affinity loss function

Due to the rapid development of deep learning, salient target detection models have achieved significant progress. However, these models ignore the center prior in the collected images, that is, there is a greater probability of being in the center area for the salient object. The hidden information in the image is not fully utilized, which hinders further improvement of the detection result. We propose a center pooling algorithm that realizes the combination of image center prior and deep learning. The algorithm dynamically modifies the size of the receptive field according to the probability of the existence of salient objects and solves the problem of ignoring the importance of different regions when using the same pooling receptive field in the entire image by the standard pooling method. At the same time, the proportion of the foreground in the image is increased, and the problem of category imbalance is alleviated. In addition, a distance-weighted affinity loss function is also proposed to supplement the keyspace information by learning the grouping or separating force between pixel pairs at different distances and increase the confidence of edge regions. To further improve the detection results, a background detection subnetwork is introduced. Through the interactive learning of the foreground and background subnetworks, the opposite relationship between the predicted maps is explored. Finally, experiments on six different related datasets are conducted, and comparisons with other models prove that our model achieves the best performance.

Incremental learning for detection in X-ray luggage perspective images.

Convolutional neural networks have achieved remarkable results in the detection of X-ray luggage contraband. However, with an increase in contraband classes and substantial artificial transformation, the offline network training method has been unable to accurately detect the rapidly growing new classes of contraband. The current model cannot incrementally learn the newly appearing classes in real time without retraining the model. When the quantity of different types of contraband is not evenly distributed in the real-time detection process, the convolution neural network that is optimized by the gradient descent method will produce catastrophic forgetting, which means learning new knowledge and forgetting old knowledge, and the detection effect on the old classes will suddenly decline. To overcome this problem, this paper proposes an incremental learning method for online continuous learning of models and incrementally learns and detects new classes in the absence of old classes in the new classes. First, we perform parameter compression on the original network by distillation to ensure stable identification of the old classes. Second, the area proposal subnetwork and object detection subnetwork are incrementally learned to obtain the recognition ability of the new classes. In addition, this paper designs a new loss function, which causes the network to avoid catastrophic forgetting and stably detect the object of the new contraband classes. To reliably verify the model, this paper produces a multi-angle dataset for security perspective images. A total of 10 classes of contraband are tested, and the interference between two object detections is analyzed by model parameters. The experimental results show that the model can stably perform new contraband object learning even when there is an uneven distribution of data types.

Non-Model-Based Monocular Pose Estimation Network for Uncooperative Spacecraft Using Convolutional Neural Network

The pose estimation of uncooperative target spacecraft is a key technique in on-orbit servicing missions, among which the method based on monocular camera with low mass and power requirements has attracted widespread attention. However, monocular pose estimation methods mostly rely on the known 3D model of the target spacecraft, and non-model-based methods have low accuracy and even output the results when there is no target spacecraft in the image. In this paper, a non-model-based monocular pose estimation network for uncooperative spacecraft based on the convolutional neural network is proposed. This network uses three sub-networks to solve the problems of pose estimation and object detection. The first sub-network, called the attitude prediction sub-network, is used to predict the relative attitude of the target spacecraft by soft classification and error quaternion regression. The second sub-network, called the position regression sub-network, is used to predict the relative position of the target spacecraft by regression. The third sub-network called the object detection sub-network is used to detect the target spacecraft to determine whether the predicted pose needs to be output. The experimental results of the pose estimation of two public spacecraft demonstrate that the proposed method can effectively detect the target spacecraft and achieve better pose estimation accuracy than previous non-model-based methods.

Leaf Spot Attention Networks Based on Spot Feature Encoding for Leaf Disease Identification and Detection

This study proposes a new attention-enhanced YOLO model that incorporates a leaf spot attention mechanism based on regions-of-interest (ROI) feature extraction into the YOLO framework for leaf disease detection. Inspired by a previous study, which revealed that leaf spot attention based on the ROI-aware feature extraction can improve leaf disease recognition accuracy significantly and outperform state-of-the-art deep learning models, this study extends the leaf spot attention model to leaf disease detection. The primary idea is that spot areas indicating leaf diseases appear only in leaves, whereas the background area does not contain useful information regarding leaf diseases. To increase the discriminative power of the feature extractor that is required in the object detection framework, it is essential to extract informative and discriminative features from the spot and leaf areas. To realize this, a new ROI-aware feature extractor, that is, a spot feature extractor was designed. To divide the leaf image into spot, leaf, and background areas, the leaf segmentation module was first pretrained, and then spot feature encoding was applied to encode spot information. Next, the ROI-aware feature extractor was connected to an ROI-aware feature fusion layer to model the leaf spot attention mechanism, and to be joined with the YOLO detection subnetwork. The experimental results confirm that the proposed ROI-aware feature extractor can improve leaf disease detection by boosting the discriminative power of the spot features. In addition, the proposed attention-enhanced YOLO model outperforms conventional state-of-the-art object detection models.

Deep learning-based pixel-level rock fragment recognition during tunnel excavation using instance segmentation model

Timely recognition of rock fragments and their morphological sizes can allow adjustment of excavation parameters during tunnel boring machine (TBM) tunnelling. Traditional manual inspection strongly relies on the experience and subjective determination of the human operators, and sieving tests cannot be conducted in real time and are energy-consuming. Rock fragments in real-world images are frequently observed against a dark background, distributed with a high size diversity, and blocked by each other. A novel deep learning-based method is proposed in this paper to achieve real-time on-site rock fragment recognition. The proposed instance segmentation model comprises two subnetworks: object detection and semantic segmentation. The object detection subnetwork is designed based on a modified single-shot detector architecture, and multilevel feature fusion, prior anchors, and self-attention modules are utilised to localise rock fragment regions. The semantic segmentation subnetwork is designed based on U-net. Down-sampling stages use the structures of the object detection subnetwork to share the extracted features of the rock fragments, and up-sampling stages employ skip connection and self-attention modules to accomplish binary segmentation in each detected bounding box. A total of 50 original images with a resolution of 4096 × 3072 were collected: 35 for training and 15 for testing. The results showed that 88.8% of the rock fragments can be recognised and that the average recall and average intersection-over-union values reach 0.87 and 0.76, respectively. Small rock fragments inevitably missed in the labelling process and extremely large ones can also be recognised. The predicted size distributions of the rock fragments fit well with the ground truth ones. Ablation experiments were conducted to further demonstrate the effectiveness of the proposed method. This study presents both visual recognition and statistical results of the size distribution of rock fragments during TBM tunnelling, which can assist in the prediction of rock properties and the adjustment of excavation parameters.

Dermo-DOCTOR: A framework for concurrent skin lesion detection and recognition using a deep convolutional neural network with end-to-end dual encoders

Background and objectiveAutomated skin lesion analysis for simultaneous detection and recognition is still challenging for inter-class homogeneity and intra-class heterogeneity, leading to low generic capability of a single convolutional neural network (CNN) with limited datasets. MethodsThis article proposes an end-to-end deep CNN-based framework for simultaneous detection and recognition of the skin lesions, named Dermo-DOCTOR, consisting of two encoders. The feature maps from two encoders are fused channel-wise, called Fused Feature Map (FFM). The FFM is utilized for decoding in the detection sub-network, concatenating each stage of two encoders’ outputs with corresponding decoder layers to retrieve the lost spatial information due to pooling in the encoders. For the recognition sub-network, the outputs of three fully connected layers, utilizing feature maps of two encoders and FFM, are aggregated to obtain a final lesion class. We train and evaluate the proposed Dermo-Doctor utilizing two publicly available benchmark datasets, such as ISIC-2016 and ISIC-2017. ResultsThe achieved segmentation results exhibit mean intersection over unions of 85.0% and 80.0% respectively for ISIC-2016 and ISIC-2017 test datasets. The proposed Dermo-DOCTOR also demonstrates praiseworthy success in lesion recognition, providing the areas under the receiver operating characteristic curves of 0.98 and 0.91 respectively for those two datasets. The experimental results show that the proposed Dermo-DOCTOR outperforms the alternative methods mentioned in the literature, designed for skin lesion detection and recognition. ConclusionAs the Dermo-DOCTOR provides better results on two different test datasets, even with limited training data, it can be an auspicious computer-aided assistive tool for dermatologists.

Visible light polarization image desmogging via Cycle Convolutional Neural Network

Visible light polarization image desmogging aims to recover the clear image solely from an input visible light polarization image with smog. It is a challenging research topic due to desmogging is an ill-posed problem. Most of existing methods effort to build desmogging physical model based on classical atmospheric scattering principle. However, compared with natural image dehazing task, the smog in visible light polarization image is very complex and diverse, which makes it hard to find accurate desmogging physical model. To address this issue, we propose a Cycle Convolutional Neural Network (CCNN), which exploits polarization characteristics explicitly to learn a visible light polarization image desmogging model in an end-to-end way. To be specific, the model computes polarization information of the visible light polarization image via Stokes equation. Object detection sub-network is utilized to detect smog regions according to the polarization information. Then an encoder–decoder sub-network with feature converter structure is proposed to generate smog-free regions. The coarse clear image is obtained by fusing the generated smog-free regions with original smog visible light polarization image. More importantly, to obtain the final clear image, the coarse clear image is considered as the input data to our model again, which makes our model in a cycle topology. Moreover, we contribute the first large-scale dataset for visible light polarization image desmogging evaluation, which contains 17,216 visible light polarization images, to validate our proposed method. On this dataset, extensive experiments demonstrate that our method can achieve the best performance in comparison with the state-of-the-art baselines.

Ship detection from scratch in Synthetic Aperture Radar (SAR) images

ABSTRACT Ship detection in Synthetic Aperture Radar (SAR) images has always been a hot topic for research. The development of Deep Neural Networks (DNNs) has strongly promoted the development of computer vision. DNNs are also increasingly applied to SAR ship detection. However, SAR ship detection still faces the following problems: (i) The network used for detection needs to be pre-trained on ImageNet, but there is a large bias between SAR images and ImageNet, which leads to training bias. (ii) The sizes of ship targets vary greatly, and many DNNs do not perform well on multi-scale and small-size SAR ship detection. Therefore, we have designed a SAR ship detector that does not require pre-training. We use DetNet as the backbone network, adopting stacked convolution instead of down-sampling to solve the problem of small object detection and adopt a feature reuse strategy to improve parameter efficiency. In addition, we introduce several branches in the proposal sub-network to provide multi-scale object detection. In the detection sub-network, we use position-sensitive region of interest pooling to improve the prediction accuracy. Experiments on SAR ship dataset prove that our method performs better than some pre-trained networks for small ship detection and complex background ship detection.