Soft Non-maximum Suppression Research Articles

An enhanced CNN-enabled learning method for promoting ship detection in maritime surveillance system

The accurate and real-time detection of moving ships has become an essential component in maritime video surveillance, leading to enhanced traffic safety and security. With the rapid development of artificial intelligence, it becomes feasible to develop intelligent techniques to promote ship detection results in maritime applications. In this work, we propose to develop an enhanced convolutional neural network (CNN) to improve ship detection under different weather conditions. To be specific, the learning and representation capacities of our network are promoted by redesigning the sizes of anchor boxes, predicting the localization uncertainties of bounding boxes, introducing the soft non-maximum suppression, and reconstructing a mixed loss function. In addition, a flexible data augmentation strategy with generating synthetically-degraded images is presented to enlarge the volume and diversity of original dataset to train learning-based ship detection methods. This strategy is capable of making our CNN-based detection results more reliable and robust under adverse weather conditions, e.g., rain, haze, and low illumination. Experimental results under different monitoring conditions demonstrate that our method significantly outperforms other competing methods (e.g., SSD, Faster R-CNN, YOLOv2 and YOLOv3) in terms of detection accuracy, robustness and efficiency. The ship detection results under poor imaging conditions have also been implemented to demonstrate the superior performance of our learning method.

YOLOv4_Drone: UAV image target detection based on an improved YOLOv4 algorithm

Advanced communications and networks have greatly improved the user experience, and unmanned aerial vehicle (UAV) are an important technology that supports people's daily life and military activities. Since target detection in UAV images is complicated by a complex background, small targets, and target occlusion, the detection accuracy of the You Only Look Once(YOLO) v4 algorithm is relatively low. Therefore, hollow convolution is used to resample the feature image to improve the feature extraction and target detection performance. In addition, the ultra-lightweight subspace attention mechanism (ULSAM) is used to derive different attention feature maps for each subspace of the feature map for multi-scale feature representation. Finally, soft non-maximum suppression (Soft-NMS) is introduced to minimize the occurrence of missed targets due to occlusion. The experimental results prove that the proposed UAV image target detection model (YOLOv4_Drone) has 5% improved to the YOLOv4 algorithm, demonstrating the effectiveness of the method.

Automatic Detection and Segmentation for Group-Housed Pigs Based on PigMS R-CNN.

Instance segmentation is an accurate and reliable method to segment adhesive pigs’ images, and is critical for providing health and welfare information on individual pigs, such as body condition score, live weight, and activity behaviors in group-housed pig environments. In this paper, a PigMS R-CNN framework based on mask scoring R-CNN (MS R-CNN) is explored to segment adhesive pig areas in group-pig images, to separate the identification and location of group-housed pigs. The PigMS R-CNN consists of three processes. First, a residual network of 101-layers, combined with the feature pyramid network (FPN), is used as a feature extraction network to obtain feature maps for input images. Then, according to these feature maps, the region candidate network generates the regions of interest (RoIs). Finally, for each RoI, we can obtain the location, classification, and segmentation results of detected pigs through the regression and category, and mask three branches from the PigMS R-CNN head network. To avoid target pigs being missed and error detections in overlapping or stuck areas of group-housed pigs, the PigMS R-CNN framework uses soft non-maximum suppression (soft-NMS) by replacing the traditional NMS to conduct post-processing selected operation of pigs. The MS R-CNN framework with traditional NMS obtains results with an F1 of 0.9228. By setting the soft-NMS threshold to 0.7 on PigMS R-CNN, detection of the target pigs achieves an F1 of 0.9374. The work explores a new instance segmentation method for adhesive group-housed pig images, which provides valuable exploration for vision-based, real-time automatic pig monitoring and welfare evaluation.

Automated detection and recognition of thyroid nodules in ultrasound images using Improve Cascade Mask R-CNN

Accurate diagnosis of thyroid nodules using ultrasonography heavily relies on the superb skills and rich experience of senior radiologists, considering the low contrast, high noise of the ultrasound image, and the diverse appearance of the nodules. Computer-aided diagnosis systems could diagnose thyroid nodules based on ultrasound characteristics to assist radiologists. However, the existing learning-based approaches for detecting and recognizing thyroid nodules have the problems of inaccurate localization and low recognition accuracy. In this study, we propose an Improved Cascade Mask R-CNN for effectively detecting and recognizing thyroid nodules. Firstly, a more effective detector is designed to better classify the ROIs and better correct the bounding boxes. Secondly, a more effective balanced L1 loss function is used to increase the gradient of the easy sample and solve the problem of imbalance between hard samples and easy samples during training. Finally, a more effective soft non-maximum suppression (Soft-NMS) method is used to set an attenuation function for adjacent bounding boxes, which solves the problem of possible missing detection in non-maximum suppression (NMS). The improved model is trained and verified by using real 1408 images collected from the known hospital. Under the localization accuracy of the IoU threshold of 0.5, the mAP reaches 87.1%, and the recognition accuracy reaches 98.67%. The experiment results show that the improved model is effective and highly valuable to help the doctors for the recognition of benign and malignant thyroid nodules.

Implementation of a Modified Faster R-CNN for Target Detection Technology of Coastal Defense Radar

The traditional method of constant false-alarm rate detection is based on the assumption of an echo statistical model. The target recognition accuracy rate and the high false-alarm rate under the background of sea clutter and other interferences are very low. Therefore, computer vision technology is widely discussed to improve the detection performance. However, the majority of studies have focused on the synthetic aperture radar because of its high resolution. For the defense radar, the detection performance is not satisfactory because of its low resolution. To this end, we herein propose a novel target detection method for the coastal defense radar based on faster region-based convolutional neural network (Faster R-CNN). The main processing steps are as follows: (1) the Faster R-CNN is selected as the sea-surface target detector because of its high target detection accuracy; (2) a modified Faster R-CNN based on the characteristics of sparsity and small target size in the data set is employed; and (3) soft non-maximum suppression is exploited to eliminate the possible overlapped detection boxes. Furthermore, detailed comparative experiments based on a real data set of coastal defense radar are performed. The mean average precision of the proposed method is improved by 10.86% compared with that of the original Faster R-CNN.

Ship Detection and Tracking in Inland Waterways Using Improved YOLOv3 and Deep SORT

Ship detection and tracking is an important task in video surveillance in inland waterways. However, ships in inland navigation are faced with accidents such as collisions. For collision avoidance, we should strengthen the monitoring of navigation and the robustness of the entire system. Hence, this paper presents ship detection and tracking of ships using the improved You Only Look Once version 3 (YOLOv3) detection algorithm and Deep Simple Online and Real-time Tracking (Deep SORT) tracking algorithm. Three improvements are made to the YOLOv3 target detection algorithm. Firstly, the Kmeans clustering algorithm is used to optimize the initial value of the anchor frame to make it more suitable for ship application scenarios. Secondly, the output classifier is modified to a single Softmax classifier to suit our ship dataset which has three ship categories and mutual exclusion. Finally, Soft Non-Maximum Suppression (Soft-NMS) is introduced to solve the deficiencies of the Non-Maximum Suppression (NMS) algorithm when screening candidate frames. Results showed the mean Average Precision (mAP) and Frame Per Second (FPS) of the improved algorithm are increased by about 5% and 2, respectively, compared with the existing YOLOv3 detecting Algorithm. Then the improved YOLOv3 is applied in Deep Sort and the performance result of Deep Sort showed that, it has greater performance in complex scenes, and is robust to interference such as occlusion and camera movement, compared to state of art algorithms such as KCF, MIL, MOSSE, TLD, and Median Flow. With this improvement, it will help in the safety of inland navigation and protection from collisions and accidents.

An Improved Algorithm for the Detection of Fastening Targets Based on Machine Vision

Object detection plays an important role in the sorting process of mechanical fasteners. Although object detection has been studied for many years, it has always been an industrial problem. Edge-based model matching is only suitable for a small range of illumination changes, and the matching accuracy is low. The optical flow method and the difference method are sensitive to noise and light, and camshift tracking is less effective in complex backgrounds. In this paper, an improved target detection method based on YOLOv3-tiny is proposed. The redundant regression box generated by the prediction network is filtered by soft nonmaximum suppression (NMS) instead of the hard decision NMS algorithm. This not only increases the size of the network structure by 52 × 52 and improves the detection accuracy of small targets but also uses the basic structure block MobileNetv2 in the feature extraction network, which enhances the feature extraction ability with the increased network layer and improves network performance. The experimental results show that the improved YOLOv3-tiny target detection algorithm improves the detection ability of bolts, nuts, screws and gaskets. The accuracy of a single type has been improved, which shows that the network greatly enhances the ability to learn objects with slightly complex features. The detection result of single shape features is slightly improved, which is higher than the recognition accuracy of other types. The average accuracy is increased from 0.813 to 0.839, an increase of two percentage points. The recall rate is increased from 0.804 to 0.821.

Intelligent Segmentation and Measurement Model for Asphalt Road Cracks Based on Modified Mask R-CNN Algorithm

Nowadays, asphalt road has dominated highways around the world. Among various defects of asphalt road, cracks have been paid more attention, since cracks often cause major engineering and personnel safety incidents. Current manual crack inspection methods are time-consuming and labor-intensive, and most segmentation methods cannot detect cracks at the pixel level. This paper proposes an intelligent segmentation and measurement model based on the modified Mask R-CNN algorithm to automatically and accurately detect asphalt road cracks. The model proposed in this paper mainly includes a convolutional neural network (CNN), an optimized region proposal network (RPN), a region of interest (RoI) Align layer, a candidate area classification network and a Mask branch of fully convolutional network (FCN). The ratio and size of anchors in the RPN are adjusted to improve the accuracy and efficiency of segmentation. Soft non-maximum suppression (Soft-NMS) algorithm is developed to improve the segmentation accuracy. A dataset including 8,689 images (512 × 512 pixels) of asphalt cracks is established and the road crack is manually marked. Transfer learning is used to initialize the model parameters in the training process. To optimize the model training parameters, multiple comparison experiments are performed, and the test results show that the mean average precision (mAP) value and F1-score of the optimal trained model are 0.952 and 0.949. Subsequently, the robustness verification test and comparative test of the trained model are conducted and the topological features of the crack are extracted. Then, the damage area, length and average width of the crack are measured automatically and accurately at pixel level. More importantly, this paper develops an automatic crack detection platform for asphalt roads to automatically extract the number, area, length and average width of cracks, which can significantly improve the crack detection efficiency for the road maintenance industry.

Intelligent Ship Detection in Remote Sensing Images Based on Multi-Layer Convolutional Feature Fusion

Intelligent detection and recognition of ships from high-resolution remote sensing images is an extraordinarily useful task in civil and military reconnaissance. It is difficult to detect ships with high precision because various disturbances are present in the sea such as clouds, mist, islands, coastlines, ripples, and so on. To solve this problem, we propose a novel ship detection network based on multi-layer convolutional feature fusion (CFF-SDN). Our ship detection network consists of three parts. Firstly, the convolutional feature extraction network is used to extract ship features of different levels. Residual connection is introduced so that the model can be designed very deeply, and it is easy to train and converge. Secondly, the proposed network fuses fine-grained features from shallow layers with semantic features from deep layers, which is beneficial for detecting ship targets with different sizes. At the same time, it is helpful to improve the localization accuracy and detection accuracy of small objects. Finally, multiple fused feature maps are used for classification and regression, which can adapt to ships of multiple scales. Since the CFF-SDN model uses a pruning strategy, the detection speed is greatly improved. In the experiment, we create a dataset for ship detection in remote sensing images (DSDR), including actual satellite images from Google Earth and aerial images from electro-optical pod. The DSDR dataset contains not only visible light images, but also infrared images. To improve the robustness to various sea scenes, images under different scales, perspectives and illumination are obtained through data augmentation or affine transformation methods. To reduce the influence of atmospheric absorption and scattering, a dark channel prior is adopted to solve atmospheric correction on the sea scenes. Moreover, soft non-maximum suppression (NMS) is introduced to increase the recall rate for densely arranged ships. In addition, better detection performance is observed in comparison with the existing models in terms of precision rate and recall rate. The experimental results show that the proposed detection model can achieve the superior performance of ship detection in optical remote sensing image.

Automatic and Robust Object Detection in X-Ray Baggage Inspection Using Deep Convolutional Neural Networks

For the purpose of ensuring public security, automatic inspection of X-ray scanners has been deployed at the entry points of many public places to detect dangerous objects. However, current surveillance systems cannot function without human supervision and intervention. In this article, we propose an effective method using deep convolutional neural networks to detect objects during X-ray baggage inspection. As a first step, a large amount of training data is generated by a specific data augmentation technique. Second, a feature enhancement module is used to improve feature extraction capabilities. Then, in order to address the foreground–background imbalance in the region proposal network, focal loss is adopted. Third, the multiscale fused region of interest is utilized to obtain more robust proposals. Finally, soft nonmaximum suppression is adopted to alleviate overlaps in baggage detection. As compared with existing algorithms, the proposed method proves that it is more accurate and robust when dealing with densely cluttered backgrounds during X-ray baggage inspection.

Bag of tricks for fabric defect detection based on Cascade R-CNN

In this paper, a bag of tricks is proposed to improve the precision of fabric defect detection. Although the general state-of-the-art convolutional neural network detection algorithm can achieve a better detection effect, in fact, the detection precision still has enough room to improve on fabric defect detection. Therefore, we propose three tricks to further improve the precision. Firstly, we use multiscale training, which scales the single input image into a number of images of different resolutions for training, so as to be able to adapt to the box distribution of different scales. Secondly, we use the dimension clusters method. By observing the distribution of the width and the height of the defect size in the fabric dataset, we find that the distribution of the defect size in the dataset is extremely unbalanced and the size span is large. We believe that the training results of the default prior boxes setting might not be optimal, so we conduct dimensional clustering for the width and height of the defect size of the dataset, so as to make the network model easier to learn. Thirdly, we use soft non-maximum suppression instead of traditional non-maximum suppression to avoid the situation that the same kinds of defect category in the dataset are overlapped and eliminated as repeated detection. With this bag of tricks, we effectively improve the precision of fabric defect detection by 8.9% mAP on the basis of the baseline of state-of-the-art convolutional neural network detection algorithm.

Convolutional neural-based algorithm for port occupancy status detection of optical distribution frames

As optical distribution frames (ODFs) carry increasing amounts of data, issues relating to their port management are becoming crucial. Image backhaul inspection has been widely accepted as an effective technology to monitor the occupancy status of ODF ports. However, the massive amount of on-site images requires a large team of off-site technologists to manually identify the occupancy status of each ODF port, which is time-consuming. We employ the you only look once version 3 (YOLOv3) network to automatically recognize ODF port occupancy. The YOLOv3 is a state-of-the-art convolutional neural network that has been shown to be very efficient for object detection in terms of processing speed and accuracy for the common objects in context. To accommodate ODF images with densely arranged small objects, high resolutions, closely spaced adjacent ports, and occlusion, we modified the original YOLOv3 with four-scale feature fusion, anchor box dimension clustering, and soft nonmaximum suppression filtering. Experiments showed a 7.38% increase in the original YOLOv3 detection accuracy rate of 91.45%. The new method can update the image backhaul inspection to automatically realize port resource management. The number of required port management technologists is considerably reduced, and the accuracy of port resources is increased, resulting in significant network investment savings.

Pedestrian detection using multi-scale squeeze-and-excitation module

Computer vision systems are major research items for autonomous vehicles. However, it is often challenging to understand the road scene, especially when objects are small and overlapping. To address these problems, this paper proposes a deep learning-based pedestrian detection method for small and overlapping objects. The proposed method adopts a parallel feature pyramid network with multi-scale feature layers, and the multi-scale squeeze-and-excitation (MSSE) module is proposed for better selection of multi-scale features. The proposed MSSE module helps to detect small objects by increasing the final feature resolution. In addition, channel-wise feature representation emphasizes important channels with reduced influence of weakly related features. Finally, the object’s proposals are regressed using soft non-maximum suppression to differentiate the overlapped objects. The experiments show significant performance enhancement with the proposed method in an ablation study.

Light-YOLOv3: fast method for detecting green mangoes in complex scenes using picking robots

When a robot picks green fruit under natural light, the color of the fruit is similar to the background; uneven lighting and fruit and leaf occlusion often affect the performance of the detection method. We take green mangoes as an experimental object. A lightweight green mangoes detection method based on YOLOv3 is proposed here. To improve the detection speed of the method, we first combine the color, texture, and shape features of green mango to design a lightweight network unit to replace the residual units in YOLOv3. Second, the improved Multiscale context aggregation (MSCA) module is used to concatenate multilayer features and make predictions, solving the problem of insufficient position information and semantic information on the prediction feature map in YOLOv3; this approach effectively improves the detection effect for the green mangoes. To address the overlap of green mangoes, soft non-maximum suppression (Soft-NMS) is used to replace non-maximum suppression (NMS), thereby reducing the missing of predicted boxes due to green mango overlaps. Finally, an auxiliary inspection green mango image enhancement algorithm (CLAHE-Mango) is proposed, is suitable for low-brightness detection environments and improves the accuracy of the green mango detection method. The experimental results show that the F1% of Light-YOLOv3 in the test set is 97.7%. To verify the performance of Light-YOLOv3 under the embedded platform, we embed one-stage methods into the Adreno 640 and Mali-G76 platforms. Compared with YOLOv3, the F1% of Light-YOLOv3 is increased by 4.5%, and the running speed is increased by 5 times, which can meet the real-time running requirements for picking robots. Through three sets of comparative experiments, we could determine that our method has the best detection results in terms of dense, backlit, direct light, night, long distance, and special angle scenes under complex lighting.

Scale-aware feature pyramid architecture for marine object detection

Marine object detection is an appealing but challengeable task in computer vision. Even though recent popular object detection algorithms perform well on common classes, they cannot acquire satisfied detection performance on marine objects because underwater images are affected by color cast and blur, and scales of the target in underwater images are usually small. These phenomena aggravate the difficulty of detection. Thus, it is urgent to design a proper structure to settle marine object detection issues. To this end, this paper proposes a novel scale-aware feature pyramid architecture named SA-FPN to extract abundant robust features on underwater images and improve the performance on marine object detection. Specifically, we design a special backbone subnetwork to improve the ability of feature extraction, which could provide richer fine-grained features for small object detection. What is more, this paper proposes a multi-scale feature pyramid to enrich the semantic features for prediction. Each feature map is enhanced by the higher level layer with context information through a top-down upsampling pathway. Through obtaining ample feature maps on underwater images, our algorithm could generate multiple bounding boxes for each target. To mitigate the reduplicative boxes and avoid miss suppression, we replace the non-maximum suppression method with soft non-maximum suppression. In this paper, we evaluate our algorithm on underwater image datasets and achieve 76.27% mAP. Meanwhile, we conduct experiments on PASCAL VOC datasets and smart unmanned vending machines datasets and get 79.13% mAP and 91.81% mAP, respectively. The experimental results reveal that our approach achieves best performance not only on marine object detection, but also on common classes.

Small-Scale Face Detection Based on Improved R-FCN

Face detection is an important basic technique for face-related applications, such as face analysis, recognition, and reconstruction. Images in unconstrained scenes may contain many small-scale faces. The features that the detector can extract from small-scale faces are limited, which will cause missed detection and greatly reduce the precision of face detection. Therefore, this study proposes a novel method to detect small-scale faces based on region-based fully convolutional network (R-FCN). First, we propose a novel R-FCN framework with the ability of feature fusion and receptive field adaptation. Second, a bottom-up feature fusion branch is established to enrich the local information of high-layer features. Third, a receptive field adaptation block (RFAB) is proposed to ensure that the receptive field can be adaptively selected to strengthen the expression ability of features. Finally, we improve the anchor setting method and adopt soft non-maximum suppression (SoftNMS) as the selection method of candidate boxes. Experimental results show that average precision for small-scale face detection of R-FCN with feature fusion branch and RFAB (RFAB-f-R-FCN) is improved by 0.8%, 2.9%, and 11% on three subsets of Wider Face compared with that of R-FCN.

Detection of tomato organs based on convolutional neural network under the overlap and occlusion backgrounds

Traditional detection methods are not sensitive to small-sized tomato organs (flowers and fruits), because the immature green tomatoes are highly similar to the background color. The overlap among fruits and the occlusion of stems and leaves on tomato organs can lead to false and missing detection, which decreases the accuracy and generalization ability of the model. Therefore, a tomato organ recognition method based on improved Feature Pyramid Network was proposed in this paper. To begin with, multi-scale feature fusion was used to fuse the detailed bottom features and high-level semantic features to detect small-sized tomato organs to improve recognition rate. And then repulsion loss was used to take place of the original smooth L1 loss function. Besides, Soft-NMS (Soft non-maximum suppression) was adopted to replace non-maximum suppression to screen the bounding boxes of tomato organs to construct a recognition model of tomato key organ. Finally, the network was trained and verified on the collected image data set. The results showed that compared with the traditional Faster R-CNN model, the performance was greatly improved (mean average precision was improved from 90.7 to 99.5%). Subsequently, the training model can be compressed so that it can be embedded into the microcontroller to develop further precise pesticide targeting application system of tomato organs and the automatic picking device.

Automatic defect detection based on improved Faster RCNN for substation equipment

Defect detection methods based on machine learning extremely accelerate the substation routine inspection process. In this paper, we propose an automatic defect detection method based on improved Faster RCNN. For one thing, random feature pyramid (RFP) structure is introduced for the highly discriminative feature map construction; for another thing, we execute the detection boxes selection by soft non-maximum suppression (SNMS), keeping the detection of defects which distribute densely. Finally, online hard example mining (OHEM) is employed to deal with the imbalance problem. Experimental results demonstrate that the proposed approach obtains competitive performance compared with state-of-the-art deep learning object detection methods.

Automated detection of defects with low semantic information in X-ray images based on deep learning

Nondestructive testing using X-ray imaging has been widely adopted in the defect detection of casting parts for quality management. Deep learning has been proved to be an effective way to detect defects in X-ray images. In this work, Feature Pyramid Network (FPN) which has been utilized broadly in many applications is adopted as our baseline. In FPN, there mainly exits two issues: firstly, down sampling operation in Convolutional Neural Network is often utilized to enhance the perception field, causing the loss of location information in feature maps, and secondly, there exists feature imbalance in feature maps and proposals. DetNet and Path Aggregation Network are adopted to solve the two shortages. To further improve the recall rate, soft Non-Maximum Suppression (soft-NMS) is adopted to remain more proposals that have high classification confidence. Defects in X-ray images of casting parts are provided with low semantic information, causing the different instances between detection results and annotations in the same area. We propose soft Intersection Over Union (soft-IOU) criterion which could evaluate several results or ground truths in the near area, making it more accurate to evaluate detection results. The experimental results demonstrate that the three proposed strategies have better performance than the baseline for our dataset.

A Specially Optimized One-Stage Network for Object Detection in Remote Sensing Images

With great significance in military and civilian applications, detecting indistinguishable small objects in wide-scale remote sensing images is still a challenging topic. In this letter, we propose a specially optimized one-stage network (SOON) focusing on extracting spatial information of high-resolution images by understanding and analyzing the combination of feature and semantic information of small objects. The SOON model consists of feature enhancement, multiscale detection, and feature fusion. The first part is implemented by constructing a receptive field enhancement (RFE) module and incorporating it into the network’s specific parts where the information of small objects mainly exists. The second part is achieved by four detectors with different sensitivities, which access to the fused and enhanced features to enable the network to make full use of features in different scales. The third part consolidates the high-level and low-level features by adopting upsampling, concatenation, and convolution operations to build a feature pyramid structure, which explicitly yields strong feature representation and semantic information. In addition, we introduce the soft-nonmaximum suppression to preserve accurate bounding boxes in the postprocessing stage for densely arranged objects. Note that the split and merge strategy and the multiscale training strategy are employed. Extensive experiments and thorough analysis are performed on the NorthWestern Polytechnical University Very-High-Resolution (NWPU VHR)-10-v2 data set and the airplane, car and ship (ACS) data set as compared with several state-of-the-art methods. The satisfactory performance in experiments verifies the effectiveness of the design and optimization.