Faster R-CNN Detector Research Articles

A Data Augmentation Approach to Distracted Driving Detection

Distracted driving behavior has become a leading cause of vehicle crashes. This paper proposes a data augmentation method for distracted driving detection based on the driving operation area. First, the class activation mapping method is used to show the key feature areas of driving behavior analysis, and then the driving operation areas are detected by the faster R-CNN detection model for data augmentation. Finally, the convolutional neural network classification mode is implemented and evaluated to detect the original dataset and the driving operation area dataset. The classification result achieves a 96.97% accuracy using the distracted driving dataset. The results show the necessity of driving operation area extraction in the preprocessing stage, which can effectively remove the redundant information in the images to get a higher classification accuracy rate. The method of this research can be used to detect drivers in actual application scenarios to identify dangerous driving behaviors, which helps to give early warning of unsafe driving behaviors and avoid accidents.

Scale Adaptive Feature Pyramid Networks for 2D Object Detection

Object detection is one of the core tasks in computer vision. Object detection algorithms often have difficulty detecting objects with diverse scales, especially those with smaller scales. To cope with this issue, Lin et al. proposed feature pyramid networks (FPNs), which aim for a feature pyramid with higher semantic content at every scale level. The FPN consists of a bottom-up pyramid and a top-down pyramid. The bottom-up pyramid is induced by a convolutional neural network as its layers of feature maps. The top-down pyramid is formed by progressive up-sampling of a highly semantic yet low-resolution feature map at the top of the bottom-up pyramid. At each up-sampling step, feature maps of the bottom-up pyramid are fused with the top-down pyramid to produce highly semantic yet high-resolution feature maps in the top-down pyramid. Despite significant improvement, the FPN still misses small-scale objects. To further improve the detection of small-scale objects, this paper proposes scale adaptive feature pyramid networks (SAFPNs). The SAFPN employs weights chosen adaptively to each input image in fusing feature maps of the bottom-up pyramid and top-down pyramid. Scale adaptive weights are computed by using a scale attention module built into the feature map fusion computation. The scale attention module is trained end-to-end to adapt to the scale of objects contained in images of the training dataset. Experimental evaluation, using both the 2-stage detector faster R-CNN and 1-stage detector RetinaNet, demonstrated the proposed approach’s effectiveness.

Training data enhancements for improving colonic polyp detection using deep convolutional neural networks

BackgroundOver the last years, the most relevant results in the context of polyp detection were achieved through deep learning techniques. However, the most common obstacles in this field are the small datasets with a reduced number of samples and the lack of data variability. This paper describes a method to reduce this limitation and improve polyp detection results using publicly available colonoscopic datasets. MethodsTo address this issue, we increased the number and variety of images from the original dataset. Our method consists on adding polyps to the dataset images. The developed algorithm performs a rigorous selection of the best region within the image to receive the polyp. This procedure preserves the realistic features of the images while creating more diverse samples for training purposes. Our method allows copying existing polyps to new non-polypoid target regions. We also develop a strategy to generate new and more varied polyps through generative adversarial neural networks. Hence, the developed approach enriches the training data, creating automatically new samples with their appropriate labels. ResultsWe applied the proposed data enhancement over a colonic polyp dataset. Thus, we can assess the effectiveness of our approach through a Faster R-CNN detection model. Performance results show improvements over the polyp detections while reducing the false-negative rate. The experimental results also show better recall metrics in comparison with both the original training set and other studies in the literature. ConclusionWe demonstrate that our proposed method has the potential to increase the data variability and number of samples in a reduced polyp dataset, improving the polyp detection rate and recall values. These results open new possibilities for advancing the study and implementation of new methods to improve computer-assisted medical image analysis.

A Unified Multi-Task Learning Architecture for Fast and Accurate Pedestrian Detection

We present a unified multi-task learning architecture for fast and accurate pedestrian detection. Different from existing methods which often focus on either a new loss function or architecture, we propose an improved multi-task convolutional neural network learning architecture to effectively and efficiently interfuse the task of pedestrian detection and semantic segmentation. To achieve this, we integrate a lightweight semantic segmentation branch to Faster R-CNN detection framework that enables end-to-end hard parameter sharing in order to boost the detection performance and maintain computational efficiency as follows. Firstly, a Semantic Segmentation to Feature Module (SS2FM) refines the convolutional features in RPN stage by integrating the features generated from the semantic segmentation branch. Secondly, a Semantic Segmentation to Confidence Module (SS2CM) refines the classification confidence in RPN stage by fusing it with the semantic segmentation confidence. We also introduce an effective anchor matching point transform to alleviate the problem of feature misalignment for heavily occluded pedestrians. The proposed unified multi-task learning architecture lends itself well to more robust pedestrian detection in diverse scenarios with negligible computation overhead. In addition, the proposed architecture can achieve high detection performance with low resolution input images, which significantly reduces the computational complexity. Experiment results on CityPersons and Caltech datasets show that our method is the fastest among all state-of-the-art pedestrian detection methods while exhibiting competitive detection performance.

Establishment and Applicability of a Diagnostic System for Advanced Gastric Cancer T Staging Based on a Faster Region-Based Convolutional Neural Network

Background: The accurate prediction of the tumor infiltration depth in the gastric wall based on enhanced CT images of gastric cancer is crucial for screening gastric cancer diseases and formulating treatment plans. Convolutional neural networks perform well in image segmentation. In this study, a convolutional neural network was used to construct a framework for automatic tumor recognition based on enhanced CT images of gastric cancer for the identification of lesion areas and the analysis and prediction of T staging of gastric cancer.Methods: Enhanced CT venous phase images of 225 patients with advanced gastric cancer from January 2017 to June 2018 were retrospectively collected. Ftable LabelImg software was used to identify the cancerous areas consistent with the postoperative pathological T stage. The training set images were enhanced to train the Faster RCNN detection model. Finally, the accuracy, specificity, recall rate, F1 index, ROC curve, and AUC were used to quantify the classification performance of T staging on this system.Results: The AUC of the Faster RCNN operating system was 0.93, and the recognition accuracies for T2, T3, and T4 were 90, 93, and 95%, respectively. The time required to automatically recognize a single image was 0.2 s, while the interpretation time of an imaging expert was ~10 s.Conclusion: In enhanced CT images of gastric cancer before treatment, the application of Faster RCNN to diagnosis the T stage of gastric cancer has high accuracy and feasibility.

A Deep-Learning-Based Real-Time Detector for Grape Leaf Diseases Using Improved Convolutional Neural Networks.

Black rot, Black measles, Leaf blight and Mites of grape are four common grape leaf diseases that seriously affect grape yield. However, the existing research lacks a real-time detecting method for grape leaf diseases, which cannot guarantee the healthy growth of grape plants. In this article, a real-time detector for grape leaf diseases based on improved deep convolutional neural networks is proposed. This article first expands the grape leaf disease images through digital image processing technology, constructing the grape leaf disease dataset (GLDD). Based on GLDD and the Faster R-CNN detection algorithm, a deep-learning-based Faster DR-IACNN model with higher feature extraction capability is presented for detecting grape leaf diseases by introducing the Inception-v1 module, Inception-ResNet-v2 module and SE-blocks. The experimental results show that the detection model Faster DR-IACNN achieves a precision of 81.1% mAP on GLDD, and the detection speed reaches 15.01 FPS. This research indicates that the real-time detector Faster DR-IACNN based on deep learning provides a feasible solution for the diagnosis of grape leaf diseases and provides guidance for the detection of other plant diseases.

Convolutional neural network–based person tracking using overhead views

In video surveillance, person tracking is considered as challenging task. Numerous computer vision, machine and deep learning–based techniques have been developed in recent years. Majority of these techniques are based on frontal view images/video sequences. The advancement of convolutional neural network reforms the way of object tracking. The network layers of convolutional neural network models trained on a number of images or video sequences improve speed and accuracy of object tracking. In this work, the generalization performance of existing pre-trained deep learning models have investigated for overhead view person detection and tracking, under different experimental conditions. The object tracking method Generic Object Tracking Using Regression Networks (GOTURN) which has been yielding outstanding tracking results in recent years is explored for person tracking using overhead views. This work mainly focused on overhead view person tracking using Faster region convolutional neural network (Faster-RCNN) in combination with GOTURN architecture. In this way, the person is first identified in overhead view video sequences and then tracked using a GOTURN tracking algorithm. Faster-RCNN detection model achieved the true detection rate ranging from 90% to 93% with a minimum false detection rate up to 0.5%. The GOTURN tracking algorithm achieved similar results with the success rate ranging from 90% to 94%. Finally, the discussion is made on output results along with future direction.

YOLO-face: a real-time face detector

Face detection is one of the important tasks of object detection. Typically detection is the first stage of pattern recognition and identity authentication. In recent years, deep learning-based algorithms in object detection have grown rapidly. These algorithms can be generally divided into two categories, i.e., two-stage detector like Faster R-CNN and one-stage detector like YOLO. Although YOLO and its varieties are not so good as two-stage detectors in terms of accuracy, they outperform the counterparts by a large margin in speed. YOLO performs well when facing normal size objects, but is incapable of detecting small objects. The accuracy decreases notably when dealing with objects that have large-scale changing like faces. Aimed to solve the detection problem of varying face scales, we propose a face detector named YOLO-face based on YOLOv3 to improve the performance for face detection. The present approach includes using anchor boxes more appropriate for face detection and a more precise regression loss function. The improved detector significantly increased accuracy while remaining fast detection speed. Experiments on the WIDER FACE and the FDDB datasets show that our improved algorithm outperforms YOLO and its varieties.

Passion fruit detection and counting based on multiple scale faster R-CNN using RGB-D images

The accurate and reliable fruit detection in orchards is one of the most crucial tasks for supporting higher level agriculture tasks such as yield mapping and robotic harvesting. However, detecting and counting small fruit is a very challenging task under variable lighting conditions, low-resolutions and heavy occlusion by neighboring fruits or foliage. To robustly detect small fruits, an improved method is proposed based on multiple scale faster region-based convolutional neural networks (MS-FRCNN) approach using the color and depth images acquired with an RGB-D camera. The architecture of MS-FRCNN is improved to detect lower-level features by incorporating feature maps from shallower convolution feature maps for regions of interest (ROI) pooling. The detection framework consists of three phases. Firstly, multiple scale feature extractors are used to extract low and high features from RGB and depth images respectively. Then, RGB-detector and depth-detector are trained separately using MS-FRCNN. Finally, late fusion methods are explored for combining the RGB and depth detector. The detection framework was demonstrated and evaluated on two datasets that include passion fruit images under variable illumination conditions and occlusion. Compared with the faster R-CNN detector of RGB-D images, the recall, the precision and F1-score of MS-FRCNN method increased from 0.922 to 0.962, 0.850 to 0.931 and 0.885 to 0.946, respectively. Furthermore, the MS-FRCNN method effectively improves small passion fruit detection by achieving 0.909 of the F1 score. It is concluded that the detector based on MS-FRCNN can be applied practically in the actual orchard environment.

Determining vehicle speed based on video using convolutional neural network

The present study solves the issue of estimating traffic flows based on video surveillance camera data. The goal is to count and classify vehicles and determine their speed. This subject area is at an early stage of development, and the study focuses on an intersection in the city of Chelyabinsk, Russia. We used a two-stage Faster R-CNN detector together with a SORT tracker to solve the set task. For the purposes of detector training and evaluation, we collected 750 video frames from over 52,000 objects located in various conditions. Experimental results show that the proposed system can count vehicles, classify them, and determine their speed with an average absolute percentage error not exceeding 22%. The data set presented in the study can further be used by other researchers as a complex test or additional training data.

Ground Truth Annotated Femoral X-Ray Image Dataset and Object Detection Based Method for Fracture Types Classification

Precise classification of femoral fractures contributes to accurate surgical strategies and better prognosis after surgery. An effective and accurate system for diagnosing femoral fractures and classifying its types will play a vital role in clinical work. This work aims to achieve the automatic detection and classification of femoral fractures in X-ray images. We build a benchmark which includes 2333 X-ray images with 9 different fracture types, and each of them is manually labeled with the ground truth boxes indicating the femoral shaft fractures and its corresponding categories according to the Association for the Study of Internal Fixation (AO). An anchor-based Faster RCNN detection model, with the backbone of ResNet-50 being constructed in a multi-resolution feature pyramid networks (FPN), is used for locating fractures regions and classifying its types. The total image level accuracy reaches 71.5%, which is higher than some of the orthopedic surgeons can get, especially young orthopedic surgeons. Therefore, it is practicable to take advantage of artificial intelligence to detect and classify the femoral shaft fractures.

Improvement of Faster-RCNN Detection Algorithms for Small Size Line Accessory Equipment

Aiming at the problem that the image of patrol inspection is affected by angle and distance, the target of small size accessory equipment in the image is small, and the recognition rate of current algorithm is low. An improved Faster-RCNN detection algorithm for small size accessory equipment of patrol image is proposed. By adjusting the structure of Faster-RCNN network, we can overcome the shortcomings of the current algorithm. Firstly, the deep residual network ResNet50 is used to replace VGG16 network. At the same time, the bottom and high-level features of convolution network are applied to the selection of candidate regions, so as to improve the utilization of effective information of targets, and then to improve the detection precision of small-sized targets. The results show that the detection accuracy of the improved Faster-RCNN model is 4.2% higher than that of the original algorithm.

Perceiving Motion From Dynamic Memory for Vehicle Detection in Surveillance Videos

Most existing video-based object detection methods utilize successful image-based object detector as a base network, and additionally exploit temporal information with either bounding-box post-processing or feature enhancement from multiple frames. However, little work has been done on directly modeling temporal motion in an efficient way for detection in surveillance videos. In this paper, a simple but effective module, denoted as motion-from-memory (MFM), is proposed to encode temporal context for improved detection in surveillance videos. With appearance features extracted from a base CNN, the MFM module maintains a dynamic memory for each input sequence and output motion features on each frame. This module costs minor additional model parameters and computations, but is very helpful for moving object detection, especially in surveillance videos. Thanks to the additional MFM module, the performance of a light-weight MobileNet-based Faster RCNN detector is boosted by 13.93% in mAP, achieving comparable performance to that of strong ResNet-50-based. When MFM is integrated into an even weaker but faster single-stage detector, it ranks the second best one among all published works when submitted to the DEETRAC vehicle detection benchmark, with 69.10% mAP, compared to 69.87% of the best one. However, when running speed is considered, the proposed method is the fastest one, running at 33 FPS with 540×960 surveillance videos on a moderate commercial GPU (NVIDIA GTX 1080Ti), which is about 3 times faster than the second fastest one.

A New CSR-DCF Tracking Algorithm based on Faster RCNN Detection Model and CSRT Tracker for Drone Data

A New CSR-DCF Tracking Algorithm based on Faster RCNN Detection Model and CSRT Tracker for Drone Data

Badminton player detection using faster region convolutional neural network

Nowadays, coaches and sport analyst are concerning about sport performance analysis through sport video match. However, they still used conventional method which is through manual observation of the full video that is very troublesome because they might miss some meaningful information presence in the video. Several previous studies have discussed about tracking ball movements, identification of player based on jersey color and number as well as player movement detection in various type of sport such as soccer and volleyball but not in badminton. Therefore, this study focused on developing an automated system using Faster Region Convolutional Neural Network (Faster R-CNN) to track the position of the badminton player from the sport broadcast video. In preparing the dataset for training and testing, several broadcast videos were converted into image frames before labelling the region which indicate the players. After that, several different trained Faster R-CNN detectors were produced from the dataset before tested with different set of videos to evaluate the detector performance. In evaluating the performance of each detector model, the average precision was obtained from precision recall graph. As a result, this study revealed that the detector successfully detects the player when the detector is being fed with more generalized dataset.

Investigation of Optimal Network Architecture for Asparagus Spear Detection in Robotic Harvesting

The University of Waikato, in collaboration with Robotics Plus Limited have developed a robotic asparagus harvester that utilises a convolutional neural network for spear detection. This paper serves as a starting point for selecting an optimal network architecture for this purpose. Specifically, this paper compared the performance of Faster RCNN (FRCNN) and Single Shot Multibox Detector (SSD) on a dataset collected by the harvesters camera systems during field trials in California. Additionally, the effect of labelling the dataset using both a single-class and multi-class paradigm were evaluated. It was found that FRCNN, trained using a single-class paradigm, had the best performance of the tested networks. This was characterized by a F1 score of 0.73, approximately 38% higher other networks tested. Multi-class labelling paradigms were found to result in approximately 27% reduction in F1 score than Single-class labelling paradigms for both FRCNN and SSD. Based on these results we conclude that FRCNN based detectors are better suited for asparagus detection than SSD based detectors.

Anchor Box Parameters and Bounding Box Overlap Ratios for the Faster R-CNN Detector in Detecting a Single Object by the Masking Background

Anchor box parameters and bounding box overlap ratios are studied in order to set them appropriately for the Faster R-CNN detector. The benchmark detection is based on monochrome images whose background may mask a small dark object. Three object detection tasks are generated, where every image either contains a small black square/rectangle or does not contain the object, representing thus class “background”. The ratios are recommended to be tried at 0.7 if this class is represented. The ratio for positive training samples is tried at a less value but greater than 0.4 for the task every image of which contains an object. The minimum anchor box size is better to try at a lesser value from a range of object sizes. The anchor box pyramid scale factor and the number of levels are better to try at 2 and 8, respectively. Subsequently, these parameters may be corrected as their influence is fuzzier than that of the ratios.

Faster R-CNN for Robust Pedestrian Detection Using Semantic Segmentation Network.

Convolutional neural networks (CNN) have enabled significant improvements in pedestrian detection owing to the strong representation ability of the CNN features. However, it is generally difficult to reduce false positives on hard negative samples such as tree leaves, traffic lights, poles, etc. Some of these hard negatives can be removed by making use of high level semantic vision cues. In this paper, we propose a region-based CNN method which makes use of semantic cues for better pedestrian detection. Our method extends the Faster R-CNN detection framework by adding a branch of network for semantic image segmentation. The semantic network aims to compute complementary higher level semantic features to be integrated with the convolutional features. We make use of multi-resolution feature maps extracted from different network layers in order to ensure good detection accuracy for pedestrians at different scales. Boosted forest is used for training the integrated features in a cascaded manner for hard negatives mining. Experiments on the Caltech pedestrian dataset show improvements on detection accuracy with the semantic network. With the deep VGG16 model, our pedestrian detection method achieves robust detection performance on the Caltech dataset.

Training and testing object detectors with virtual images

In the area of computer vision, deep learning has produced a variety of state-of-the-art models that rely on massive labeled data. However, collecting and annotating images from the real world is too demanding in terms of labor and money investments, and is usually inflexible to build datasets with specific characteristics, such as small area of objects and high occlusion level. Under the framework of Parallel Vision, this paper presents a purposeful way to design artificial scenes and automatically generate virtual images with precise annotations. A virtual dataset named ParallelEye is built, which can be used for several computer vision tasks. Then, by training the DPM U+0028 Deformable parts model U+0029 and Faster R-CNN detectors, we prove that the performance of models can be significantly improved by combining ParallelEye with publicly available real-world datasets during the training phase. In addition, we investigate the potential of testing the trained models from a specific aspect using intentionally designed virtual datasets, in order to discover the flaws of trained models. From the experimental results, we conclude that our virtual dataset is viable to train and test the object detectors.

Training Deep Residual Network with Funnel Online Hard Examples Mining Method on Object Detector

The image sets always contains most of the simple samples and a small number of difficult samples. If the network can automatically select difficult samples, the performance will be improved. In the field of object detection, R-CNN, Fast-RCNN (FRCN) and Faster R-CNN (FRCNN) are regarded as the leader of the detection network architectures. They focus on solving the time efficiency problem, but did not automatically choose difficult samples. The recent Online Hard Examples Mining (OHEM) method is a good solution to this problem. But for small sample sets, the effectiveness is not good enough. In this paper, the feature extraction layer as well as the difficult mining method are designed and applied on Faster-RCNN detector. The network proposed here achieved 78.3% mAP on the PASCAL VOC2007 test set, which are 5.1 point higher than that of FRCNN (VOC2007, 73.2%).