Region-based Convolutional Neural Network Method Research Articles

Detection of the Corn Kernel Breakage Rate Based on an Improved Mask Region-Based Convolutional Neural Network

Real-time knowledge of kernel breakage during corn harvesting plays a significant role in the adjustment of operational parameters of corn kernel harvesters. (1) Transfer learning by initializing the DenseNet121 network with pre-trained weights for training and validating a dataset of corn kernels was adopted. Additionally, the feature extraction capability of DenseNet121 was improved by incorporating the attention mechanism of a Convolutional Block Attention Module (CBAM) and a Feature Pyramid Network (FPN) structure. (2) The quality of intact and broken corn kernels and their pixels were found to be coupled, and a linear regression model was established using the least squares method. The results of the test showed that: (1) The MAPb50 and MAPm50 of the improved Mask Region-based Convolutional Neural Network (RCNN) model were 97.62% and 98.70%, in comparison to the original Mask Region-based Convolutional Neural Network (RCNN) model, which were improved by 0.34% and 0.37%, respectively; the backbone FLOPs and Params were 3.09 GMac and 9.31 M, and the feature extraction time was 206 ms; compared to the original backbone, these were reduced by 3.87 GMac and 17.32 M, respectively. The training of the obtained prediction weights for the detection of a picture of the corn kernel took 76 ms, so compared to the Mask RCNN model, it was reduced by 375 ms; based on the concept of transfer learning, the improved Mask RCNN model converged twice as quickly with the loss function using pre-training weights than the loss function without pre-training weights during training. (2) The coefficients of determination R2 of the two models, when the regression models of the pixels and the quality of intact and broken corn kernels were analyzed, were 0.958 and 0.992, respectively. These findings indicate a strong correlation between the pixel characteristics and the quality of corn kernels. The improved Mask RCNN model was used to segment mask pixels to calculate the corn kernel breakage rate. The verified error between the machine vision and the real breakage rate ranged from −0.72% to 0.65%, and the detection time of the corn kernel breakage rate was only 76 ms, which could meet the requirements for real-time detection. According to the test results, the improved Mask RCNN method had the advantages of a fast detection speed and high accuracy, and can be used as a data basis for adjusting the operation parameters of corn kernel harvesters.

Automatic Pavement Crack Identification Based on an Improved C-Mask Region-Based Convolutional Neural Network Model

A pavement crack identification method based on an improved C-mask region-based convolutional neural network (R-CNN) model is proposed to solve problems whereby existing crack recognition algorithms exhibit low accuracy and cannot perform detection and segmentation tasks simultaneously. The crack dataset that was collected in this study included three categories: transverse cracks, longitudinal cracks, and alligator cracks. The model integrates the detection task and the segmentation task into one model, and segments the crack pixels in the generated detection box while achieving target positioning. Firstly, based on the mask R-CNN model, the improved C-mask R-CNN method is designed, which improves the quality of the region proposal box by combining the detectors that are cascaded with different intersections over union thresholds, and achieves accurate crack location under high-threshold detection. Secondly, the ratio of the anchor in the model is adjusted, and a series of optimization parameters and experimental comparisons are carried out for the improved model to realize the segmentation of the crack pixels in the generated detection box during the crack location. The effectiveness of the proposed model is verified, and finally, an evaluation method for cracks is proposed. Furthermore, the calculation of the crack geometric parameters is completed. The experimental results demonstrate that the mean average precision (mAP) of the C-mask R-CNN model detection part reached 95.4%, and the mAP of the segmentation part reached 93.5%. Moreover, the proposed model is convenient for researchers to deploy and implement.

Detection of circuit components on hand-drawn circuit images by using faster R-CNN method

In this study, one of deep learning methods, which has been very popular in recent years, is employed for the detection and classification of circuit components in hand-drawn circuit images. Each circuit component located in different positions on the scanned images of hand-drawn circuits, which are frequently used in electrical and electronics engineering, is considered as a separate object. In order to detect the components on the circuit image, Faster Region Based Convolutional Neural Network (R-CNN) method is used instead of conventional methods. With the Faster R-CNN method, which has been developed in recent years to detect and classify objects, preprocessing on image data is minimized, and the feature extraction phase is done automatically. In the study, it is aimed to detect and classify four different circuit components in the scanned images of hand-drawn circuits with high accuracy by using the Python programming language on the Google Colab platform. The circuit components to be detected on the hand-drawn circuits are specified as resistor, inductor, capacitor, and voltage source. For the training of the model used, a data set was created by collecting 800 circuit images consisting of hand drawings of different people. For the detection of the components, the pretrained Faster R-CNN Inception V2 model was used after fine tuning and arrangements depending on the process requirements. The model was trained in 50000 epochs, and the success of the trained model has been tested on the circuits drawn in different styles on the paper. The trained model was able to detect circuit components quickly and with a high rate of performance. In addition, the loss graphics of the model were examined. The proposed method shows its efficiency by quickly detecting each of the 4 different circuit components on the image and classifying them with high performance.

Automated segmentation for whole human eye OCT image using RM multistage Mask R-CNN

Accurate segmentation of structures in whole human eye optical coherence tomography (OCT) images can improve the accuracy of disease diagnosis and ophthalmic biological parameter measurement. However, due to the complex features of the cornea, lens, and retina in the whole human eye OCT image and external interference, it is difficult to precisely extract the object features, which restricts the segmentation accuracy of whole human eye OCT images. A relation module (RM) multistage Mask Region-based Convolutional Neural Network (R-CNN) method for whole human eye OCT image segmentation is established. Based on Mask R-CNN, modulated deformable convolution is employed to produce an irregular receptive field and improve the adaptability of the network to the whole human eye OCT image’s object deformation. In the feature map extraction, the RM is combined to learn the position relation feature map of the human eye structures, which is utilized to enhance feature maps. A multistage mask is constructed in the segmentation branch, and the error is corrected through iterations of the mask to improve the segmentation accuracy of the whole human eye OCT image. Combined with the above structures, the RM multistage Mask R-CNN method is constructed for whole human eye OCT image segmentation. The model is trained by whole human eye OCT images and is applied to realize highly accurate whole human eye OCT image segmentation. Comparison experiments with K-means, U-net, and Feature Pyramid Networks (FPN)-deformable-mask R-CNN are performed to verify the segmentation accuracy and robustness of the proposed method to complex deformation and external interference.

A Page Object Detection Method Based on Mask R-CNN

Page object detection is crucial for document understanding. Different granularities for objects can result in different performances. In this study, block level region object detection is considered among the inherent hierarchical structure for document images. Inspired by Mask R-CNN (Region-based Convolutional Neural Networks) method, an end to end network is proposed to perform object classification, bounding box identification, and page object mask generation at the same time. Latex based synthetic document generation is designed for enlarging the training data. A large number of synthetic page images are generated for training to alleviate the insufficient dataset problem. Compared with existing page object competition methods, the proposed method achieves better results, with mAP of 0.917 on page objects such as table, figure and maths detection.

Improved mask R-CNN-based cloud masking method for remote sensing images

ABSTRACT Clouds lead to missing or distorted land-related information in impacted areas in optical remote sensing images. Cloud masking, which labels cloud-contaminated pixels, forms the basis for subsequent image utilization, such as excluding the distorted pixel or filling in the missing area. However, due to the diverse spectral, textural, and shape characteristics of different clouds and complicated combinations with the underlying land surfaces, cloud masking has become a challenge in remote sensing image processing. In recent years, the Mask region-based convolutional neural network (R-CNN) method, which performs instance segmentation from a complex background and generates a pixelwise mask for the object of interest, has been used widely in object segmentation tasks. When the Mask R-CNN method is used for cloud masking, the mask result has certain problems, such as failing to extract uncommon clouds and outputting inaccurate mask boundaries for large clouds. To address these problems, we introduce two strategies, group training and boundary optimization, to improve the Mask R-CNN. For group training, samples are divided into several groups. The samples in the first group are used for the initial training, and the samples in the next group are used for evaluation. Only samples with missing or falsely detected clouds are used for tuning the classifier; then, these processes are repeated until all groups have been used or the detection precision becomes stable. For boundary optimization, a block-by-block mask strategy is adopted to guarantee that clouds with diverse sizes have similar performances. Finally, two open data sets and one data set labelled by ourselves are selected to test the proposed method, and the results demonstrate that our method can produce cloud masks for different cloud types and diverse underlying land surfaces and can achieve high accuracies, thereby providing an effective alternative for cloud masking. Compared with the original Mask R-CNN method, our method improves the average recall, average precision, and intersection over union by 5.88%, 2.4%, and 0.071 in pixel level, respectively, demonstrating the effectiveness of our improvement.

Local keypoint-based Faster R-CNN

Region-based Convolutional Neural Network (R-CNN) detectors have achieved state-of-the-art results on various challenging benchmarks. Although R-CNN has achieved high detection performance, the research of local information in producing candidates is insufficient. In this paper, we design a Keypoint-based Faster R-CNN (K-Faster) method for object detection. K-Faster incorporates local keypoints in Faster R-CNN to improve the detection performance. In detail, a sparse descriptor, which first detects the points of interest in a given image and then samples a local patch and describes its invariant features, is first employed to produce keypoints. All 2-combinations of the produced keypoints are second selected to generate keypoint anchors, which are helpful for object detection. The heterogeneously distributed anchors are then encoded in feature maps based on their areas and center coordinates. Finally, the keypoint anchors are coupled with the anchors produced by Faster R-CNN, and the coupled anchors are used for Region Proposal Network (RPN) training. Comparison experiments are implemented on PASCAL VOC 07/12 and MS COCO. The experimental results show that our K-Faster approach not only increases the mean Average Precision (mAP) performance but also improves the positioning precision of the detected boxes.

Region-Based CNN Method with Deformable Modules for Visually Classifying Concrete Cracks

Cracks are often the most intuitive indicators for assessing the condition of in-service structures. Intelligent detection methods based on regular convolutional neural networks (CNNs) have been widely applied to the field of crack detection in recently years; however, these methods exhibit unsatisfying performance on the detection of out-of-plane cracks. To overcome this drawback, a new type of region-based CNN (R-CNN) crack detector with deformable modules is proposed in the present study. The core idea of the method is to replace the traditional regular convolution and pooling operation with a deformable convolution operation and a deformable pooling operation. The idea is implemented on three different regular detectors, namely the Faster R-CNN, region-based fully convolutional networks (R-FCN), and feature pyramid network (FPN)-based Faster R-CNN. To examine the advantages of the proposed method, the results obtained from the proposed detector and corresponding regular detectors are compared. The results show that the addition of deformable modules improves the mean average precisions (mAPs) achieved by the Faster R-CNN, R-FCN, and FPN-based Faster R-CNN for crack detection. More importantly, adding deformable modules enables these detectors to detect the out-of-plane cracks that are difficult for regular detectors to detect.

An Optimized Faster R-CNN Method Based on DRNet and RoI Align for Building Detection in Remote Sensing Images

In recent years, the increase of satellites and UAV (unmanned aerial vehicles) has multiplied the amount of remote sensing data available to people, but only a small part of the remote sensing data has been properly used; problems such as land planning, disaster management and resource monitoring still need to be solved. Buildings in remote sensing images have obvious positioning characteristics; thus, the detection of buildings can not only help the mapping and automatic updating of geographic information systems but also have guiding significance for the detection of other types of ground objects in remote sensing images. Aiming at the deficiency of traditional building remote sensing detection, an improved Faster R-CNN (region-based Convolutional Neural Network) algorithm was proposed in this paper, which adopts DRNet (Dense Residual Network) and RoI (Region of Interest) Align to utilize texture information and to solve the region mismatch problems. The experimental results showed that this method could reach 82.1% mAP (mean average precision) for the detection of landmark buildings, and the prediction box of building coordinates was relatively accurate, which improves the building detection results. Moreover, the recognition of buildings in a complex environment was also excellent.

OTHR-13. A DEEP LEARNING APPROACH TO DETECT CANCER METASTASES TO THE BRAIN IN MRI

BACKGROUND AND OBJECTIVE: Brain metastases have been found to account for one-fourth of all cancer metastases seen in clinics. Magnetic resonance imaging (MRI) is widely used for detecting brain metastases. Accurate detection of the brain metastases is critical to design radiotherapy to treat the cancer and monitor their progression or response to the therapy and prognosis. However, finding metastases on brain MRI is very challenging as many metastases are small and manifest as objects of weak contrast on the images. In this work we present a deep learning approach integrated with a classification scheme to detect cancer metastases to the brain on MRI. MATERIALS AND METHODS: We retrospectively extracted 101 metastases patients, equal to 1535 metastases on 10192 slices of images in a total of 336 scans from our PACS and manually marked the lesions on T1-weighted contrast enhanced MRI as the ground-truth. We then randomly separated the cases into training, validation, and test sets for developing and optimizing the deep learning neural network. We designed a 2-step computer-aided detection (CAD) pipeline by first applying a fast region-based convolutional neural network method (R-CNN) to sequentially process each slice of an axial brain MRI to find abnormal hyper-intensity that may correspond to a brain metastasis and, second, applying a random under sampling boost (RUSBoost) classification method to reduce the false positive metastases. RESULTS: The computational pipeline was tested on real brain images. A sensitivity of 97.28% and false positive rate of 36.25 per scan over the images were achieved by using the proposed method. CONCLUSION: Our results demonstrated the deep learning-based method can detect metastases in very challenging cases and can serve as CAD tool to help radiologists interpret brain MRIs in a time-constrained environment.

Automatic Vehicle License Plate Extraction Using Region-Based Convolutional Neural Networks and Morphological Operations

The number and range of the candidate vehicle license plate (VLP) region affects the result of the VLP extraction symmetrically. Therefore, in order to improve the VLP extraction rate, many candidate VLP regions are selected. However, there is a problem that the processing time increases symmetrically. In this paper, we propose a method that allows detecting a vehicle license plate in the real-time mode. To do this, the proposed method makes use of the region-based convolutional neural network (R-CNN) method and morphological operations. The R-CNN method is a deep learning method that selects a large number of candidate regions from an input image and compares them to determine whether objects of interest are included. However, this method has limitations when used in real-time processing. Therefore, to address this limitation in the proposed method, while selecting a candidate vehicle region, the selection range is reduced based on the size and position of the vehicle in the input image; hence, processing can be performed quickly. A vehicle license plate is detected by performing a morphological operation based on the edge pixel distribution of the detected vehicle region. Experimental results show that the detection rate of vehicles is approximately 92% in real road environments, and the detection rate of vehicle license plates is approximately 83%.

Accelerate proposal generation in R-CNN methods for fast pedestrian extraction

PurposeThe purpose of this study is to develop a novel region-based convolutional neural networks (R-CNN) approach that is more efficient while at least as accurate as existing R-CNN methods. In this way, the proposed method, namely R2-CNN, provides a more powerful tool for pedestrian extraction for person re-identification, which involve a huge number of images and pedestrian needs to be extracted efficiently to meet the real-time requirement.Design/methodology/approachThe proposed R2-CNN is tested on two types of data sets. The first one the USC Pedestrian Detection data set, which consists of three sub-sets USC-A, UCS-B and USC-C, with respect to their characteristics. This data set is used to test the performance of R2-CNN in the pedestrian extraction task. The speed and performance of the investigated algorithms were collected. The second data set is the PASCAL VOC 2007 data set, which is a common benchmark data set for object detection. This data set was used to analyze characteristics of R2-CNN in the case of general object detection task.FindingsThis study proposes a novel R-CNN method that is both more efficient and more accurate than existing methods. The method, when used as an object detector, would facilitate the data preprocessing stage of person re-identification.Originality/valueThe study proposes a novel approach for object detection, which shows advantages in both efficiency and accuracy for pedestrian detection task. It contributes to both data preprocessing for person re-identification and the research on deep learning.

Developing an Algorithm for Buildings Extraction and Determining Changes from Airborne LiDAR, and Comparing with R-CNN Method from Drone Images

The world has experienced urban changes rapidly, and this phenomenon encourages authors to contribute to the United Nations sustainable development goals (SDGs) 2030 and geospatial information. This study presents a proposed algorithm of change detection and extracting the borders of buildings. This proposed algorithm provides a set of instructions to describe the method of solving the problem of how extracting the boundary of buildings from the light detection and ranging (LiDAR) input data incorporating with the firefly and ant colony algorithms. The method has used two different epochs to compare buildings and to identify the type of changes in selected buildings. These changes are based on the newly built or demolished buildings. We also used drone images and mask the region-based convolutional neural network (R-CNN) method to compare the results of roof extraction of buildings vs. the proposed algorithm. This study shows that the proposed algorithm identifies the changes of all buildings with higher accuracy of extracting border of buildings than the existing methods, successfully. This study also determines that the amount of root mean square error (RMSE) is 2.40 m2 when we use LiDAR. This proposed algorithm contributes to identifying rapidly changed buildings, and it is helpful for global geospatial information of urban management that can add best practice and solution toward the UN SDGs connectivity dilemma of urban settlement, resilience, and sustainability.

Automatic license plate detection and recognition framework to enhance security applications

We develop an automatic license plate recognition (ALPR) system for enhancing the investigation capabilities of law enforcing agencies to monitor suspicious vehicles. The recognition performance of real-time ALPR systems is affected to great extent in challenging conditions such as varying illumination, angle-of-view, different sizes of plates, changing contrast, and shadows. Moreover, character segmentation step sensitivity to plate resolution, size of characters, occluded characters, and width between characters makes it difficult to properly isolate the character, which further degrades recognition accuracy. In the first step of the proposed framework, a plate is localized using the faster region-based convolutional neural network method. In the second step, our study proposes a segmentation-free plate recognition approach that applies an adaptive boosting method with linear discriminant analysis for feature selection followed by matching the plates with a database for suspected vehicles and information retrieval. Simulation results show that the proposed framework is more robust to illumination variations, low-resolution images, different orientations, and variable license plate sizes than the conventional ones.

Performance evaluation of faster R-CNN on GPU for object detection

This paper presents a performance evaluation of Faster Region-based Convolutional Neural Network method with different parameters to observe the mean average precision. Faster R-CNN replaces the previous proposal method with Region Proposal Network to complete the network. RPN predicts object bounds and its scores at each region making it a fully convolutional network. RPN produces almost cost-free region proposals since the network shares fully image convolutional features with detection network. The use of this technique improve training and testing speed and mean average precision (mAP) compared to SPPnet. It can achieves approximately 10ms per image for object detection and time cost in region proposal. The dataset used to train and test is on VOC 2007. This technique is implemented in MATLAB R2017a using Caffe on NVidia GTX 1060 and GTX 1080.

Toward Fast and Accurate Vehicle Detection in Aerial Images Using Coupled Region-Based Convolutional Neural Networks

Vehicle detection in aerial images, being an interesting but challenging problem, plays an important role for a wide range of applications. Traditional methods are based on sliding-window search and handcrafted or shallow-learning-based features with heavy computational costs and limited representation power. Recently, deep learning algorithms, especially region-based convolutional neural networks (R-CNNs), have achieved state-of-the-art detection performance in computer vision. However, several challenges limit the applications of R-CNNs in vehicle detection from aerial images: 1) vehicles in large-scale aerial images are relatively small in size, and R-CNNs have poor localization performance with small objects; 2) R-CNNs are particularly designed for detecting the bounding box of the targets without extracting attributes; 3) manual annotation is generally expensive and the available manual annotation of vehicles for training R-CNNs are not sufficient in number. To address these problems, this paper proposes a fast and accurate vehicle detection framework. On one hand, to accurately extract vehicle-like targets, we developed an accurate-vehicle-proposal-network (AVPN) based on hyper feature map which combines hierarchical feature maps that are more accurate for small object detection. On the other hand, we propose a coupled R-CNN method, which combines an AVPN and a vehicle attribute learning network to extract the vehicle's location and attributes simultaneously. For original large-scale aerial images with limited manual annotations, we use cropped image blocks for training with data augmentation to avoid overfitting. Comprehensive evaluations on the public Munich vehicle dataset and the collected vehicle dataset demonstrate the accuracy and effectiveness of the proposed method.