Region Proposal Network Research Articles

An Efficient and Accurate Cross-Domain Object Detection Method Using One-Level Feature and Domain Adaptation

To improve the accuracy of cross-domain object detection, the existing unsupervised domain adaptation (UDA) object detection methods mostly use Feature Pyramid Network (FPN), multiple Region Proposal Network (RPN), and multiple domain classifier, but these methods lead to complex network structures, slow model convergence, and low detection efficiency. To solve the above problems, this paper proposes an Efficient and Accurate Cross-domain Object Detection Method Using One-level Feature and Domain Adaptation (OFDA). This method realizes one-level feature object detection through feature fusion and divide-and-conquer technology; realizes overfitting feature suppression and unsupervised domain adaptation through domain-specific suppression and domain feature alignment technology; realizes background feature suppression through the Objectness branch, which replaces the time-consuming Region Proposal Network (RPN) structure and improves the efficiency of unsupervised adaptive detection. The paper verifies the feasibility and superiority of the proposed method by the comparative experiments and ablation experiments of multiple datasets. The proposed OFDA method not only improves the efficiency of object detection, but also ensures the accuracy of cross-domain detection.

Research on a Traffic Sign Recognition Method under Small Sample Conditions.

Traffic signs are updated quickly, and there image acquisition and labeling work requires a lot of manpower and material resources, so it is difficult to provide a large number of training samples for high-precision recognition. Aiming at this problem, a traffic sign recognition method based on FSOD (few-shot object learning) is proposed. This method adjusts the backbone network of the original model and introduces dropout, which improves the detection accuracy and reduces the risk of overfitting. Secondly, an RPN (region proposal network) with improved attention mechanism is proposed to generate more accurate target candidate boxes by selectively enhancing some features. Finally, the FPN (feature pyramid network) is introduced for multi-scale feature extraction, and the feature map with higher semantic information but lower resolution is merged with the feature map with higher resolution but weaker semantic information, which further improves the detection accuracy. Compared with the baseline model, the improved algorithm improves the 5-way 3-shot and 5-way 5-shot tasks by 4.27% and 1.64%, respectively. We apply the model structure to the PASCAL VOC dataset. The results show that this method is superior to some current few-shot object detection algorithms.

Cork classification based on multi-scale faster-RCNN with machine vision

In traditional industrial production, the classification of cork mainly adopts manual operation. Due to round cork blocks with complex textures, manual sorting is time-consuming and often leads to inefficiency. It will lead to inefficiency and a high error rate. Therefore, an automatic cork sorting system is urgently required to replace manual operation. In this paper, an advanced deep-learning method based on machine vision is adopted to detect the surface of corks and classify them automatically. To robustly detect the texture of cork profile, a Multiple Scale Faster R-CNN based on the upper and lower layers (UMS-FRCNN) approach is proposed. The network consists of a region proposal component and the region-of-interest (RoI) detection component. There are two main contributions in our proposed network, which play a significant role in achieving state-of-the-art performance in cork classification. Firstly, the multi-scale information is grouped in two aspects: region suggestion and RoI detection, to detect on small cork texture regions. Secondly, the convolution Region Proposal Network (RPN) is used to generate the candidate regions of the UMS-FRCNN model to improve the classification accuracy. The experimental results show that our method not only can accurately classify the corks but also has better performance than the Faster RCNN. Compared with other methods, our method has the advantages of high precision, less time-consuming, and no dependence labeling. Our work has been successfully utilized in a cork production plant.

An active foveated gaze prediction algorithm based on a Bayesian ideal observer

Predicting human eye movements is a crucial task for understanding human behavior and has numerous applications in machine vision. Most current models for predicting eye movements are data-driven and require large datasets of recorded eye movements, which can be expensive and time-consuming to collect. In this paper, we present a novel theory-based model for predicting eye movements in a foveated visual system that maximizes information gain at each fixation. Our model uses a region-proposal network and eccentricity-based max pooling to account for the loss of detail in peripheral vision. We apply our model to predict human fixations in a visual search task for objects in real-world scenes. Unlike data-driven models, our model does not require training on large eye movement datasets and can generalize to any set of natural images and targets. We evaluate the generalization capability of our model by demonstrating its results on two publicly available visual search datasets, Ehinger and COCO-search18, without any further training on those datasets. Our model outperforms or performs comparably to data-driven models that are directly trained on human eye movement datasets.

A Few-Shot Defect Detection Method for Transmission Lines Based on Meta-Attention and Feature Reconstruction

In tasks of transmission line defect detection, traditional object detection algorithms are ineffective, with few training samples of defective components. Meta-learning uses multi-task learning as well as fine-tuning to learn common features in different tasks, which has the ability to adapt to new tasks quickly, shows good performance in few-shot object detection, and has good generalization in new tasks. For this reason, we proposed a few-shot defect detection method (Meta PowerNet) with a Meta-attention RPN and Feature Reconstruction Module for transmission lines based on meta-learning. First, in the stage of region proposal, a new region proposal network (Meta-Attention Region Proposal Network, MA-RPN) is designed to fuse the support set features and the query set features to filter the noise in anchor boxes. In addition, it has the ability to focus on the subtle texture features of smaller-sized objects by fusing low-level features from the query set. Second, in the meta-feature construction stage, we designed a meta-learner with the defect feature reconstruction module as the core to capture and focus on the defect-related feature channels. The experimental results show that under the condition, there are only 30 training objects for various types of component defects. The method achieves 72.5% detection accuracy for component defects, which is a significant improvement compared with other mainstream few-shot object detection. Meanwhile, the MA-RPN designed in this paper can be used in other meta-learning object detection models universally.

Estimation method and research of fruit glycemic load index based on the fusion SE module Faster R-CNN

In recent years, with the close connection between the field of artificial intelligence and life, people's quality of life has been improved, and food choices have increased. However, how to understand one's own dietary structure and sugar intake and obtain the correct glycemic load index reference has become a more concerned issue for diabetic patients. This paper studies how to measure the dietary intake of diabetic patients. Our purpose is to identify fruits with high sweetness and low glycemic load value, to help diabetic patients give correct and reasonable reference to their daily diet. For better research, we constructed a fruit dataset for studying the daily diet of diabetic patients. At the same time, we improve the Faster R-CNN network for fruit recognition, introduce an attention mechanism module in the feature extraction stage, adjust the anchor aspect ratio of the Region Proposal Network (RPN), and perform a fusion update operation in the fully connected layer. Experiments show that, compared with existing methods, our method im-proves the precision and average recall rate by 2.47% and 3.06%, respectively. Then we designed a volume estimation method, and then obtained the glycemic load index of the fruit, and finally gave an eating opinion based on this value.

Arbitrary-Oriented Object Detection in Aerial Images with Dynamic Deformable Convolution and Self-Normalizing Channel Attention

Objects in aerial images often have arbitrary orientations and variable shapes and sizes. As a result, accurate and robust object detection in aerial images is a challenging problem. In this paper, an arbitrary-oriented object detection method for aerial images, based on Dynamic Deformable Convolution (DDC) and Self-normalizing Channel Attention Mechanism (SCAM), is proposed; this method uses ReResNet-50 as the backbone network to extract rotation-equivariant features. First, DDC is proposed as a replacement for the conventional convolution operation in the Convolutional Neural Network (CNN) in order to cope with various shapes, sizes and arbitrary orientations of the objects. Second, SCAM embedded into the high layer of ReResNet-50, which allows the network to enhance the important feature channels and suppress the irrelevant ones. Finally, Rotation Regions of Interest (RRoI) are generated based on a Region Proposal Network (RPN) and a RoI Transformer (RT), and the RoI-wise classification and bounding box regression are realized by Rotation-invariant RoI Align (RiRoI Align). The proposed method is comprehensively evaluated on three publicly available benchmark datasets. The mean Average Precision (mAP) can reach 80.91%, 92.73% and 94.1% on DOTA-v1.0, DOTA-v1.5 and HRSC2016 datasets, respectively. The experimental results show that, when compared with the state-of-the-arts methods, the proposed method can achieve superior detection accuracy.

Robust online learning based on siamese network for ship tracking

The complex and changeable inland river scenes resulting out of frequent occlusions of ships in the available tracking methods are not accurate enough to estimate the motion state of the target ship leading to object tracking drift or even loss. In view of this, an attempt is made to propose a robust online learning ship tracking algorithm based on the Siamese network and the region proposal network. Firstly, the algorithm combines the off-line Siamese network classification score and the online classifier score for discriminative learning, and establishes an occlusion determination mechanism according to the classification the fusion score. When the target is in the occlusion state, the target template is not updated, and the global search mechanism is activated to relocate the target, thereby avoiding object tracking drift. Secondly, an efficient adaptive online update strategy, UpdateNet, is introduced to improve the template degradation in the tracking process. Finally, on comparing the state-of-the-art tracking algorithms on the inland river ship datasets, the experimental results of the proposed algorithm show strong robustness in occlusion scenarios with an accuracy and success rate of 56.8% and 57.2% respectively. Supportive source codes for this research are publicly available at https://github.com/Libra-jing/SiamOL.

Region‐based fully convolutional networks with deformable convolution and attention fusion for steel surface defect detection in industrial Internet of Things

AbstractNext‐generation 6G networks will fully drive the development of the industrial Internet of Things. Steel surface defect detection as an important application in industrial Internet of Things has recently received increasing attention from the military industry, the aviation industry and other fields, which is closely related to the quality of industrial production products. However, many typical convolutional neural networks‐based methods are insensitive to the problem of unclear boundaries. In this article, the authors develop a region‐based fully convolutional networks with deformable convolution and attention fusion to adaptively learn salient features for steel surface defect detection. Specifically, deformable convolution is applied into selectively replace the standard convolution in the backbone of the region‐based fully convolutional networks, which performs significantly in scenarios with unclear defect boundaries. Moreover, convolutional block attention module is utilised in region proposal network to further enhance detection accuracy. The proposed architecture is demonstrated on two popular steel defect detection benchmarks, including NEU‐DET and GC10‐DET, which can effectively present the performance of steel surface defect detection by abundant experiments. The mean average precision on two datasets reaches 80.9% and 66.2%. The average precision of defect crazing, inclusion, patches, pitted‐surface, rolled‐in scale and scratches on NEU‐DET is 58.2%, 82.3%, 95.7%, 85.6%, 75.9%, and 87.9% respectively.

A COMPARATIVE STUDY OF DEEP LEARNING TECHNIQUES FOR BOLL ROT DISEASE DETECTION IN COTTON CROPS

Early detection of plant diseases helps to prevent loss of productivity and overcomes the shortcomings of continuous human monitoring. To solve these problems, many researchers have already completed their work to identify the diseases automatically, rapidly, and with greater accuracy using deep learning methods. This research combines deep learning with agriculture by developing a system for identifying cotton boll rot. We used two states of art pre-trained models SSD with MobileNet-V2 and Faster R-CNN with Inception -V2, which can locate boll rot attacks in cotton crops. It will be determined how much damage our crops have sustained. The trained model achieved 65% and 89% accuracy, respectively. The accuracy results for disease identification demonstrated that the deep network model is prospective and can significantly influence effective disease identification. It may also have the potential for disease detection in real-world agricultural systems of interest, region proposal networks, convolutional neural networks; deep neural networks; bounding boxes; support vector machines.

Enhance the Accuracy of Landslide Detection in UAV Images Using an Improved Mask R-CNN Model: A Case Study of Sanming, China.

Extracting high-accuracy landslide areas using deep learning methods from high spatial resolution remote sensing images is a hot topic in current research. However, the existing deep learning algorithms are affected by background noise and landslide scale effects during the extraction process, leading to poor feature extraction effects. To address this issue, this paper proposes an improved mask regions-based convolutional neural network (Mask R-CNN) model to identify the landslide distribution in unmanned aerial vehicles (UAV) images. The improvement of the model mainly includes three aspects: (1) an attention mechanism of the convolutional block attention module (CBAM) is added to the backbone residual neural network (ResNet). (2) A bottom-up channel is added to the feature pyramidal network (FPN) module. (3) The region proposal network (RPN) is replaced by guided anchoring (GA-RPN). Sanming City, China was selected as the study area for the experiments. The experimental results show that the improved model has a recall of 91.4% and an accuracy of 92.6%, which is 12.9% and 10.9% higher than the original Mask R-CNN model, respectively, indicating that the improved model is more effective in landslide extraction.

Overlapped tobacco shred image segmentation and area computation using an improved Mask RCNN network and COT algorithm.

The classification of the four tobacco shred varieties, tobacco silk, cut stem, expanded tobacco silk, and reconstituted tobacco shred, and the subsequent determination of tobacco shred components, are the primary tasks involved in calculating the tobacco shred blending ratio. The identification accuracy and subsequent component area calculation error directly affect the composition determination and quality of the tobacco shred. However, tiny tobacco shreds have complex physical and morphological characteristics; in particular, there is substantial similarity between the expanded tobacco silk and tobacco silk varieties, and this complicates their classification. There must be a certain amount of overlap and stacking in the distribution of tobacco shreds on the actual tobacco quality inspection line. There are 24 types of overlap alone, not to mention the stacking phenomenon. Self-winding does not make it easier to distinguish such varieties from the overlapped types, posing significant difficulties for machine vision-based tobacco shred classification and component area calculation tasks. This study focuses on two significant challenges associated with identifying various types of overlapping tobacco shreds and acquiring overlapping regions to calculate overlapping areas. It develops a new segmentation model for tobacco shred images based on an improved Mask region-based convolutional neural network (RCNN). Mask RCNN is used as the segmentation network's mainframe. Convolutional network and feature pyramid network (FPN) in the backbone are replaced with Densenet121 and U-FPN, respectively. The size and aspect ratios of anchors parameters in region proposal network (RPN) are optimized. An algorithm for the area calculation of the overlapped tobacco shred region (COT) is also proposed, which is applied to overlapped tobacco shred mask images to obtain overlapped regions and calculate the overlapped area. The experimental results showed that the final segmentation accuracy and recall rates are 89.1% and 73.2%, respectively. The average area detection rate of 24 overlapped tobacco shred samples increases from 81.2% to 90%, achieving high segmentation accuracy and overlapped area calculation accuracy. This study provides a new implementation method for the type identification and component area calculation of overlapped tobacco shreds and a new approach for other similar overlapped image segmentation tasks.

Developing a Deep Learning-Based Defect Detection System for Ski Goggles Lenses

Ski goggles help protect the eyes and enhance eyesight. The most important part of ski goggles is their lenses. The quality of the lenses has leaped with technological advances, but there are still defects on their surface during manufacturing. This study develops a deep learning-based defect detection system for ski goggles lenses. The first step is to design the image acquisition model that combines cameras and light sources. This step aims to capture clear and high-resolution images on the entire surface of the lenses. Next, defect categories are identified, including scratches, watermarks, spotlight, stains, dust-line, and dust-spot. They are labeled to create the ski goggles lenses defect dataset. Finally, the defects are automatically detected by fine-tuning the mobile-friendly object detection model. The mentioned defect detection model is the MobileNetV3 backbone used in a feature pyramid network (FPN) along with the Faster-RCNN detector. The fine-tuning includes: replacing the default ResNet50 backbone with a combination of MobileNetV3 and FPN; adjusting the hyper-parameter of the region proposal network (RPN) to suit the tiny defects; and reducing the number of the output channel in FPN to increase computational performance. Our experiments demonstrate the effectiveness of defect detection; additionally, the inference speed is fast. The defect detection accuracy achieves a mean average precision (mAP) of 55%. The work automatically integrates all steps, from capturing images to defect detection. Furthermore, the lens defect dataset is publicly available to the research community on GitHub. The repository address can be found in the Data Availability Statement section.

A Framework of Faster CRNN and VGG16-Enhanced Region Proposal Network for Detection and Grade Classification of Knee RA

We developed a framework to detect and grade knee RA using digital X-radiation images and used it to demonstrate the ability of deep learning approaches to detect knee RA using a consensus-based decision (CBD) grading system. The study aimed to evaluate the efficiency with which a deep learning approach based on artificial intelligence (AI) can find and determine the severity of knee RA in digital X-radiation images. The study comprised people over 50 years with RA symptoms, such as knee joint pain, stiffness, crepitus, and functional impairments. The digitized X-radiation images of the people were obtained from the BioGPS database repository. We used 3172 digital X-radiation images of the knee joint from an anterior-posterior perspective. The trained Faster-CRNN architecture was used to identify the knee joint space narrowing (JSN) area in digital X-radiation images and extract the features using ResNet-101 with domain adaptation. In addition, we employed another well-trained model (VGG16 with domain adaptation) for knee RA severity classification. Medical experts graded the X-radiation images of the knee joint using a consensus-based decision score. We trained the enhanced-region proposal network (ERPN) using this manually extracted knee area as the test dataset image. An X-radiation image was fed into the final model, and a consensus decision was used to grade the outcome. The presented model correctly identified the marginal knee JSN region with 98.97% of accuracy, with a total knee RA intensity classification accuracy of 99.10%, with a sensitivity of 97.3%, a specificity of 98.2%, a precision of 98.1%, and a dice score of 90.1% compared with other conventional models.

Adapting a Swin Transformer for License Plate Number and Text Detection in Drone Images

The use of drones and unmanned aerial vehicles has significantly increased in various real-world applications such as monitoring illegal car parking, tracing vehicles, controlling traffic jams, and chasing vehicles. However, accurate detection of license plate numbers in drone images becomes complex and challenging due to variations in height distances and oblique angles during image capturing, unlike most existing methods that focus on normal images for text/license plate number detection. To address this issue, this work proposes a new model for license plate number detection in drone images using Swin transformer. The Swin transformer is chosen due to its special properties such as higher accuracy, efficiency, and fewer computations, making it suitable for license plate number/text detection in drone images. To further improve the performance of the proposed model under adverse conditions such as degradations, poor quality, and occlusion, the proposed work incorporates a maximally stable extremal region-based regional proposal network to represent text data in the images. Experimental results on both normal license plates and drone images demonstrate the superior performance of the proposed model over state-of-the-art methods.

Vehicle Detection for Accident Prevention

Abstract: One of the key components of the smart traffic concept is vehicle detection and tracking. Modern city planning and development is impossible to achieve without a thorough understanding of the city's existing traffic flows. Surveillance video is an underutilized source of traffic data that can be discovered using a wide range of information technology tools and solutions, including machine learning techniques. A critical step in these systems is robust and reliable vehicle detection. It employs an object detection model to locate vehicles in images captured by an outdoor surveillance camera. A review of recent vision-based on-road vehicle detection systems is presented in this paper. YOLO generates a region proposal network and classifies these region proposals at the same time. The purpose of this paper is to address the issues raised above. This model has good performance in object detection.

Segmentation of Unsound Wheat Kernels Based on Improved Mask RCNN.

The grade of wheat quality depends on the proportion of unsound kernels. Therefore, the rapid detection of unsound wheat kernels is important for wheat rating and evaluation. However, in practice, unsound kernels are hand-picked, which makes the process time-consuming and inefficient. Meanwhile, methods based on traditional image processing cannot divide adherent particles well. To solve the above problems, this paper proposed an unsound wheat kernel recognition algorithm based on an improved mask RCNN. First, we changed the feature pyramid network (FPN) to a bottom-up pyramid network to strengthen the low-level information. Then, an attention mechanism (AM) module was added between the feature extraction network and the pyramid network to improve the detection accuracy for small targets. Finally, the regional proposal network (RPN) was optimized to improve the prediction performance. Experiments showed that the improved mask RCNN algorithm could identify the unsound kernels more quickly and accurately while handling adhesion problems well. The precision and recall were 86% and 91%, respectively, and the inference time on the test set with about 200 targets for each image was 7.83 s. Additionally, we compared the improved model with other existing segmentation models, and experiments showed that our model achieved higher accuracy and performance than the other models, laying the foundation for wheat grading.

AD-SiamRPN: Anti-Deformation Object Tracking via an Improved Siamese Region Proposal Network on Hyperspectral Videos

Object tracking using Hyperspectral Images (HSIs) obtains satisfactory result in distinguishing objects with similar colors. Yet, the tracking algorithm tends to fail when the target undergoes deformation. In this paper, a SiamRPN based hyperspectral tracker is proposed to deal with this problem. Firstly, a band selection method based on a genetic optimization method is designed for rapidly reducing the redundancy of information in HSIs. Specifically, three bands with highest joint entropy are selected. To solve the problem that the information of the template in the SiamRPN model decays over time, an update network is trained on the dataset from general objective tracking benchmark, which can obtain effective cumulative templates. The use of cumulative templates with spectral information makes it easier to track the deformed target. In addition, transfer learning of the pre-trained SiamRPN is designed to obtain a better model for HSIs. The experimental results show that the proposed tracker can obtain good tracking results over the entire public dataset, and that it is better than the other popular trackers when the target’s deformation is qualitatively and quantitatively compared, achieving an overall success rate of 57.5% and a deformation challenge success rate of 70.8%.

Lung nodule detection in chest CT images based on vision transformer network with Bayesian optimization

PurposeLung cancer patients are usually diagnosed by the Computed tomography (CT) modality with the help of radiologists. Frequently, radiologists manually analyse CT images, which is a big task that consumes more time and results in inaccuracy. Therefore, this study proposes an automated computer-aided diagnosis (CAD) scheme based on Vision Transformer architecture with Bayesian Optimisation for lung nodule detection to assist radiologists in decision-making. MethodsThe Vision Transformer architecture relies on the shifted window-based vision transformer encoders to improve the processing and propagation efficiency of the lung nodule features. The baseline detection model is developed by integrating shifted window-based vision transformer for deep nodule feature extractions and a region proposal network to predict candidate nodules with low network complexity. The Bayesian Optimisation with the Gaussian method was used to optimise the hyperparameter values from the baseline model suitable for lung nodule detection on chest CT images. ResultsWe used 888 CT images with 10-fold cross-validation from the Lung Nodule Analysis 2016 (LUNA16) public dataset to validate the proposed network. Free-response receiver operating characteristics and competition performance metrics (CPM) was used for the performance assessment. The proposed architecture obtained the highest detection sensitivity of 98.39% and the CPM score of 0.909 with few network parameters. ConclusionThis study proposes a CAD system based on a vision transformer network determined by Bayesian Optimisation for lung nodule detection in chest CT images. The empirical results prove the proposed algorithm's effectiveness compared to the state-of-the-art methods.

3D Street Object Detection from Monocular Images Using Deep Learning and Depth Information

In this study, we present a three-dimensional (3D) object detection algorithm based on monocular images by constructing an end-to-end network, that incorporates depth information. The entire network consists of three parts. The first part includes the basic object detection neural network as the main body, that uses the region proposal network to obtain the two-dimensional (2D) region proposal of the object. The second part is the depth estimation branch network, that obtains the depth information of the object pixels and calculates the corresponding 3D point cloud. In the last part, concatenated features obtained from the aforementioned two parts are fed into the fully-connected layers. Subsequently, 2D and 3D detection results are obtained. Compared with certain existing methods, the accuracy of the detection results is improved in this study.