Region Proposal Network Research Articles

Deep Learning Algorithm for Optimized Sensor Data Fusion in Fault Diagnosis and Tolerance

Environmental perception is one of the key technologies to realize autonomous vehicles. The fault diagnosis process involves identifying the fault that occurred or the cause of the out-of-control condition. Here, the major objective is to locate problems in detection by analysing previous data or sequential patterns of data that cause failure. This study evaluates the use of deep learning for improved sensor data fusion in fault identification and tolerance using the KITTI dataset. The input video from the dataset has been transformed to frames through median filtering. Next, feature extraction is applied to a preprocessed image, resulting in the fusion of sensor data. Data fusion is then carried out utilizing an enhanced RPN (region proposal network). The enhanced RPN also has a loss function (object detection loss, bounding box loss and target classification loss), an estimate of ROI and feature extraction network (FEN). Through the use of the COOT connected blue monkey optimization (CCBMO) model, the weight of the optimally enhanced RPN is established. Next, using global non-maximum suppression with both global and local confidence, fault identification and tolerance are carried out. From the analysis, it clearly shows that proposed method accomplished better results in terms of accuracy, precision and specificity of 97.78%, 93.76% and 93.43%, respectively, when compared with various conventional models with respect to diverse performance measures.

Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentation.

Few-Shot Instance Segmentation (FSIS) requires detecting and segmenting novel classes with limited support examples. Existing methods based on Region Proposal Networks (RPNs) face two issues: 1) overfitting suppresses novel class objects and 2) dual-branch models require complex spatial correlation strategies to prevent spatial information loss when generating class prototypes. We introduce a unified framework, Reference Twice (RefT), to exploit the relationship between support and query features for FSIS and related tasks. Our three main contributions are: 1) a novel transformer-based baseline that avoids overfitting, offering a new direction for FSIS; 2) demonstrating that support object queries encode key factors after base training, allowing query features to be enhanced twice at both feature and query levels using simple cross-attention, thus avoiding complex spatial correlation interaction; and 3) introducing a class-enhanced base knowledge distillation loss to address the issue of DETR-like models struggling with incremental settings due to the input projection layer, enabling easy extension to incremental FSIS. Extensive experimental evaluations on the COCO dataset under three FSIS settings demonstrate that our method performs favorably against existing approaches across different shots, e.g., +8.2/+9.4 performance gain over state-of-the-art methods with 10/30-shots.

Toward Automated Detection of Microbleeds with Anatomical Scale Localization Using Deep Learning

Cerebral Microbleeds (CMBs) are chronic deposits of small blood products in the brain tissues, which have explicit relation to various cerebrovascular diseases depending on their anatomical location, including cognitive decline, intracerebral hemorrhage, and cerebral infarction. However, manual detection of CMBs is a time consuming and error-prone process because of their sparse and tiny structural properties. The detection of CMBs is commonly affected by the presence of many CMB mimics that cause a high false-positive rate (FPR), such as calcifications and pial vessels. This paper proposes a novel 3D deep learning framework that not only detects CMBs but also identifies their anatomical location in the brain (i.e., lobar, deep, and infratentorial regions). For the CMBs detection task, we propose a single end-to-end model by leveraging the 3D U-Net as a backbone with Region Proposal Network (RPN). To significantly reduce the false positives within the same single model, we develop a new scheme, containing Feature Fusion Module (FFM) that detects small candidates utilizing contextual information and Hard Sample Prototype Learning (HSPL) that mines CMB mimics and generates additional loss term called concentration loss using Convolutional Prototype Learning (CPL). For the anatomical localization task, we exploit the 3D U-Net segmentation network to segment anatomical structures of the brain. This task not only identifies to which region the CMBs belong but also eliminates some false positives from the detection task by leveraging anatomical information. We utilize Susceptibility-Weighted Imaging (SWI) and phase images as 3D input to efficiently capture 3D information. The results show that the proposed RPN that utilizes the FFM and HSPL outperforms the baseline RPN and achieves a sensitivity of 94.66% vs. 93.33% and an average number of false positives per subject (FPavg) of 0.86 vs. 14.73. Furthermore, the anatomical localization task enhances the detection performance by reducing the FPavg to 0.56 while maintaining the sensitivity of 94.66%.

Log Volume Measurement and Counting Based on Improved Cascade Mask R-CNN and Deep SORT

Logs require multiple verifications to ensure accurate volume and quantity measurements. Log end detection is a crucial step in measuring log volume and counting logs. Currently, this task primarily relies on the Mask R-CNN instance segmentation model. However, the Feature Pyramid Network (FPN) in Mask R-CNN may compromise accuracy due to feature redundancy during multi-scale fusion, particularly with small objects. Moreover, counting logs in a single image is challenging due to their large size and stacking. To address the above issues, we propose an improved log segmentation model based on Cascade Mask R-CNN. This method uses ResNet for multi-scale feature extraction and integrates a hierarchical Convolutional Block Attention Module (CBAM) to refine feature weights and enhance object emphasis. Then, a Region Proposal Network (RPN) is employed to generate log segmentation proposals. Finally, combined with Deep SORT, the model tracks log ends in video streams and counts the number of logs in the stack. Experiments demonstrate the effectiveness of our method, achieving an average precision (AP) of 82.3, APs of 75.3 for small, APm of 70.9 for medium, and APl of 86.2 for large objects. These results represent improvements of 1.8%, 3.7%, 2.6%, and 1.4% over Mask R-CNN, respectively. The detection rate reached 98.6%, with a counting accuracy of 95%. Compared to manually measured volumes, our method shows a low error rate of 4.07%.

Improved Faster R-CNN Based on Multi-Scale Module for Detecting Weak Ground Targets in Remote Sensing Images

In order to optimize the detection capability of small and weak ground targets in remote sensing images, a remote sensing image ground weak target detection method based on multi-scale module improved Faster R-CNN is proposed. By adjusting the Region Proposal Network (RPN) and Region of Interest (RoI) pooling layers, the model is better suited for feature extraction and classification of small targets. Generate feature maps at multiple scales by introducing additional convolutional layers to process inputs at different resolutions, with each layer focusing on targets within a specific size range. These multi-scale features are integrated and input into RPN to provide richer target proposal characteristics. By using image transformation techniques such as scaling, cropping, and rotation, the training dataset is expanded to simulate different small target scenes, enhancing the model’s adaptability to changes in target size, direction, and environment. The experimental results show that the proposed method not only outperforms other models in accuracy, recall, and F1 score, but also maintains high consistency and stability across different datasets. Its significant advantage in mean accuracy (MAP) further proves its reliability and effectiveness in remote sensing applications.

Lightweight mask R-CNN for instance segmentation and particle physical property analysis in multiphase flow

A lightweight Mask R-CNN instance segmentation model was developed here to analyze particle size and shape accurately and quickly. Firstly, a hybrid Depthwise Dilated Convolutional Network (DDNet) is proposed, and the feature pyramid layers and the shared convolutional layers of the region proposal network are simplified, reducing the model complexity while ensuring robust feature extraction capabilities. Then, segmentation accuracy is significantly improved without sacrificing computational speed and performance by introducing the Dice loss function and clustering algorithm. Experimental results show that the model parameters are significantly reduced by 49.46%, and the segmentation speed increases from 2.15 FPS (frames per second) to 5.88 FPS. Meanwhile, the segmentation accuracy (AP50) increased from 90.56% to 91.21%. In addition, it was proven that the particle size distribution and shape could be analyzed accurately and rapidly with the proposed model, providing essential information for multiphase flow process optimization and equipment design in industrial applications.

KPTr: Key point transformer for LiDAR-based 3D object detection

Accurate 3D environmental perception is crucial for ensuring the safety of autonomous vehicles. However, many existing point-based 3D object detection methods rely only on independent point features to extract spatial information and ignore the dependencies between points. Meanwhile, these methods usually use max-pooling operations to aggregate neighboring point features without considering the stability of local geometric information. To address these issues, this paper proposes a novel two-stage LiDAR-based key point transformer that consists of a region proposal network module and a region-based transformer module to enhance 3D detection performance. Firstly, unprocessed raw point clouds are taken as the input of the model to preserve detailed spatial structural information. Secondly, in the region proposal network an enhanced PointNet++ is used to segment foreground points and generate high-quality first-stage proposals. Then, the representation of proposals is improved by the region-based transformer module, which constructs long-range key point cloud relationship features within the regions of interest and diffuses spatial information into the feature channel layers. Additionally experiments on the KITTI dataset show that detection performance of the proposed method surpasses most mainstream 3D object detection methods and achieves a detection speed of 14.28 Hz, striking a good balance between detection accuracy and speed. Finally, the proposed method is deployed on real vehicle hardware platforms for online detection, thereby proving its industrial value.

Incremental RPN: Hierarchical Region Proposal Network for Apple Leaf Disease Detection in Natural Environments.

Apple leaf diseases can seriously affect apple production and quality, and accurately detecting them can improve the efficiency of disease monitoring. Owing to the complex natural growth environment, apple leaf lesions may be easily confused with background noise, leading to poor performance. In this study, a cascaded Incremental Region Proposal Network (Inc-RPN) is proposed to accurately detect apple leaf diseases in natural environments. The proposed Inc-RPN has a two-layer RPN architecture, where the precursor RPN is leveraged to generate diseased leaf proposals, and the successor RPN focuses on extracting target disease spots based on diseased leaf proposals. In the successor RPN, a low-level feature aggregation module is designed to fully utilize the bridged features and preserve the semantic information of the target disease spots. An incremental module is also leveraged to extract aggregated diseased leaf features and target disease spot features. Finally, a novel position anchor generator is designed to generate anchors based on diseased leaf proposals. The experimental results show that the proposed Inc-RPN performs very well on the FALD_CED and Apple Leaf Disease datasets, showing that it can accurately perform apple leaf disease detection tasks.

People Detection Using Artificial Intelligence with Panchromatic Satellite Images

The detection of people in urban environments from satellite imagery can be employed in a variety of applications, such as urban planning, business management, crisis management, military operations, and security. A WorldView-3 satellite image of Prague was processed. Several variants of feature-extracting networks, referred to as backbone networks, were tested alongside the Faster R–CNN model. This model combines region proposal networks with object detection, offering a balance between speed and accuracy that is well suited for dense and varied urban environments. Data augmentation was used to increase the robustness of the models, which contributed to the improvement of classification results. Achieving a high level of accuracy is an ongoing challenge due to the low spatial resolution of available imagery. An F1 score of 54% was achieved using data augmentation, a 15 cm buffer, and a maximum distance limit of 60 cm.

Open-world object detection: A solution based on reselection mechanism and feature disentanglement

Traditional object detection algorithms operate within a closed set, where the training data may not cover all real-world objects. Therefore, the issue of open-world object detection has attracted significant attention. Open-world object detection faces two major challenges: “neglecting unknown objects” and “misclassifying unknown objects as known ones.” In our study, we address these challenges by utilizing the Region Proposal Network (RPN) outputs to identify potential unknown objects with high object scores that do not overlap with ground truth annotations. We introduce the reselection mechanism, which separates unknown objects from the background. Subsequently, we employ the simulated annealing algorithm to disentangle features of unknown and known classes, guiding the detector’s learning process. Our method has improved on multiple evaluation metrics such as U-mAP, U-recall, and UDP, greatly alleviating the challenges faced by open world object detection.

Versatile Teacher: A class-aware teacher–student framework for cross-domain adaptation

Addressing the challenge of domain shift between datasets is vital in maintaining model performance. In the context of cross-domain object detection, the teacher–student framework, a widely-used semi-supervised model, has shown significant accuracy improvements. However, existing methods often overlook class differences, treating all classes equally, resulting in suboptimal results. Furthermore, the integration of instance-level alignment with a one-stage detector, essential due to the absence of a Region Proposal Network (RPN), remains unexplored in this framework. In response to these shortcomings, we introduce a novel teacher–student model named Versatile Teacher (VT). VT differs from previous works by considering class-specific detection difficulty and employing a two-step pseudo-label selection mechanism, referred to as Class-aware Pseudo-label Adaptive Selection (CAPS), to generate more reliable pseudo labels. These labels are leveraged as saliency matrices to guide the discriminator for targeted instance-level alignment. Our method demonstrates promising results on three benchmark datasets, and extends the alignment methods for widely-used one-stage detectors, presenting significant potential for practical applications. Code is available at https://github.com/RicardooYoung/VersatileTeacher.

Superpixel perception graph neural network for intelligent defect detection of aero-engine blade

Aero-engine is the core component of aircraft and other spacecraft. The high-speed rotating blades provide power by sucking in air and fully combusting, and various defects will inevitably occur, threatening the operation safety of aero-engine. Therefore, regular inspections are essential for such a complex system. However, existing traditional technology which is borescope inspection is labor-intensive, time-consuming, and experience-dependent. To endow this technology with intelligence, a novel superpixel perception graph neural network (SPGNN) is proposed by utilizing a multi-stage graph convolutional network (MSGCN) for feature extraction and superpixel perception region proposal network (SPRPN) for region proposal. First, to capture complex and irregular textures, the images are transformed into a series of patches, to obtain their graph representations. Then, MSGCN composed of several GCN blocks extracts graph structure features and performs graph information processing at graph level. Last but not least, the SPRPN is proposed to generate perceptual bounding boxes by fusing graph representation features and superpixel perception features. Therefore, the proposed SPGNN always implements feature extraction and information transmission at the graph level in the whole SPGNN pipeline, to alleviate the reduction of receptive field and information loss. To verify the effectiveness of SPGNN, we construct a simulated blade dataset with 3000 images. A public aluminum dataset is also used to validate the performances of different methods. The experimental results demonstrate that the proposed SPGNN has superior performance compared with the state-of-the-art methods.

DOUNet: Dynamic Optimization and Update Network for Oriented Object Detection

Object detection can accurately identify and locate targets in images, serving basic industries such as agricultural monitoring and urban planning. However, targets in remote sensing images have random rotation angles, which hinders the accuracy of remote sensing image object detection algorithms. In addition, due to the long-tailed distribution of detected objects in remote sensing images, the network finds it difficult to adapt to imbalanced datasets. In this article, we designed and proposed the Dynamic Optimization and Update network (DOUNet). By introducing adaptive rotation convolution to replace 2D convolution in the Region Proposal Network (RPN), the features of rotating targets are effectively extracted. To address the issues caused by imbalanced data, we have designed a long-tail data detection module to collect features of tail categories and guide the network to output more balanced detection results. Various experiments have shown that after two stages of feature learning and classifier learning, our designed network can achieve optimal performance and perform better in detecting imbalanced data.

Weld seam object detection system based on the fusion of 2D images and 3D point clouds using interpretable neural networks

This study introduces a novel approach that addresses the limitations of existing methods by integrating 2D image processing with 3D point cloud analysis, enhanced by interpretable neural networks. Unlike traditional methods that rely on either 2D or 3D data alone, our approach leverages the complementary strengths of both data types to improve detection accuracy in environments adversely affected by welding spatter and smoke. Our system employs an improved Faster R-CNN model with a ResNet50 backbone for 2D image analysis, coupled with an innovative orthogonal plane intersection line extraction algorithm for 3D point cloud processing. By incorporating explainable components such as visualizable feature maps and a transparent region proposal network, we address the “black box” issue common in deep learning models.This architecture enables a more transparent decision-making process, providing technicians with necessary insights to understand and trust the system’s outputs. The Faster-RCNN structure is designed to break down the object detection process into distinct, understandable steps, from initial feature extraction to final bounding box refinement. This fusion of 2D-3D data analysis and interpretability not only improves detection performance but also sets a new standard for transparency and reliability in automated welding systems, facilitating wider adoption in industrial applications.

VIDF-Net: A Voxel-Image Dynamic Fusion method for 3D object detection

In recent years, multi-modal fusion methods have shown excellent performance in the field of 3D object detection, which select the voxel centers and globally fuse with image features across the scene. However, these approaches exist two issues. First, The distribution of voxel density is highly heterogeneous due to the discrete volumes. Additionally, there are significant differences in the features between images and point clouds. Global fusion does not take into account the correspondence between these two modalities, which leads to the insufficient fusion. In this paper, we propose a new multi-modal fusion method named Voxel-Image Dynamic Fusion (VIDF). Specifically, VIDF-Net is composed of the Voxel Centroid Mapping module (VCM) and the Deformable Attention Fusion module (DAF). The Voxel Centroid Mapping module is used to calculate the centroid of voxel features and map them onto the image plane, which can locate the position of voxel features more effectively. We then use the Deformable Attention Fusion module to dynamically calculates the offset of each voxel centroid from the image position and combine these two modalities. Furthermore, we propose Region Proposal Network with Channel-Spatial Aggregate to combine channel and spatial attention maps for improved multi-scale feature interaction. We conduct extensive experiments on the KITTI dataset to demonstrate the outstanding performance of proposed VIDF network. In particular, significant improvements have been observed in the Hard categories of Cars and Pedestrians, which shows the significant effectiveness of our approach in dealing with complex scenarios.

Brain tumor image segmentation method using hybrid attention module and improved mask RCNN

To meet the needs of automated medical analysis of brain tumor magnetic resonance imaging, this study introduces an enhanced instance segmentation method built upon mask region-based convolutional neural network. By incorporating squeeze-and-excitation networks, a channel attention mechanism, and concatenated attention neural network, a spatial attention mechanism, the model can more adeptly focus on the critical regions and finer details of brain tumors. Residual network-50 combined attention module and feature pyramid network as the backbone network to effectively capture multi-scale characteristics of brain tumors. At the same time, the region proposal network and region of interest align technology were used to ensure that the segmentation area matched the actual tumor morphology. The originality of the research lies in the deep residual network that combines attention mechanism with feature pyramid network to replace the backbone based on mask region convolutional neural network, achieving an improvement in the efficiency of brain tumor feature extraction. After a series of experiments, the precision of the model is 90.72%, which is 0.76% higher than that of the original model. Recall was 91.68%, an increase of 0.95%; Mean Intersection over Union was 94.56%, an increase of 1.39%. This method achieves precise segmentation of brain tumor magnetic resonance imaging, and doctors can easily and accurately locate the tumor area through the segmentation results, thereby quickly measuring the diameter, area, and other information of the tumor, providing doctors with more comprehensive diagnostic information.

Improved region proposal network for enhanced few-shot object detection

Despite significant success of deep learning in object detection tasks, the standard training of deep neural networks requires access to a substantial quantity of annotated images across all classes. Data annotation is an arduous and time-consuming endeavor, particularly when dealing with infrequent objects. Few-shot object detection (FSOD) methods have emerged as a solution to the limitations of classic object detection approaches based on deep learning. FSOD methods demonstrate remarkable performance by achieving robust object detection using a significantly smaller amount of training data. A challenge for FSOD is that instances from novel classes that do not belong to the fixed set of training classes appear in the background and the base model may pick them up as potential objects. These objects behave similarly to label noise because they are classified as one of the training dataset classes, leading to FSOD performance degradation. We develop a semi-supervised algorithm to detect and then utilize these unlabeled novel objects as positive samples during the FSOD training stage to improve FSOD performance. Specifically, we develop a hierarchical ternary classification region proposal network (HTRPN) to localize the potential unlabeled novel objects and assign them new objectness labels to distinguish these objects from the base training dataset classes. Our improved hierarchical sampling strategy for the region proposal network (RPN) also boosts the perception ability of the object detection model for large objects. We test our approach and COCO and PASCAL VOC baselines that are commonly used in FSOD literature. Our experimental results indicate that our method is effective and outperforms the existing state-of-the-art (SOTA) FSOD methods. Our implementation is provided as a supplement to support reproducibility of the results https://github.com/zshanggu/HTRPN.11Early partial results of this work is presented in the 2023 ICCV Workshop on Visual Continual Learning (Shangguan and Rostami, 2023).

A Nesting Segmenting and Categorizing Model for Diabetic Wound Depth Detection in Foot Using Convolutional Neural Networks

Background: Traditional methods of Diabetic Foot Ulcers (DFUs) assessment and segmentation using computer vision and Machine Learning (ML) have faced limitations in accuracy and automation. Recent advancements in Deep Learning (DL) offer potential improvements, but challenges remain in the precise quantification and categorization of DFUs. Methods: This study explores a framework combining Mask Region-based Convolutional Neural Networks (R-CNN) with a nested structure for DFU segmentation. The approach involved collecting and preprocessing DFU images from Xiangya Hospital and the DFU Challenge (DFUC) 2020. A Region Proposal Network (RPN) was employed to identify potential regions of interest, which were then analyzed using Mask R-CNN with ResNet-FPN as the backbone. The model aimed to enhance feature extraction and categorization of DFUs through a layered approach that integrates regional proposals and segmentation masks. Results: The proposed method achieved superior results in DFU categorization compared to existing techniques. The use of Mask R-CNN with ResNet-FPN improved segmentation accuracy, with the framework achieving notable enhancements in F-score and accuracy metrics—0.79%, 16.30%, and 3.24% improvements over other algorithms. The method effectively learned and categorized complex DFU structures, providing a more accurate assessment of wound severity levels. The performance was validated against various models, including R-CNN, YOLO, DetNet, and EfficientDet, demonstrating the robustness of the proposed method. Conclusion: The integration of Mask R-CNN with a nested feature extraction framework offers a promising advancement in DFU segmentation. This approach addresses existing limitations in automated DFU assessment by improving accuracy and consistency.

Rupiah Banknotes Detection Comparison of The Faster R-CNN Algorithm and YOLOv5

Money is an essential part of human life. Humans are never separated from activities related to money. As time goes by, money is not only a means of transactions between humans but also between humans and machines. Machines can recognize money in various ways, including object detection. Object detection is one of the most popular branches of computer vision. There are many methods for carrying out object detection, such as Faster R-CNN and YOLO. Faster R-CNN has been widely used in various fields to perform object detection tasks. Faster R-CNN has advantages over its predecessor because it uses a Region Proposal Network (RPN) as a substitute for selective search, which requires less compilation time. YOLO (You Only Look Once) is the most frequently used object detection method. This method divides the image into grids; each part of the grid predicts objects and their probabilities. The main advantages of YOLO are its high speed and ability to recognize objects in various conditions and positions with reasonably high accuracy. This research compares the Faster R-CNN algorithm model using the ResNet-50 architecture with YOLOv5 to recognize rupiah banknotes. The dataset used is 1120 images consisting of 8 classes. The YOLOv5 model trained on RGB data had the best results, with calculation accuracy reaching 1. Test results on three images also showed suitable results. The hope is that this research can be applied in other research to build a system for recognizing rupiah banknotes.

Intelligent cotter pins defect detection for electrified railway based on improved faster R-CNN and dilated convolution

The cotter pin (CP) is a vital fastener for the catenary support components (CSCs) of high-speed electrified railways. Due to the vibration and excitation caused by the passing of railway vehicles, some CPs may be broken or fallen off over time, which poses a significant safety hazard to the railway systems. Currently, the CP defect detection is primarily conducted by humans, which is inefficient and inconsistent. Therefore, there is an urgent need for automatic CP defect detection to ensure railway safety. However, this task is very challenging as it requires covering hundreds or thousands of miles in limited times when the railway stops running. To this end, we first design a traffic track intelligent imaging device to capture catenary images at various angles at high speed. Then, inspired by the success of deep learning-based object detection, we develop a CP detection model based on an improved Faster R-CNN with a multi-scale region proposal network (MS-RPN) and propose the positive sample adaptive loss function (PSALF) to enhance detection accuracy. Finally, we propose a module to recognize the CP defect based on dilated convolution. The experimental results show that our method can effectively detect the CP defect in the catenary image, achieving 99.05 % precision and 98.40 % recall rate on CP defect detection. Furthermore, CP detection method and CP defect detection are significantly faster than baseline method, with FPS improvements of 2.76 and 24.67, respectively, thus making it more suitable for real-time applications in railway systems.