Cascade R-CNN Research Articles

Metal Surface Defect Detection Method Based on Improved Cascade R-CNN

Abstract In contemporary industrial systems, ensuring the quality of object surfaces has become an essential and inescapable aspect of factory inspections. Cascade regional convolutional neural network (cascade R-CNN), an object detection and instance segmentation algorithm based on deep learning, has been widely applied in numerous industrial applications. Nonetheless, there is still space for improving the detection of defects on metal surfaces. The cascade R-CNN does not currently have good results in metal defect detection, but after improving it by combining some newly proposed modules, cascade R-CNN has a good performance. This article proposes an enhanced metal defect detection method based on cascade R-CNN. Specifically, the improved backbone network is employed to acquire the features of images, which enables more precise localization. Additionally, up and down sampling is combined to extract multiscale defect feature maps, and contrast histogram equalization enhancement is utilized to tackle the issue of unclear contrast in the data. Experimental results demonstrate that the proposed approach achieves a mean average precision (mAP) of 0.754 on the NEU-DET dataset and outperforms the cascade R-CNN model by 9.2%.

Two-Stage Pedestrian Detection Model Using a New Classification Head for Domain Generalization

Pedestrian detection based on deep learning methods have reached great success in the past few years with several possible real-world applications including autonomous driving, robotic navigation, and video surveillance. In this work, a new neural network two-stage pedestrian detector with a new custom classification head, adding the triplet loss function to the standard bounding box regression and classification losses, is presented. This aims to improve the domain generalization capabilities of existing pedestrian detectors, by explicitly maximizing inter-class distance and minimizing intra-class distance. Triplet loss is applied to the features generated by the region proposal network, aimed at clustering together pedestrian samples in the features space. We used Faster R-CNN and Cascade R-CNN with the HRNet backbone pre-trained on ImageNet, changing the standard classification head for Faster R-CNN, and changing one of the three heads for Cascade R-CNN. The best results were obtained using a progressive training pipeline, starting from a dataset that is further away from the target domain, and progressively fine-tuning on datasets closer to the target domain. We obtained state-of-the-art results, MR−2 of 9.9, 11.0, and 36.2 for the reasonable, small, and heavy subsets on the CityPersons benchmark with outstanding performance on the heavy subset, the most difficult one.

Defect detection method for key area guided transmission line components based on knowledge distillation

Introduction: The aim of this paper is to address the problem of the limited number of defect images for both metal tools and insulators, as well as the small range of defect features.Methods: A defect detection method for key area-guided transmission line components based on knowledge distillation is proposed. First, the PGW (Prediction-Guided Weighting) module is introduced to improve the foreground target distillation region, and the distillation range is precisely concentrated in the position of the first k feature pixels with the highest quality score in the form of a mask. The feature knowledge of defects of hardware and insulators is used as the focus for the teacher network to guide the student network. Then, the GcBlock module is used to capture the relationship between the target defects of the hardware and the transmission lines in the background, and the overall relationship information of the image is used to promote the students’ network to learn the teacher’s network perception ability of the relationship information. Finally, the classification task mask and regression task mask generated by the PGW module, combined with the overall image relationship loss, form a distillation loss function for network training to improve the accuracy of students’ network detection accuracy.Results and Discussion: The effectiveness of the proposed method is verified by using self-build metal fittings and insulator defect data sets. The experimental results show that the student network mAP_50 (Mean Average Precision at 50) in the Faster R-CNN model with the knowledge distillation algorithm added in this paper increases by 8.44%, and the RetinaNet model increases by 2.6%. The Cascade R-CNN model improved by 5.28%.

UAV Image Small Object Detection Based on RSAD Algorithm

There are many small objects in UAV images, and the object scale varies greatly. When the SSD algorithm detects them, the backbone network’s feature extraction capabilities are poor; it does not fully utilize the semantic information in the deeper feature layer, and it does not give enough consideration to the little items in the loss function, which result in serious missing object detection and low object detection accuracy. To tackle these issues, a new algorithm called RSAD (Resnet Self-Attention Detector) that takes advantage of the self-attention mechanism has been proposed. The proposed RSAD algorithm utilises the residual structure of the ResNet-50 backbone network, which is more capable of feature extraction, in order to extract deeper features from UAV image information. It then utilises the SAFM (Self-Attention Fusion Module) to reshape and concatenate the shallow and deep features of the backbone network, selectively weighted by attention units, ensuring the efficient fusion of features to provide rich semantic features for small object detection. Lastly, it introduces the Focal Loss loss function, which adjusts the corresponding parameters to enhance the contribution of small objects to the detection model. The ablation experiments show that the mAP of RSAD is 10.6% higher than that of the SSD model, with SAFM providing the highest mAP enhancement of 7.4% and ResNet-50 and Focal Loss providing 1.3% and 1.9% enhancements, respectively. The detection speed is only reduced by 3FPS, but it meets the real-time requirement. Comparison experiments show that in terms of mAP, it is far ahead of Faster R-CNN, Cascade R-CNN, RetinaNet, CenterNet, YOLOv5s, and YOLOv8n, which are the mainstream object detection models; In terms of FPS, it slightly inferior to YOLOv5s and YOLOv8n. Thus, RSAD has a good balance between detection speed and accuracy, and it can facilitate the advancement of the UAV to complete object detection tasks in different scenarios.

An Enhanced Target Detection Algorithm for Maritime Search and Rescue Based on Aerial Images

Unmanned aerial vehicles (UAVs), renowned for their rapid deployment, extensive data collection, and high spatial resolution, are crucial in locating distressed individuals during search and rescue (SAR) operations. Challenges in maritime search and rescue include missed detections due to issues including sunlight reflection. In this study, we proposed an enhanced ABT-YOLOv7 algorithm for underwater person detection. This algorithm integrates an asymptotic feature pyramid network (AFPN) to preserve the target feature information. The BiFormer module enhances the model’s perception of small-scale targets, whereas the task-specific context decoupling (TSCODE) mechanism effectively resolves conflicts between localization and classification. Using quantitative experiments on a curated dataset, our model outperformed methods such as YOLOv3, YOLOv4, YOLOv5, YOLOv8, Faster R-CNN, Cascade R-CNN, and FCOS. Compared with YOLOv7, our approach enhances the mean average precision (mAP) from 87.1% to 91.6%. Therefore, our approach reduces the sensitivity of the detection model to low-lighting conditions and sunlight reflection, thus demonstrating enhanced robustness. These innovations have driven advancements in UAV technology within the maritime search and rescue domains.

Improving table detection for document images using boundary

Locating tables in document images is the first step to extracting table information, and high location precision is required. The dominant approach of table detection is based on an object detection algorithm, and the detector defines the prediction task as a regression problem, which inevitably leads to positioning errors. To address this issue, this paper presents an approach called Border Line Correction (BLC) to refine the rough prediction results of the original detector through the table boundary lines extracted from the document image. Our approach transforms the regression task into a classification problem, thus avoiding the inherent regression error of the object detection algorithm. Traditional annotation methods are inadequate for table detection tasks as they fail to capture the completeness and purity of the detection results. Therefore, this study treats the correct position of a table as a tolerance region. Additionally, to overcome the limitations of existing datasets in the materials domain, we collected 1183 samples from scientific literature in the materials field and created the MatTab dataset, annotating the tables with tolerance regions. This paper use Cascade RCNN with Swin Transformer as baseline models, and BLC is utilized to optimize the detection results. Experimental results demonstrate significant improvements with BLC at an IOU of 0.95 on the MatTab, ICDAR2019, and ICDAR2017 datasets. In MatTab, the percentage of correctly detected complete and pure tables increased from 72.3% to 82.1%.

Microbial Colony Detection Based on Deep Learning

In clinical drug sensitivity experiments, it is necessary to plate culture pathogenic bacteria and pick suitable colonies for bacterial solution preparation, which is a process that is currently carried out completely by hand. Moreover, the problems of plate contamination, a long culture period, and large image annotation in colony plate image acquisition can lead to a small amount of usable data. To address the issues mentioned above, we adopt a deep learning approach and conduct experiments on the AGAR dataset. We propose to use style transfer to extend the trainable dataset and successfully obtain 4k microbial colony images using this method. In addition, we introduce the Swin Transformer as a feature extraction network in the Cascade Mask R-CNN model architecture to better extract the feature information of the images. After our experimental comparison, the model achieves a mean Average Precision (mAP) of 61.4% at the Intersection over Union (IoU) [0.50:0.95]. This performance surpasses that of the Cascade R-CNN with HRNet, which is the top-performing model in experiments conducted on the AGAR dataset, by a margin of 2.2%. Furthermore, we perform experiments using YOLOv8x on the AGAR dataset, which results in a mAP of 76.7%.

MFMDet: multi-scale face mask detection using improved Cascade rcnn

MFMDet: multi-scale face mask detection using improved Cascade rcnn

Effective Fabric Defect Detection Model for High-Resolution Images

The generation of defects during fabric production impacts fabric quality, and fabric production factories can greatly benefit from the automatic detection of fabric defects. Although object detection algorithms based on convolutional neural networks can be quickly developed, fabric defect detection encounters several problems. Accordingly, a fabric defect detection model based on Cascade R-CNN was proposed in this study. We also proposed block recognition and detection box merging algorithms to achieve complete defect detection in high-resolution images. We implemented a Switchable Atrous Convolution layer to enhance the feature extraction ability of ResNet-50 and upgraded the Feature Pyramid Network to improve the detection accuracy of small defects. Experimental results demonstrated that the proposed model can efficiently perform defect detection in fabric.

An automatic fine-grained violence detection system for animation based on modified faster R-CNN

Animation drama and movies are important entertainment sources for children and young people. However, many of these entertainment videos introduce ideas such as violence, fighting, abuse, and car accidents, which are not suitable for children and young people. In order to prevent children from viewing these videos, different countries' government bodies put restrictions on age on these videos. Violence detection is mostly done using manual inspection, which is time and resource consuming and cannot achieve a very good result. Therefore, artificial technologies such as deep learning and machine learning can detect violence in a video. Bleeding is one of the distinct characteristics of violent actions presented in such videos. This paper aims to propose a system that can detect a given video's blood in real-time. Researchers have proposed different image processing-based techniques to detect violence, but their proposed systems lack precision and real-time detection. To this end, a deep learning-based approach is proposed to detect violence in a given video and images. A Faster R-CNN model is modified due to the sophisticated characteristics of the violence in the cartoon and animation images to achieve the highest performance. The backbone of the proposed violence detection model is changed to a modified RegNet model to extract the features from the frame. The standard inner lateral connection is replaced with the modulated deformable convolutional (MDC) layer to extract deformable feature maps. A novel distributed attention module (DAM) is proposed on the feature pyramid network to improve the performance of the feature extraction. The multiscale Region of interest (ROI) Align is adapted to improve the detection performance of violence in a different scenario. Moreover, the classification approach is integrated with the detection model to classify the different levels of violence in a given frame. We have explored state-of-the-art object detection methods such as Faster R-CNN, Cascade R-CNN, yolov3-spp, SSD, FCOS, and yolov5 to detect violence in a particular frame of a video. We compared the different models' performances and found out that the modified Faster R-CNN is appropriate for real-time blood detection in a given video or image with high accuracy. This study makes a significant contribution to detecting violence in animation videos which will help different platforms and government bodies to regulate entertainment content.

Bird Object Detection: Dataset Construction, Model Performance Evaluation, and Model Lightweighting.

The application of object detection technology has a positive auxiliary role in advancing the intelligence of bird recognition and enhancing the convenience of bird field surveys. However, challenges arise due to the absence of dedicated bird datasets and evaluation benchmarks. To address this, we have not only constructed the largest known bird object detection dataset, but also compared the performances of eight mainstream detection models on bird object detection tasks and proposed feasible approaches for model lightweighting in bird object detection. Our constructed bird detection dataset of GBDD1433-2023, includes 1433 globally common bird species and 148,000 manually annotated bird images. Based on this dataset, two-stage detection models like Faster R-CNN and Cascade R-CNN demonstrated superior performances, achieving a Mean Average Precision (mAP) of 73.7% compared to one-stage models. In addition, compared to one-stage object detection models, two-stage object detection models have a stronger robustness to variations in foreground image scaling and background interference in bird images. On bird counting tasks, the accuracy ranged between 60.8% to 77.2% for up to five birds in an image, but this decreased sharply beyond that count, suggesting limitations of object detection models in multi-bird counting tasks. Finally, we proposed an adaptive localization distillation method for one-stage lightweight object detection models that are suitable for offline deployment, which improved the performance of the relevant models. Overall, our work furnishes an enriched dataset and practice guidelines for selecting suitable bird detection models.

An improved block-level approach for tunnel lining crack detection

Abstract Tunnel lining cracks pose a great safety risk to safe operation. Due to severe illumination conditions and lining cracks, which are usually thinner with varied shapes compared to cracks generated in pavement, bridge, and other transportation infrastructure, the lining crack inspection at the block level is a challenging task. In this study, to address this issue, we propose a novel Swin Transformer-based network including Shape-IoU method to enhance the perception of the tunnel lining crack and the refinement of the crack. To validate the superiority and robustness of the proposed model, a total of six classic object detection models (i.e. Cascade Mask R-CNN, Cascade R-CNN, Faster R-CNN, feature selective anchor-free module, fully convolutional one-stage object detection, neural architecture search—fully convolutional one-stage object detection) are adopted for model training, validation, and testing with the customized inspection dataset, which includes 1,200 high resolution tunnel lining inspection images. With the training results and the visualization results, our proposed model has shown excellent performance across multiple metrics. Specifically, the enhanced Swin Transformer model achieves a remarkable 96.10 per cent on the mAP50 metric, surpassing the original model by 0.80 per cent. Moreover, it exhibits an accelerated detection speed of 2.4 tasks/s compared to the previous rate of 1.43 tasks/s. The results demonstrate that the proposed methodology in this paper significantly enhances both detection accuracy and speed of the model, paving the way for field application in the near future.

An improved cascade RCNN detection method for key components and defects of transmission lines

AbstractOverhead transmission line detection based on deep learning of aerial images taken by UAVs has been widely investigated. Despite its success, it is limited by several factors, including inappropriate evaluation criteria and dramatic scaling of components in the images. To mitigate these issues, a relative mean Average Precision evaluation index is proposed to accurately measure the model's detection performance for smaller objects. A data enhancement strategy including multi‐scale transformation is adopted to alleviate the problem of drastic scaling. The existing Cascade RCNN target detection technology is enhanced by incorporating Swin‐v2 and a balanced feature pyramid to improve feature characterization capabilities, while side‐aware boundary localization is utilized to improve the positioning accuracy of the model. Experimental results demonstrate that the proposed method outperforms state‐of‐the‐art methods on CPLID and achieves 7.8%, 11.8%, and 5.5% higher detection accuracy than the baseline for mAP50, relative small and medium mAP, respectively. Additionally, the paper discusses the influence of adopted data enhancement on the robustness of the model.

Decoupled Edge Guidance Network for Automatic Checkout.

Automatic checkout (ACO) aims at correctly generating complete shopping lists from checkout images. However, the domain gap between the single product in training data and multiple products in checkout images endows ACO tasks with a major difficulty. Despite remarkable advancements in recent years, resolving the significant domain gap remains challenging. It is possibly because networks trained solely on synthesized images may struggle to generalize well to realistic checkout scenarios. To this end, we propose a decoupled edge guidance network (DEGNet), which integrates synthesized and checkout images via a supervised domain adaptation approach and further learns common domain representations using a domain adapter. Specifically, an edge embedding module is designed for generating edge embedding images to introduce edge information. On this basis, we develop a decoupled feature extractor that takes original images and edge embedding images as input to jointly utilize image information and edge information. Furthermore, a novel proposal divide-and-conquer strategy (PDS) is proposed for the purpose of augmenting high-quality samples. Through experimental evaluation, DEGNet achieves state-of-the-art performance on the retail product checkout (RPC) dataset, with checkout accuracy (cAcc) results of 93.47% and 95.25% in the average mode of faster RCNN and cascade RCNN frameworks, respectively. Codes are available at https://github.com/yourbikun/DEGNet.

Deep-Learning-Based Detection of Transmission Line Insulators

At this stage, the inspection of transmission lines is dominated by UAV inspection. Insulators, as essential equipment for transmission line equipment, are susceptible to various factors during UAV detection, and their detection results often lead to leakages and false detection. Combining deep learning detection algorithms with the UAV transmission line inspection system can effectively solve the current sensing problem. To improve the recognition accuracy of insulator detection, the MS-COCO pre-training strategy that combines the FPN module with a cascading R-CNN algorithm based on the ResNeXt-101 network is proposed. The purpose of this paper is to systematically and comprehensively analyze mainstream isolator detection algorithms at the current stage and to verify the effectiveness of the improved Cascade R-CNN X101 model by combining the mAP (mean Average Precision) value and other related evaluation indices. Compared with Faster R-CNN, Retina Net, and other detection algorithms, the model is highly accurate and can effectively deal with the false detection, leakage, and non-recognition of the environment in online special detection. The research in this paper provides a new idea for intelligent fault detection of transmission line insulators and has some reference value for engineering applications.

Welding defects classification by weakly supervised semantic segmentation

Radiographic non-destructive evaluation (NDE) is an essential technique for understanding defects in welds. These radiographs require certified workers to interpret them to identify the presence of defects. Recent deep learning techniques, primarily semantic segmentation, could help welding defect detection and classification. Using image segmentation technology to obtain performance evaluations of the presence, location, and size of defects can improve the stability of defect evaluations while saving a great deal of time. However, supervised instance segmentation requires many manually implemented pixel-level annotations, dramatically increasing the difficulty and cost of achieving non-destructive evaluations. In our work, the weakly supervised semantic segmentation based on the Cut-Cascade RCNN model is used to classify defects. The cascade RCNN obtains the region of interest (ROI) and classification information. In the ROI, adaptive threshold segmentation of the defects is implemented, and the image is filtered to obtain the mask information. The accuracy of using the Cut-Cascade RCNN model in our x-ray dataset size can reach 90.15%.

Dead Laying Hens Detection Using TIR-NIR-Depth Images and Deep Learning on a Commercial Farm.

In large-scale laying hen farming, timely detection of dead chickens helps prevent cross-infection, disease transmission, and economic loss. Dead chicken detection is still performed manually and is one of the major labor costs on commercial farms. This study proposed a new method for dead chicken detection using multi-source images and deep learning and evaluated the detection performance with different source images. We first introduced a pixel-level image registration method that used depth information to project the near-infrared (NIR) and depth image into the coordinate of the thermal infrared (TIR) image, resulting in registered images. Then, the registered single-source (TIR, NIR, depth), dual-source (TIR-NIR, TIR-depth, NIR-depth), and multi-source (TIR-NIR-depth) images were separately used to train dead chicken detecting models with object detection networks, including YOLOv8n, Deformable DETR, Cascade R-CNN, and TOOD. The results showed that, at an IoU (Intersection over Union) threshold of 0.5, the performance of these models was not entirely the same. Among them, the model using the NIR-depth image and Deformable DETR achieved the best performance, with an average precision (AP) of 99.7% (IoU = 0.5) and a recall of 99.0% (IoU = 0.5). While the IoU threshold increased, we found the following: The model with the NIR image achieved the best performance among models with single-source images, with an AP of 74.4% (IoU = 0.5:0.95) in Deformable DETR. The performance with dual-source images was higher than that with single-source images. The model with the TIR-NIR or NIR-depth image outperformed the model with the TIR-depth image, achieving an AP of 76.3% (IoU = 0.5:0.95) and 75.9% (IoU = 0.5:0.95) in Deformable DETR, respectively. The model with the multi-source image also achieved higher performance than that with single-source images. However, there was no significant improvement compared to the model with the TIR-NIR or NIR-depth image, and the AP of the model with multi-source image was 76.7% (IoU = 0.5:0.95) in Deformable DETR. By analyzing the detection performance with different source images, this study provided a reference for selecting and using multi-source images for detecting dead laying hens on commercial farms.

Complementary Feature Pyramid Network for Object Detection

The way of constructing a robust feature pyramid is crucial for object detection. However, existing feature pyramid methods, which aggregate multi-level features by using element-wise sum or concatenation, are inefficient to construct a robust feature pyramid. The reason is that these methods cannot be effective in discriminating the relevant semantics of objects. In this article, we propose a Complementary Feature Pyramid Network (CFPN) to aggregate multi-level features selectively and efficiently by exploring complementary information between multi-level features. Specifically, a Spatial Complementary Module (SCM) and a Channel Complementary Module (CCM) are designed and embedded in CFPN to enhance useful information and suppress irrelevant information during feature fusions along spatial and channel dimensions, respectively. CFPN is a generic feature extractor, as evidenced by its seamless integration into single-stage, two-stage, and end-to-end object detectors. Experiments conducted on the COCO and Pascal VOC datasets demonstrate that integrating our CFPN into RetinaNet, Faster RCNN, Cascade RCNN, and Sparse RCNN obtains consistent performance improvements with negligible overheads. Code and models are available at: https://github.com/VIPLab-CQU/CFPN .

PNANet: Probabilistic Two-Stage Detector Using Pyramid Non-Local Attention

Object detection algorithms require compact structures, reasonable probability interpretability, and strong detection ability for small targets. However, mainstream second-order object detectors lack reasonable probability interpretability, have structural redundancy, and cannot fully utilize information from each branch of the first stage. Non-local attention can improve sensitivity to small targets, but most of them are limited to a single scale. To address these issues, we propose PNANet, a two-stage object detector with a probability interpretable framework. We propose a robust proposal generator as the first stage of the network and use cascade RCNN as the second stage. We also propose a pyramid non-local attention module that breaks the scale constraint and improves overall performance, especially in small target detection. Our algorithm can be used for instance segmentation after adding a simple segmentation head. Testing on COCO and Pascal VOC datasets as well as practical applications demonstrated good results in both object detection and instance segmentation tasks.

A novel method for intelligent analysis of rice yield traits based on LED transmission imaging and cloud computing

Rice yield traits extraction is necessary for rice breeding and genetic analysis, in which the discrimination of filled and unfilled grain was the bottleneck. The traditional methods including wind separating and water sinking are inefficient, unreliable, and poorly reproducible, meanwhile the machine vision methods based on X-ray and 3D structured imaging have the disadvantages of high cost, ionizing radiation, and poor scalability. To address the above problems, this study has proposed a novel method for intelligent analysis of rice yield traits based on LED transmission imaging and cloud application. The study has found that the red-light LED transmission imaging was able to obtain high-contrast images of filled and unfilled grains and had the advantages of radiation-free, low-cost, and high-efficiency imaging. Using 3,200 rice grain images as the training set, the results showed that the YOLOv5 model outperformed the Faster RCNN, RetinaNet, SSD, and Cascade RCNN models, and its filled and unfilled grain recognition precision was 98.94% and 90.96% on the 800 test sets, while the recall rate was 97.91% and 94.94%. To further improve the recognition accuracy of the YOLOv5 model, five model optimization methods were adopted in the study, including TPH, CBAM, adding a small target detection layer, using Bi-FPN and Multi-NMS. The results proved that the accuracy of unfilled grain recognition had significantly improved, the recognition precision increased by 4.54% to 95.44%, and the recall rate increased by 0.77% to 95.71%. Finally, this study has designed the specified algorithms based on the recognized rectangular box, which could accurately measure filled and unfilled grain numbers, seed setting rate and grain shape parameters. Moreover, the study has designed a local Qt software and a cloud-based website, which can support the simultaneous input of multiple images with high real-time performance and good scalability, providing a practical novel tool for rice breeding and genetic analysis.