Faster R-CNN Algorithm Research Articles

Generating real-world-like labelled synthetic datasets for construction site applications

Having synthetic image generation and automatic labelling as two separate processes remains one of the main limitations of automatic generation of large real-world synthetic datasets. To overcome this drawback, a methodology to perform both tasks in a simultaneous and automatic manner is proposed. To resemble real-world scenarios, a diverse set of rendering configurations of illumination, locations, and sizes are presented. For testing, three synthetic datasets (S, M and SM), oriented to the construction field, were generated. Faster R-CNN, RetinaNet, and YoloV4 detection algorithms were used to independently evaluate the datasets using the COCO evaluation metrics and the PascalVOC Mean Average Accuracy metric. Results show that, in general, the S dataset performed 1.2% better in the evaluation metrics and that the SM dataset obtained better plots of training and validation loss curves in each detector; highlighting the combinational usage of images with single and multiple objects as a better generalisation approach.

Robustness Analysis of Traffic Sign Recognization based on ResNet

Autonomous driving has always been an important research topic and application task of artificial intelligence, which has attracted the attention of a large number of researchers. As an important component of the environmental perception module in autonomous driving tasks, traffic sign recognition can help drivers understand road information in a timely manner and avoid potentially dangerous driving operations. To this end, accurate recognition of traffic signs is crucial from both a strategic and a practical point of view. The early recognition technology of traffic signs is mainly based on the detection of color and shape, whose recognition accuracy is limited due to the fading and deformation of traffic signs. Numerous researchers have successfully used these deep learning-based object identification algorithms for traffic sign detection and recognition thanks to the development of the Faster R-CNN and YOLO series algorithms. However, we argue that few efforts focus on the recognition performance of the model in different scenarios, which is especially important in the process of autonomous driving. Based on this observation, this paper first constructs the ResNet model. Part of the image recognition accuracy predicted by the model reached 99%. Afterward, the robustness of the model is explored by simulating complex scenes by changing illumination and noise, and it is proved that the model has good generalization ability and practical application ability.

Research on Transformer Equipment Identification Based on Improved Fast RCNN Algorithm

Because of the problems such as the long work cycle and manual efficiency faced by the traditional manual inspection and video monitoring inspection methods of substations, this paper proposes an electrical equipment identification algorithm that improves the Fast RCNN network to realize automatic identification and positioning of electrical equipment. We used affine transformation, background fusion, hue and saturation change, clipping and filling methods to enrich images and change the feature extraction network from VGG16 to ResNet50 with residual structure so that the feature extraction network can focus on specific anchor frames, and then used K-means clustering algorithm in the regional recommendation network to increase the convergence speed of the model, in this way, the improved anchoring height-to-width ratio is more suitable for detecting the image of power equipment. The experimental simulation shows that the average detection accuracy of the improved model is better than that of the original model, with the average accuracy mAP reaching 90.04%, and the image detection speed reaching 11 frames/second.

Analysis and Detection of Transmission Lines Based on Twin Reality

Now life has been transformed and upgraded to the direction of intelligence, digitalization and informatization along with the advancement of scientific and technological information. The traditional detection of transmission lines is completed by the professional maintenance personnel of the power grid through manual detection of lines. This method is inefficient and has a certain threat to the life safety of detection personnel. The development of deep learning and computational vision supplies a fresh idea for detection of transmission lines. Therefore, this paper analyzes the traditional Faster R-CNN based on twin reality scene modeling technology. Then aiming at the shortcomings of traditional algorithms, the Cascade R-CNN algorithm is proposed to complete the detection of transmission lines based on twin reality system. It compares the accuracy and other indicators of Cascade R-CNN and Faster R-CNN algorithms. The effectiveness of two algorithms for analysis and detection of transmission lines is verified through experiments. Finally, the results indicates that the system using Cascade R-CNN has higher prediction accuracy, and has better practicability for analysis and detection of transmission lines.

Banana Pseudostem Visual Detection Method Based on Improved YOLOV7 Detection Algorithm

Detecting banana pseudostems is an indispensable part of the intelligent management of banana cultivation, which can be used in settings such as counting banana pseudostems and smart fertilization. In complex environments, dense and occlusion banana pseudostems pose a significant challenge for detection. This paper proposes an improved YOLOV7 deep learning object detection algorithm, YOLOV7-FM, for detecting banana pseudostems with different growth conditions. In the loss optimization part of the YOLOV7 model, Focal loss is introduced, to optimize the problematic training for banana pseudostems that are dense and sheltered, so as to improve the recognition rate of challenging samples. In the data augmentation part of the YOLOV7 model, the Mixup data augmentation is used, to improve the model’s generalization ability for banana pseudostems with similar features to complex environments. This paper compares the AP (average precision) and inference speed of the YOLOV7-FM algorithm with YOLOX, YOLOV5, YOLOV3, and Faster R-CNN algorithms. The results show that the AP and inference speed of the YOLOV7-FM algorithm is higher than those models that are compared, with an average inference time of 8.0 ms per image containing banana pseudostems and AP of 81.45%. This improved YOLOV7-FM model can achieve fast and accurate detection of banana pseudostems.

Research on steel rail surface defects detection based on improved YOLOv4 network.

The surface images of steel rails are extremely difficult to detect and recognize due to the presence of interference such as light changes and texture background clutter during the acquisition process. To improve the accuracy of railway defects detection, a deep learning algorithm is proposed to detect the rail defects. Aiming at the problems of inconspicuous rail defects edges, small size and background texture interference, the rail region extraction, improved Retinex image enhancement, background modeling difference, and threshold segmentation are performed sequentially to obtain the segmentation map of defects. For the classification of defects, Res2Net and CBAM attention mechanism are introduced to improve the receptive field and small target position weights. The bottom-up path enhancement structure is removed from the PANet structure to reduce the parameter redundancy and enhance the feature extraction of small targets. The results show the average accuracy of rail defects detection reaches 92.68%, the recall rate reaches 92.33%, and the average detection time reaches an average of 0.068 s per image, which can meet the real-time of rail defects detection. Comparing the improved method with the mainstream target detection algorithms such as Faster RCNN, SSD, YOLOv3 and other algorithms, the improved YOLOv4 has excellent comprehensive performance for rail defects detection, the improved YOLOv4 model obviously better than several others in P r , R c , and F1 value, and can be well-applied to rail defect detection projects.

An Artificial Intelligence Model for Teeth Segmentation and Numbering on Orthopantomograms.

To develop a Deep Learning (DL) Artificial Intelligence (AI) model for the instance segmentation and numbering of teeth on Orthopantomograms (OPGs). Forty OPGs were manually annotated to lay down the ground truth for training two Convolutional Neural Networks (CNNs), U-net, and Faster RCNN. These algorithms were concurrently trained and validated on a dataset of 1,280 teeth (40 OPGs) each. The U-net algorithm was trained on OPGs specifically annotated with fluid margins to label all 32 teeth via instance segmentation allowing each tooth to be denoted as a separate entity from the surrounding structures. Simultaneously, teeth were also numbered as per the Fédération Dentaire Internationale (FDI) numbering system, using bounding boxes to train Faster RCNN. Consequently, both trained CNNs were combined to develop an AI model capable of segmenting and numbering all teeth on an OPG. The performance of the U-net algorithm was determined using various performance metrics including precision=88.8%, accuracy=88.2%, re-call=87.3%, F-1 score=88%, dice index=92.3% and Intersection over Union (IoU)=86.3%. The performance metrics of the Faster RCNN algorithm were determined using overlap accuracy=30.2 bounding boxes (out of a possible of 32 boxes) and classifier accuracy of labels=93.8%. The instance segmentation and teeth numbering results of our trained AI model were close to the ground truth, holding a promising future for its incorporation into clinical dental practice. The ability of an AI model to automatically identify teeth on OPGs will aid dentists with diagnosis and treatment planning thus increasing efficiency.

Weed Identification in Soybean Seedling Stage Based on Optimized Faster R-CNN Algorithm

Soybean in the field has a wide range of intermixed weed species and a complex distribution status, and the weed identification rate of traditional methods is low. Therefore, a weed identification method is proposed based on the optimized Faster R-CNN algorithm for the soybean seedling. Three types of weed datasets, including soybean, with a total of 9816 photos were constructed, and cell phone photo data were used for training and recognition. Firstly, by comparing the classification effects of ResNet50, VGG16, and VGG19, VGG19 was identified as the best backbone feature extraction network for model training. Secondly, an attention mechanism was embedded after the pooling layer in the second half of VGG19 to form the VGG19-CBAM structure, which solved the problem of low attention to the attention target during model training using the trained Faster R-CNN algorithm to identify soybean and weeds in the field under the natural environment and compared with two classical target detection algorithms, SSD and Yolov4. The experimental results show that the Faster R-CNN algorithm using VGG19-CBAM as the backbone feature extraction network can effectively identify soybeans and weeds in complex backgrounds. The average recognition speed for a single image is 336 ms, and the average recognition accuracy is 99.16%, which is 5.61% higher than before optimization, 2.24% higher than the SSD algorithm, and 1.24% higher than the Yolov4 algorithm. Therefore, this paper’s optimized target detection model is advantageous and can provide a scientific method for accurate identification and monitoring of grass damage.

Cervical cell classification with deep-learning algorithms.

Cervical cancer is a serious threat to the lives and health of women. The accurate analysis of cervical cell smear images is an important diagnostic basis for cancer identification. However, pathological data are often complex and difficult to analyze accurately because pathology images contain a wide variety of cells. To improve the recognition accuracy of cervical cell smear images, we propose a novel deep-learning model based on the improved Faster R-CNN, shallow feature enhancement networks, and generative adversarial networks. First, we used a global average pooling layer to enhance the robustness of the data feature transformation. Second, we designed a shallow feature enhancement network to improve the localization and recognition of weak cells. Finally, we established a data augmentation network to improve the detection capability of the model. The experimental results demonstrate that our proposed methods are superior to CenterNet, YOLOv5, and Faster R-CNN algorithms in some aspects, such as shorter time consumption, higher recognition precision, and stronger adaptive ability. Its maximum accuracy is 99.81%, and the overall mean average precision is 89.4% for the SIPaKMeD and Herlev datasets. Our method provides a useful reference for cervical cell smear image analysis. The missed diagnosis rate and false diagnosis rate are relatively high for cervical cell smear images of different pathologies and stages. Therefore, our algorithms need to be further improved to achieve a better balance. We will use a hyperspectral microscope to obtain more spectral data of cervical cells and input them into deep-learning models for data processing and classification research. First, we sent training samples of cervical cells into our proposed deep-learning model. Then, we used the proposed model to train eight types of cervical cells. Finally, we utilized the trained classifier to test the untrained samples and obtained the classification results. Fig 1. Deep-learning cervical cell classification framework.

Classification of Long-Bone Fractures Using Modified Faster RCNN for X-Ray Images

Objectives: Methods in machine learning have been shown to be an essential tool for the diagnosis and treatment of disease. The scientists are constantly looking for new technology that might improve current clinical practise. One of the most well-liked study fields continues to be computerized bone fracture identification and categorization. Furthermore, identifying cracks and determining where they occur have been challenges for historically developed approaches for the long bone fracture method. Methods: We propose a new approach for automatic fracture detection in X-Ray images. This approach is built on top of two-stage fracture detection deep learning algorithm called Faster R-CNN with a major modification of using rotated bounding box. The described procedure is divided into four main steps: X-ray images are used to: I identify the bone contour; (ii) identify fracture-points or fractures; (iii) find a similar set of shapes that are compatible with the identification of cracks; and (iv) classify and thoroughly evaluate the fracture-type using bounding box. Finding: The resolution process uses the stretched numerical conventional line techniques (RDS), arcs, discrete curvature, and shape directory, among other mathematical characteristics of digital curves. We assessed the suggested model’s performance in terms of classification and detection. We divide x-ray pictures of bone fractures into 2 groups, fracture and non-fracture, and we also use a rectangular box to identify the location of fractures. Besides, an additional benefit of rotated bounding box is that it can provide relative information on the orientation and length of fracture without the further segmentation and measurement step. Novelty: This study develops a new approach to automatically and accurately detect fractures in X-Ray images. The proposed approach is developed on the basis of Faster R-CNN algorithm with a major modification for the task of rotated bounding box prediction. Keywords: Long bone; X-Ray images; Classification; Detection; Faster RCNN; Fracture; Non-Fracture; Types

Research on application of helmet wearing detection improved by YOLOv4 algorithm.

Aiming at the problem that the model of YOLOv4 algorithm has too many parameters and the detection effect of small targets is poor, this paper proposes an improved helmet fitting detection model based on YOLOv4 algorithm. Firstly, this model improves the detection accuracy of small targets by adding multi-scale prediction and improving the structure of PANet network. Then, the improved depth-separable convolution was used to replace the standard 3 × 3 convolution, which greatly reduced the model parameters without reducing the detection ability of the model. Finally, the k_means clustering algorithm is used to optimize the prior box. The model was tested on the self-made helmet dataset helmet_dataset. Experimental results show that compared with the safety helmet detection model based on Faster RCNN algorithm, the improved YOLOv4 algorithm has faster detection speed, higher detection accuracy and smaller number of model parameters. Compared with the original YOLOv4 model, the mAP of the improved YOLOv4 algorithm is increased by 0.49%, reaching 93.05%. The number of model parameters was reduced by about 58%, to about 105 MB. The model reasoning speed is 35 FPS. The improved YOLOv4 algorithm can meet the requirements of helmet wearing detection in multiple scenarios.

Automatic pothole detection

Road irregularities, such as potholes and road cracks, are a common hazard of daily commute and transportation. Automobile owners everywhere have long worried about the high price of fixing damage caused by potholes. Many accidents caused by potholes can be avoided if road repair agencies can respond quickly. Thus, an automated pothole detection system may help improve this issue. In this work, automatic pothole detection is performed using Faster R-CNN, Sparse R-CNN, YOLOv3, YOLOv5, and YOLOv7 computer vision algorithms. For this purpose, the dataset of road damage images collected in the Slovak region was used. The use of country-specific data is of great importance since the accuracy of pothole detection is dependent on the country of origin of the data. In this work, YOLOv7 achieved the highest detection accuracy of 0.884 (mAP@0.5) followed by YOLOv5 with the accuracy of 0.854 (mAP@0.5) on the pothole dataset.

Optimizing Internet of Things-Based Intelligent Transportation System’s Information Acquisition Using Deep Learning

This work first discusses the Intelligent Transportation System (ITS)-oriented dynamic and static Information Acquisition Models (IAMs) and explains the information collection mechanism of the Internet of Things (IoT)-based ITS. The goal is to improve travel conditions and contribute to a better urban environment. In order to do so, the Faster Region-based Convolutional Neural Network (Faster R-CNN) is introduced to extract the IoT-based ITS’s electronic data features. It is observed that the Faster R-CNN has excellent recall and accuracy in extracting the features from the ITS electronic data sets. Specifically, the Faster R-CNN’s average recall and accuracy reach 83.89% and 86.79%. The accuracy is 6.20% higher than the R-CNN method. Thus, the Faster R-CNN algorithm features more robust and reliable performance for collecting and analyzing ITS data. Overall, this work examines ITS-oriented electronic information collection and automatic detection against the technological background of applying Computer Vision, Deep Learning, and IoT in urban traffic management. In particular, it explains the IoT-based ITS’s electronic information collection mechanism under Deep Learning (Faster R-CNN). The finding offers a theoretical foundation for implementing Deep Learning technologies in collecting ITS-oriented big data and smart city construction.

Research on improved character recognition technology of wagon number based on Faster-RCNN

Aiming at the characteristics of wagon number character printing with location features, we propose a wagon number character recognition technique based on the Faster-RCNN algorithm for the output layer to improve the accuracy of recognition by eliminating and filling the misjudged and missed characters in the output layer of image recognition. Using the relative position of each wagon number character to form a relative position matrix, the correct candidate frame is output by calculating the similarity of the relative position matrix of the candidate frame and the standard frame, and the correct wagon number is retrieved by comparing with the wagon number in the database.

Defect Detection of Composite Material Terahertz Image Based on Faster Region-Convolutional Neural Networks

Terahertz (THz) nondestructive testing (NDT) technology has been increasingly applied to the internal defect detection of composite materials. However, the THz image is affected by background noise and power limitation, leading to poor THz image quality. The recognition rate based on traditional machine vision algorithms is not high. The above methods are usually unable to determine surface defects in a timely and accurate manner. In this paper, we propose a method to detect the internal defects of composite materials by using terahertz images based on a faster region-convolutional neural networks (faster R-CNNs) algorithm. Terahertz images showing internal defects in composite materials are first acquired by a terahertz time-domain spectroscopy system. Then the terahertz images are filtered, the blurred images are removed, and the remaining images are enhanced with data and annotated with image defects to create a dataset consistent with the internal defects of the material. On the basis of the above work, an improved faster R-CNN algorithm is proposed in this paper. The network can detect various defects in THz images by changing the backbone network, optimising the training parameters, and improving the prior box algorithm to improve the detection accuracy and efficiency of the network. By taking the commonly used composite sandwich structure as a representative, a sample with typical defects is designed, and the image data are obtained through the test. Comparing the proposed method with other existing network methods, the former proves to have the advantages of a short training time and high detection accuracy. The results show that the mean average precision (mAP) without data enhancement reached 95.50%, and the mAP with data enhancement reached 98.35% and exceeded the error rate of human eye detection (5%). Compared with the original faster R-CNN algorithm of 84.39% and 85.12%, the improvement is 11.11% and 10.23%, respectively, which demonstrates superb feature extraction capability and reduces the occurrence of network errors and omissions.

An improved faster R-CNN algorithm for assisted detection of lung nodules

The morbidity and mortality of lung cancer are increasing rapidly in every country in the world, and pulmonary nodules are the main symptoms of lung cancer in the early stage. If we can diagnose pulmonary nodules in time at the early stage and follow up and treat suspicious patients, we can effectively reduce the incidence of lung cancer. CT (Computed Tomography) has been applied to the screening of many diseases because of its high resolution. Pulmonary nodules show white round shadows in CT images. With the popularity of CT equipment, doctors need to review a large number of imaging results every day. Doctors will misjudge and miss the lesions because of reviewing CT scanning results for a long time. At this time, the method of automatic detection of pulmonary nodules by computer can relieve the pressure of doctors in reviewing CT scan results.Traditional lung nodule detection methods, such as gray threshold method and region growing method, divide the detection process into two steps: extracting candidate regions and eliminating false regions. In addition, the traditional detection method can only operate on a single image, which leads to the inability of this method to detect the batch scanning results in real time. With the continuous development of computer equipment performance and artificial intelligence, the relationship between medical image processing and deep learning is getting closer and closer. In deep learning, object detection methods such as Faster R-CNN、YOLO can complete parallel detection of batch images, and deep structure can fully extract the features of input images. Compared with traditional lung nodule detection methods, it has the characteristics of high efficiency and high precision. Faster R-CNN is a classical and high-precision two-stage object detection method. In this paper, an improved Faster R-CNN model is proposed. On the basis of Faster R-CNN, multi-scale training strategy is used to fully mine the features of different scale spaces and perform path augmentation on lower-dimensional features, which improves the small object detection ability of the model. Through Online Hard Example Mining (OHEM), the loss value is used to quantify the difficulty of candidate region detection, and the training times of the region to be detected are adaptively adjusted. Make full use of prior information to customize the size and proportion of preset boundary anchor boxes. Using deformable convolution to improve the visual field to enhance the global features and enhance the ability to extract the feature information of pulmonary nodules in the same scale space.The new model was tested on LUNA16 (Lung Nodule Analysis 2016) dataset. The detection precision of the improved Faster R-CNN model for pulmonary nodules increased from 76.4% to 90.7%, and the recall rate increased from 40.1% to 56.8% Compared with the mainstream object detection algorithms YOLOv3 and Cascade R-CNN, the improved model is superior to the above models in every index.

Dam Crack Image Detection Model on Feature Enhancement and Attention Mechanism

Dam crack detection can effectively avoid safety accidents of dams. To solve the problem that the dam crack image samples are not available and the traditional algorithm detects cracks with low accuracy, we provide a dam crack image detection model based on crack feature enhancement and attention mechanism. Firstly, we expand the dam crack image dataset through a generative adversarial network based on crack feature enhancement (Cracks Enhancements GAN, CE-GAN). It can fully expand the dam crack data samples and improve the quality of the training data. Secondly, we propose a crack image detection model based on the attention mechanism (Attention-based Faster-RCNN, AF-RCNN). The attention mechanism is added in the crack detection module to give different weights to the proposal boxes around the crack target and fuse the candidate boxes with high weights to accurately detect the crack target location. The experimental results show that our algorithm achieves 81.07% mAP on the expanded dam crack dataset, which is 8.39% higher than the original Faster-RCNN algorithm. The detection accuracy is significantly improved compared with other traditional dam crack detection algorithm models.

Insulator image autonomous recognition and defect intelligent detection based on multispectral image

Insulator determines the insulation level and power supply reliability of transmission lines. The traditional operation and maintenance method of insulators has a large workload. This paper presents an insulator recognition and fault diagnosis system based on image recognition and machine learning. Firstly, the composite insulators in complex backgrounds have been identified by Faster RCNN algorithm, which helps to extract the image of insulators by drone shot. Then, the cracking of umbrella skirts has been carried out by means of image processing. Also, the contamination of composite insulator umbrella skirts is also identified. The appropriate feature quantity by Fisher’s discrimination is recommended, and the insulator contamination level has been identified by S component. Lastly, analyzing the infrared image of the insulator provides the basis for the normalization of insulator temperature rise detection results in different environments. Insulator image autonomous recognition and defect intelligent detection, which is based on deep reinforcement learning, is helpful for the operation, maintenance and repair of line insulators.

Remote sensing landslide target detection method based on improved Faster R-CNN

To timely detect landslide hazards to start emergency rescue, an improved Faster R-CNN algorithm is proposed for remote sensing image landslide detection. First, the gamma transform and Gaussian filtering methods of image enhancement are used to improve the quality of the images. Second, the effect of batchsize size on the model is eliminated using the group normalization method. Finally, multiscale feature fusion is performed by adding a feature pyramid network structure to optimize the extracted landslide small target features, and then the backbone network is set as deep residual shrinkage network 50 to make the model more focused on information useful for landslide detection. The experimental results show that the improved model improves the accuracy rate as well as the average precision by 8.8% and 8.4%, respectively, compared with the unimproved Faster R-CNN, and compared with the first-stage models, such as you only look once version 4 and single-shot detector, which verify the superiority of the model in our study and can detect landslide targets well.

Sugarcane stem node detection and localization for cutting using deep learning.

In order to promote sugarcane pre-cut seed good seed and good method planting technology, we combine the development of sugarcane pre-cut seed intelligent 0p99oposeed cutting machine to realize the accurate and fast identification and cutting of sugarcane stem nodes. In this paper, we proposed an algorithm to improve YOLOv4-Tiny for sugarcane stem node recognition. Based on the original YOLOv4-Tiny network, the three maximum pooling layers of the original YOLOv4-tiny network were replaced with SPP (Spatial Pyramid Pooling) modules, which fuse the local and global features of the images and enhance the accurate localization ability of the network. And a 1×1 convolution module was added to each feature layer to reduce the parameters of the network and improve the prediction speed of the network. On the sugarcane dataset, compared with the Faster-RCNN algorithm and YOLOv4 algorithm, the improved algorithm yielded an mean accuracy precision (MAP) of 99.11%, a detection accuracy of 97.07%, and a transmission frame per second (fps) of 30, which can quickly and accurately detect and identify sugarcane stem nodes. In this paper, the improved algorithm is deployed in the sugarcane stem node fast identification and dynamic cutting system to achieve accurate and fast sugarcane stem node identification and cutting in real time. It improves the seed cutting quality and cutting efficiency and reduces the labor intensity.