R-CNN Algorithm Research Articles

Multidefect Detection Tool for Large-Scale PV Plants: Segmentation and Classification

Unmanned aerial vehicles (UAVs) with high-resolution optical and infrared ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IR</i> ) imaging have been introduced in recent years to perform inexpensive and fast inspections in operation and maintenance activities of solar power plants, reducing the labor needed, while lowering the on-site inspection time. Even though UAVs can acquire images extremely quickly, the analysis of those images is still a time-consuming procedure that should be performed by a trained professional. Therefore, a computer vision approach may be used to accelerate image analysis. In this work, a dataset of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IR</i> images was created from a 10-MW solar power plant and a comparative analysis between mask R- convolutional neural network (CNN) and U-Net was performed for two experiments. Concerning the defective module segmentation, the mask R-CNN algorithm achieved a mean average precision at intersection over union (IoU) = 0.50 of 0.96, using augmentation data. Regarding the segmentation and classification of failure type, the algorithm reached a value of 0.88 considering the same evaluation metric and data augmentation. When compared to the U-Net in terms of IoU, the mask R-CNN outperformed it with 0.87 and 0.83 for the first and second experiments, respectively.

Transfer Learning Approaches for Leaf Diseased Portion Detection

Abstract—Overview - In this study, The difficulties in iden- tifying and classifying diseased areas on leaves prompted us to create a deep block attention solid-state drive (SSD) for identifying diseased areas on leaves and classifying their severity. We propose a novel leaf disease portion method. Squeeze- and- excitation SSD (Se-SSD), and DBA-SSD approaches are recommended for finding plant leaves.SSD feature extraction network and channel attention mechanism are coupled by Se SSD, VGG feature extraction network is enhanced by DB SSD, and channel attention mechanism is coupled by DBA SSD. Convolutional layers trained on Image Net images using the VGG model are transferred to this model, reducing training time and speeding up the training process. On the other hand, the collected images containing plant leaves are randomly split into a training set and a test setup .1:1 ratio. After careful consideration, we chose the Leaf Village dataset because it contains imagery relevant to the study area. This collection contains leaf images, including images of cotton leaves. Image data is enhanced using data enhancement techniques such as horizontal flipping and histogram equalization. In this study, we examine and contrast the enhanced performance of VGG16 and RCNN with that of the more conventional Faster RCNN and Single Shot target identification approaches operating in the same setting. Comparing its performance with that of other target identification algorithms, we find that SSD and faster RCNN outperforms other algorithms in terms of algorithm. Index Terms—VGG16 Architecture, SSD Algorithm, Faster- RCNN Algorithm,Leaf diseased portion

Research on steel rail surface defects detection based on improved YOLOv4 network.

The surface images of steel rails are extremely difficult to detect and recognize due to the presence of interference such as light changes and texture background clutter during the acquisition process. To improve the accuracy of railway defects detection, a deep learning algorithm is proposed to detect the rail defects. Aiming at the problems of inconspicuous rail defects edges, small size and background texture interference, the rail region extraction, improved Retinex image enhancement, background modeling difference, and threshold segmentation are performed sequentially to obtain the segmentation map of defects. For the classification of defects, Res2Net and CBAM attention mechanism are introduced to improve the receptive field and small target position weights. The bottom-up path enhancement structure is removed from the PANet structure to reduce the parameter redundancy and enhance the feature extraction of small targets. The results show the average accuracy of rail defects detection reaches 92.68%, the recall rate reaches 92.33%, and the average detection time reaches an average of 0.068 s per image, which can meet the real-time of rail defects detection. Comparing the improved method with the mainstream target detection algorithms such as Faster RCNN, SSD, YOLOv3 and other algorithms, the improved YOLOv4 has excellent comprehensive performance for rail defects detection, the improved YOLOv4 model obviously better than several others in P r , R c , and F1 value, and can be well-applied to rail defect detection projects.

Car Detection from Very High-Resolution UAV Images Using Deep Learning Algorithms

It is important to determine car density in parking lots, especially in hospitals, large enterprises, and residential areas, which are used intensively, in terms of executing existing management systems and making precise plans for the future. In this study, cars in parking lots were detected using high-resolution unmanned aerial vehicle (UAV) images with deep learning methods. We tested the performance of the two approaches by determining the number of cars in a parking lot using the You Only Look Once (YOLOv3) and Mask Region–Based Convolutional Neural Networks (Mask R-CNN) approaches as deep learning methods and the deep learning tool of Esri ArcGIS Pro. High-resolution UAV images were processed by photogrammetry and used as input products for the R-CNN and YOLOv3 algorithm. Recall, F1 score, precision ratio/uncertainty accuracy, and average producer accuracy of products automatically extracted with the algorithm were determined as 0.862/0.941, 0.874/0.946, 0.885/0.951, and 0.776/0.897 for R-CNN and YOLOv3, respectively.

An Artificial Intelligence Model for Teeth Segmentation and Numbering on Orthopantomograms.

To develop a Deep Learning (DL) Artificial Intelligence (AI) model for the instance segmentation and numbering of teeth on Orthopantomograms (OPGs). Forty OPGs were manually annotated to lay down the ground truth for training two Convolutional Neural Networks (CNNs), U-net, and Faster RCNN. These algorithms were concurrently trained and validated on a dataset of 1,280 teeth (40 OPGs) each. The U-net algorithm was trained on OPGs specifically annotated with fluid margins to label all 32 teeth via instance segmentation allowing each tooth to be denoted as a separate entity from the surrounding structures. Simultaneously, teeth were also numbered as per the Fédération Dentaire Internationale (FDI) numbering system, using bounding boxes to train Faster RCNN. Consequently, both trained CNNs were combined to develop an AI model capable of segmenting and numbering all teeth on an OPG. The performance of the U-net algorithm was determined using various performance metrics including precision=88.8%, accuracy=88.2%, re-call=87.3%, F-1 score=88%, dice index=92.3% and Intersection over Union (IoU)=86.3%. The performance metrics of the Faster RCNN algorithm were determined using overlap accuracy=30.2 bounding boxes (out of a possible of 32 boxes) and classifier accuracy of labels=93.8%. The instance segmentation and teeth numbering results of our trained AI model were close to the ground truth, holding a promising future for its incorporation into clinical dental practice. The ability of an AI model to automatically identify teeth on OPGs will aid dentists with diagnosis and treatment planning thus increasing efficiency.

Object Recognition System for the Visually Impaired: A Deep Learning Approach using Arabic Annotation

Object detection is an important computer vision technique that has increasingly attracted the attention of researchers in recent years. The literature to date in the field has introduced a range of object detection models. However, these models have largely been English-language-based, and there is only a limited number of published studies that have addressed how object detection can be implemented for the Arabic language. As far as we are aware, the generation of an Arabic text-to-speech engine to utter objects’ names and their positions in images to help Arabic-speaking visually impaired people has not been investigated previously. Therefore, in this study, we propose an object detection and segmentation model based on the Mask R-CNN algorithm that is capable of identifying and locating different objects in images, then uttering their names and positions in Arabic. The proposed model was trained on the Pascal VOC 2007 and 2012 datasets and evaluated on the Pascal VOC 2007 testing set. We believe that this is one of a few studies that uses these datasets to train and test the Mask R-CNN model. The performance of the proposed object detection model was evaluated and compared with previous object detection models in the literature, and the results demonstrated its superiority and ability to achieve an accuracy of 83.9%. Moreover, experiments were conducted to evaluate the performance of the incorporated translator and TTS engines, and the results showed that the proposed model could be effective in helping Arabic-speaking visually impaired people understand the content of digital images.

Multispecies individual tree crown extraction and classification based on BlendMask and high-resolution UAV images

Accurate detection and segmentation of individual trees from unmanned aerial vehicle images is critical for forestry resource surveys and accurate forest management. Deep learning methods have been used for studies of individual tree crown segmentation, classification, and number of trees in mixed coniferous and broad-leaved forests, but the accuracy needs to be improved. Therefore, this study uses BlendMask, a simpler and more efficient algorithm that combines Mask R-CNN and Yolact algorithms to effectively combine instance-level information with semantic information at a finer granularity level, greatly improving crown segmentation accuracy and classification results. Three coniferous species and five broad-leaved species unmanned aerial vehicle images collected from the Jing Yue multispecies ecological forestry site in Changping District, Beijing, were used as the dataset, and the results were compared with Yolact and Mask R-CNN. The results show that the method described in this work has the highest Kappa coefficient (0.89) and overall accuracy (92.14%) in the test set. For segmentation accuracy, coniferous species’ producer’s accuracy was 0.91 to 0.95, whereas that of broad-leaved species was 0.89 to 0.92. For species classification, the F1-score and mean average precision for coniferous species were greater than 91%, whereas those for broad-leaved species were 77.64% to 85.63%. The accuracy of extracting stand density in low and medium canopy density stands was 0.9909 and 0.9422, respectively, whereas that in high canopy density stands was 0.8913. This study shows that the BlendMask model has a good effect in studying the classification of multiple tree species, the segmentation of individual tree crowns, and the statistics of the number of trees in complex forest areas. Compared with broad-leaved forests and high canopy density stands, this model is more suitable for coniferous forest and medium and low canopy density stand scenarios. This study provides an important tool for obtaining more accurate species classification, canopy segmentation, and resource inventory results in complex forest areas.

Cervical cell classification with deep-learning algorithms.

Cervical cancer is a serious threat to the lives and health of women. The accurate analysis of cervical cell smear images is an important diagnostic basis for cancer identification. However, pathological data are often complex and difficult to analyze accurately because pathology images contain a wide variety of cells. To improve the recognition accuracy of cervical cell smear images, we propose a novel deep-learning model based on the improved Faster R-CNN, shallow feature enhancement networks, and generative adversarial networks. First, we used a global average pooling layer to enhance the robustness of the data feature transformation. Second, we designed a shallow feature enhancement network to improve the localization and recognition of weak cells. Finally, we established a data augmentation network to improve the detection capability of the model. The experimental results demonstrate that our proposed methods are superior to CenterNet, YOLOv5, and Faster R-CNN algorithms in some aspects, such as shorter time consumption, higher recognition precision, and stronger adaptive ability. Its maximum accuracy is 99.81%, and the overall mean average precision is 89.4% for the SIPaKMeD and Herlev datasets. Our method provides a useful reference for cervical cell smear image analysis. The missed diagnosis rate and false diagnosis rate are relatively high for cervical cell smear images of different pathologies and stages. Therefore, our algorithms need to be further improved to achieve a better balance. We will use a hyperspectral microscope to obtain more spectral data of cervical cells and input them into deep-learning models for data processing and classification research. First, we sent training samples of cervical cells into our proposed deep-learning model. Then, we used the proposed model to train eight types of cervical cells. Finally, we utilized the trained classifier to test the untrained samples and obtained the classification results. Fig 1. Deep-learning cervical cell classification framework.

Weed Identification in Soybean Seedling Stage Based on Optimized Faster R-CNN Algorithm

Soybean in the field has a wide range of intermixed weed species and a complex distribution status, and the weed identification rate of traditional methods is low. Therefore, a weed identification method is proposed based on the optimized Faster R-CNN algorithm for the soybean seedling. Three types of weed datasets, including soybean, with a total of 9816 photos were constructed, and cell phone photo data were used for training and recognition. Firstly, by comparing the classification effects of ResNet50, VGG16, and VGG19, VGG19 was identified as the best backbone feature extraction network for model training. Secondly, an attention mechanism was embedded after the pooling layer in the second half of VGG19 to form the VGG19-CBAM structure, which solved the problem of low attention to the attention target during model training using the trained Faster R-CNN algorithm to identify soybean and weeds in the field under the natural environment and compared with two classical target detection algorithms, SSD and Yolov4. The experimental results show that the Faster R-CNN algorithm using VGG19-CBAM as the backbone feature extraction network can effectively identify soybeans and weeds in complex backgrounds. The average recognition speed for a single image is 336 ms, and the average recognition accuracy is 99.16%, which is 5.61% higher than before optimization, 2.24% higher than the SSD algorithm, and 1.24% higher than the Yolov4 algorithm. Therefore, this paper’s optimized target detection model is advantageous and can provide a scientific method for accurate identification and monitoring of grass damage.

Classification of Long-Bone Fractures Using Modified Faster RCNN for X-Ray Images

Objectives: Methods in machine learning have been shown to be an essential tool for the diagnosis and treatment of disease. The scientists are constantly looking for new technology that might improve current clinical practise. One of the most well-liked study fields continues to be computerized bone fracture identification and categorization. Furthermore, identifying cracks and determining where they occur have been challenges for historically developed approaches for the long bone fracture method. Methods: We propose a new approach for automatic fracture detection in X-Ray images. This approach is built on top of two-stage fracture detection deep learning algorithm called Faster R-CNN with a major modification of using rotated bounding box. The described procedure is divided into four main steps: X-ray images are used to: I identify the bone contour; (ii) identify fracture-points or fractures; (iii) find a similar set of shapes that are compatible with the identification of cracks; and (iv) classify and thoroughly evaluate the fracture-type using bounding box. Finding: The resolution process uses the stretched numerical conventional line techniques (RDS), arcs, discrete curvature, and shape directory, among other mathematical characteristics of digital curves. We assessed the suggested model’s performance in terms of classification and detection. We divide x-ray pictures of bone fractures into 2 groups, fracture and non-fracture, and we also use a rectangular box to identify the location of fractures. Besides, an additional benefit of rotated bounding box is that it can provide relative information on the orientation and length of fracture without the further segmentation and measurement step. Novelty: This study develops a new approach to automatically and accurately detect fractures in X-Ray images. The proposed approach is developed on the basis of Faster R-CNN algorithm with a major modification for the task of rotated bounding box prediction. Keywords: Long bone; X-Ray images; Classification; Detection; Faster RCNN; Fracture; Non-Fracture; Types

Automatic pothole detection

Road irregularities, such as potholes and road cracks, are a common hazard of daily commute and transportation. Automobile owners everywhere have long worried about the high price of fixing damage caused by potholes. Many accidents caused by potholes can be avoided if road repair agencies can respond quickly. Thus, an automated pothole detection system may help improve this issue. In this work, automatic pothole detection is performed using Faster R-CNN, Sparse R-CNN, YOLOv3, YOLOv5, and YOLOv7 computer vision algorithms. For this purpose, the dataset of road damage images collected in the Slovak region was used. The use of country-specific data is of great importance since the accuracy of pothole detection is dependent on the country of origin of the data. In this work, YOLOv7 achieved the highest detection accuracy of 0.884 (mAP@0.5) followed by YOLOv5 with the accuracy of 0.854 (mAP@0.5) on the pothole dataset.

Optimizing Internet of Things-Based Intelligent Transportation System’s Information Acquisition Using Deep Learning

This work first discusses the Intelligent Transportation System (ITS)-oriented dynamic and static Information Acquisition Models (IAMs) and explains the information collection mechanism of the Internet of Things (IoT)-based ITS. The goal is to improve travel conditions and contribute to a better urban environment. In order to do so, the Faster Region-based Convolutional Neural Network (Faster R-CNN) is introduced to extract the IoT-based ITS’s electronic data features. It is observed that the Faster R-CNN has excellent recall and accuracy in extracting the features from the ITS electronic data sets. Specifically, the Faster R-CNN’s average recall and accuracy reach 83.89% and 86.79%. The accuracy is 6.20% higher than the R-CNN method. Thus, the Faster R-CNN algorithm features more robust and reliable performance for collecting and analyzing ITS data. Overall, this work examines ITS-oriented electronic information collection and automatic detection against the technological background of applying Computer Vision, Deep Learning, and IoT in urban traffic management. In particular, it explains the IoT-based ITS’s electronic information collection mechanism under Deep Learning (Faster R-CNN). The finding offers a theoretical foundation for implementing Deep Learning technologies in collecting ITS-oriented big data and smart city construction.

Tobacco Leaf Segmentation Based on Improved MASK RCNN Algorithm and SAM Model

Tobacco Leaf Segmentation Based on Improved MASK RCNN Algorithm and SAM Model

Research on application of helmet wearing detection improved by YOLOv4 algorithm.

Aiming at the problem that the model of YOLOv4 algorithm has too many parameters and the detection effect of small targets is poor, this paper proposes an improved helmet fitting detection model based on YOLOv4 algorithm. Firstly, this model improves the detection accuracy of small targets by adding multi-scale prediction and improving the structure of PANet network. Then, the improved depth-separable convolution was used to replace the standard 3 × 3 convolution, which greatly reduced the model parameters without reducing the detection ability of the model. Finally, the k_means clustering algorithm is used to optimize the prior box. The model was tested on the self-made helmet dataset helmet_dataset. Experimental results show that compared with the safety helmet detection model based on Faster RCNN algorithm, the improved YOLOv4 algorithm has faster detection speed, higher detection accuracy and smaller number of model parameters. Compared with the original YOLOv4 model, the mAP of the improved YOLOv4 algorithm is increased by 0.49%, reaching 93.05%. The number of model parameters was reduced by about 58%, to about 105 MB. The model reasoning speed is 35 FPS. The improved YOLOv4 algorithm can meet the requirements of helmet wearing detection in multiple scenarios.

Picture Processing Optimization Technology Based on Mask R-CNN Algorithm

Picture processing is applied in all kind of fields, such as space science research, medical imaging, photography art. Because the human vision system is a complex nonlinear dynamic system, the traditional image enhancement methods often can not meet the requirements of real-time in practical applications. In the face of a large number of images, it is of great practical significance to obtain practical information from these data. With the rapid development of computer CnTech, image compression coding CnTech has also made great progress. And through the Mask R-CNN algorithm for effective segmentation and image recognition, can help people face massive image information, to their own needs as the goal, to achieve efficient retrieval, analysis, induction. The traditional methods of image classification by manually selecting features or manually extracting template matching have some defects. In this paper, a Mask R-CNN image optimization processing CnTech is proposed, which can accurately identify images and has higher accuracy for image feature extraction. It is meaningful to achieve automatic and accurate segmentation of microscopic images. Through training on COCO data set, it is verified through experiments that there are problems in image segmentation and recognition. In Mask R-CNN model, resource consumption is reduced and the efficiency of image segmentation and recognition is improved. Compared with the current cutting-edge algorithms, the image object detection on the strength of Mask R-CNN model has significant advantages.

Defect Detection of Composite Material Terahertz Image Based on Faster Region-Convolutional Neural Networks

Terahertz (THz) nondestructive testing (NDT) technology has been increasingly applied to the internal defect detection of composite materials. However, the THz image is affected by background noise and power limitation, leading to poor THz image quality. The recognition rate based on traditional machine vision algorithms is not high. The above methods are usually unable to determine surface defects in a timely and accurate manner. In this paper, we propose a method to detect the internal defects of composite materials by using terahertz images based on a faster region-convolutional neural networks (faster R-CNNs) algorithm. Terahertz images showing internal defects in composite materials are first acquired by a terahertz time-domain spectroscopy system. Then the terahertz images are filtered, the blurred images are removed, and the remaining images are enhanced with data and annotated with image defects to create a dataset consistent with the internal defects of the material. On the basis of the above work, an improved faster R-CNN algorithm is proposed in this paper. The network can detect various defects in THz images by changing the backbone network, optimising the training parameters, and improving the prior box algorithm to improve the detection accuracy and efficiency of the network. By taking the commonly used composite sandwich structure as a representative, a sample with typical defects is designed, and the image data are obtained through the test. Comparing the proposed method with other existing network methods, the former proves to have the advantages of a short training time and high detection accuracy. The results show that the mean average precision (mAP) without data enhancement reached 95.50%, and the mAP with data enhancement reached 98.35% and exceeded the error rate of human eye detection (5%). Compared with the original faster R-CNN algorithm of 84.39% and 85.12%, the improvement is 11.11% and 10.23%, respectively, which demonstrates superb feature extraction capability and reduces the occurrence of network errors and omissions.

An improved faster R-CNN algorithm for assisted detection of lung nodules

The morbidity and mortality of lung cancer are increasing rapidly in every country in the world, and pulmonary nodules are the main symptoms of lung cancer in the early stage. If we can diagnose pulmonary nodules in time at the early stage and follow up and treat suspicious patients, we can effectively reduce the incidence of lung cancer. CT (Computed Tomography) has been applied to the screening of many diseases because of its high resolution. Pulmonary nodules show white round shadows in CT images. With the popularity of CT equipment, doctors need to review a large number of imaging results every day. Doctors will misjudge and miss the lesions because of reviewing CT scanning results for a long time. At this time, the method of automatic detection of pulmonary nodules by computer can relieve the pressure of doctors in reviewing CT scan results.Traditional lung nodule detection methods, such as gray threshold method and region growing method, divide the detection process into two steps: extracting candidate regions and eliminating false regions. In addition, the traditional detection method can only operate on a single image, which leads to the inability of this method to detect the batch scanning results in real time. With the continuous development of computer equipment performance and artificial intelligence, the relationship between medical image processing and deep learning is getting closer and closer. In deep learning, object detection methods such as Faster R-CNN、YOLO can complete parallel detection of batch images, and deep structure can fully extract the features of input images. Compared with traditional lung nodule detection methods, it has the characteristics of high efficiency and high precision. Faster R-CNN is a classical and high-precision two-stage object detection method. In this paper, an improved Faster R-CNN model is proposed. On the basis of Faster R-CNN, multi-scale training strategy is used to fully mine the features of different scale spaces and perform path augmentation on lower-dimensional features, which improves the small object detection ability of the model. Through Online Hard Example Mining (OHEM), the loss value is used to quantify the difficulty of candidate region detection, and the training times of the region to be detected are adaptively adjusted. Make full use of prior information to customize the size and proportion of preset boundary anchor boxes. Using deformable convolution to improve the visual field to enhance the global features and enhance the ability to extract the feature information of pulmonary nodules in the same scale space.The new model was tested on LUNA16 (Lung Nodule Analysis 2016) dataset. The detection precision of the improved Faster R-CNN model for pulmonary nodules increased from 76.4% to 90.7%, and the recall rate increased from 40.1% to 56.8% Compared with the mainstream object detection algorithms YOLOv3 and Cascade R-CNN, the improved model is superior to the above models in every index.

Insulator image autonomous recognition and defect intelligent detection based on multispectral image

Insulator determines the insulation level and power supply reliability of transmission lines. The traditional operation and maintenance method of insulators has a large workload. This paper presents an insulator recognition and fault diagnosis system based on image recognition and machine learning. Firstly, the composite insulators in complex backgrounds have been identified by Faster RCNN algorithm, which helps to extract the image of insulators by drone shot. Then, the cracking of umbrella skirts has been carried out by means of image processing. Also, the contamination of composite insulator umbrella skirts is also identified. The appropriate feature quantity by Fisher’s discrimination is recommended, and the insulator contamination level has been identified by S component. Lastly, analyzing the infrared image of the insulator provides the basis for the normalization of insulator temperature rise detection results in different environments. Insulator image autonomous recognition and defect intelligent detection, which is based on deep reinforcement learning, is helpful for the operation, maintenance and repair of line insulators.

Identifikasi Objek Cagar Budaya Candi Mahligai Berbasis Citra Digital Menggunakan Mask R-CNN

Object detection is determining the existence of an object and its scope and location in an image. Excessive historical relics and it is generally difficult for humans to remember all the names. One of the fields of science that supports in facilitating human work is confidence score. The main problem in confidence score is how to recognize images and make use of the images they capture. The purpose of this research is to identify the image of Mahligai Temple. This research uses the Deep Learning Mask R-CNN algorithm as a detection of image objects that can recognize cultural heritage sites. The application is carried out on the image of Mahligai Temple. The results using Confussion Matrix and obtained an accuracy of 82.0%, precision 78.57% and recall 88.0%. Single Class identification done successfully at the threshold value 0.776 with error percentage of 18%.

Remote sensing landslide target detection method based on improved Faster R-CNN

To timely detect landslide hazards to start emergency rescue, an improved Faster R-CNN algorithm is proposed for remote sensing image landslide detection. First, the gamma transform and Gaussian filtering methods of image enhancement are used to improve the quality of the images. Second, the effect of batchsize size on the model is eliminated using the group normalization method. Finally, multiscale feature fusion is performed by adding a feature pyramid network structure to optimize the extracted landslide small target features, and then the backbone network is set as deep residual shrinkage network 50 to make the model more focused on information useful for landslide detection. The experimental results show that the improved model improves the accuracy rate as well as the average precision by 8.8% and 8.4%, respectively, compared with the unimproved Faster R-CNN, and compared with the first-stage models, such as you only look once version 4 and single-shot detector, which verify the superiority of the model in our study and can detect landslide targets well.