Faster Region-based Convolutional Neural Network Research Articles

Location Recognition of Indoor Firefighting Facilities based on RGB-D Camera and 3D LiDAR

Recently, as fires frequently occur in large buildings, digital twin (DT) technology that enables remote and real-time monitoring and control similar to the real world environment is being studied as a disaster response technology in buildings. In order to use DT technology, it is essential to collect the spatial data of actual building indoor environments and firefighting facilities. This study proposes an indoor spatial data collecting system that can generate the modeling data inside the building and location data of firefighting facilities using laser imaging detection and ranging (LiDAR) and RGB-D cameras. First, point clouds from three-dimensional (3D) LiDAR and the FAST-LIO2 (Fast LiDAR-Inertial Odometry) algorithm are used to obtain odometry information in an indoor environment. The firefighting facilities located inside the building are detected using RGB images and the deep learning model Faster region-based convolutional neural network (R-CNN) with Inception V2 architecture trained using RGB images of four types of firefighting facilities: fire extinguishers, fire hydrants, exit signs, and fire detectors. When a firefighting facility is detected, the relative distance between the RGB-D camera and the firefighting facility is calculated through the depth image and intrinsic parameters of the RGB-D camera. Afterwards, odometry information obtained from FAST-LIO2 and the relative distance are combined to obtain the 3D location of the firefighting facility. Point clouds of the FAST-LIO2 algorithm are then converted into models of the building indoor environment. Through this method, spatial data of an actual building can be constructed and used with DT technology.

Noninvasive Tracking of Every Individual in Unmarked Mouse Groups Using Multi-Camera Fusion and Deep Learning.

Accurate and efficient methods for identifying and tracking each animal in a group are needed to study complex behaviors and social interactions. Traditional tracking methods (e.g., marking each animal with dye or surgically implanting microchips) can be invasive and may have an impact on the social behavior being measured. To overcome these shortcomings, video-based methods for tracking unmarked animals, such as fruit flies and zebrafish, have been developed. However, tracking individual mice in a group remains a challenging problem because of their flexible body and complicated interaction patterns. In this study, we report the development of a multi-object tracker for mice that uses the Faster region-based convolutional neural network (R-CNN) deep learning algorithm with geometric transformations in combination with multi-camera/multi-image fusion technology. The system successfully tracked every individual in groups of unmarked mice and was applied to investigate chasing behavior. The proposed system constitutes a step forward in the noninvasive tracking of individual mice engaged in social behavior.

An Artificial-Intelligence-Based Novel Rice Grade Model for Severity Estimation of Rice Diseases

The pathogens such as fungi and bacteria can lead to rice diseases that can drastically impair crop production. Because the illness is difficult to control on a broad scale, crop field monitoring is one of the most effective methods of control. It allows for early detection of the disease and the implementation of preventative measures. Disease severity estimation based on digital picture analysis, where the pictures are obtained from the rice field using mobile devices, is one of the most effective control strategies. This paper offers a method for quantifying the severity of three rice crop diseases (brown spot, blast, and bacterial blight) that can determine the stage of plant disease. A total of 1200 images of rice illnesses and healthy images make up the input dataset. With the help of agricultural experts, the diseased zone was labeled according to the disease type using the Make Sense tool. More than 75% of the images in the dataset correspond to one disease label, healthy plants represent more than 15%, and multiple diseases represent 5% of the images labeled. This paper proposes a novel artificial intelligence rice grade model that uses an optimized faster-region-based convolutional neural network (FRCNN) approach to calculate the area of leaf instances and the infected regions. EfficientNet-B0 architecture was used as a backbone as the network shows the best accuracy (96.43%). The performance was compared with the CNN architectures: VGG16, ResNet101, and MobileNet. The model evaluation parameters used to measure the accuracy are positive predictive value, sensitivity, and intersection over union. This severity estimation method can be further deployed as a tool that allows farmers to obtain perfect predictions of the disease severity level based on lesions in the field conditions and produce crops more organically.

Diagnosing the Leukemia using Faster Region based Convolutional Neural Network

It is difficult to building deep learning algorithms for identifying chronic diseases. One of the must difficulties facing the system of diagnosing leukemia is the irregular shape and twisted nucleus in white blood cells (WBCs) without cleaning and segmentation of cells by Appling filters. Moreover, it is challenge to identify and classify the WBC at once time which is considered the essential step of leukemia diagnosing. This paper proposed system only based on deep learning algorithms. The modified Faster R-CNN (Faster Region based-Convolutional Neural Networks) algorithm is used to detect and classify WBCs. The system is achieved a high accuracy training on the database tacked directly from microscope which used in [4].

A new super resolution Faster R-CNN model based detection and classification of urine sediments

The diagnosis of urinary tract infections and kidney diseases using urine microscopy images has gained significant attention of medical community in recent years. These images are usually created by physicians’ own rule of thumb manually. However, this manual urine sediment analysis is usually labor-intensive and time-consuming. In addition, even when physicians carefully examine an image, an erroneous cell recognition may occur due to some optical illusions. In order to achieve cell recognition in low-resolution urine microscopy images with a higher level of accuracy, a new super resolution Faster Region-based Convolutional Neural Network (Faster R-CNN) method is proposed. It aims to increase resolution in low-resolution urine microscopy images using self-similarity based single image super resolution which was used during the pre-processing. De-noising based Wiener filter and Discrete Wavelet Transform (DWT) are used to de-noise high resolution images, respectively, to increase the level of accuracy for image recognition. Finally, for the feature extraction and classification stages, AlexNet, VGFG16 and VGG19 based Faster R-CNN models are used for the recognition and detection of multi-class cells. The model yielded accuracy rates are 98.6%, 96.4% and 96.2% respectively.

Effective Parameters to Design an Automatic Parking System

The automated parking system is an extensive branch of smart transport systems. The smartness of such systems is determined by different parameters such as parking maneuver planning. Coding this control system includes vehicle parking and understanding the environment. A high-quality classification mask has been used on each sample to analyze the automated vehicle parking parameters. Mask region-based convolutional neural networks (R-CNN) was taught using a small computational workload titled faster R-CNN that operates in five frames per second. In this paper, the rapidly-exploring random tree (RRT) method was used for routing the parking space and a nonlinear model predictive control (NMPC) controller was added to develop this system. We add the line detection algorithm commands to the mask R-CNN algorithm. The results can be useful to design a secure automatic parking system as well as a powerful perception system.

Deep Learning Object Detection for Image Analysis of Cherry Fruit Fly (Rhagoletis cerasi L.) on Yellow Sticky Traps

Insect populations appear with a high spatial, temporal and type-specific diversity in orchards. One of the many monitoring tools for pest management is the manual assessment of sticky traps. However, this type of assessment is laborious and time-consuming so that only a few locations can be controlled in an orchard. The aim of this study is to test state-of-the art object detection algorithms from deep learning to automatically detect cherry fruit flies (Rhagoletis cerasi), a common insect pest in cherry plantations, within images from yellow sticky traps. An image annotation database was built with images taken from yellow sticky traps with more than 1600 annotated cherry fruit flies. For better handling in the computational algorithms, the images were augmented to smaller ones by the known image preparation methods “flipping” and “cropping” before performing the deep learning. Five deep learning image recognition models were tested including Faster Region-based Convolutional Neural Network (R-CNN) with two different methods of pretraining, Single Shot Detector (SSD), RetinaNet, and You Only Look Once version 5 (YOLOv5). R‑CNN and RetinaNet models outperformed other ones with a detection average precision of 0.9. The results indicate that deep learning can act as an integral component of an automated system for high-throughput assessment of pest insects in orchards. Therefore, this can reduce the time for repetitive and laborious trap assessment but also increase the observed amount of sticky traps

Damage detection in slab structures based on two-dimensional curvature mode shape method and Faster R-CNN

This paper proposes a novel method based on the two-dimensional (2D) curvature mode shape method, Convolutional Neural Networks (CNN), and Faster Region-based Convolutional Neural Networks (faster R-CNN) for detecting damage in slab structures. The 2D curvature mode shapes could be measured directly or calculated from the measured mode shapes using the central difference method. The damage indicator is defined as the absolute differences between the 2D curvature mode shapes of the damaged and intact slabs. The contour plot is chosen to convert the damage indicators into images. Four hundred damage scenarios are created using a Finite Element (FE) model of the slab. Images created from those damage scenarios are then used to train CNN and faster R-CNN. Four damage types are considered in this research, namely a single small hole, a single big hole, two small holes, and two large holes. After training, CNN can predict the damage types and faster R-CNN can predict the bounding boxes around the damaged areas. A test sample set is created to test the performance of the proposed method. The effect of noise in the mode shape data is considered. Results show that the classification accuracy for damage type is high. The overlap ratios between the predicted bounding boxes and the real damaged areas are more than 40% for 80% of tested scenarios. Furthermore, the low influence of noise on the predicted results is investigated. The proposed method is robust and has great potential for application to real structures.

Damaged apple detection with a hybrid YOLOv3 algorithm

This paper proposes an improved You Only Look Once (YOLOv3) algorithm for automatically detecting damaged apples to promote the automation of the fruit processing industry. In the proposed method, a clustering method based on Rao-1 algorithm is introduced to optimize anchor box sizes. The clustering method uses the intersection over the union to form the objective function and the most representative anchor boxes are generated for normal and damaged apple detection. To verify the feasibility and effectiveness of the proposed method, real apple images collected from the Internet are employed. Compared with the generic YOLOv3 and Fast Region-based Convolutional Neural Network (Fast R-CNN) algorithms, the proposed method yields the highest mean average precision value for the test dataset. Therefore, it is practical to apply the proposed method for intelligent apple detection and classification tasks.

CNN-based object detection and growth estimation of plum fruit (Prunus mume) using RGB and depth imaging techniques

Modern people who value healthy eating habits have shown increasing interest in plum (Prunus mume) fruits, primarily owing to their nutritiousness and proven efficacy. As consumption increases, it becomes important to monitor work to prevent Prunus mume fruits from falling out. Moreover, determining the growth status of Prunus mume is also crucial and is attracting increasing attention. In this study, convolutional neural network (CNN)-based deep learning object detection was developed using RGBD images collected from Prunus mume farms. These RGBD images consider various environments, including the depth information of objects in the outdoor field. A faster region-based convolutional neural network (R-CNN), EfficientDet, Retinanet, and Single Shot Multibox Detector (SSD) were applied for detection, and the performance of all models was estimated by comparing their respective computing speeds and average precisions (APs). The test results show that the EfficientDet model is the most accurate, and SSD MobileNet is the fastest among the four models. In addition, the algorithm was developed to acquire the growth status of P. mume fruits by applying the coordinates and score values of bounding boxes to the depth map. Compared to the diameters of the artificial Prunus mume fruits used as the experimental group, the calculated diameters were very similar to those of the artificial objects. Collectively, the results demonstrate that the CNN-based deep learning Prunus mume detection and growth estimation method can be applied to real farmlands.

Radio source-component association for the LOFAR Two-metre Sky Survey with region-based convolutional neural networks

Context. Radio loud active galactic nuclei (RLAGNs) are often morphologically complex objects that can consist of multiple, spatially separated, components. Only when the spatially separated radio components are correctly grouped together can we start to look for the corresponding optical host galaxy and infer physical parameters such as the size and luminosity of the radio object. Existing radio detection software to group these spatially separated components together is either experimental or based on assumptions that do not hold for current generation surveys, such that, in practice, astronomers often rely on visual inspection to resolve radio component association. However, applying visual inspection to all the hundreds of thousands of well-resolved RLAGNs that appear in the images from the Low Frequency Array (LOFAR) Two-metre Sky Survey (LoTSS) at 144 MHz, is a daunting, time-consuming process, even with extensive manpower. Aims. Using a machine learning approach, we aim to automate the radio component association of large (>15 arcsec) radio components. Methods. We turned the association problem into a classification problem and trained an adapted Fast region-based convolutional neural network to mimic the expert annotations from the first LoTSS data release. We implemented a rotation data augmentation to reduce overfitting and simplify the component association by removing unresolved radio sources that are likely unrelated to the large and bright radio components that we consider using predictions from an existing gradient boosting classifier. Results. For large (>15 arcsec) and bright (>10 mJy) radio components in the LoTSS first data release, our model provides the same associations for 85.3% ± 0.6 of the cases as those derived when astronomers perform the association manually. When the association is done through public crowd-sourced efforts, a result similar to that of our model is attained. Conclusions. Our method is able to efficiently carry out manual radio-component association for huge radio surveys and can serve as a basis for either automated radio morphology classification or automated optical host identification. This opens up an avenue to study the completeness and reliability of samples of radio sources with extended, complex morphologies.

Identifying threat objects using faster region-based convolutional neural networks (faster R-CNN)

Automated detection of threat objects in a security X-ray image is vital to prevent unwanted incidents in busy places like airports, train stations, and malls. The manual method of threat object detection is time-consuming and tedious. Also, the person on duty can overlook the threat objects due to limited time in checking every person’s belongings. As a solution, this paper presents a faster region-based convolutional neural network (Faster R-CNN) object detector to automatically identify threat objects in an X-ray image using the IEDXray dataset. The dataset was composed of scanned X-ray images of improvised explosive device (IED) replicas without the main charge. This paper extensively evaluates the Faster R-CNN architecture in threat object detection to determine which configuration can be used to improve the detection performance. Our findings showed that the proposed method could identify three classes of threat objects in X-ray images. In addition, the mean average precision (mAP) of the threat object detector could be improved by increasing the input image's image resolution but sacrificing the detector's speed. The threat object detector achieved 77.59% mAP and recorded an inference time of 208.96 ms by resizing the input image to 900 × 1536 resolution. Results also showed that increasing the bounding box proposals did not significantly improve the detection performance. The mAP using 150 bounding box proposals only achieved 75.65% mAP, and increasing the bounding box proposal twice reduced the mAP to 72.22%.

MCI-frcnn: A deep learning method for topological micro-domain boundary detection.

Chromatin structural domains, or topologically associated domains (TADs), are a general organizing principle in chromatin biology. RNA polymerase II (RNAPII) mediates multiple chromatin interactive loops, tethering together as RNAPII-associated chromatin interaction domains (RAIDs) to offer a framework for gene regulation. RAID and TAD alterations have been found to be associated with diseases. They can be further dissected as micro-domains (micro-TADs and micro-RAIDs) by clustering single-molecule chromatin-interactive complexes from next-generation three-dimensional (3D) genome techniques, such as ChIA-Drop. Currently, there are few tools available for micro-domain boundary identification. In this work, we developed the MCI-frcnn deep learning method to train a Faster Region-based Convolutional Neural Network (Faster R-CNN) for micro-domain boundary detection. At the training phase in MCI-frcnn, 50 images of RAIDs from Drosophila RNAPII ChIA-Drop data, containing 261 micro-RAIDs with ground truth boundaries, were trained for 7days. Using this well-trained MCI-frcnn, we detected micro-RAID boundaries for the input new images, with a fast speed (5.26 fps), high recognition accuracy (AUROC = 0.85, mAP = 0.69), and high boundary region quantification (genomic IoU = 76%). We further applied MCI-frcnn to detect human micro-TADs boundaries using human GM12878 SPRITE data and obtained a high region quantification score (mean gIoU = 85%). In all, the MCI-frcnn deep learning method which we developed in this work is a general tool for micro-domain boundary detection.

Deep learning-based diagnosis of stifle joint diseases in dogs.

In this retrospective, analytical study, we developed a deep learning-based diagnostic model that can be applied to canine stifle joint diseases and compared its accuracy with that achieved by veterinarians to verify its potential as a reliable diagnostic method. A total of 2382 radiographs of the canine stifle joint from cooperative animal hospitals were included in a dataset. Stifle joint regions were extracted from the original images using the faster region-based convolutional neural network (R-CNN) model, and the object detection accuracy was evaluated. Four radiographic findings: patellar deviation, drawer sign, osteophyte formation, and joint effusion, were observed in the stifle joint and used to train a residual network (ResNet) classification model. Implant and growth plate groups were analyzed to compare the classification accuracy against the total dataset. All deep learning-based classification models achieved target accuracies exceeding 80%, which is comparable to or slightly less than those achieved by veterinarians. However, in the case of drawer signs, further research is necessary to improve the low sensitivity of the model. When the implant group was excluded, the classification accuracy significantly improved, indicating that the implant acted as a distraction. These results indicate that deep learning-based diagnoses can be expected to become useful diagnostic models in veterinary medicine.

Improving liver lesions classification on CT/MRI images based on Hounsfield Units attenuation and deep learning

The early sign detection of liver lesions plays an extremely important role in preventing, diagnosing, and treating liver diseases. In fact, radiologists mainly consider Hounsfield Units to locate liver lesions. However, most studies focus on the analysis of unenhanced computed tomography images without considering an attenuation difference between Hounsfield Units before and after contrast injection. Therefore, the purpose of this work is to develop an improved method for the automatic detection and classification of common liver lesions based on deep learning techniques and the variations of the Hounsfield Units density on computed tomography scans. We design and implement a multi-phase classification model developed on the Faster Region-based Convolutional Neural Networks (Faster R–CNN), Region-based Fully Convolutional Networks (R–FCN), and Single Shot Detector Networks (SSD) with the transfer learning approach. The model considers the variations of the Hounsfield Unit density on computed tomography scans in four phases before and after contrast injection (plain, arterial, venous, and delay). The experiments are conducted on three common types of liver lesions including liver cysts, hemangiomas, and hepatocellular carcinoma. Experimental results show that the proposed method accurately locates and classifies common liver lesions. The liver lesions detection with Hounsfield Units gives high accuracy of 100%. Meanwhile, the lesion classification achieves an accuracy of 95.1%. The promising results show the applicability of the proposed method for automatic liver lesions detection and classification. The proposed method improves the accuracy of liver lesions detection and classification compared with some preceding methods. It is useful for practical systems to assist doctors in the diagnosis of liver lesions. In our further research, an improvement can be made with big data analysis to build real-time processing systems and we expand this study to detect lesions from all parts of the human body, not just the liver.

Deep Learning Approach For Objects Detection in Underwater Pipeline Images

ABSTRACT In this paper, we present automatic, deep-learning methods for pipeline detection in underwater environments. Seafloor pipelines are critical infrastructure for oil and gas transport. The inspection of those pipelines is required to verify their integrity and determine the need for maintenance. Underwater conditions present a harsh environment that is challenging for image recognition due to light refraction and absorption, poor visibility, scattering, and attenuation, often causing poor image quality. Modern machine-learning object detectors utilize Convolutional Neural Network (CNN), requiring a training dataset of sufficient quality. In the paper, six different deep-learning CNN detectors for underwater object detection were trained and tested: five are based on the You Only Look Once (YOLO) architectures (YOLOv4, YOLOv4-Tiny, CSP-YOLOv4, YOLOv4@Resnet, YOLOv4@DenseNet), and one on the Faster Region-based CNN (RCNN) architecture. The models’ performances were evaluated in terms of detection accuracy, mean average precision (mAP), and processing speed measured with the Frames Per Second (FPS) on a custom dataset containing underwater pipeline images. In the study, the YOLOv4 outperformed other models for underwater pipeline object detection resulting in an mAP of 94.21% with the ability to detect objects in real-time. Based on the literature review, this is one of the pioneering works in this field.

Social Distancing Monitoring System using Deep Learning

COVID-19 has been declared a pandemic in the world by 2020. One way to prevent COVID-19 disease, as the World Health Organization (WHO) suggests, is to keep a distance from other people. It is advised to stay at least 1 meter away from others, even if they do not appear to be sick. The reason is that people can also be the virus carrier without having any symptoms. Thus, many countries have enforced the rules of social distancing in their Standard Operating Procedure (SOP) to prevent the virus spread. Monitoring the social distance is challenging as this requires authorities to carefully observe the social distancing of every single person in a surrounding, especially in crowded places. Real-time object detection can be proposed to improve the efficiency in monitoring the social distance SOP inspection. Therefore, in this paper, object detection using a deep neural network is proposed to help the authorities monitor social distancing even in crowded places. The proposed system uses the You Only Look Once (YOLO) v4 object detection models for the detection. The proposed system is tested on the MS COCO image dataset with a total of 330,000 images. The performance of mean average precision (mAP) accuracy and frame per second (FPS) of the proposed object detection is compared with Faster Region-based Convolutional Neural Network (R-CNN) and Multibox Single Shot Detector (SSD) model. Finally, the result is analyzed among all the models.

Machine Recognition of Map Point Symbols Based on YOLOv3 and Automatic Configuration Associated with POI

This study is oriented towards machine autonomous mapping and the need to improve the efficiency of map point symbol recognition and configuration. Therefore, an intelligent recognition method for point symbols was developed using the You Only Look Once Version 3 (YOLOv3) algorithm along with the Convolutional Block Attention Module (CBAM). Then, the recognition results of point symbols were associated with the point of interest (POI) to achieve automatic configuration. To quantitatively analyze the recognition effectiveness of this study algorithm and the comparison algorithm for map point symbols, the recall, precision and mean average precision (mAP) were employed as evaluation metrics. The experimental results indicate that the recognition efficiency of point symbols is enhanced compared to the original YOLOv3 algorithm, and that the mAP is increased by 0.55%. Compared to the Single Shot MultiBox Detector (SSD) algorithm and Faster Region-based Convolutional Neural Network (Faster RCNN) algorithm, the precision, recall rate, and mAP all performed well, achieving 97.06%, 99.72% and 99.50%, respectively. On this basis, the recognized point symbols are associated with POI, and the coordinate of point symbols are assigned through keyword matching and enrich their attribute information. This enables automatic configuration of point symbols and achieves a relatively good effect of map configuration.

Feature representation improved Faster R-CNN model for high-efficiency pavement crack detection

Two optimization methods are proposed to improve faster region-based convolutional neural network (Faster R-CNN), which are (1) restructuring Faster R-CNN's backbone network and the classification and regression (C&R) network using residual networks and (2) designing the feature ensemble structure for Faster R-CNN to combine the shallow with deep feature maps of the backbone. In addition, this paper proposed a method to evaluate the model's performance, which is pixel mean value ( Pmean) distribution of different channel feature maps, and quantitatively evaluate the feature representation capability of the model. Experimental results show that mean average precision (mAP) of the model optimized by the first method can reach 86.5%, which is 1.9% higher than that of baseline. However, mAP of the model optimized by the second method reaches 87.5%, which is 2.9% higher than the baseline model. The Pmean statistics of each channel feature map extracted by different backbones show that the model accuracy is higher when the Pmean of its channel feature maps is bigger, which can effectively improve the interpretability of the model accuracy.

Identification of the Angle Errors of the LED Parallel-Light Module in PCB Exposure Device by Using Neural Network Learning Algorithms

For the smart manufacturing development of printed-circuit-board (PCB) exposure devices, the LED parallel-light (LPL) module is investigated and the angle errors of those LPL units are identified by neural network learning algorithms. At present, in PCB manufacturing, most circuit boards use photoresist covering etching. After exposure and development, unwanted copper foil is etched and removed to make circuit boards. The exposure process is its key process, and the equipment used in this process is an exposure machine. The LPL unit is designed and the LPL exposure module is searched under the principle of higher irradiance uniformity. The learning data of supervised learning for the convolutional neural network (CNN) include a 2D irradiance distribution image constructed by the ray tracing simulation tool. In these supervised learning data, all units of LPL-EM are randomly added with a self-specific angle error. By using Fast Region-based CNN, the identification of the multi-LPL module with the specific errors of inclination and azimuth angle is verified. Those results preliminarily illustrate that supervised learning techniques should be able to help identify the errors of inclination and azimuth angle for the single LPL unit and multi-light module of PCB exposure devices. In other words, this technology should serve as a reference for the development of the PCB exposure process towards smart manufacturing.