Faster R-CNN Architecture Research Articles

A comprehensive review of methods based on deep learning for diabetes-related foot ulcers.

BackgroundDiabetes mellitus (DM) is a chronic disease with hyperglycemia. If not treated in time, it may lead to lower limb amputation. At the initial stage, the detection of diabetes-related foot ulcer (DFU) is very difficult. Deep learning has demonstrated state-of-the-art performance in various fields and has been used to analyze images of DFUs.ObjectiveThis article reviewed current applications of deep learning to the early detection of DFU to avoid limb amputation or infection.MethodsRelevant literature on deep learning models, including in the classification, object detection, and semantic segmentation for images of DFU, published during the past 10 years, were analyzed.ResultsCurrently, the primary uses of deep learning in early DFU detection are related to different algorithms. For classification tasks, improved classification models were all based on convolutional neural networks (CNNs). The model with parallel convolutional layers based on GoogLeNet and the ensemble model outperformed the other models in classification accuracy. For object detection tasks, the models were based on architectures such as faster R-CNN, You-Only-Look-Once (YOLO) v3, YOLO v5, or EfficientDet. The refinements on YOLO v3 models achieved an accuracy of 91.95% and the model with an adaptive faster R-CNN architecture achieved a mean average precision (mAP) of 91.4%, which outperformed the other models. For semantic segmentation tasks, the models were based on architectures such as fully convolutional networks (FCNs), U-Net, V-Net, or SegNet. The model with U-Net outperformed the other models with an accuracy of 94.96%. Taking segmentation tasks as an example, the models were based on architectures such as mask R-CNN. The model with mask R-CNN obtained a precision value of 0.8632 and a mAP of 0.5084.ConclusionAlthough current research is promising in the ability of deep learning to improve a patient’s quality of life, further research is required to better understand the mechanisms of deep learning for DFUs.

An Intelligent Rice Yield Trait Evaluation System Based on Threshed Panicle Compensation.

High-throughput phenotyping of yield-related traits is meaningful and necessary for rice breeding and genetic study. The conventional method for rice yield-related trait evaluation faces the problems of rice threshing difficulties, measurement process complexity, and low efficiency. To solve these problems, a novel intelligent system, which includes an integrated threshing unit, grain conveyor-imaging units, threshed panicle conveyor-imaging unit, and specialized image analysis software has been proposed to achieve rice yield trait evaluation with high throughput and high accuracy. To improve the threshed panicle detection accuracy, the Region of Interest Align, Convolution Batch normalization activation with Leaky Relu module, Squeeze-and-Excitation unit, and optimal anchor size have been adopted to optimize the Faster-RCNN architecture, termed ‘TPanicle-RCNN,’ and the new model achieved F1 score 0.929 with an increase of 0.044, which was robust to indica and japonica varieties. Additionally, AI cloud computing was adopted, which dramatically reduced the system cost and improved flexibility. To evaluate the system accuracy and efficiency, 504 panicle samples were tested, and the total spikelet measurement error decreased from 11.44 to 2.99% with threshed panicle compensation. The average measuring efficiency was approximately 40 s per sample, which was approximately twenty times more efficient than manual measurement. In this study, an automatic and intelligent system for rice yield-related trait evaluation was developed, which would provide an efficient and reliable tool for rice breeding and genetic research.

Vehicle Detection from Aerial Images with Object and Motion Detection

Moving vehicle detection is one of important issues in surveillance and traffic monitoring applications for aerial images. In this study, a vehicle detection method is proposed by combining motion and object detection. A method based on background modeling and subtraction is applied for motion detection, while Faster-RCNN architecture is used for object detection. Motion detection result is enhanced with the proposed superpixel based refinement method. Experimental study shows that performance of motion detection increases about 8\% for $F_1$ metric with the proposed post processing method. Object detection, motion detection and superpixel segmentation methods interact with each other in parallel processes with the proposed software architecture, which significantly increases the working speed of the method. In last step of the proposed method, each vehicle is tracked with the kalman filter. The performance of proposed method is evaluated on the VIVID dataset. The performance evaluation shows that proposed method increases $F_1$ and recall values significantly compared to the motion and object detection methods alone. It also outperforms SCBU and MCD methods which are widely used for performance comparison in motion detection studies in the literature

Prediction of material properties of propellant waste modified bricks through microstructures by Topographic independent component analysis coupled 3D Convolution neural networks

PurposeDeciphering materials properties from the textural information of microstructural images is still a challenge. ObjectiveProposes a Topographic Independent Component Analysis coupled with 3-D Convolutional Neural Network (TICA- 3DCNN) architecture that processes the features from scanning electron microscope (SEM) images to predict the material properties of high energy propellant (HEP) modified bricks. MethodFirst, high-energy modified bricks with ten different HEP additives fractions is prepared, and their microstructural information is acquired with SEM. Then, SEM images of each group are pre-processed and subjected to the TICA analysis to decipher the distinct higher-order microstructural dependencies, such as co-activation of components. These microstructural co-activation components are subsequently ordered and passed into 3D CNN architecture to map them with the material properties such as water absorption, compressive strength, density, and porosity. An extreme learning machine is employed as a fully connected classification layer in the 3D-CNN architecture. Finally, the performance of the proposed TICA - 3DCNN approach is compared with the traditional 2D CNN frameworks such as Inception-v4 and Faster RCNN. ResultsProposed TICA-3DCNN architecture has performed with higher sensitivity and specificity than Inception-v4 and Faster RCNN architecture in deciphering microstructural textural properties and mapping them with materials properties. ConclusionThe Proposed model can be used in the construction and building materials to predict material properties through microstructural features.

Hyperparameters Tuning of Faster R-CNN Deep Learning Transfer for Persistent Object Detection in Radar Images

In previous work, a methodology was proposed to obtain a sea surface object detection model based on the FasterR-CNN architecture using Sperry Marine commercial navigation radar images. Unfortunately, the percentage of recall using the validation dataset was 75.76% with a minimum score for true positives of 7% due to a network overfitting problem. In this research, the overfitting problem is solved by comparing three experiments. Each experiment consists of the combinations of different hyperparameters within the Faster RCNN architecture. The main hyperparameters modified to improve the performance of the model were weights initialization and the optimizer. The results finally achieved, show a significant improvement in relation to the previous work. The improved persistent object detection model shows a recall of 93.94% with a minimum score for true positives of 98%.

Evaluation of Deep Learning-Based Automated Detection of Primary Spine Tumors on MRI Using the Turing Test.

BackgroundRecently, the Turing test has been used to investigate whether machines have intelligence similar to humans. Our study aimed to assess the ability of an artificial intelligence (AI) system for spine tumor detection using the Turing test.MethodsOur retrospective study data included 12179 images from 321 patients for developing AI detection systems and 6635 images from 187 patients for the Turing test. We utilized a deep learning-based tumor detection system with Faster R-CNN architecture, which generates region proposals by Region Proposal Network in the first stage and corrects the position and the size of the bounding box of the lesion area in the second stage. Each choice question featured four bounding boxes enclosing an identical tumor. Three were detected by the proposed deep learning model, whereas the other was annotated by a doctor; the results were shown to six doctors as respondents. If the respondent did not correctly identify the image annotated by a human, his answer was considered a misclassification. If all misclassification rates were >30%, the respondents were considered unable to distinguish the AI-detected tumor from the human-annotated one, which indicated that the AI system passed the Turing test.ResultsThe average misclassification rates in the Turing test were 51.2% (95% CI: 45.7%–57.5%) in the axial view (maximum of 62%, minimum of 44%) and 44.5% (95% CI: 38.2%–51.8%) in the sagittal view (maximum of 59%, minimum of 36%). The misclassification rates of all six respondents were >30%; therefore, our AI system passed the Turing test.ConclusionOur proposed intelligent spine tumor detection system has a similar detection ability to annotation doctors and may be an efficient tool to assist radiologists or orthopedists in primary spine tumor detection.

Feature distribution alignments for object detection in the thermal domain

Infrared imaging has recently played an important role in a wide range of applications including video surveillance, robotics and night vision. However, the manufacturing cost of high-resolution infrared cameras is more expensive regarding similar quality in visible cameras. This could explain the fact that thermal databases are less available compared to visible ones. In this paper, we mainly emphasis the need for aligning features from visible and thermal domains for object detection in order to ensure effective results in both domains without the need to retrain data and to perform additional annotations. To address that, we incorporate feature distribution alignments into faster R-CNN architecture at different levels. The resulting proposed adaptive detector has the advantage of covering different aspects of the domain shift in order to improve the overall performance. Using KAIST and FLIR ADAS datasets, the effectiveness of the proposed detector is assessed and better results are obtained compared to the baseline detector and to the obtained results by other existing works. Our code is available at https://github.com/AmineMarnissi/UDAT.

Intensive Cold-Air Invasion Detection and Classification with Deep Learning in Complicated Meteorological Systems

Faster R-CNN architecture is used to solve the problems of moving path uncertainty, changeable coverage, and high complexity in cold-air induced large-scale intensive temperature-reduction (ITR) detection and classification, since those problems usually lead to path identification biases as well as low accuracy and generalization ability of recognition algorithm. In this paper, an improved recognition method of national ITR (NITR) path in China based on faster R-CNN in complicated meteorological systems is proposed. Firstly, quality control of the original dataset of strong cooling processes is carried out by means of data filtering. Then, according to the NITR standard and the characteristics of NITR, the NITR dataset in China is established by the intensive temperature-reduction areas located through spatial transformation. Meanwhile, considering that the selection of regularization parameters of Softmax classification method will cause the problem of probability calculation, support vector machine (SVM) is used for path classification to enhance the confidence of classification. Finally, the improved faster R-CNN model is used to identify, classify, and locate the path of NITR events. The experimental results show that, compared to other models, the improved faster R-CNN algorithm greatly improves the performance of NITR’s path recognition, especially for the mixed NITR paths and single NITR paths. Therefore, the improved faster R-CNN model has fast calculation speed, high recognition accuracy, good robustness, and generalization ability of NITR path recognition.

Unsupervised thermal-to-visible domain adaptation method for pedestrian detection

Pedestrian detection is a common task in the research area of video analysis and its results lay the foundations of a wide range of applications. It is commonly known that under challenging illumination and weather conditions, conventional visible cameras perform poorly and this limitation could be catered using thermal imagery. But, due to the fact that annotated thermal datasets are less available compared to the visible ones, in this paper we emphasis the need for leveraging information from the visible domain to perform detection in the thermal domain at no additional annotation cost. Precisely, we propose a domain adaptation method by incorporating feature distribution alignments into Faster R-CNN architecture at different levels and at two phases of the network. The resulting proposed adaptive detector has the advantage of covering different aspects of the domain shift in order to improve the overall performance. The proposed detector is evaluated on KAIST multispectral dataset and the obtained results demonstrate its effectiveness by improving the adaptability in the thermal domain. Also, by means of comparisons to other existing works, better results are obtained. Additional experiments are conducted on other datasets to further justify the obtained results.

Brain tumor detection from MRI images with using proposed deep learning model: the partial correlation-based channel selection

A brain tumor is an abnormal growth of a mass or cell in the brain. Early diagnosis of the tumor significantly increases the chances of successful treatment. Artificial intelligence-based systems can detect the tumor in early stages. In this way, it could be possible to detect a tumor and resolve this problem that may endanger human life early. In the study, the partial correlation-based channel selection formula was presented that allowed the selection of the most prominent feature that differs from the other studies in the literature. Additionally, the multi-channel convolution structure was proposed for the feature network phase of the Faster R-CNN architecture. In the proposed model, the most prominent features were obtained from the multi-channel selection structure in the feature network phase with the channel selection formula in the channel selection layer. The architecture was applied for the early detection of possible brain tumors, which are a severe risk for human life. Within the present study, the brain tumor was classified applying the proposed multi-channel Faster R-CNN based model with three different open-access datasets. VGG-16, faster region-based convolutional neural network (Faster R-CNN), DenseNet-201, Resnet-50, and SRN models, which are popular deep learning architectures, were applied to the same problem to compare the results and demonstrate the efficiency of the proposed model. Accuracy, sensitivity, and processing times of the applied methods were measured to demonstrate the models? performance and efficiency. As a result, the highest accuracy rates were obtained using the proposed model as 98.31%, 99.6%, and 99.8% for three datasets. In addition, it was compared with related studies in the literature to demonstrate the proposed model's applicability. The proposed model's accuracy and performance proved to be higher than in the other studies.

Major nocturnal Pest classification model using Faster RCNN architecture of Deep learning

Agriculture research improves the quality and quantity of crops, but pests degrade them. Pesticides are used to prevent these pests from reproducing. However, excessive pesticide use is extremely detrimental to both production and the environment. As a result, initial pest detection is required. We analyzed the most frequently used methodologies in order to determine the most appropriate technique for the first diagnosis and early detection of significant nocturnal flying pests such as White Grub, Helicoverpa, and Spodoptera. We identified and analyzed three frequently used deep learning meta-architectures (Faster R-CNN, SSD Inception, and SSD Mobilenet) for object detection using a small Pest dataset. The faster RCNN meta-architecture outperforms other meta-architectures. To address the issue of class imbalance, we used image augmentation with a Faster RCNN meta-architecture. The proposed work demonstrates how to classify Nocturnal Pests using a Faster RCNN of Deep Learning with a better accuracy performance on a limited dataset and utilization as decision-making tool based on classified results.

Reinforced Neighbour Feature Fusion Object Detection with Deep Learning

Neural networks have enabled state-of-the-art approaches to achieve incredible results on computer vision tasks such as object detection. However, previous works have tried to improve the performance in various object detection necks but have failed to extract features efficiently. To solve the insufficient features of objects, this work introduces some of the most advanced and representative network models based on the Faster R-CNN architecture, such as Libra R-CNN, Grid R-CNN, guided anchoring, and GRoIE. We observed the performance of Neighbour Feature Pyramid Network (NFPN) fusion, ResNet Region of Interest Feature Extraction (ResRoIE) and the Recursive Feature Pyramid (RFP) architecture at different scales of precision when these components were used in place of the corresponding original members in various networks obtained on the MS COCO dataset. Compared to the experimental results after replacing the neck and RoIE parts of these models with our Reinforced Neighbour Feature Fusion (RNFF) model, the average precision (AP) is increased by 3.2 percentage points concerning the performance of the baseline network.

A self-attention based faster R-CNN for polyp detection from colonoscopy images

At present, the incidence rate of colorectal cancer (CRC) is increasing year by year. It has always affected people's physical and mental health and quality of life. How to improve the detection ability of polyp plays a key role in colonoscopy. In order to solve these problems, in this paper, we first enhance the contrast of the input image by well distinguishing the foreground from the background in order to improve the saliency of the polyp regions. Then, we feed the enhanced image into an improved Faster R-CNN architecture comprised three processing modules for feature extraction, region proposal generation, and polyp detection, respectively. In order to further improve the quality, as well as the feature abstraction capability of the feature maps produced by the feature extraction network, we append an attention module to attend to the useful feature channels and weaken the contributions of the helpless feature channels. The experimental results demonstrate that the accuracy of the proposed polyp detection network is greatly improved compared with the existing algorithms, and the network not only can accurately identify polyps of varying sizes and conditions in single polyp images, but also can achieve excellent performance in handling multiple polyp images. This paper will be greatly helpful to alleviate the missed diagnosis of clinicians in the process of endoscopic examination and disease treatment, as well as providing effective assistance for the early diagnosis, treatment and prevention of the CRC, which is also of great significance to the clinical work of physicians.

A deep learning based approach for the detection of diseases in pepper and potato leaves

The present study proposes a Faster R-CNN Object Detection Approach with GoogLeNet Classifier (Faster R-CNN-GC) using image stitching, Faster R-CNN and GoogLeNet to detect pepper and potato leaves as well as leaf diseases in them. It is widely known that for a successful object detection performance, Faster R-CNN requires performing image labelling on a very high number of data, which will later train Faster R-CNN. However, this process is often very time-consuming. The present study mainly aims to shorten this process by designing an object detection approach which benefits from Faster R-CNN and GoogLeNet architecture. Firstly, Faster R-CNN and GoogLeNet were trained. Later, for the testing process, some of two-piece images were combined using an image stitching approach. Finally, using Faster R-CNN and GoogLeNet, pepper and potato leaves are detected and diseases are written on them. In addition, the proposed system was compared with Faster R-CNN Object Detection Approach with AlexNet Classifier (Faster R-CNN-AC), Faster R-CNN Object Detection Approach with SequezeNet Classifier (Faster R-CNN-SC) and Faster R-CNN. The findings of the experimental studies demonstrated that Faster R-CNN-GC displayed a higher object detection performance compared to other approaches.

Understanding Natural Disaster Scenes from Mobile Images Using Deep Learning

With the ubiquitous use of mobile imaging devices, the collection of perishable disaster-scene data has become unprecedentedly easy. However, computing methods are unable to understand these images with significant complexity and uncertainties. In this paper, the authors investigate the problem of disaster-scene understanding through a deep-learning approach. Two attributes of images are concerned, including hazard types and damage levels. Three deep-learning models are trained, and their performance is assessed. Specifically, the best model for hazard-type prediction has an overall accuracy (OA) of 90.1%, and the best damage-level classification model has an explainable OA of 62.6%, upon which both models adopt the Faster R-CNN architecture with a ResNet50 network as a feature extractor. It is concluded that hazard types are more identifiable than damage levels in disaster-scene images. Insights are revealed, including that damage-level recognition suffers more from inter- and intra-class variations, and the treatment of hazard-agnostic damage leveling further contributes to the underlying uncertainties.

Simultaneous regression‐based spatial coverage estimation and object detection with deep learning

Object detection has been in the focus of researchers within varying applications propelled by the recent advances in deep learning and neural networks. Many applications require both detection of class instances as well as a quantification of the spatial coverage of the class instances. While the performance of deep learning approaches for these tasks has been extensively studied there has not been much effort into creating a unified network structure to achieve both goals. The purpose of this paper is to present a regressor to the faster R-CNN architecture that can help quantify the spatial coverage estimation of some detected object. The goal of the regressor is to provide a reproducible result of the spatial coverage. To demonstrate the developed architecture, an example use-case of land cover estimation is used. The experiments conducted in this paper show that the network does not sacrifice object detection accuracy, and indicate that the network is able to estimate the spatial coverage of six different types of land.

A pilot study of a deep learning approach to submerged primary tooth classification and detection.

The aim of the study was to compare the success and reliability of an artificial intelligence (AI) application in the detection and classification of submerged teeth in panoramic radiographs. Convolutional neural network (CNN) algorithms were used to detect and classify submerged molars. The detection module, based on the stateof- the-art Faster R-CNN architecture, processed a radiograph to define the boundaries of submerged molars. A separate testing set was used to evaluate the diagnostic performance of the system and compare it with that of experts in the field. The success rate of the classification and identification of the system was high when evaluated according to the reference standard. The system was extremely accurate in its performance in comparison with observers. The performance of the proposed computeraided diagnosis solution is comparable to that of experts. It is useful to diagnose submerged molars with an AI application to prevent errors. In addition, this will facilitate the diagnoses of pediatric dentists.

Improved accuracy of pest detection using augmentation approach with Faster R-CNN

In agriculture, Pests are decreasing agricultural productivity. Identifying a pest is a challenging process and subject to expert opinion. Nowadays, lots of work carried out for automatic pest detection. It becomes possible because of emerging Deep Learning’s object detection architectures. This paper shows the multi-class pest detection using Faster R-CNN architecture and compared the performance results of image augmentation with focused on the accuracy performance along with small dataset. We have used Horizontal Flip and 90 Degree Rotation augmentation parameters for solving class imbalance problem. We found that trained pest detection model with augmentation options can perform better with an accuracy of 91.02% using Faster R-CNN architecture.

BirdNet+: Two-Stage 3D Object Detection in LiDAR Through a Sparsity-Invariant Bird’s Eye View

Autonomous navigation relies upon an accurate understanding of the elements in the surroundings. Among the different on-board perception tasks, 3D object detection allows the identification of dynamic objects that cannot be registered by maps, being key for safe navigation. Thus, it often requires the use of LiDAR data, which is able to faithfully represent the scene geometry. However, although raw laser point clouds contain rich features to perform object detection, more compact representations such as the bird’s eye view (BEV) projection are usually preferred in order to meet the time requirements of the control loop. This paper presents an end-to-end object detection network based on the well-known Faster R-CNN architecture that uses BEV images as input to produce the final 3D boxes. Our regression branches can infer not only the axis-aligned bounding boxes but also the rotation angle, height, and elevation of the objects in the scene. The proposed network provides state-of-the-art results for car, pedestrian, and cyclist detection with a single forward pass when evaluated on the KITTI 3D Object Detection Benchmark, with an accuracy that exceeds 64% mAP 3D for the Moderate difficulty. Further experiments on the challenging nuScenes dataset show the generalizability of both the method and the proposed BEV representation against different LiDAR devices and across a wider set of object categories by being able to reach more than 30% mAP with a single LiDAR sweep and almost 40% mAP with the usual 10-sweep accumulation.

Flower Detection Using Object Analysis: New Ways to Quantify Plant Phenology in a Warming Tundra Biome

Rising temperatures caused by global warming are affecting the distributions of many plant and animal species across the world. This can lead to structural changes in entire ecosystems, and serious, persistent environmental consequences. However, many of these changes occur in vast and poorly accessible biomes and involve myriad species. As a consequence, conventional methods of measurement and data analysis are resource-intensive, restricted in scope, and in some cases, intractable for measuring species changes in remote areas. In this article, we introduce a method for detecting flowers of tundra plant species in large data sets obtained by aerial drones, making it possible to understand ecological change at scale, in remote areas. We focus on the sedge species E. vaginatum that is dominant at the investigated tundra field site in the Canadian Arctic. Our system is a modified version of the Faster R-CNN architecture capable of real-world plant phenology analysis. Our model outperforms experienced human annotators in both detection and counting, recording much higher recall and comparable level of precision, regardless of the image quality caused by varying weather conditions during the data collection. (K. Stanski, GitHub - karoleks4/flower-detection: Flower detection using object analysis: New ways to quantify plant phenology in a warming tundra biome. GitHub. Accessed: Sep. 17, 2021. [Online]. Available: https://github.com/karoleks4/flower-detection .)