Region Convolutional Neural Network Research Articles

Research on Induction Measurement of Operators Approaching Charged Bodies in Complex Operation Scenarios

Aiming at the uneven distribution of the electric field and the difficulty in achieving low false alarms for operators approaching charged bodies in the complex operating environment in the power system, a mask region convolutional neural network (MR-CNN) based complex operation scene recognizer is first constructed in this paper to perform image feature extraction, target box regression, and classification training. Secondly, a simulated annealing-based mirror-image charge method is proposed to calculate the electric field distribution characteristics. The simulated annealing algorithm is used to optimize the number of charges of the mirror-image charges method to minimize the weighted sum of the calculation error and the electric field strength calculation time. Finally, the feasibility and effectiveness of the proposed algorithm are verified through simulation experiments.

Neural Network Image Segmentation for Sign Language Interpretation

The use of neural networks to recognize and classify objects in images is a popular field in computer science. It is highly likely that an object in an image chosen for classification will have a representation matrix with significantly less pixels than the background or other elements of the image. As a result, the initial plan would be to divide or segment that object from the other portions of the image that are not essential for categorization. This also serves as the study's objective, for which we employ segmentation to separate the components essential to the classification procedure and assess any room for improvement in the final classification outcome. Mask Region Convolutional Neural Network was the model used for segmentation, and Convolutional Neural Network was the model used for classification. The study's findings demonstrate a notable improvement in the classification in the case of sign language. Further advancement of image segmentation models implies better more accurate results for classification models once they are combined. Keywords— Neural network, Image segmentation, Sign language, Classification, Mask Regional Convolutional Neural Network.

A domain specific knowledge extraction transformer method for multisource satellite-borne SAR images ship detection

Multisource satellite-borne synthetic aperture radar (SAR) images have different probability distributions. Traditional supervised learning, consequently, cannot achieve good test performance on one novel satellite-borne SAR dataset while training a good model on another existing satellite-borne SAR dataset. In this article, a domain adaptation (DA) Transformer object detection method is proposed to solve the unlabeled multisource satellite-borne SAR image object detection problem. Unlike existing DA methods based on convolutional neural network (CNN) that focus more on multi-level local feature extraction, we choose to use Vision Transformer (ViT) Faster region CNN (FRCNN) as the baseline network to cope with the extraction of global features of SAR images. Then, two classification tokens are used to learn the mapping of different domains and fully extract domain-specific knowledge, generating two different feature spaces that rely on the original label and pseudo-label to train the source and target domains feature spaces, respectively. Besides, the pseudo-label of target domain is also refined and reconstructed by feature clustering in order to improve the accuracy of target domain knowledge. Finally, the original detection head of FRCNN is employed to detect the target domain SAR image objects. Extensive experiments on image datasets from multisource satellite-borne SAR such as Gaofen-3, TerraSAR-X, Sentinel-1, and RadarSAT-2 show that compared to the other state of the art (SOTA) methods, the proposed method can achieve the greatest object detection accuracy. Especially, taking the recently proposed Transformer-based method as an example, our method has more than 5% improvement in accuracy and more than 16% reduction in training time.

Research on garment flat multi-component recognition based on Mask R -CNN

The automatic recognition of garment flat information has been widely researched through computer vision. However, the unapparent visual feature and low recognition accuracy pose serious challenges to the application. Herein, inspired by multi-object instance segmentation, the method of mask region convolutional neural network (Mask R-CNN) for garment flat multi-component is proposed in this paper. The steps include feature enhancement, attribute annotation, feature extraction, and bounding box regression and recognition. First, the Laplacian was employed to enhance the image feature, and the Polygon annotated component attributes to reduce the interaction interference. Next, the ResNet was applied to realize identity mapping to characterize redundant information of components. Finally, the feature map was entered into two branches to achieve bounding box regression and recognition. The results demonstrated that the proposed method could realize multi-component recognition effectively. Compared with the unenhanced feature, the mAP increased by 2.27%, reaching 97.87%, and the average F1 was 0.958. Compared to VGGNet and MobileNet, the ResNet backbone used for Mask R-CNN could improve the mAP by 11.55%. Mask R-CNN was more robust than the state-of-the-art methods and more suitable for garment flat multi-component recognition.

Advanced Computer Vision-Based Subsea Gas Leaks Monitoring: A Comparison of Two Approaches.

Recent years have witnessed the increasing risk of subsea gas leaks with the development of offshore gas exploration, which poses a potential threat to human life, corporate assets, and the environment. The optical imaging-based monitoring approach has become widespread in the field of monitoring underwater gas leakage, but the shortcomings of huge labor costs and severe false alarms exist due to related operators' operation and judgment. This study aimed to develop an advanced computer vision-based monitoring approach to achieve automatic and real-time monitoring of underwater gas leaks. A comparison analysis between the Faster Region Convolutional Neural Network (Faster R-CNN) and You Only Look Once version 4 (YOLOv4) was conducted. The results demonstrated that the Faster R-CNN model, developed with an image size of 1280 × 720 and no noise, was optimal for the automatic and real-time monitoring of underwater gas leakage. This optimal model could accurately classify small and large-shape leakage gas plumes from real-world datasets, and locate the area of these underwater gas plumes.

Polarization Orientation Method Based on Remote Sensing Image in Cloudy Weather

Autonomous navigation technology is a core technology for intelligent operation, allowing the vehicles to perform tasks without relying on external information, which effectively improves the concealability and reliability. In this paper, based on the previous research on the bionic compound eye, a multi-channel camera array with different polarization degrees was used to construct the atmospheric polarization state measurement platform. A polarization trough threshold segmentation algorithm was applied to study the distribution characteristics and characterization methods of polarization states in atmospheric remote sensing images. In the extracted polarization feature map, the tilting suggestion box was obtained based on the multi-direction window extraction network (similarity-based region proposal networks, SRPN) and the rotation of the suggestion box (Rotation Region of interests, RRoIs). Fast Region Convolutional Neural Networks (RCNN) was used to screen the suggestion boxes, and the Non-maximum suppression (NMS) method was used to select the angle, corresponding to the label of the suggestion box with the highest score, as the solar meridian azimuth in the vehicle coordinate system. The azimuth angle of the solar meridian in the atmospheric coordinate system can be calculated by the astronomical formula. Finally, the final heading angle can be obtained according to the conversion relationship between the coordinate systems. By fitting the measured data based on the least Square method, the slope K value is −1.062, RMSE (Root Mean Square Error) is 6.984, and the determination coefficient R-Square is 0.9968. Experimental results prove the effectiveness of the proposed algorithm, and this study can construct an autonomous navigation algorithm with high concealment and precision, providing a new research idea for the research of autonomous navigation technology.

Cascade Learning Embedded Vision Inspection of Rail Fastener by Using a Fault Detection IoT Vehicle

Fastener needs to be monitored and inspected periodically to ensure the rail’s safety due to its easily damaged accessory for railway infrastructure. Recently, Industrial Internet of Things (IIoT) and artificial intelligence (AI)-based visual inspection techniques have been exploited to realize the online inspection of fastener’s fault by using a fault detection IoT vehicle that is mounted with multitype sensors and cameras according to the design of our research team. However, instead of traditional artificial inspection, the AI-based automatic fastener inspection approach is still faced with some challenges, for example, collection of enough samples of faulted fastener. In this article, we propose a cascade learning embedded vision inspection method of rail fastener based on the deep convolutional neural network (DCNN). The proposed method has two steps: 1) region position and 2) fault detection. First, a modified single shot multibox detector (SSD) model is adopted to locate the fastener regions from the captured railway images. Then, a key component detection (KCD) method based on the improved faster region convolutional neural network (RCNN) is proposed to realize the detection of faulted fastener. Extensive experiments are conducted to demonstrate the performance of the proposed method. The experiment results show that the proposed method achieves an average precision of 95.38% and an average recall of 98.62% on fastener detection, which is much better than the manual operation.

Spatial ocean wave height prediction with CNN mixed-data deep neural networks using random field simulated bathymetry

With climate change impacts like sea level rise and changing storms, proper prediction of significant wave height (SWH) becomes increasingly important for coastal protection and marine disaster prevention. In the coastal areas of the North Sea, the morphodynamically changing ebb-tidal delta (ETD) sandbanks cause non-linear wave propagation. Therefore, consideration of spatial dependencies using bathymetric data is essential for accurate machine learning predictions. We developed a novel two-dimensional mixed-data deep convolutional neural network (CNN) for spatial SWH prediction in the nearshore area of Norderney, Germany. To overcome the problem of limited bathymetry data, dynamic ETD sandbank morphologies were simulated using random fields and used as model input for the first time. The regional mixed-data CNN was also trained and tested with in-situ metocean input data from 2004 to 2017 and SWAN-modeled ground truth wave fields as output. The proposed CNN architecture outperformed other benchmark models on the test data (RMSE = 0.097 m, R2 = 0.977, MAAPE = 6.7%). Further validation on 59 buoy measurements revealed very similar accuracy of the CNN and SWAN. Compared to the commonly used numerical SWAN model, the trained CNN reduced the computational cost by a factor of more than 300000, making it an efficient surrogate predictive model.

Perbandingan Kinerja Algoritma YOLO Dan RCNN Pada Deteksi Plat Nomor Kendaraan

Vehicle license plate detection is a system used to automatically recognize and identify license plates on passing vehicles. This system is useful to assist in managing traffic, reducing crime, and increasing the efficiency of the transportation system. However, in its implementation, license plate detection often encounters difficulties due to variations in shape, color, and lighting conditions. In addition, vehicle license plates are also cut off or distorted in digital images, which can complicate the detection process. To solve these problems, an effective method is needed to identify objects from digital images. This study aims to evaluate and compare the performance of two popular algorithms used for vehicle license plate detection, namely YOLO (You only look once) and RCNN (Region Convolutional Neural network). The performance evaluation of the two algorithms uses the same dataset so that their accuracy can be compared. The results showed that the YOLOv4 algorithm has a higher level of detection accuracy than RCNN with an accuracy of 96% and 87,8%, respectively. Based on these results it can be concluded that YOLOv4 is more suitable for use in vehicle license plate detection applications with prioritized speeds.

Adaptive Non-Maximum Suppression for improving performance of Rumex detection

A crucial post-processing stage in numerous object detection methods is Non-Maximum Suppression (NMS). The key idea of this technique is to rank the detected bounding boxes according to their scores. Subsequently, selecting the bounding box with the maximum score represents the one best-fitted to the object and suppresses the remaining significant boxes. Conventional NMS suffers from locating objects with accurate bounding boxes as there are multiple boxes in a certain region. This issue reduces the detection performance of automated weed applications in the real world. Weed detection methods based on Region-Convolutional Neural Network (R-CNN) frameworks remain suffer from a lack of detection rate due to overlapping and occlusion leaves issues. This paper presents an Ensemble-Region Convolutional Neural Networks (E-RCNN) model of three state-of-the-art networks to detect Rumex obtusifolius L. (R. obtu.) weeds under various conditions, especially overlapping. The proposed E-RCNN model is used due to its novelty of using ensemble classifiers with the combination of three extractors at its backbone. Adaptive Non-Maximum Suppression (ANMS) is proposed with the Region Proposal Network (RPN) to enhance the detection performance of overlapping and occluded objects by overcoming the drawbacks of conventional Non-Maximum Suppression (NMS). A hybrid model of three CNN extractor networks is used as the backbone in the classification stage. Thus, integrating three networks into one robust model increases the recognition capability by extracting additional useful features more efficiently than those from an individual network. For detection, RPN is used to generate multi-proposed boxes, whereas ANMS is used to select the best box that has a high score rate to match the target object. Our proposed model has trained and tested two standard benchmarking datasets of Rumex weeds under real-world data. The proposed model tested each dataset separately to evaluate the detection rate in terms of Intersection over Union (IoU). For comparing the evaluation of the detection rate, AlexNet, Single-Shot Detector (SSD), DetectNet and Faster R-CNN with conventional NMS models are used to compare the results.

Clinical Decision Support System for Geriatric Dental Treatment Using a Bayesian Network and a Convolutional Neural Network.

The aim of this study was to evaluate the performance of a clinical decision support system (CDSS) for therapeutic plans in geriatric dentistry. The information that needs to be considered in a therapeutic plan includes not only the patient's oral health status obtained from an oral examination, but also other related factors such as underlying diseases, socioeconomic characteristics, and functional dependency. A Bayesian network (BN) was used as a framework to construct a model of contributing factors and their causal relationships based on clinical knowledge and data. The faster R-CNN (regional convolutional neural network) algorithm was used to detect oral health status, which was part of the BN structure. The study was conducted using retrospective data from 400 patients receiving geriatric dental care at a university hospital between January 2020 and June 2021. The model showed an F1-score of 89.31%, precision of 86.69%, and recall of 82.14% for the detection of periodontally compromised teeth. A receiver operating characteristic curve analysis showed that the BN model was highly accurate for recommending therapeutic plans (area under the curve = 0.902). The model performance was compared to that of experts in geriatric dentistry, and the experts and the system strongly agreed on the recommended therapeutic plans (kappa value = 0.905). This research was the first phase of the development of a CDSS to recommend geriatric dental treatment. The proposed system, when integrated into the clinical workflow, is expected to provide general practitioners with expert-level decision support in geriatric dental care.

Automatic Detection of Corrosion in Large-Scale Industrial Buildings Based on Artificial Intelligence and Unmanned Aerial Vehicles

In recent years, Artificial Intelligence (AI) provided essential tools to enhance the productivity of activities related to civil engineering, particularly in design, construction, and maintenance. In this framework, the present work proposes a novel AI computer vision methodology for automatically identifying the corrosion phenomenon on roofing systems of large-scale industrial buildings. The proposed method can be incorporated into computational packages for easier integration by the industry to enhance the inspection activities’ performance. For this purpose, a dedicated image database with more than 8k high-resolution aerial images was developed for supervised training. An Unmanned Aerial Vehicle (UAV) was used to acquire remote georeferenced images safely and efficiently. The corrosion anomalies were manually annotated using a segmentation strategy summing up 18,381 instances. These anomalies were identified through instance segmentation using the Mask based Region-Convolution Neural Network (Mask R-CNN) framework adjusted to the created dataset. Some adjustments were performed to enhance the performance of the classification model, particularly defining an adequate input image size, data augmentation strategy, Intersection over a Union (IoU) threshold during training, and type of backbone network. The inferences show promising results, with correct detections even under complex backgrounds, poor illumination conditions, and instances of significantly reduced dimensions. Furthermore, in scenarios without a roofing system, the model proved reliable, not producing any false positive occurrences. The best model achieved metrics’ values equal to 65.1% for the bounding box detection Average Precision (AP) and 59.2% for the mask AP, considering an IoU of 50%. Regarding classification metrics, the precision and recall were equal to 85.8% and 84.0%, respectively. The developed methodology proved to be extremely valuable for guiding infrastructure managers in taking physically informed decisions based on the real assets condition.

Deep Learning-Assisted Droplet Digital PCR for Quantitative Detection of Human Coronavirus.

Since coronavirus disease 2019 (COVID-19) pandemic rapidly spread worldwide, there is an urgent demand for accurate and suitable nucleic acid detection technology. Although the conventional threshold-based algorithms have been used for processing images of droplet digital polymerase chain reaction (ddPCR), there are still challenges from noise and irregular size of droplets. Here, we present a combined method of the mask region convolutional neural network (Mask R-CNN)-based image detection algorithm and Gaussian mixture model (GMM)-based thresholding algorithm. This novel approach significantly reduces false detection rate and achieves highly accurate prediction model in a ddPCR image processing. We demonstrated that how deep learning improved the overall performance in a ddPCR image processing. Therefore, our study could be a promising method in nucleic acid detection technology.

Evaluation of bactericidal effects of silver hydrosol nanotherapeutics against Enterococcus faecium 1449 drug resistant biofilms.

Silver (Ag) nanoparticles (NPs) are well documented for their broad-spectrum bactericidal effects. This study aimed to test the effect of bioactive Ag-hydrosol NPs on drug-resistant E. faecium 1449 strain and explore the use of artificial intelligence (AI) for automated detection of the bacteria. The formation of E. faecium 1449 biofilms in the absence and presence of Ag-hydrosol NPs at different concentrations ranging from 12.4 mg/L to 123 mg/L was evaluated using a 3-dimentional culture system. The biofilm reduction was evaluated using the confocal microscopy in addition to the Transmission Electronic Microscopy (TEM) visualization and spectrofluorimetric quantification using a Biotek Synergy Neo2 microplate reader. The cytotoxicity of the NPs was evaluated in human nasal epithelial cells using the MTT assay. The AI technique based on Fast Regional Convolutional Neural Network architecture was used for the automated detection of the bacteria. Treatment with Ag-hydrosol NPs at concentrations ranging from 12.4 mg/L to 123 mg/L resulted in 78.09% to 95.20% of biofilm reduction. No statistically significant difference in biofilm reduction was found among different batches of Ag-hydrosol NPs. Quantitative concentration-response relationship analysis indicated that Ag-hydrosol NPs exhibited a relative high anti-biofilm activity and low cytotoxicity with an average EC50 and TC50 values of 0.0333 and 6.55 mg/L, respectively, yielding an average therapeutic index value of 197. The AI-assisted TEM image analysis allowed automated detection of E. faecium 1449 with 97% ~ 99% accuracy. Conclusively, the bioactive Ag-hydrosol NP is a promising nanotherapeutic agent against drug-resistant pathogens. The AI-assisted TEM image analysis was developed with the potential to assess its treatment effect.

Tapped area detection and new tapping line location for natural rubber trees based on improved mask region convolutional neural network.

Aiming at the problem that the rubber tapping robot finds it difficult to accurately detect the tapped area and locate the new tapping line for natural rubber trees due to the influence of the rubber plantation environment during the rubber tapping operation, this study proposes a method for detecting the tapped area and locating the new tapping line for natural rubber trees based on the improved mask region convolutional neural network (Mask RCNN). First, Mask RCNN was improved by fusing the attention mechanism into the ResNeXt, modifying the anchor box parameters, and adding a tiny fully connected layer branch into the mask branch to realize the detection and rough segmentation of the tapped area. Then, the fine segmentation of the existing tapping line was realized by combining edge detection and logic operation. Finally, the existing tapping line was moved down a certain distance along the center line direction of the left and right edge lines of the tapped area to obtain the new tapping line. The tapped area detection results of 560 test images showed that the detection accuracy, segmentation accuracy, detection average precision, segmentation average precision, and intersection over union values of the improved Mask RCNN were 98.23%, 99.52%, 99.6%, 99.78%, and 93.71%, respectively. Compared with other state-of-the-art approaches, the improved Mask RCNN had better detection and segmentation performance, which could better detect and segment the tapped area of natural rubber trees under different shooting conditions. The location results of 560 new tapping lines under different shooting conditions showed that the average location success rate of new tapping lines was 90% and the average location time was 0.189 s. The average values of the location errors in the x and y directions were 3 and 2.8 pixels, respectively, and the average value of the total location error was 4.5 pixels. This research not only provides a location method for the new tapping line for the rubber tapping robot but also provides theoretical support for the realization of rubber tapping mechanization and automation.

Prognosticating various acute covid lung disorders from COVID-19 patient using chest CT Images

The global spread of coronavirus illness has surged dramatically, resulting in a catastrophic pandemic situation. Despite this, accurate screening remains a significant challenge due to difficulties in categorizing infection regions and the minuscule difference between typical pneumonia and COVID (Coronavirus Disease) pneumonia. Diagnosing COVID-19 using the Mask Regional-Convolutional Neural Network (Mask R-CNN) is proposed to classify the chest computerized tomographic (CT) images into COVID-positive and COVID-negative. Covid-19 has a direct effect on the lungs, causing damage to the alveoli, which leads to various lung complications. By fusing multi-class data, the severity level of the patients can be classified using the meta-learning few-shot learning technique with the residual network with 50 layers deep (ResNet-50) as the base classifier. It has been tested with the outcome of COVID positive chest CT image data. From these various classes, it is possible to predict the onset possibilities of acute COVID lung disorders such as sepsis, acute respiratory distress syndrome (ARDS), COVID pneumonia, COVID bronchitis, etc. The first method of classification is proposed to diagnose whether the patient is affected by COVID-19 or not; it achieves a mean Average Precision (mAP) of 91.52% and G-mean of 97.69% with 98.60% of classification accuracy. The second method of classification is proposed for the detection of various acute lung disorders based on severity provide better performance in all the four stages, the average accuracy is of 95.4%, the G-mean for multiclass achieves 94.02%, and the AUC is 93.27% compared with the cutting-edge techniques. It enables healthcare professionals to correctly detect severity for potential treatments.

Vision‐aided three‐dimensional damage quantification and finite element model geometric updating for reinforced concrete structures

AbstractThis article presents a vision‐aided framework to achieve three‐dimensional (3D) concrete damage quantification and finite element (FE) model geometric updating for reinforced concrete structures. The framework can process images and point clouds to extract damage information and update it in an FE model. First, a mask region convolutional neural network was used to realize highly precise damage detection and segmentation based on images. Second, a 3D point cloud was adopted in conjunction with the processed images for 3D damage qualification. The model‐updating method enables an FE model to delete concrete elements to update the variations in volume caused by structural damage. This framework supports interaction with mainstream FE software for further analysis. To demonstrate the efficiency of the proposed framework, it was used in an experiment on a reinforced‐concrete shear wall.

Marker-less tracking system for multiple mice using Mask R-CNN.

Although the appropriate evaluation of mouse behavior is crucial in pharmacological research, most current methods focus on single mouse behavior under light conditions, owing to the limitations of human observation and experimental tools. In this study, we aimed to develop a novel marker-less tracking method for multiple mice with top-view videos using deep-learning-based techniques. The following stepwise method was introduced: (i) detection of mouse contours, (ii) assignment of identifiers (IDs) to each mouse, and (iii) correction of mis-predictions. The behavior of C57BL/6 mice was recorded in an open-field arena, and the mouse contours were manually annotated for hundreds of frame images. Then, we trained the mask regional convolutional neural network (Mask R-CNN) with all annotated images. The mouse contours predicted by the trained model in each frame were assigned to IDs by calculating the similarities of every mouse pair between frames. After assigning IDs, correction steps were applied to remove the predictive errors semi-automatically. The established method could accurately predict two to four mice for first-look videos recorded under light conditions. The method could also be applied to videos recorded under dark conditions, extending our ability to accurately observe and analyze the sociality of nocturnal mice. This technology would enable a new approach to understand mouse sociality and advance the pharmacological research.

Robust Deep Transfer Learning Based Object Detection and Tracking Approach

At present days, object detection and tracking concepts have gained more importance among researchers and business people. Presently, deep learning (DL) approaches have been used for object tracking as it increases the performance and speed of the tracking process. This paper presents a novel robust DL based object detection and tracking algorithm using Automated Image Annotation with ResNet based Faster regional convolutional neural network (R-CNN) named (AIA-FRCNN) model. The AIA-RFRCNN method performs image annotation using a Discriminative Correlation Filter (DCF) with Channel and Spatial Reliability tracker (CSR) called DCF-CSRT model. The AIA-RFRCNN model makes use of Faster RCNN as an object detector and tracker, which involves region proposal network (RPN) and Fast R-CNN. The RPN is a full convolution network that concurrently predicts the bounding box and score of different objects. The RPN is a trained model used for the generation of the high-quality region proposals, which are utilized by Fast R-CNN for detection process. Besides, Residual Network (ResNet 101) model is used as a shared convolutional neural network (CNN) for the generation of feature maps. The performance of the ResNet 101 model is further improved by the use of Adam optimizer, which tunes the hyperparameters namely learning rate, batch size, momentum, and weight decay. Finally, softmax layer is applied to classify the images. The performance of the AIA-RFRCNN method has been assessed using a benchmark dataset and a detailed comparative analysis of the results takes place. The outcome of the experiments indicated the superior characteristics of the AIA-RFRCNN model under diverse aspects.

Detection and tracking of vehicles using license plate recognition

The density of vehicles on roads has increased manifold over the last few decades. A seamless system to detect and track a particular vehicle can solve many problems, like traffic congestion, etc. This paper proposes the use of real-time data taken from closed-circuit televisions to detect and track the movements of vehicles. The feature extraction and classification are completely done by the process of faster RCNN (regional convolutional neural networks). The core work is based on region proposal networks. CNN furthers the generation of features; classification is done separately, but it is aided by RCNN, which includes deep learning as well as training the neural network. This is used along with the Kanade Lukas Tomasi algorithm to tack the desired features extracted. This method is simulated on MATLAB software.