Mask R-CNN Algorithm Research Articles

Mountainous village relocation planning with 3D GIS virtual imaging space model and neural network

ABSTRACT China’s mountainous village relocation planning faces the problems of extensive survey ranges and high travel costs, and current planning still depends on traditional field visits and census data survey method. To improve efficiency and control costs, digital survey methods are proposed to support efficient data collection and produce data with accurate spatial reference . This research discusses a specific survey method for relocation planning using virtual imaging space modeling based on 3D GIS combined with neural network algorithm. A digital village model of a pilot village is established by oblique photography and WEBGIS technology, while the improved Mask R-CNN algorithm is adopted to identify and classify different buildings in the village. A relocation plan including the number of buildings in each class and the reclamation area is provided and the cost and value of the plan is fully evaluated. The results demonstrate that compared with the traditional survey method, the proposed method of digital survey can better address the issue of remote mountain areas, including provide more accurate, complete and sustainable information, make the surveying and planning more convenient, efficient and smart and improve the objectivity of village relocation planning.

High-Level Semantic Feature Detector: Pedestrian Detection Based On Improved Mask R-CNN Algorithm

Pedestrian detection is a crucial topic that must be solved for a variety of reasons, including its usefulness in the domains of advanced mechanics, car safety, and surveillance. A significant portion of the progress over the last several years has been driven by the ease with which people may test their hypotheses on publicly available datasets and offer forth workable answers. In the current day, as deep learning approaches, it is common practise to use sliding-window classifiers in bespoke or anchor-based expectations for object recognition. To keep up with the rapid pace of development, this paper presents a new perspective in which object recognition is conceived of as a semantic object detection task at an undeniable level, and it also introduces refined evaluation metrics that demonstrate how commonly used per-window measures are ineffective and can fail to predict performance on full images. The proposed hybrid model combines the strengths of MobileNet and ResNet50's skip connection with Faster RCNN to analyse the whole image and pull out the relevant characteristics for detection. However, the suggested approach departs from the standard, low-level provisioning and instead focuses on a higher-level detection. Therefore, this research simplifies pedestrian identification using convolutions, reducing it to a focus and scale expectation job. As a result, the suggested method is straightforward, offers competitive accuracy and remarkable speed, and inspires the development of a novel, appealing pedestrian detector.

Deep Learning Techniques for Ear Diseases Based on Segmentation of the Normal Tympanic Membrane.

Otitis media is a common infection worldwide. Owing to the limited number of ear specialists and rapid development of telemedicine, several trials have been conducted to develop novel diagnostic strategies to improve the diagnostic accuracy and screening of patients with otologic diseases based on abnormal otoscopic findings. Although these strategies have demonstrated high diagnostic accuracy for the tympanic membrane (TM), the insufficient explainability of these techniques limits their deployment in clinical practice. We used a deep convolutional neural network (CNN) model based on the segmentation of a normal TM into five substructures (malleus, umbo, cone of light, pars flaccida, and annulus) to identify abnormalities in otoscopic ear images. The mask R-CNN algorithm learned the labeled images. Subsequently, we evaluated the diagnostic performance of combinations of the five substructures using a three-layer fully connected neural network to determine whether ear disease was present. We obtained the receiver operating characteristic (ROC) curve of the optimal conditions for the presence or absence of eardrum diseases according to each substructure separately or combinations of substructures. The highest area under the curve (0.911) was found for a combination of the malleus, cone of light, and umbo, compared with the corresponding areas under the curve of 0.737-0.873 for each substructure. Thus, an algorithm using these five important normal anatomical structures could prove to be explainable and effective in screening abnormal TMs. This automated algorithm can improve diagnostic accuracy by discriminating between normal and abnormal TMs and can facilitate appropriate and timely referral consultations to improve patients' quality of life in the context of primary care.

Obstacle Detection Using Monocular Camera with Mask R-CNN Method

An autonomous car is a car that can operate without being controlled by humans. Autonomous cars must be able to detect obstacles so that the car does not hit objects that are on the path to be traversed. Therefore, it takes a variety of sensors to determine the surrounding conditions. The sensors commonly used in autonomous cars are cameras and LiDAR. Compared to LiDAR, the camera has a relatively long detection distance, lower cost, and can be used to classify objects. In this final project, the monocular camera and Mask R-CNN algorithm are used to create a system that can detect obstacles in the form of cars, motorcycles, and humans. The system will generate segmentation instances, bounding boxes, classifications, distance, and width estimation for each detected object. By using a custom dataset that is created manually it fits perfectly with the surrounding environment. The system used can produce a Mean Average Precision of 0.81, a Mean Average Recall of 0.89, an F1 score of 0.86, and a Mean Absolute Percentage Error of 13.4% for the distance estimator. The average detection speed of each image is 0.29 seconds.

ReID-DeePNet: A Hybrid Deep Learning System for Person Re-Identification

Person re-identification has become an essential application within computer vision due to its ability to match the same person over non-overlapping cameras. However, it is a challenging task because of the broad view of cameras with a large number of pedestrians appearing with various poses. As a result, various approaches of supervised model learning have been utilized to locate and identify a person based on the given input. Nevertheless, several of these approaches perform worse than expected in retrieving the right person in real-time over multiple CCTVs/camera views. This is due to inaccurate segmentation of the person, leading to incorrect classification. This paper proposes an efficient and real-time person re-identification system, named ReID-DeePNet system. It is based on fusing the matching scores generated by two different deep learning models, convolutional neural network and deep belief network, to extract discriminative feature representations from the pedestrian image. Initially, a segmentation procedure was developed based on merging the advantages of the Mask R-CNN and GrabCut algorithm to tackle the adverse effects caused by background clutter. Afterward, the two different deep learning models extracted discriminative feature representations from the pedestrian segmented image, and their matching scores were fused to make the final decision. Several extensive experiments were conducted, using three large-scale and challenging person re-identification datasets: Market-1501, CUHK03, and P-DESTRE. The ReID-DeePNet system achieved new state-of-the-art Rank-1 and mAP values on these three challenging ReID datasets.

Detecting the Flame Front Evolution in Spark-Ignition Engine under Lean Condition Using the Mask R-CNN Approach

In the wake of previous works, the authors propose a new approach for the identification and evolution of the flame front in an optical SI engine. Currently, it is an essential prerogative to characterize the capability of innovative igniters to guarantee earlier flame development in critical operating conditions, such as ultra-lean mixture, towards which automotive research is moving to deal with the ever more stringent regulations on pollutant emissions. The core of the new approach lies in the R-CNN Mask method. The latter consists of a conceptually simple and general framework for object instance segmentation. It can efficiently detect objects contained in an image while simultaneously generating a high-quality segmentation mask for each instance. In particular, the aim this work is to develop an automatized algorithm for detecting, as objectively as possible, the flame front evolution of lean/ultra-lean mixtures ignited by low-temperature plasma-based ignition systems. The capability of the Mask R-CNN algorithm to automatically estimate the binarized area, without setting a defined binarized threshold, allows us to perform an analysis of the flame front evolution completely independent from the user interpretation. Mask R-CNN can detect the kernel in advance and can identify events as regular combustions instead of misfires or anomalies if compared to other traditional approaches. These features make the proposed method the most suitable option to analysis the real behavior of the innovative ignition systems at critical operating conditions.

Research on Image Identification Method of Rock Thin Slices in Tight Oil Reservoirs Based on Mask R-CNN

Terrestrial tight oil has extremely strong diagenesis heterogeneity, so a large number of rock thin slices are needed to reveal the real microscopic pore-throat structure characteristics. In addition, difficult identification, high cost, long time, strong subjectivity and other problems exist in the identification of tight oil rock thin slices, and it is difficult to meet the needs of fine description and quantitative characterization of the reservoir. In this paper, a method for identifying the characteristics of rock thin slices in tight oil reservoirs based on the deep learning technique was proposed. The present work has the following steps: first, the image preprocessing technique was studied. The original image noise was removed by filtering, and the image pixel size was unified by a normalization technique to ensure the quality of samples; second, the self-labeling image data augmentation technique was constructed to solve the problem of sparse samples; third, the Mask R-CNN algorithm was introduced and improved to synchronize the segmentation and recognition of rock thin slice components in tight oil reservoirs; Finally, it was demonstrated through experiments that the SMR method has significant advantages in accuracy, execution speed and migration.

Smarat tag for social distance and face mask detection

Today the biggest problem the world is facing above all the natural disasters is the Covid-19. It's been more than a year but the solution to the issue is still at a far fetch. However, still have few ways to control the outbreak as instructed by the WHO (World Health Organization). A few among them are wearing a mask and maintaining social distance. The objective of the project is to detect face masks in a public gathering or an event. The algorithm used in the project to achieve the objective is MobileNet V2. An image of a few people wearing a mask and without wearing a mask is used as an input dataset. There are few processes involved in achieving the objective of the project that include pre-processing, data augmentation, training, testing and image segmentation. After the processes, with the help of the Mask R-CNN algorithm, will get a segmented image of the input dataset of people wearing a mask and people not wearing a mask. Then, the model is implemented using a webcam where get result of people wearing a mask and not wearing a mask along with the accuracy in percentage.

Artificial Intelligence-Assisted Image Analysis of Acetaminophen-Induced Acute Hepatic Injury in Sprague-Dawley Rats.

Although drug-induced liver injury (DILI) is a major target of the pharmaceutical industry, we currently lack an efficient model for evaluating liver toxicity in the early stage of its development. Recent progress in artificial intelligence-based deep learning technology promises to improve the accuracy and robustness of current toxicity prediction models. Mask region-based CNN (Mask R-CNN) is a detection-based segmentation model that has been used for developing algorithms. In the present study, we applied a Mask R-CNN algorithm to detect and predict acute hepatic injury lesions induced by acetaminophen (APAP) in Sprague-Dawley rats. To accomplish this, we trained, validated, and tested the model for various hepatic lesions, including necrosis, inflammation, infiltration, and portal triad. We confirmed the model performance at the whole-slide image (WSI) level. The training, validating, and testing processes, which were performed using tile images, yielded an overall model accuracy of 96.44%. For confirmation, we compared the model’s predictions for 25 WSIs at 20× magnification with annotated lesion areas determined by an accredited toxicologic pathologist. In individual WSIs, the expert-annotated lesion areas of necrosis, inflammation, and infiltration tended to be comparable with the values predicted by the algorithm. The overall predictions showed a high correlation with the annotated area. The R square values were 0.9953, 0.9610, and 0.9445 for necrosis, inflammation plus infiltration, and portal triad, respectively. The present study shows that the Mask R-CNN algorithm is a useful tool for detecting and predicting hepatic lesions in non-clinical studies. This new algorithm might be widely useful for predicting liver lesions in non-clinical and clinical settings.

Analgesic Effects of Dexmedetomidine Combined with Spinal and Epidural Anesthesia Nursing on Prostate Hyperplasia Patients after Transurethral Resection of Prostate by Intelligent Algorithm-Based Magnetic Resonance Imaging.

To analyze the investigation of the application effects of different doses of dexmedetomidine (Dex) with combined spinal and epidural anesthesia nursing on analgesia after transurethral resection of prostate (TURP) by intelligent algorithm-based magnetic resonance imaging (MRI), MRI imaging segmentation model of mask regions with convolutional neural network (Mask R-CNN) features was proposed in the research. Besides, the segmentation effects of Mask R-CNN, U-net, and V-net algorithms were compared and analyzed. Meanwhile, a total of 184 patients receiving TURP were selected as the research objects, and they were divided into A, B, C, and D groups based on random number table method, each group including 46 cases. Patients in each group were offered different doses of Dex, and visual analogue scale (VAS) and Ramsay scores of different follow-up visit time, use of other analgesics, the incidence of postoperative cystospasm, and nursing satisfaction of patients in four groups were compared. The results demonstrated that Dice similarity coefficient (DSC) value, specificity, and positive predictive value of Mask R-CNN algorithm were 0.623 ± 0.084, 98.61%, and 69.57%, respectively, all of which were higher than those of U-net and V-net algorithms. Pain VAS scores and the incidence of cystospasm at different time periods of groups B and C were both significantly lower than those of group D (P < 0.05). Ramsay scores of groups B and C at 8 hours, 12 hours, 24 hours, and 48 hours after the operation were all remarkably higher than those in group D (P < 0.05). Besides, nursing satisfaction of groups B and C was obviously superior to that in group D, and the difference demonstrated statistical meaning (P < 0.05). The differences revealed that Dex showed excellent analgesic and sedative effects and could effectively reduce the incidence of complications after TURP, including cystospasm and nausea. In addition, it helped improve nursing satisfaction and patient prognosis.

Damage Detection of Insulators in Catenary Based on Deep Learning and Zernike Moment Algorithms

The intelligent damage detection of catenary insulators is one of the key steps in maintaining the safe and stable operation of railway traction power supply systems. However, traditional deep learning algorithms need to train a large number of images with damage features, which are hard to obtain; and feature-matching algorithms have limitations in anti-complex background interference, affecting the accuracy of damage detection. The current work proposes a method that combines deep learning and Zernike moment algorithms. The Mask R-CNN algorithm is firstly used to identify the catenary insulators to realize the region proposal of the insulators. After image preprocessing, the Zernike moment algorithm is used to replace the existing Hu moment algorithm to extract more detailed insulator contour features, then the similarity value and its standard deviation are further calculated, so as to complete the damage detection of the catenary insulator. The experimental results show that the mean average precision of insulator identification can reach 96.4%, and the Zernike moment algorithm has an accuracy of 93.36% in judging the damage of insulators. Compared with the existing Hu moment algorithm, the accuracy is increased by 10.94%, which provides a new method for the automatic detection of damaged insulators in catenary and even other scenarios.

Instance segmentation scheme for roofs in rural areas based on Mask R-CNN

Rooftop detection has numerous applications such as change detection in human settlements, land encroachments, planning routes to rural areas and estimation of solar generation potential of cities. Detecting the number, type and shape of building roofs form part of preliminary procedures to perform a variety of tasks for making decisions. Assessment of rooftops in rural areas is an important task in the estimation of potential solar generation and sizing of solar PV systems which has proven to be very challenging due to the different quality, lighting conditions and resolution of aerial and satellite images. In this research, we implement a mask RCNN algorithm using TensorFlow Object Detection API to detect the rooftop of buildings in a typical rural settlement. The average precision and recall values (@ IoU = 0.5:0.05:0.95) of the trained model were 85% and 88.2% respectively. The results of the experiment show that the approach can effectively and accurately detect and segment rural rooftops from high-resolution aerial images.

Deep learning based approach for the instance segmentation of clayey soil desiccation cracks

The identification of clayey soil desiccation cracks is an important practical issue in geotechnical engineering and engineering geology. The desiccation cracks can dramatically increase the hydraulic conductivity and deteriorate the mechanical performances of clayey soils. Traditionally, the analysis of soil desiccation cracks relies on visual inspection and image processing techniques, which lack automation and intelligence. Therefore, there is an increasing need for an automated algorithm to meet accuracy and efficiency requirements for various engineering scenarios. In this study, a state-of-the-art deep-learning algorithm, Mask R-CNN, was utilized for the clayey soil crack detection, localization and segmentation. A comprehensive dataset including 1200 annotated crack images of 256 × 256 resolution was prepared for the algorithm training and validation. The proposed Mask R-CNN algorithm achieved precision, recall and F1 score of 73.29%, 82.76% and 77.74%, respectively. Besides, the algorithm gained a mean localization accuracy (APbb) of 64.14% and a mean segmentation accuracy (APm) of 47.59%. The detection performance of the Mask R-CNN was also compared with that of the U-Net under three different scenarios. The test results have demonstrated the superiority of the Mask R-CNN over the U-Net algorithm in crack detection, localization and segmentation.

Intelligent Intersection Vehicle and Pedestrian Detection Based on Convolutional Neural Network

The preprocessed images are input to a pretrained neural network to obtain the corresponding feature mapping, and the corresponding region of interest is set for each point in the feature mapping to obtain multiple candidate feature regions; subsequently, these candidate feature regions are fed into a region proposal network and a deep residual network for binary classification and BB regression, and some of the candidate feature regions are filtered out, and the remaining feature regions are subjected to ROIAIign operation; finally, classification, BB regression, and mask generation are performed on these feature regions, and full convolutional nerve network operation is performed in each feature region and output. To further identify the specific model of the vehicle, this paper proposes a multifeature model recognition method that fuses the improved model with the optimized Mask R-CNN algorithm. A vehicle local feature dataset including vehicle badges, lights, air intake grille, and whole vehicle outline is established to simplify the network structure of model. Meanwhile, its detection frame generation process and the adjustment rules of overlapping frame confidence in nonmaximum suppression are improved for coarse vehicle localization. Then, the generated vehicle detection frames after localization are output to the Mask R-CNN algorithm after further optimizing the RPN structure. The localized vehicle detection frames are then output to the Mask R-CNN algorithm after further optimization of the RPN structure for local feature recognition, and good recognition results are achieved. Finally, this paper establishes a distributed server-based vehicle recognition system, which mainly includes database module, file module, feature extraction and matching module, message queue module, WEB module, and vehicle detection module. Due to the limitations of traditional region generation methods, this paper provides a brief analysis of the region generation network in the Faster R-CNN algorithm and details the loss calculation principle of the output layer.

On the use of deep learning for fish species recognition and quantification on board fishing vessels

The development and effective compliance of efficient fishing policies that guarantee both the sustainability of marine resources and fishing activity is one of the main challenges that policymakers nowadays face. At EU level, successful implementation of the Common Fisheries Policy (CFP) depends, at a large extent, on the capacity to quantify catches on board commercial vessels. Because of the large number of fishing vessels and the high number of trips to be monitored classic, monitoring methods, mainly based on inspections, are not effective. Therefore, the use of electronic devices to quantify fishing catches is gaining relevance. The data provided by such devices, in combination with mathematical models, may be used to assess the state of the different fishing stocks and to optimize the fishing activity. In this work, we consider different algorithms based on Deep Learning (DL) for species identification and length estimation. On the one hand, for the instance segmentation task, we have adapted the Mask R-CNN algorithm to the problem of fish species identification. On the other hand, the MobileNet-V1 convolutional neural network is used for the estimation of the length of each individual. The results show that, when overlapping among individuals is moderate to low, both the identification and length estimation models are able to satisfactorily quantify the catch. In situations where overlapping among individuals is large, results need further improvements.

A VR-based CAD 3D Modeling System for Smart Home Products

There is still ample room for further optimization of the application effect of virtual reality (VR) in smart products, and further expansion of the reconstruction research of smart home products. Hence, the related algorithms of spatial reconstruction and three-dimensional (3D) Computer-Aided Design (CAD) item reconstruction is proposed based on the relevant concepts of CAD and convolutional neural network. The algorithms aim to construct the virtual application scene and the simulation state of product usage by VR technology, and stimulate customers’ purchase decision. Through the experiments, it is found that the Fully Convolutional Network optimized by Visual Geometry Group Network can effectively realize scene reconstruction and item reconstruction. The average overall recognition accuracy of scene segmentation is 91.3%, and the mean Intersection over Union is 63.4%, showing a better recognition effect than other similar algorithms. Moreover, in the detection of indoor items, the maximum detection accuracy of the MaskR-CNN algorithm is 0.89, and the detection effect is also better than that of other algorithms. The experiments achieve a good spatial reconstruction structure, with the average voxel Prec rates in unsheltered space and sheltered space of 94.4% and 55.6%, and the average voxel Recall of 92.2% and 36.5%, respectively. The experimental results indicate an excellent spatial reconstruction effect. This study can provide a more scientific reference for subsequent VR system research.

Vision-based method for tracking workers by integrating deep learning instance segmentation in off-site construction

Vision-based tracking of construction workers is the fundamental step for many automated applications in off-site construction. This research proposes a vision-based method for tracking workers in off-site construction by integrating deep learning instance segmentation. The proposed method consists of three main modules including instance segmentation, instance association, and instance assignment. The instance segmentation module applies the Mask R-CNN algorithm to extract masks and bounding boxes of workers from videos. Then, an association matrix is constructed at each two consecutive frames based on the mask intersection-over-union and Kalman filtering. Finally, the instance assignment module solves the association matrix to produce tracking results. In experiments, the proposed method achieved the multiple object tracking accuracy of 96.4% and multiple object tracking precision of 86.2%. The testing results indicate the developed method can successfully track multiple workers when facing the challenges of occlusions and scale variations, etc.

Foreign object detection for railway ballastless trackbeds: A semisupervised learning method

This paper proposes a semisupervised algorithm for detecting foreign objects in ballastless beds based on the improved deep SVDD (Support Vector Data Description) algorithm. First, we use the improved Mask R-CNN algorithm to extract the rail and fastener areas in images, assuming that no foreign object exists in the rail and fastener areas. Second, we deepen the backbone network of the deep SVDD to enhance its ability to extract deep semantics from complex images. We perform pure color coverage processing with different colors and mean blur processing with different blur kernels on the rails and fastener regions extracted by the improved Mask R-CNN. The results show that the AUC (Area Under the Curve) of our improved deep SVDD algorithm is 89.23% and improves the AUC compared to that of the original model by 11.09%.

Mask-RCNN recognition method of composite fold shape in ultrasound images

Wrinkle defects will be inevitably produced during composite manufacturing and the in-service life of composite structures. Because of their diverse morphological changes and small deformation, it is difficult to manually identify the wrinkle with important errors. In order to improve the inspection efficiency, a Mask-RCNN algorithm is proposed to detect and classify different forms of wrinkle defects in composites based on phased array images. Carbon fiber composite laminates are prepared first in different forms of wrinkle defects. Secondly, the ultrasonic phased array is used to collect full matrix data. The longitudinal scanning image of the composite laminate is then obtained through the wavenumber imaging algorithm. According to the geometric characteristics of the folds in the geological layer, the wrinkles in the composite laminate are divided into three categories, and the relationship between the wrinkle shape and the material damage degree is established. The Mask-RCNN algorithm is finally proposed for automaticaly detecting and classifying the wrinkle defects. The introduction of semantic segmentation in this algorithm can help to reveal the positions and shapes of wrinkle defects. The experimental results show that the accuracies of Mask-RCNN in the recognition of different forms of wrinkles reach 92.1%, 90.9%, and 93.3%, respectively, and the classification and recognition of wrinkles are accurate and effective. It provides theoretical support for the integration and automation of data acquisition-imaging-defect recognition in composite industries.

Boosted Mask R-CNN algorithm for accurately detecting strawberry plant canopies in the fields from low-altitude drone images

The research constructs a novel structure by integrating two parts of online object detection pipelines into the current state-of-the-art Mask R-CNN algorithm to improve the detection performance. The DJI Mavic Air drone is used to collect low-altitude sensing images of strawberry plant canopies. The data augmentation method is employed to feed more instances into the original image dataset to boost the generalization and robustness of the detection model in the training procedure. A ResNet50 backbone combined with a feature pyramid network is presented to extract the features of strawberry plant canopies. The online hard example mining algorithm is introduced to mine hard samples to learn rich features and update model weights. Soft non-maximum suppression based on recursive application on the remaining detection boxes within the predefined overlap threshold is proposed to improve the performance of identifying the large complex overlapping area of strawberry plant canopies. The qualitative results demonstrate that the improved detection model had an AP50 of 96.9 and an AR of 78.5 on the test set, which are approximately 30% higher than the original values.