Mask R-CNN Network Research Articles

Fusion PCAM R-CNN of Automatic Segmentation for Magnetic Flux Leakage Defects.

Magnetic leakage detection technology plays an important role in the long-oil pipeline. Automatic segmentation of defecting images is crucial for the detection of magnetic flux leakage (MFL) works. At present, accurate segmentation for small defects has always been a difficult problem. In contrast to the state-of-the-art MFL detection methodologies based on convolution neural network (CNN), an optimization method is devised in our study by integrating mask region-based CNN (Mask R-CNN) and information entropy constraint (IEC). To be precise, the principal component analysis (PCA) is utilized to improve the feature learning and network segmentation ability of the convolution kernel. The similarity constraint rule of information entropy is proposed to be inserted into the convolution layer in the Mask R-CNN network. The Mask R-CNN optimizes the convolutional kernel with similar weights or higher similarity, meanwhile, the PCA network reduces the dimension of the feature image to reconstruct the original feature vector. As such, the feature extraction of MFL defects is optimized in the convolution check. The research results can be applied in the field of MFL detection.

HiSEG: Human assisted instance segmentation

Instance segmentation is a form of image detection which has a range of applications, such as object refinement, medical image analysis, and image/video editing, all of which demand a high degree of accuracy. However, this precision is often beyond the reach of what even state-of-the-art, fully automated instance segmentation algorithms can deliver. The performance gap becomes particularly prohibitive for small and complex objects. Practitioners typically resort to fully manual annotation, which can be a laborious process. In order to overcome this problem, we propose a novel approach to enable more precise predictions and generate higher-quality segmentation masks for high-curvature, complex and small-scale objects. Our human-assisted segmentation method, HiSEG, augments the existing Strong Mask R-CNN network to incorporate human-specified partial boundaries. We also present a dataset of hand-drawn partial object boundaries, which we refer to as “human attention maps”. In addition, the Partial Sketch Object Boundaries (PSOB) dataset contains hand-drawn partial object boundaries which represent curvatures of an object’s ground truth mask with several pixels. Through extensive evaluation using the PSOB dataset, we show that HiSEG outperforms state-of-the art methods such as Mask R-CNN, Strong Mask R-CNN, Mask2Former, and Segment Anything, achieving respective increases of +42.0, +34.9, +29.9, and +13.4 points in APMask metrics for these four models. We hope that our novel approach will set a baseline for future human-aided deep learning models by combining fully automated and interactive instance segmentation architectures.

Implementation of surface crack detection method for nuclear fuel pellets by weakly supervised learning

ABSTRACT Surface cracks are one of the primary defects in nuclear fuel pellets, posing a significant hazard to nuclear safety production. Deep learning-based methods recently developed for crack detection are typically trained using a supervised learning, with detection performance dependent on pixel-level annotations, which suffer from high labeling costs and low efficiency. To address this issue, we propose a Weakly Supervised Crack Detection (WSCD) network for surface crack detection of nuclear fuel pellets. The method adopts bounding-box annotations instead of pixel-level annotations, and rapidly generates pixel-level pseudo-labels using the designed Local Fusion Segmentation (LFS) module. Leveraging the Mask-RCNN network as the backbone, the network introduces spatial attention mechanisms to optimize feature extraction networks, enhancing the extraction capability of multi-scale crack features. Lastly, a novel loss function is optimized to address sample imbalance issues and expedite network convergence. Experimental results on the established crack dataset demonstrate that the proposed method improves labeling efficiency by approximately 20 times, achieving a segmentation accuracy (IoU) of 81.9%, which reaches 92.2% of the segmentation accuracy achieved by supervised learning and outperforms that the IoU (80.3%) of other supervised crack segmentation network for nuclear fuel pellets, and meeting the precision requirements of nuclear fuel pellets production lines.

A Measurement Method for Body Parameters of Mongolian Horses Based on Deep Learning and Machine Vision

The traditional manual methods for measuring Mongolian horse body parameters are not very safe, have low levels of automation, and cannot effectively ensure animal welfare. This research proposes a method for extracting target Mongolian horse body parameters based on deep learning and machine vision technology. Firstly, Swin Transformer is used as the backbone feature extraction network of Mask R-CNN model, and the CNN-based differentiated feature clustering model is added to minimize the loss of similarity and spatial continuity between pixels, thereby improving the robustness of the model while reducing error pixels and optimizing the rough mask boundary output. Secondly, an improved Harris algorithm and a polynomial fitting method based on contour curves are applied to determine the positions of various measurement points on the horse mask and calculate various body parameters. The accuracy of the proposed method was tested using 20 Mongolian horses. The experimental results show that compared with the original Mask R-CNN network, the PA (pixel accuracy) and MIoU (mean intersection over union) of the optimized model results increased from 91.46% and 84.72% to 98.72% and 95.36%, respectively. The average relative errors of shoulder height, withers height, chest depth, body length, croup height, shoulder angle, and croup angle were 4.01%, 2.98%, 4.86%, 2.97%, 3.06%, 4.91%, and 5.21%, respectively. The research results can provide technical support for assessing body parameters related to the performance of horses under natural conditions, which is of great significance for improving the refinement and welfare of Mongolian horse breeding techniques.

A Novel Boundary-Guided Global Feature Fusion Module for Instance Segmentation

The task of instance segmentation is widely acknowledged as being one of the most formidable challenges in the field of computer vision. Current methods have low utilization of boundary information, especially in dense scenes with occlusion and complex shapes of object instances, the boundary information may become ineffective. This results in coarse object boundary masks that fail to cover the entire object. To address this challenge, we are introducing a novel method called boundary-guided global feature fusion (BGF) which is based on the Mask R-CNN network. We designed a boundary branch that includes a Boundary Feature Extractor (BFE) module to extract object boundary features at different stages. Additionally, we constructed a binary image dataset containing instance boundaries for training the boundary branch. We also trained the boundary branch separately using a dedicated dataset before training the entire network. We then input the Mask R-CNN features and boundary features into a feature fusion module where the boundary features provide shape information needed for detection and segmentation. Finally, we use a global attention module (GAM) to further fuse features. Through extensive experiments, we demonstrate that our approach outperforms state-of-the-art instance segmentation algorithms, producing finer and more complete instance masks while also improving model capability.

Controlling Crystal Growth of a Rare Earth Element Scandium Salt in Antisolvent Crystallization

Recovering scandium from hydrometallurgical residue bears the potential of a better supply of an industry depending on imports from countries with more mineral resources than Europe. To recover scandium from unused metal production residue, strip liquors from a solvent extraction process are treated with an antisolvent to crystallize the ammonium scandium fluoride salt (NH4)3ScF6 with high product yields. However, high local supersaturation leads to strong nucleation, resulting in small crystals, which are difficult to handle in the subsequent solid-liquid separation. Reducing local supersaturation makes it possible to reduce nucleation and control crystal growth. Key operation parameters are the concentration of ethanol in the feed and its addition rate. The concentration of the antisolvent in the feed causes a shorter mixing time in the proximity of the antisolvent inlet, which leads to a smaller local supersaturation and therefore less nucleation and more crystal growth. Lowering the antisolvent addition rate enhances this effect. The crystal size distribution during and at the end of the fed-batch process is analyzed by SEM imagery of sampled and dried crystals. To produce reproducible crystal size distribution from SEM images the neural network Mask R-CNN has been trained for the automated crystal detection and size analysis.

Application of the convolutional network Mask R-CNN for the estimation of the body weight of the guinea pig

La inteligencia artificial puede contribuir en el seguimiento del ciclo productivo del cuy mediante la aplicación de redes convolucionales, siendo una necesidad la estimación de su peso. Este estudio se enfocó en la aplicación de la red convolucional Mask R-CNN, utilizando una aplicación móvil como herramienta para la captura de imágenes. La metodología abarcó las siguientes etapas: i) revisión bibliográfica, ii) recolección de datos (imágenes y pesos de cuyes), iii) procesamiento de imágenes mediante aumento de datos, iv) construcción de un dataset (selección de imágenes y transformación de datos), v) adaptación y entrenamiento de la red convolucional, vi) análisis de los resultados para validar su desempeño, y finalmente, vii) implementación de una aplicación móvil como herramienta de estimación de peso. Se logró recopilar un conjunto de 6244 imágenes de cuyes con sus respectivos pesos y máscaras, junto con la adaptación de la red Mask R-CNN. Estas tareas condujeron a una correlación de R2 = 80,2% con el conjunto de validación, así como al desarrollo de un prototipo funcional capaz de estimar el peso de los cuyes utilizando la cámara de un teléfono celular.

Automatic tooth periodontal ligament segmentation of cone beam computed tomography based on instance segmentation network

ObjectiveThe three-dimensional morphological structures of periodontal ligaments (PDLs) are important data for periodontal, orthodontic, prosthodontic, and implant interventions. This study aimed to employ a deep learning (DL) algorithm to segment the PDL automatically in cone-beam computed tomography (CBCT). MethodThis was a retrospective study. We randomly selected 389 patients and 1734 axial CBCT images from the CBCT database, and designed a fully automatic PDL segmentation computer-aided model based on instance segmentation Mask R-CNN network. The labels of the model training were ‘teeth’ and ‘alveolar bone’, and the ‘PDL’ is defined as the region where the ‘teeth’ and ‘alveolar bone’ overlap. The model’s segmentation performance was evaluated using CBCT data from eight patients outside the database. ResultsQualitative evaluation indicates that the PDL segmentation accuracy of incisors, canines, premolars, wisdom teeth, and implants reached 100%. The segmentation accuracy of molars was 96.4%. Quantitative evaluation indicates that the mIoU and mDSC of PDL segmentation were 0.667 ± 0.015 (>0.6) and 0.799 ± 0.015 (>0.7) respectively. ConclusionThis study analysed a unique approach to AI-driven automatic segmentation of PDLs on CBCT imaging, possibly enabling chair-side measurements of PDLs to facilitate periodontists, orthodontists, prosthodontists, and implantologists in more efficient and accurate diagnosis and treatment planning.

A Mask R-CNN Network for Wide-Area Mining Subsidence Automatic Detection With InSAR Observations

A Mask R-CNN Network for Wide-Area Mining Subsidence Automatic Detection With InSAR Observations

A Novel Mobile Vision Based Technique for 3D Human Pose Estimation

In this work, we propose a novel technique for accurately constructing 3D human poses based on mobile phone camera recordings. From the originally recorded video frames by a mobile phone camera, firstly a mask R-CNN network is applied to detect the human body and extract 2D body skeletons. Based on the 2D skeletons, a temporal convolutional network (TCN) is then applied to lift 2D skeletons to 3D ones for the 3D human pose estimation. From the experimental evaluations, it is shown that 3D human poses can be accurately reconstructed by the proposed technique in this work based on mobile phone camera recordings while the reconstruction result is very close to the one by a specialized motion capture system.

AN IMPROVED MASK R-CNN: EXTRACTION OF DOOR AND WINDOW INSTANCES ON VILLAGE BUILDING FAÇADE IMAGES

Abstract. Rapid access to the basic structure of village buildings is conducive to the investigation of the load-bearing bodies of village houses and provides data support for disaster assessment and post-disaster rescue and reconstruction. The development of computer vision technology provides new ideas and tools for identifying and extracting basic structures of housing buildings. Considering that the original Mask R-CNN ignores the spatial association and relationship of door and window elements, an advanced deep learning model based on Mask R-CNN network is proposed in this paper to detect and segment the door and window structure from the façade images. The improved network architectures integrate the attention mechanism with the original network, containing an improved Coordinate Attention(CA) module and a relationship module-based head network. The experimental results show that the Average Precision(AP) value of the backbone combined with the improved CA module is increased by 0.7% and 0.7% on regression and segmentation tasks respectively, compared with the original Mask R-CNN network. In the head network based on the relationship module, the calculation strategy of the relational module proposed in this paper increases the AP values of detection and segmentation from 76.7% and 77.7% to 80.6% and 80.0%, respectively.

Automatic segmentation of vocal tract articulators in real-time magnetic resonance imaging

Background and ObjectivesThe characterization of the vocal tract geometry during speech interests various research topics, including speech production modeling, motor control analysis, and speech therapy design. Real-time MRI is a reliable and non-invasive tool for this purpose. In most cases, it is necessary to know the contours of the individual articulators from the glottis to the lips. Several techniques have been proposed for segmenting vocal tract articulators, but most are limited to specific applications. Moreover, they often do not provide individualized contours for all soft-tissue articulators in a multi-speaker configuration. MethodsA Mask R-CNN network was trained to detect and segment the vocal tract articulator contours in two real-time MRI (RT-MRI) datasets with speech recordings of multiple speakers. Two post-processing algorithms were then proposed to convert the network's outputs into geometrical curves. Nine articulators were considered: the two lips, tongue, soft palate, pharynx, arytenoid cartilage, epiglottis, thyroid cartilage, and vocal folds. A leave-one-out cross-validation protocol was used to evaluate inter-speaker generalization. The evaluation metrics were the point-to-closest-point distance and the Jaccard index (for articulators annotated as closed contours). ResultsThe proposed method accurately segmented the vocal tract articulators, with an average root mean square point-to-closest-point distance of less than 2.2mm for all the articulators in the leave-one-out cross-validation setting. The minimum P2CPRMS was 0.91mm for the upper lip, and the maximum was 2.18mm for the tongue. The Jaccard indices for the thyroid cartilage and vocal folds were 0.60 and 0.61, respectively. Additionally, the method adapted to a new subject with only ten annotated samples. ConclusionsOur research introduced a method for individually segmenting nine non-rigid vocal tract articulators in real-time MRI movies. The software is openly available as an installable package to the speech community. It is designed to develop speech applications and clinical and non-clinical research in fields that require vocal tract geometry, such as speech, singing, and human beatboxing.

Lumen segmentation using a Mask R-CNN in carotid arteries with stenotic atherosclerotic plaque

In patients at high risk for ischemic stroke, clinical carotid ultrasound is often used to grade stenosis, determine plaque burden and assess stroke risk. Analysis currently requires a trained sonographer to manually identify vessel and plaque regions, which is time and labor intensive. We present a method for automatically determining bounding boxes and lumen segmentation using a Mask R-CNN network trained on sonographer assisted ground-truth carotid lumen segmentations. Automatic lumen segmentation also lays the groundwork for developing methods for accurate plaque segmentation, and wall thickness measurements in cases with no plaque. Different training schemes are used to identify the Mask R-CNN model with the highest accuracy. Utilizing a single-channel B-mode training input, our model produces a mean bounding box intersection over union (IoU) of 0.81 and a mean lumen segmentation IoU of 0.75. However, we encountered errors in prediction when the jugular vein is the most prominently visualized vessel in the B-mode image. This was due to the fact that our dataset has limited instances of B-mode images with both the jugular vein and carotid artery where the vein is dominantly visualized. Additional training datasets are anticipated to mitigate this issue.

An Edge-Guided Deep Learning Solar Panel Hotspot Thermal Image Segmentation Algorithm

To overcome the deficiencies in segmenting hot spots from thermal infrared images, such as difficulty extracting the edge features, low accuracy, and a high missed detection rate, an improved Mask R-CNN photovoltaic hot spot thermal image segmentation algorithm has been proposed in this paper. Firstly, the edge image features of hot spots were extracted based on residual neural networks. Secondly, by combining the feature pyramid structure, an edge-guided feature pyramid structure was designed, and the hot spot edge features were injected into a Mask R-CNN network. Thirdly, an infrared spatial attention module was introduced into the Mask R-CNN network when feature extraction and the infrared features of the detected hot spots were enhanced. Fourthly, the size ratio of the candidate frames was adjusted self-adaptively according to the structural characteristics of the aspect ratio of the hot spots. Finally, the validation experiments were conducted, and the results demonstrated that the hot spot contours of thermal infrared images were enhanced through the algorithm proposed in this paper, and the segmentation accuracy was significantly improved.

Detecting H i Galaxies with Deep Neural Networks in the Presence of Radio Frequency Interference

In the neutral hydrogen (H i) galaxy survey, a significant challenge is to identify and extract the H i galaxy signal from the observational data contaminated by radio frequency interference (RFI). For a drift-scan survey, or more generally a survey of a spatially continuous region, in the time-ordered spectral data, the H i galaxies and RFI all appear as regions that extend an area in the time-frequency waterfall plot, so the extraction of the H i galaxies and RFI from such data can be regarded as an image segmentation problem, and machine-learning methods can be applied to solve such problems. In this study, we develop a method to effectively detect and extract signals of H i galaxies based on a Mask R-CNN network combined with the PointRend method. By simulating FAST-observed galaxy signals and potential RFI impact, we created a realistic data set for the training and testing of our neural network. We compared five different architectures and selected the best-performing one. This architecture successfully performs instance segmentation of H i galaxy signals in the RFI-contaminated time-ordered data, achieving a precision of 98.64% and a recall of 93.59%.

MCBM-SLAM: An Improved Mask-Region-Convolutional Neural Network-Based Simultaneous Localization and Mapping System for Dynamic Environments

Current research on SLAM can be divided into two parts according to the research scenario: SLAM research in dynamic scenarios and SLAM research in static scenarios. Research is now relatively well established for static environments. However, in dynamic environments, the impact of moving objects leads to inaccurate positioning accuracy and poor robustness of SLAM systems. To address the shortcomings of SLAM systems in dynamic environments, this paper develops a series of solutions to address these problems. First, an attention-based Mask R-CNN network is used to ensure the reliability of dynamic object extraction in dynamic environments. Dynamic feature points are then rejected based on the mask identified by the Mask R-CNN network, and a preliminary estimate of the camera pose is made. Secondly, in order to enhance the picture matching quality and efficiently reject the mismatched points, this paper proposes an image mismatching algorithm incorporating adaptive edge distance with grid motion statistics. Finally, static feature points on dynamic objects are re-added using motion constraints and chi-square tests, and the camera’s pose is re-estimated. The SLAM algorithm of this paper was run on the KITTI and TUM-RGBD datasets, respectively, and the results show that the SLAM algorithm of this paper outperforms the ORB-SLAM2 algorithm for sequences containing more dynamic objects in the KITTI dataset. On the TUM-RGBD dataset, the Dyna-SLAM algorithm increased localization accuracy by an average of 71.94% when compared to the ORB-SLAM2 method, while the SLAM algorithm in this study increased localization accuracy by an average of 78.18% when compared to the ORB-SLAM2 algorithm. When compared to the Dyna-SLAM technique, the SLAM algorithm in this work increased average positioning accuracy by 6.24%, proving that it is superior to Dyna-SLAM.

Nondestructive detection of key phenotypes for the canopy of the watermelon plug seedlings based on deep learning

Nondestructive measurement technology of phenotype can provide substantial phenotypic data support for applications such as seedling breeding, management, and quality testing. The current method of measuring seedling phenotypes mainly relies on manual measurement which is inefficient, subjective and destroys samples. Therefore, the paper proposes a nondestructive measurement method for the canopy phenotype of the watermelon plug seedlings based on deep learning. The Azure Kinect was used to shoot canopy color images, depth images, and RGB-D images of the watermelon plug seedlings. The Mask-RCNN network was used to classify, segment, and count the canopy leaves of the watermelon plug seedlings. To reduce the error of leaf area measurement caused by mutual occlusion of leaves, the leaves were repaired by CycleGAN, and the depth images were restored by image processing. Then, the Delaunay triangulation was adopted to measure the leaf area in the leaf point cloud. The YOLOX target detection network was used to identify the growing point position of each seedling on the plug tray. Then the depth differences between the growing point and the upper surface of the plug tray were calculated to obtain plant height. The experiment results show that the nondestructive measurement algorithm proposed in this paper achieves good measurement performance for the watermelon plug seedlings from the 1 true-leaf to 3 true-leaf stages. The average relative error of measurement is 2.33% for the number of true leaves, 4.59% for the number of cotyledons, 8.37% for the leaf area, and 3.27% for the plant height. The experiment results demonstrate that the proposed algorithm in this paper provides an effective solution for the nondestructive measurement of the canopy phenotype of the plug seedlings.

Lightweight Strawberry Instance Segmentation on Low-Power Devices for Picking Robots

Machine vision plays a great role in localizing strawberries in a complex orchard or greenhouse for picking robots. Due to the variety of each strawberry (shape, size, and color) and occlusions of strawberries by leaves and stems, precisely locating each strawberry brings a great challenge to the vision system of picking robots. Several methods have been developed for localizing strawberries, based on the well-known Mask R-CNN network, which, however, are not efficient running on the picking robots. In this paper, we propose a simple and highly efficient framework for strawberry instance segmentation running on low-power devices for picking robots, termed StrawSeg. Instead of using the common paradigm of “detection-then-segment”, we directly segment each strawberry in a single-shot manner without relying on object detection. In our model, we design a novel feature aggregation network to merge features with different scales, which employs a pixel shuffle operation to increase the resolution and reduce the channels of features. Experiments on the open-source dataset StrawDI_Db1 demonstrate that our model can achieve a good trade-off between accuracy and inference speed on a low-power device.

KLASIFIKASI RAS ANJING MENGGUNAKAN METODE CONVOLUTIONAL NEURAL NETWORK DENGAN ARSITEKTUR VGG-16

Dogs are lovable animals that bring companionship and fun to any household, but things to consider before getting a dog are their daily needs including food, shelter, and animal care, as well as affection and physical and mental stimulation. This research was conducted to introduce dog breeds and how to care for each breed. This system is built using Mask R-CNN and Convolutional Neural Network (CNN), a deep learning architecture.
 Mask R-CNN model is used for detecting and cropping dog in images, trained using Microsoft Common Object of Context (MS COCO) dataset. CNN is used for classification of dog breeds in images and is trained using 17.513 images of 17 different breeds. Result for Mask R-CNN show the detection accuracy for dogs has 74% using test images, and CNN show the identification accuracy using test images has 82% accuracy, and for CNN using cropped images has 87% accuracy.

Locating Tea Bud Keypoints by Keypoint Detection Method Based on Convolutional Neural Network

Tea is one of the most consumed beverages in the whole world. Premium tea is a kind of tea with high nutrition, quality, and economic value. This study solves the problem of detecting premium tea buds in automatic plucking by training a modified Mask R-CNN network for tea bud detection in images. A new anchor generation method by adding additional anchors and the CIoU loss function were used in this modified model. In this study, the keypoint detection branch was optimized to locate tea bud keypoints, which, containing a fully convolutional network (FCN), is also built to locate the keypoints of bud objects. The built convolutional neural network was trained through our dataset and obtained an 86.6% precision and 88.3% recall for the bud object detection. The keypoint localization had a precision of 85.9% and a recall of 83.3%. In addition, a dataset for the tea buds and picking points was constructed in study. The experiments show that the developed model can be robust for a range of tea-bud-harvesting scenarios and introduces the possibility and theoretical basis for fully automated tea bud harvesting.