Improved Mask R-CNN Research Articles

Segmentation Method of Zanthoxylum bungeanum Cluster Based on Improved Mask R-CNN

The precise segmentation of Zanthoxylum bungeanum clusters is crucial for developing picking robots. An improved Mask R-CNN model was proposed in this study for the segmentation of Zanthoxylum bungeanum clusters in natural environments. Firstly, the Swin-Transformer network was introduced into the model’s backbone as the feature extraction network to enhance the model’s feature extraction capabilities. Then, the SK attention mechanism was utilized to fuse the detailed information into the mask branch from the low-level feature map of the feature pyramid network (FPN), aiming to supplement the image detail features. Finally, the distance intersection over union (DIOU) loss function was adopted to replace the original bounding box loss function of Mask R-CNN. The model was trained and tested based on a self-constructed Zanthoxylum bungeanum cluster dataset. Experiments showed that the improved Mask R-CNN model achieved 84.0% and 77.2% in detection mAP50box and segmentation mAP50mask, respectively, representing a 5.8% and 4.6% improvement over the baseline Mask R-CNN model. In comparison to conventional instance segmentation models, such as YOLACT, Mask Scoring R-CNN, and SOLOv2, the improved Mask R-CNN model also exhibited higher segmentation precision. This study can provide valuable technology support for the development of Zanthoxylum bungeanum picking robots.

A Temperature Monitoring Method for Sensor Arrays Based on Temperature Mapping and Improved Mask R-CNN

A Temperature Monitoring Method for Sensor Arrays Based on Temperature Mapping and Improved Mask R-CNN

Long-term video activity monitoring and anomaly alerting of group-housed pigs

The activity level of group-housed pigs can reflect their daily behavioral patterns and health conditions. Currently, most pig farms mainly rely on manual daily patrols to monitor the activities of group-housed pigs, which is labor-intensive and inaccurate. In this paper, we propose a computer vision-based method for the automatic monitoring and alerting of group-housed pigs activity anomaly. We established a pig segmentation model based on an improved Mask R-CNN incorporating the CBAM attention mechanism and CIoU loss function for segmenting pigs from video frames. The CBAM attention mechanism enables the model to focus more effectively on target regions, while CIoU loss function guides precise learning of object boundaries, outperforming the traditional IoU loss. Grounded on the segmentation results, we invocated a frame difference method to quantify the activity level of group-housed pigs. Subsequently, we adopted the K-means clustering algorithm to explore the daily behaviors of these pigs. Furthermore, we constructed a SARIMA model using historical data to proactively forecast potential shifts in activity levels. The experimental results demonstrated that the accuracy and recall rate of the pig instance segmentation model were 98.7% and 97.9%, respectively. The SARIMA model performed well in predicting activity levels of group-housed pigs with differing health status. For the pig herds under healthy status, the Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) were 6.01 and 7.67 respectively, while for the pig herds under unhealthy status, the MAE and RMSE were 6.41 and 9.30 respectively. We proposed an effectively method to monitor daily behavioral patterns and anomalies of group-housed pigs, providing valuable references for pig farm management.

Detection of Straw Coverage under Conservation Tillage Based on an Improved Mask Regional Convolutional Neural Network (Mask R-CNN)

Conservation tillage, a crucial method for protecting soil fertility, depends heavily on maintaining adequate straw coverage. The current method of straw coverage detection relies primarily on manual measurement, which is both time-consuming and laborious. This paper introduces a novel straw coverage detection approach based on an improved mask regional convolutional neural network (Mask R-CNN) algorithm. Several images of wheat straw-covered fields were taken, and the dataset was augmented using techniques like image inversion, contrast enhancement, Gaussian noise addition, and translation after cropping the original images. These fields use a crop rotation cycle of wheat and corn. Subsequently, the straw images were annotated using the Labelme annotation tool to obtain the available straw instance segmentation dataset. The Mask R-CNN algorithm was improved by refining the mask generation network structure through a multi-feature fusion strategy, which interweaves features from both the encoder and the mask generation network, enhancing the model’s ability to capture detailed and shape information of the straw. Lastly, using the mask information output by the improved Mask R-CNN algorithm, the straw coverage was calculated by counting the proportion of pixels within each segmented region. In the results, compared to the original Mask R-CNN algorithm, our improved Mask R-CNN algorithm achieved an average improvement of 7.8% in segmentation accuracy, indicating that the improved Mask R-CNN algorithm offers superior segmentation performance. Thus, the new algorithm can achieve straw coverage detection with higher accuracy and can provide a reference for other agricultural applications.

Intelligent Detection of Tunnel Leakage Based on Improved Mask R-CNN

The instance segmentation model based on deep learning has addressed the challenges in intelligently detecting water leakage in shield tunneling. Due to the limited generalization ability of the baseline model, occurrences of missed detections, false detections, and repeated detections are encountered during the actual detection of tunnel water leakage. This paper adopts Mask R-CNN as the baseline model and introduces a mask cascade strategy to enhance the quality of positive samples. Additionally, the backbone network in the model is replaced with RegNetX to enlarge the model’s receptive field, and MDConv is introduced to enhance the model’s feature extraction capability in the edge receptive field region. Building upon these improvements, the proposed model is named Cascade-MRegNetX. The backbone network MRegNetX features a symmetrical block structure, which, when combined with deformable convolutions, greatly assists in extracting edge features from corresponding regions. During the dataset preprocessing stage, we augment the dataset through image rotation and classification, thereby improving both the quality and quantity of samples. Finally, by leveraging pre-trained models through transfer learning, we enhance the robustness of the target model. This model can effectively extract features from water leakage areas of different scales or deformations. Through instance segmentation experiments conducted on a dataset comprising 766 images of tunnel water leakage, the experimental results demonstrate that the improved model achieves higher precision in tunnel water leakage mask detection. Through these enhancements, the detection effectiveness, feature extraction capability, and generalization ability of the baseline model are improved. The improved Cascade-MRegNetX model achieves respective improvements of 7.7%, 2.8%, and 10.4% in terms of AP, AP0.5, and AP0.75 compared to the existing Cascade Mask R-CNN model.

Intelligent Diagnosis of Concrete Defects Based on Improved Mask R-CNN

Intelligent Diagnosis of Concrete Defects Based on Improved Mask R-CNN

Research on Key Algorithm for Sichuan Pepper Pruning Based on Improved Mask R-CNN

Research on Key Algorithm for Sichuan Pepper Pruning Based on Improved Mask R-CNN

Early gastric cancer detection and lesion segmentation based on deep learning and gastroscopic images

Gastric cancer is a highly prevalent disease that poses a serious threat to public health. In clinical practice, gastroscopy is frequently used by medical practitioners to screen for gastric cancer. However, the symptoms of gastric cancer at different stages of advancement vary significantly, particularly in the case of early gastric cancer (EGC). The manifestations of EGC are often indistinct, leading to a detection rate of less than 10%. In recent years, researchers have focused on leveraging deep learning algorithms to assist medical professionals in detecting EGC and thereby improve detection rates. To enhance the ability of deep learning to detect EGC and segment lesions in gastroscopic images, an Improved Mask R-CNN (IMR-CNN) model was proposed. This model incorporates a “Bi-directional feature extraction and fusion module” and a “Purification module for feature channel and space” based on the Mask R-CNN (MR-CNN). Our study includes a dataset of 1120 images of EGC for training and validation of the models. The experimental results indicate that the IMR-CNN model outperforms the original MR-CNN model, with Precision, Recall, Accuracy, Specificity and F1-Score values of 92.9%, 95.3%, 93.9%, 92.5% and 94.1%, respectively. Therefore, our proposed IMR-CNN model has superior detection and lesion segmentation capabilities and can effectively aid doctors in diagnosing EGC from gastroscopic images.

A deep semantic network-based image segmentation of soybean rust pathogens.

Asian soybean rust is a highly aggressive leaf-based disease triggered by the obligate biotrophic fungus Phakopsora pachyrhizi which can cause up to 80% yield loss in soybean. The precise image segmentation of fungus can characterize fungal phenotype transitions during growth and help to discover new medicines and agricultural biocides using large-scale phenotypic screens. The improved Mask R-CNN method is proposed to accomplish the segmentation of densely distributed, overlapping and intersecting microimages. First, Res2net is utilized to layer the residual connections in a single residual block to replace the backbone of the original Mask R-CNN, which is then combined with FPG to enhance the feature extraction capability of the network model. Secondly, the loss function is optimized and the CIoU loss function is adopted as the loss function for boundary box regression prediction, which accelerates the convergence speed of the model and meets the accurate classification of high-density spore images. The experimental results show that the mAP for detection and segmentation, accuracy of the improved algorithm is improved by 6.4%, 12.3% and 2.2% respectively over the original Mask R-CNN algorithm. This method is more suitable for the segmentation of fungi images and provide an effective tool for large-scale phenotypic screens of plant fungal pathogens.

A Method for Calculating the Leaf Area of Pak Choi Based on an Improved Mask R-CNN

The leaf area of pak choi is a critical indicator of growth rate, nutrient absorption, and photosynthetic efficiency, and it is required to be precisely measured for an optimal agricultural output. Traditional methods often fail to deliver the necessary accuracy and efficiency. We propose a method for calculating the leaf area of pak choi based on an improved Mask R-CNN. We have enhanced Mask R-CNN by integrating an advanced attention mechanism and a two-layer fully convolutional network (FCN) into its segmentation branch. This integration significantly improves the model’s ability to detect and segment leaf edges with increased precision. By extracting the contours of reference objects, the conversion coefficient between the pixel area and the actual area is calculated. Using the mask segmentation output from the model, the area of each leaf is calculated. Experimental results demonstrate that the improved model achieves mean average precision (mAP) scores of 0.9136 and 0.9132 in detection and segmentation tasks, respectively, representing improvements of 1.01% and 1.02% over the original Mask R-CNN. The model demonstrates excellent recognition and segmentation capabilities for pak choi leaves. The error between the calculation result of the segmented leaf area and the actual measured area is less than 4.47%. These results indicate that the proposed method provides a reliable segmentation and prediction performance. It eliminates the need for detached leaf measurements, making it suitable for real-life leaf area measurement scenarios and providing valuable support for automated production technologies in plant factories.

Rapid detection of non-normal teeth on dental X-ray images using improved Mask R-CNN with attention mechanism.

Dental health has been getting increased attention. Timely detection of non-normal teeth (caries, residual root, retainer, teeth filling, etc.) is of great importance for people's health, well-being, and quality of life. This work proposes a rapid detection of non-normal teeth based on improved Mask R-CNN, aiming to achieve comprehensive screening of non-normal teeth on dental X-ray images. An improved Mask R-CNN based on attention mechanism was used to develop a non-normal teeth detection method trained on a high-quality annotated dataset, which can segment the whole mask of each non-normal tooth on the dental X-ray image immediately. The average precision (AP) of the proposed non-normal teeth detection was 0.795 with an intersection-over-union of 0.5 and max detections (maxDets) of 32, which was higher than that of the typical Mask R-CNN method (AP = 0.750). In addition, validation experiments showed that the evaluation metrics (AP, recall, precision-recall (P-R) curve) of the proposed method were superior to those of the Mask R-CNN method. Furthermore, the experimental results indicated that proposed method exhibited a high sensitivity (95.65%) in detecting secondary caries. The proposed method took about 0.12s to segment non-normal teeth on one dental X-ray image using the laptop (8G memory, NVIDIA RTX 3060 graphics processing unit), which was much faster than conventional manual methods. The proposed method enhances the accuracy and efficiency of abnormal tooth diagnosis for practitioners, while also facilitating early detection and treatment of dental caries to substantially lower patient costs. Additionally, it can enable rapid and objective evaluation of student performance in dental examinations.

A Paddy Image Segmentation Method Based on Improved Mask R-CNN

A Paddy Image Segmentation Method Based on Improved Mask R-CNN

An Improved Mask R-CNN Micro-Crack Detection Model for the Surface of Metal Structural Parts.

Micro-crack detection is an essential task in critical equipment health monitoring. Accurate and timely detection of micro-cracks can ensure the healthy and stable service of equipment. Aiming at improving the low accuracy of the conventional target detection model during the task of detecting micro-cracks on the surface of metal structural parts, this paper built a micro-cracks dataset and explored a detection performance optimization method based on Mask R-CNN. Firstly, we improved the original FPN structure, adding a bottom-up feature fusion path to enhance the information utilization rate of the underlying feature layer. Secondly, we added the methods of deformable convolution kernel and attention mechanism to ResNet, which can improve the efficiency of feature extraction. Lastly, we modified the original loss function to optimize the network training effect and model convergence rate. The ablation comparison experiments shows that all the improvement schemes proposed in this paper have improved the performance of the original Mask R-CNN. The integration of all the improvement schemes can produce the most significant performance improvement effects in recognition, classification, and positioning simultaneously, thus proving the rationality and feasibility of the improved scheme in this paper.

Fabric defect detection and classification via deep learning-based improved Mask RCNN

Fabric defect detection and classification via deep learning-based improved Mask RCNN

Clustering and Segmentation of Adhesive Pests in Apple Orchards Based on GMM-DC

The segmentation of individual pests is a prerequisite for pest feature extraction and identification. To address the issue of pest adhesion in the apple orchard pest identification process, this research proposed a pest adhesion image segmentation method based on Gaussian Mixture Model with Density and Curvature Weighting (GMM-DC). First, in the HSV color space, an image was desaturated by adjusting the hue and inverting to mitigate threshold crossing points. Subsequently, threshold segmentation and contour selection methods were used to separate the image background. Next, a shape factor was introduced to determine the regions and quantities of adhering pests, thereby determining the number of model clustering clusters. Then, point cloud reconstruction was performed based on the color and spatial distribution features of the pests. To construct the GMM-DC segmentation model, a spatial density (SD) and spatial curvature (SC) information function were designed and embedded in the GMM. Finally, experimental analysis was conducted on the collected apple orchard pest images. The results showed that GMM-DC achieved an average accurate segmentation rate of 95.75%, an average over-segmentation rate of 2.83%, and an average under-segmentation rate of 1.42%. These results significantly outperformed traditional image segmentation methods. In addition, the original and improved Mask R-CNN models were used as recognition models, and the mean Average Precision was used as the evaluation metric. Recognition experiments were conducted on pest images with and without the proposed method. The results show the mean Average Precision for pest images segmented with the proposed method as 92.43% and 96.75%. This indicates an improvement of 13.01% and 12.18% in average recognition accuracy, respectively. The experimental results demonstrate that this method provides a theoretical and methodological foundation for accurate pest identification in orchards.

An Edge-Guided Deep Learning Solar Panel Hotspot Thermal Image Segmentation Algorithm

To overcome the deficiencies in segmenting hot spots from thermal infrared images, such as difficulty extracting the edge features, low accuracy, and a high missed detection rate, an improved Mask R-CNN photovoltaic hot spot thermal image segmentation algorithm has been proposed in this paper. Firstly, the edge image features of hot spots were extracted based on residual neural networks. Secondly, by combining the feature pyramid structure, an edge-guided feature pyramid structure was designed, and the hot spot edge features were injected into a Mask R-CNN network. Thirdly, an infrared spatial attention module was introduced into the Mask R-CNN network when feature extraction and the infrared features of the detected hot spots were enhanced. Fourthly, the size ratio of the candidate frames was adjusted self-adaptively according to the structural characteristics of the aspect ratio of the hot spots. Finally, the validation experiments were conducted, and the results demonstrated that the hot spot contours of thermal infrared images were enhanced through the algorithm proposed in this paper, and the segmentation accuracy was significantly improved.

Multi-scale coal and gangue detection in dense state based on improved Mask RCNN

Coal and gangue intelligent sorting accounts for a vital proportion of modern coal mines, which takes on critical significance in boosting clean coal utilization. Aiming at the problem of low recognition accuracy that arises from the large diversity in particle size of raw coal and the adhesion and half-occlusion between them, A Mask RCNN-based instance segmentation network for coal and gangue image is proposed. First, multi-branch parallel feature extraction bottlenecks are developed to build a lightweight backbone with a mixed attention mechanism and self-correcting convolution incorporated to enhance the feature representation of targets. Subsequently, a necknet is developed, aggregating the context feature information of the backbone from channel and space to enhance the position and boundary information of targets. Lastly, the effectiveness of this method is fully validated by ablation and test experiments using self-built coal and gangue RGB image datasets include varieties of characteristics as input. As indicated by the experimental results, the above-described method is capable of enhancing the segmentation ability of stacked and adhered coal and gangue and reducing the missed detection rate of small targets. ISNet_CG achieves mAP, mIoU, F1-scores, and FPS of 98.0, 96.5, 0.987, and 24.2 FPS, respectively, such that the correct category and location can be provided for coal and gangue in a dense state.

Extraction of Pine Wilt Disease Regions Using UAV RGB Imagery and Improved Mask R-CNN Models Fused with ConvNeXt

Pine wilt disease (PWD) is one of the most concerning diseases in forestry and poses a considerable threat to forests. Since the deep learning approach can interpret the raw images acquired by UAVs, it provides an effective means for forest health detection. However, the fact that only PWD can be detected but not the degree of infection can be evaluated hinders forest management, so it is necessary to establish an effective method to accurately detect PWD and extract regions infected by PWD. Therefore, a Mask R-CNN-based PWD detection and extraction algorithm is proposed in this paper. Firstly, the extraction of image features is improved by using the advanced ConvNeXt network. Then, it is proposed to change the original multi-scale structure to PA-FPN and normalize it by using GN and WS methods, which effectively enhances the data exchange between the bottom and top layers under low Batch-size training. Finally, a branch is added to the Mask module to improve the ability to extract objects using fusion. In addition, a PWD region extraction module is proposed in this paper for evaluating the damage caused by PWD. The experimental results show that the improved method proposed in this paper can achieve 91.9% recognition precision, 90.2% mapping precision, and 89.3% recognition rate of the affected regions on the PWD dataset. It can effectively identify the distribution of diseased pine trees and calculate the damage proportion in a relatively accurate way to facilitate the management of forests.

SMR-RS: An Improved Mask R-CNN Specialized for Rolled Rice Stubble Row Segmentation

As a highly productive rice, ratoon rice is widely planted worldwide, but the rolling of rice stubble in mechanical harvesting severely limits its total yield; based on this, some scholars have proposed rolled rice stubble righting machines. However, limited by the uncertainty of the field environment, the machine’s localization accuracy of the target needs to be improved. To address this problem, real-time detection of rolled rice stubble rows is a prerequisite. Therefore, this paper introduces a deep learning method for the first time to achieve this. To this end, we presented a novel approach to improve a model that is used for the simplification of Mask R-CNN, which does not require any modules to be added or replaced on the original model. Firstly, two branches in the second stage were deleted, and the region proposals output from the stage was used directly as the mask generation region, and segmentation performance was substantially improved after a simple optimization of the region proposals. Further, the contribution of the feature map was counted, and the backbone network was simplified accordingly. The resulting SMR-RS model was still able to perform instance segmentation and has better segmentation performance than Mask R-CNN and other state-of-the-art models while significantly reducing the average image processing time and hardware consumption.

Research on stacked ore detection based on improved Mask RCNN under complex background

Research on stacked ore detection based on improved Mask RCNN under complex background