Instance Segmentation Research Articles

Mask R-CNN and Centroid Tracking Algorithm to Process UAV Based Thermal-RGB Video for Drylot Cattle Heat Stress Monitoring

This study developed and evaluated an algorithm for processing thermal-RGB video feeds captured by an unmanned aerial vehicle (UAV) to automate heat stress monitoring in cattle housed in the drylots. The body surface temperature (BST) of individual cows was used as an indicator of heat stress. UAV data were collected using RGB and thermal infrared imagers, respectively, at 2 and 6.67 cm per pixel spatial resolution in Spring 2023 (dataset-1) and Summer 2024 (dataset-2). Study sites were two commercial drylots in Washington State. The custom algorithms were developed to: (1) detect and localize individual cows using a Mask R-CNN-based instance segmentation model combined with centroid tracking; and (2) extract BST by averaging the thermal-imagery pixels for each of the segmented cows. The algorithm showed higher detection accuracy with RGB images as input (F1 score: 0.89) compared to thermal (F1 score: 0.64). BST extraction with combined RGB and thermal imaging approach required corrections for alignment problems associated with differences in optics, imaging field of view, resolution, and lens properties. Consequently, thermal imaging-only approach was adopted for assessing real-time cow localization and BST estimation. Operating at one frame per second, algorithm successfully detected 72.4% and 81.65% of total cows in video frames from dataset-1 (38 s) and -2 (48 s), respectively. The mean absolute difference between algorithm output and ground truth (BSTGT) was 2.1 °C (dataset-1) and 3.3 °C (dataset-2), demonstrating satisfactory performance. With further refinements, this approach could be a viable tool for real-time heat stress monitoring in large-scale drylot production systems.

An Improved Instance Segmentation Method for Complex Elements of Farm UAV Aerial Survey Images.

Farm aerial survey layers can assist in unmanned farm operations, such as planning paths and early warnings. To address the inefficiencies and high costs associated with traditional layer construction, this study proposes a high-precision instance segmentation algorithm based on SparseInst. Considering the structural characteristics of farm elements, this study introduces a multi-scale attention module (MSA) that leverages the properties of atrous convolution to expand the sensory field. It enhances spatial and channel feature weights, effectively improving segmentation accuracy for large-scale and complex targets in the farm through three parallel dense connections. A bottom-up aggregation path is added to the feature pyramid fusion network, enhancing the model's ability to perceive complex targets such as mechanized trails in farms. Coordinate attention blocks (CAs) are incorporated into the neck to capture richer contextual semantic information, enhancing farm aerial imagery scene recognition accuracy. To assess the proposed method, we compare it against existing mainstream object segmentation models, including the Mask R-CNN, Cascade-Mask, SOLOv2, and Condinst algorithms. The experimental results show that the improved model proposed in this study can be adapted to segment various complex targets in farms. The accuracy of the improved SparseInst model greatly exceeds that of Mask R-CNN and Cascade-Mask and is 10.8 and 12.8 percentage points better than the average accuracy of SOLOv2 and Condinst, respectively, with the smallest number of model parameters. The results show that the model can be used for real-time segmentation of targets under complex farm conditions.

EF yolov8s: A Human–Computer Collaborative Sugarcane Disease Detection Model in Complex Environment

The precise identification of disease traits in the complex sugarcane planting environment not only effectively prevents the spread and outbreak of common diseases but also allows for the real-time monitoring of nutrient deficiency syndrome at the top of sugarcane, facilitating the supplementation of relevant nutrients to ensure sugarcane quality and yield. This paper proposes a human–machine collaborative sugarcane disease detection method in complex environments. Initially, data on five common sugarcane diseases—brown stripe, rust, ring spot, brown spot, and red rot—as well as two nutrient deficiency conditions—sulfur deficiency and phosphorus deficiency—were collected, totaling 11,364 images and 10 high-definition videos captured by a 4K drone. The data sets were augmented threefold using techniques such as flipping and gamma adjustment to construct a disease data set. Building upon the YOLOv8 framework, the EMA attention mechanism and Focal loss function were added to optimize the model, addressing the complex backgrounds and imbalanced positive and negative samples present in the sugarcane data set. Disease detection models EF-yolov8s, EF-yolov8m, EF-yolov8n, EF-yolov7, and EF-yolov5n were constructed and compared. Subsequently, five basic instance segmentation models of YOLOv8 were used for comparative analysis, validated using nutrient deficiency condition videos, and a human–machine integrated detection model for nutrient deficiency symptoms at the top of sugarcane was constructed. The experimental results demonstrate that our improved EF-yolov8s model outperforms other models, achieving mAP_0.5, precision, recall, and F1 scores of 89.70%, 88.70%, 86.00%, and 88.00%, respectively, highlighting the effectiveness of EF-yolov8s for sugarcane disease detection. Additionally, yolov8s-seg achieves an average precision of 80.30% with a smaller number of parameters, outperforming other models by 5.2%, 1.9%, 2.02%, and 0.92% in terms of mAP_0.5, respectively, effectively detecting nutrient deficiency symptoms and addressing the challenges of sugarcane growth monitoring and disease detection in complex environments using computer vision technology.

Asymmetric patch sampling for contrastive learning

Asymmetric appearance between positive pair effectively reduces the risk of representation degradation in contrastive learning. However, there are still a mass of appearance similarities between positive pair constructed by the existing methods, thus inhibiting the further representation improvement. To address the above issue, we propose a novel asymmetric patch sampling strategy, which significantly reduces the appearance similarities but retains the image semantics. Specifically, dual patch sampling strategies are respectively applied to the given image. First, sparse patch sampling is conducted to obtain the first view, which reduces spatial redundancy of image and allows a more asymmetric view. Second, a selective patch sampling is proposed to construct another view with large appearance discrepancy relative to the first one. Due to the inappreciable appearance similarities between positive pair, the trained model is encouraged to capture the similarities on semantics, instead of low-level ones.Experimental results demonstrate that our method significantly outperforms the existing self-supervised learning methods on ImageNet-1K and CIFAR datasets, e.g., 2.5% finetuning accuracy improvement on CIFAR100. Furthermore, our method achieves state-of-the-art performance on downstream tasks, object detection and instance segmentation on COCO. Additionally, compared to other self-supervised methods, our method is more efficient on both memory and computation during pretraining. The source code and the trained weights are available at https://github.com/visresearch/aps.

Zero-shot image segmentation for monitoring thermal conditions of individual cage-free laying hens

Body temperature is a critical indicator of the health and productivity of egg-laying chickens and other domesticated animals. Recent advancements in thermography allow for precise surface temperature measurement without physical contact with animals, reducing animal stress from human handling. Gold standard temperature analysis via thermography requires manual selection of limited points for an object of interest, which could be time-consuming and inadequate for representing the comprehensive thermal profile of a chicken’s body. The objective of this study was to leverage and optimize a zero-shot artificial intelligence technology for the automatic segmentation of individual cage-free laying hens within thermal images, providing insights into their overall thermal conditions. A zero-shot image segmentation model (Segment Anything, “SAM”) was modified by replacing manual selections of target points with automatic selection of the initial point using pre-processing techniques (e.g., thresholding) in each thermal image. The model was also incorporated with post-processing techniques integrated with a machine learning classifier to improve segmentation accuracy. Three versions of modified SAM models (i.e., SAM, FastSAM, and MobileSAM), two common instance segmentation algorithms (i.e., YOLOv8 and Mask R-CNN), and two foundation segmentation models (i.e., U2-Net and ISNet) were comparatively evaluated to determine the optimal one for bird segmentation from thermal images. A total of 1,917 thermal images were collected from cage-free laying hens (Hy-Line W-36) at 77–80 weeks of age. The image dataset exhibited considerable variations such as feathers, bird movement, body gestures, and the specific conditions of cage-free facilities. The experimental results demonstrate that the modified SAM did not only surpass the six zero-shot models—YOLOv8, Mask R-CNN, FastSAM, MobileSAM, U2Net, and ISNet—but also outperformed other modified SAM-based models (Modified FastSAM and Modified MobileSAM) in terms of hen detection performance, achieving a success rate of 84.4 %, and in segmentation performance, with an intersection over union of 85.5 %, recall of 91.0 %, and an F1 score of 92.3 %. The optimal model, modified SAM, was pipelined to extract statistics including the averages (°C) of mean (27.03, 27.04, 28.53, 26.68), median (26.27, 26.84, 28.28, 26.78), 25th percentile (25.33, 25.61, 27.26, 25.53), and 75th percentile (28.04, 27.95, 29.22, 27.55) of surface body temperature of individual laying hens in thermal images for each week. More statistics of hen body surface temperature can be extracted based on the segmentation results. The developed pipeline is a useful tool for automatically evaluating the thermal conditions of individual birds.

DAMM for the detection and tracking of multiple animals within complex social and environmental settings

Accurate detection and tracking of animals across diverse environments are crucial for studying brain and behavior. Recently, computer vision techniques have become essential for high-throughput behavioral studies; however, localizing animals in complex conditions remains challenging due to intra-class visual variability and environmental diversity. These challenges hinder studies in naturalistic settings, such as when animals are partially concealed within nests. Moreover, current tools are laborious and time-consuming, requiring extensive, setup-specific annotation and training procedures. To address these challenges, we introduce the 'Detect-Any-Mouse-Model' (DAMM), an object detector for localizing mice in complex environments with minimal training. Our approach involved collecting and annotating a diverse dataset of single- and multi-housed mice in complex setups. We trained a Mask R-CNN, a popular object detector in animal studies, to perform instance segmentation and validated DAMM’s performance on a collection of downstream datasets using zero-shot and few-shot inference. DAMM excels in zero-shot inference, detecting mice and even rats, in entirely unseen scenarios and further improves with minimal training. Using the SORT algorithm, we demonstrate robust tracking, competitive with keypoint-estimation-based methods. Notably, to advance and simplify behavioral studies, we release our code, model weights, and data, along with a user-friendly Python API and a Google Colab implementation.

Segmentation Method of Zanthoxylum bungeanum Cluster Based on Improved Mask R-CNN

The precise segmentation of Zanthoxylum bungeanum clusters is crucial for developing picking robots. An improved Mask R-CNN model was proposed in this study for the segmentation of Zanthoxylum bungeanum clusters in natural environments. Firstly, the Swin-Transformer network was introduced into the model’s backbone as the feature extraction network to enhance the model’s feature extraction capabilities. Then, the SK attention mechanism was utilized to fuse the detailed information into the mask branch from the low-level feature map of the feature pyramid network (FPN), aiming to supplement the image detail features. Finally, the distance intersection over union (DIOU) loss function was adopted to replace the original bounding box loss function of Mask R-CNN. The model was trained and tested based on a self-constructed Zanthoxylum bungeanum cluster dataset. Experiments showed that the improved Mask R-CNN model achieved 84.0% and 77.2% in detection mAP50box and segmentation mAP50mask, respectively, representing a 5.8% and 4.6% improvement over the baseline Mask R-CNN model. In comparison to conventional instance segmentation models, such as YOLACT, Mask Scoring R-CNN, and SOLOv2, the improved Mask R-CNN model also exhibited higher segmentation precision. This study can provide valuable technology support for the development of Zanthoxylum bungeanum picking robots.

DIO-SLAM: A Dynamic RGB-D SLAM Method Combining Instance Segmentation and Optical Flow.

Feature points from moving objects can negatively impact the accuracy of Visual Simultaneous Localization and Mapping (VSLAM) algorithms, while detection or semantic segmentation-based VSLAM approaches often fail to accurately determine the true motion state of objects. To address this challenge, this paper introduces DIO-SLAM: Dynamic Instance Optical Flow SLAM, a VSLAM system specifically designed for dynamic environments. Initially, the detection thread employs YOLACT (You Only Look At CoefficienTs) to distinguish between rigid and non-rigid objects within the scene. Subsequently, the optical flow thread estimates optical flow and introduces a novel approach to capture the optical flow of moving objects by leveraging optical flow residuals. Following this, an optical flow consistency method is implemented to assess the dynamic nature of rigid object mask regions, classifying them as either moving or stationary rigid objects. To mitigate errors caused by missed detections or motion blur, a motion frame propagation method is employed. Lastly, a dense mapping thread is incorporated to filter out non-rigid objects using semantic information, track the point clouds of rigid objects, reconstruct the static background, and store the resulting map in an octree format. Experimental results demonstrate that the proposed method surpasses current mainstream dynamic VSLAM techniques in both localization accuracy and real-time performance.

Instance segmentation of faces and mouth-opening degrees based on improved YOLOv8 method

Instance segmentation of faces and mouth-opening degrees based on improved YOLOv8 method

Practical guidelines for cell segmentation models under optical aberrations in microscopy

Cell segmentation is essential in biomedical research for analyzing cellular morphology and behavior. Deep learning methods, particularly convolutional neural networks (CNNs), have revolutionized cell segmentation by extracting intricate features from images. However, the robustness of these methods under microscope optical aberrations remains a critical challenge. This study evaluates cell image segmentation models under optical aberrations from fluorescence and bright field microscopy. By simulating different types of aberrations, including astigmatism, coma, spherical aberration, trefoil, and mixed aberrations, we conduct a thorough evaluation of various cell instance segmentation models using the DynamicNuclearNet (DNN) and LIVECell datasets, representing fluorescence and bright field microscopy cell datasets, respectively. We train and test several segmentation models, including the Otsu threshold method and Mask R-CNN with different network heads (FPN, C3) and backbones (ResNet, VGG, Swin Transformer), under aberrated conditions. Additionally, we provide usage recommendations for the Cellpose 2.0 Toolbox on complex cell degradation images. The results indicate that the combination of FPN and SwinS demonstrates superior robustness in handling simple cell images affected by minor aberrations. In contrast, Cellpose 2.0 proves effective for complex cell images under similar conditions. Furthermore, we innovatively propose the Point Spread Function Image Label Classification Model (PLCM). This model can quickly and accurately identify aberration types and amplitudes from PSF images, assisting researchers without optical training. Through PLCM, researchers can better apply our proposed cell segmentation guidelines. This study aims to provide guidance for the effective utilization of cell segmentation models in the presence of minor optical aberrations and pave the way for future research directions.

Application of Instance Segmentation to Identifying Insect Concentrations in Data from an Entomological Radar

Entomological radar is one of the most effective tools for monitoring insect migration, capable of detecting migratory insects concentrated in layers and facilitating the analysis of insect migration behavior. However, traditional entomological radar, with its low resolution, can only provide a rough observation of layer concentrations. The advent of High-Resolution Phased Array Radar (HPAR) has transformed this situation. With its high range resolution and high data update rate, HPAR can generate detailed concentration spatiotemporal distribution heatmaps. This technology facilitates the detection of changes in insect concentrations across different time periods and altitudes, thereby enabling the observation of large-scale take-off, landing, and layering phenomena. However, the lack of effective techniques for extracting insect concentration data of different phenomena from these heatmaps significantly limits detailed analyses of insect migration patterns. This paper is the first to apply instance segmentation technology to the extraction of insect data, proposing a method for segmenting and extracting insect concentration data from spatiotemporal distribution heatmaps at different phenomena. To address the characteristics of concentrations in spatiotemporal distributions, we developed the Heatmap Feature Fusion Network (HFF-Net). In HFF-Net, we incorporate the Global Context (GC) module to enhance feature extraction of concentration distributions, utilize the Atrous Spatial Pyramid Pooling with Depthwise Separable Convolution (SASPP) module to extend the receptive field for understanding various spatiotemporal distributions of concentrations, and refine segmentation masks with the Deformable Convolution Mask Fusion (DCMF) module to enhance segmentation detail. Experimental results show that our proposed network can effectively segment concentrations of different phenomena from heatmaps, providing technical support for detailed and systematic studies of insect migration behavior.

Automated fish counting system based on instance segmentation in aquaculture

Non-invasive methods of fish counting are crucial for the effective management of aquaculture and aquatic resources. However, accurate and reliable estimation of fish populations in real-time is a significant challenge due to severe occlusion, body deformation, and rapid movement of fish. Recent advancements in computer vision and deep learning algorithms offer substantial potential to revolutionize fish counting methodologies, thereby enhancing conservation efforts and promoting the sustainable management of aquatic resources. This paper presents a comprehensive dataset of 256,580 fish instances annotated with mask information at the pixel level. To address the challenges associated with geometric variations in fish size, pose, perspective, and body shape, we propose a lightweight instance segmentation model based on YOLOv8, which integrates the CSP bottleneck with dual convolutions and the deformable convolutional networks (DConv). In addition, an efficient feature fusion network incorporating multi-scale information was introduced to improve segmentation accuracy by enhancing the representation of feature information, and a decoupled head with attention mechanisms was designed to detect and segment fish bodies in real time. Our proposed approach achieved an outstanding mAP50 of 98.1 % at 60.6 FPS, with model parameters and floating point operations per second (FLOPS) amounting to 25.4 M and 96.9 G, respectively. The instance segmentation-based counting system exhibited impressive performance metrics, with Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and R-Square (R2) values of 1.19, 1.65, and 0.992, respectively. The proposed method demonstrates strong potential for practical application as an intelligent solution for biomass estimation in recirculating aquaculture systems.

An Efficient and Low-Cost Deep Learning-Based Method for Counting and Sizing Soybean Nodules

Soybeans are an essential source of food, protein, and oil worldwide, and the nodules on their root systems play a critical role in nitrogen fixation and plant growth. In this study, we tackled the challenge of limited high-resolution image quantities and the constraints on model learning by innovatively employing image segmentation technology for an in-depth analysis of soybean nodule phenomics. Through a meticulously designed segmentation algorithm, we broke down large-resolution images into numerous smaller ones, effectively improving the model’s learning efficiency and significantly increasing the available data volume, thus laying a solid foundation for subsequent analysis. In terms of model selection and optimization, after several rounds of comparison and testing, YOLOX was identified as the optimal model, achieving an accuracy of 91.38% on the test set with an R2 of up to 86%, fully demonstrating its efficiency and reliability in nodule counting tasks. Subsequently, we utilized YOLOV5 for instance segmentation, achieving a precision of 93.8% in quickly and accurately extracting key phenotypic indicators such as the area, circumference, length, and width of the nodules, and calculated the statistical properties of these indicators. This provided a wealth of quantitative data for the morphological study of soybean nodules. The research not only enhanced the efficiency and accuracy of obtaining nodule phenotypic data and reduced costs but also provided important scientific evidence for the selection and breeding of soybean materials, highlighting its potential application value in agricultural research and practical production.

Fine classification of rice fields in high-resolution remote sensing images

Fine-grained management of rice fields can enhance the yield and quality of rice crops. Challenges in achieving fine classification include interference from similar vegetation, the irregularity of natural field shapes, and complex scale variations. This paper introduces Rice Attention Cascade Network (RACNet), for the fine classification of rice fields in high-resolution satellite remote sensing imagery. The network employs the Hybrid Task Cascade network as the base framework and uses spectral and indices mixed multimodal data as input to reinforce the feature differentiation of similar vegetation. Initially, a Channel Attention Deformable-ResNet (CAD-ResNet) was designed to enhance the feature representation of rice on different channels. Deformable convolution improves the ability of CAD-ResNet to capture irregular field shapes. Then, to address the issue of complex scale changes, the multi-scale features extracted by the CAD-ResNet are progressively fused using an Asymptotic Feature Pyramid, reducing the loss of scale information between non-adjacent layers. Experiments on the Meishan rice dataset show that the proposed method is capable of accurate instance segmentation for fragmented or irregularly shaped rice fields. The evaluation metric AP50 of RACNet reaches 50.8%.

The accuracy of deep learning models for diagnosing maxillary fungal ball rhinosinusitis.

To assess the accuracy of deep learning models for the diagnosis of maxillary fungal ball rhinosinusitis (MFB) and to compare the accuracy, sensitivity, specificity, precision, and F1-score with a rhinologist. Data from 1539 adult chronic rhinosinusitis (CRS) patients who underwent paranasal sinus computed tomography (CT) were collected. The overall dataset consisted of 254 MFB cases and 1285 non-MFB cases. The CT images were constructed and labeled to form the deep learning models. Seventy percent of the images were used for training the deep-learning models, and 30% were used for testing. Whole image analysis and instance segmentation analysis were performed using three different architectures: MobileNetv3, ResNet50, and ResNet101 for whole image analysis, and YOLOv5X-SEG, YOLOv8X-SEG, and YOLOv9-C-SEG for instance segmentation analysis. The ROC curve was assessed. Accuracy, sensitivity (recall), specificity, precision, and F1-score were compared between the models and a rhinologist. Kappa agreement was evaluated. Whole image analysis showed lower precision, recall, and F1-score compared to instance segmentation. The models exhibited an area under the ROC curve of 0.86 for whole image analysis and 0.88 for instance segmentation. In the testing dataset for whole images, the MobileNet V3 model showed 81.00% accuracy, 47.40% sensitivity, 87.90% specificity, 66.80% precision, and a 67.20% F1 score. Instance segmentation yielded the best evaluation with YOLOv8X-SEG showing 94.10% accuracy, 85.90% sensitivity, 95.80% specificity, 88.90% precision, and an 89.80% F1-score. The rhinologist achieved 93.5% accuracy, 84.6% sensitivity, 95.3% specificity, 78.6% precision, and an 81.5% F1-score. Utilizing paranasal sinus CT imaging with enhanced localization and constructive instance segmentation in deep learning models can be the practical promising deep learning system in assisting physicians for diagnosing maxillary fungal ball.

Brain tumor image segmentation method using hybrid attention module and improved mask RCNN

To meet the needs of automated medical analysis of brain tumor magnetic resonance imaging, this study introduces an enhanced instance segmentation method built upon mask region-based convolutional neural network. By incorporating squeeze-and-excitation networks, a channel attention mechanism, and concatenated attention neural network, a spatial attention mechanism, the model can more adeptly focus on the critical regions and finer details of brain tumors. Residual network-50 combined attention module and feature pyramid network as the backbone network to effectively capture multi-scale characteristics of brain tumors. At the same time, the region proposal network and region of interest align technology were used to ensure that the segmentation area matched the actual tumor morphology. The originality of the research lies in the deep residual network that combines attention mechanism with feature pyramid network to replace the backbone based on mask region convolutional neural network, achieving an improvement in the efficiency of brain tumor feature extraction. After a series of experiments, the precision of the model is 90.72%, which is 0.76% higher than that of the original model. Recall was 91.68%, an increase of 0.95%; Mean Intersection over Union was 94.56%, an increase of 1.39%. This method achieves precise segmentation of brain tumor magnetic resonance imaging, and doctors can easily and accurately locate the tumor area through the segmentation results, thereby quickly measuring the diameter, area, and other information of the tumor, providing doctors with more comprehensive diagnostic information.

Artificial intelligence-based ship hull plate corrosion monitoring using Convolutional Neural Network (CNN): comparison of YOLOv8 and Detectron2 architecture models

ABSTRACT Detecting and monitoring structural damage is crucial to prevent significant damage to ship structures caused by corrosion. This study aims to measure the performance of corrosion detection using Convolutional Neural Networks (CNN) by comparing YOLOv8 and Detectron2 models analyzed at various data augmentation settings. The results show that Detectron2 with the ResNet101 backbone performs better than Detectron2 ResNet50 and YOLOv8 models. The result found that applied augmentation data settings, such as vertical & horizontal flip, rotation, and increasing & decreasing colour saturation, highlight their importance in training robust corrosion detection models. These findings contribute to developing corrosion detection methods, showing that the Detectron2 model with instance segmentation can overcome the complexity and variety of corrosion shapes more effectively than YOLOv8 model. The instance segmentation model has proven to be the more effective CNN model for detecting abstract objects, such as corrosion damage in ship hulls.

Sparse Convolution-Based 6D Pose Estimation for Robotic Bin-Picking With Point Clouds

Abstract Estimating the orientation and position of objects is a crucial step in robotic bin-picking tasks. The challenge lies in the fact that, in real-world scenarios, a diverse array of objects is often randomly stacked, resulting in significant occlusion. This study introduces an innovative approach aimed at predicting 6D poses by processing point clouds through a two-stage neural network. In the initial stage, a network for scenes with low-textured environments is designed. Its purpose is to perform instance segmentation and provide an initial pose estimation. Entering the second stage, a pose refinement network is suggested. This network is intended to enhance the precision of pose prediction, building upon the output from the first stage. To tackle the challenge of resource-intensive annotation, a simulation technique is employed to generate a synthetic dataset. Additionally, a dedicated software tool has been developed to annotate real point cloud datasets. In practical experiments, our method demonstrated superior performance compared to baseline methods such as PointGroup and Iterative Closest Point. This superiority is evident in both segmentation accuracy and pose refinement. Moreover, practical grasping experiments have underscored the method's efficacy in real-world industrial robot bin-picking applications. The results affirm its capability to successfully address the challenges produced by occluded and randomly stacked objects.

Cotton-YOLO-Seg: An Enhanced YOLOV8 Model for Impurity Rate Detection in Machine-Picked Seed Cotton

The detection of the impurity rate in machine-picked seed cotton is crucial for precision agriculture. This study proposes a novel Cotton-YOLO-Seg cotton-impurity instance segmentation algorithm based on the you only look once version 8 small segmentation model (Yolov8s-Seg). The algorithm achieves precise pixel-level segmentation of cotton and impurities in seed cotton images and establishes a detection model for the impurity rate, enabling accurate detection of the impurity rate in machine-picked cotton. The proposed algorithm removes the Pyramid 4 (P4) feature layer and incorporates Multi-Scale Convolutional Block Attention (MSCBCA) that integrates the Convolutional Block Attention Module (CBAM) and Multi-Scale Convolutional Attention (MSCA) into the Faster Implementation of Cross Stage Partial Bottleneck with 2 Convolutions (C2f) module of the feature extraction network, forming a novel C2f_MSCBCA module. The SlimNeck structure is introduced in the feature fusion network by replacing the P4 feature layer with the small-target detection layer Pyramid 2 (P2). Additionally, transfer learning is employed using the Common Objects in Context (COCO) instance segmentation dataset. The analysis of 100 groups of cotton image samples shows that the Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) for impurity rate detection are 0.29%, 0.33%, and 3.70%, respectively, which are reduced by 52.46%, 48.44%, and 53.75% compared to the Yolov8s-seg model. The Precision (P), Recall (R), and mean Average Precision at an intersection over union of 0.5 (mAP@0.5) are 85.4%, 78.4%, and 80.8%, respectively, which are improved by 4.2%, 6.2%, and 6.4% compared to Yolov8s-seg model, significantly enhancing the segmentation performance of minor impurities. The Cotton-YOLO-Seg model demonstrates practical significance for precisely detecting the impurity rate in machine-picked seed cotton.

PIS-Net: Efficient weakly supervised instance segmentation network based on annotated points for rice field weed identification

Weed damage in rice fields is one of the main causes of reduced rice yields and quality. Accurate and efficient weed identification is the prerequisite for realizing intelligent and precise weeding in paddies. Recently, Vision Transformers (ViTs) have emerged with superior performance on computer vision tasks compared to the convolutional neural network (CNN)-based models. However, the lack of fully labeled weed datasets hinders the potential application of deep learning models in weed identification. To address the above issues, this study customizes a novel point-supervised instance segmentation network (PIS-Net) for weakly supervised instance segmentation of weeds in rice fields. More correctly, we first propose a novel instance segmentation point labeling scheme that utilizes randomly generated annotation points within each instance, aiming to decrease both labeling time and difficulty. Additionally, to make optimal use of point labels, this study puts forth a mask generation strategy based on the adaptive selection of pyramid levels. In this sense, the network model can flexibly choose the pyramid level expected to generate the most suitable instance mask based on the network's reliability. Finally, we establish the pseudo label refinement network (PLR-Net) to refine rough instance masks. The proposed PIS-Net utilizes 13 randomly generated annotation points for each instance, yet achieving an AP of 38.5 and an AP50 of 68.3, which is superior to the baseline mask-R-CNN with an AP of 8.2 and AP50 of 6.9, achieving 90 % fully supervised performance. This method effectively utilizes point labels, annotated with high efficiency, as a robust source of weak supervision to address challenges in weed data annotation and the low accuracy of existing weakly supervised models. Experiments show that the point annotation scheme of the PIS-Net is faster than full-object mask annotation, and the AP is also higher than the current semi-supervised weed segmentation model, enjoying high potentials in practical paddy fields.