Mean Pixel Accuracy Research Articles

Lesion segmentation using 3D scan and deep learning for the evaluation of facial portwine stain birthmarks

BackgroundPortwine stain (PWS) birthmarks are congenital vascular malformations. The quantification of PWS area is an important step in lesion classification and treatment evaluation. AimsThe aim of this study was to evaluate the combination of 3D scan with deep learning for automated PWS area quantization. Materials and methodsPWS color was measured using a portable spectrophotometer. PWS patches (29.26–45.82 cm2) of different color and shape were generated for 2D and 3D PWS model. 3D images were acquired by a handheld 3D scanner to create texture maps. For semantic segmentation, an improved DeepLabV3+ network was developed for PWS lesion extraction from texture mapping of 3D images. In order to achieve accurate extraction of lesion regions, the convolutional block attention module (CBAM) and DENSE were introduced and the network was trained under Ranger optimizer. The performance of different backbone networks for PWS lesion extraction were also compared. ResultsIDeepLabV3+ (Xception) showed the best results in PWS lesion extraction and area quantification. Its mean Intersection over Union (MIou) was 0.9797, Mean Pixel Accuracy (MPA) 0.9908, Accuracy 0.9989, Recall 0.9886 and F1-score 0.9897, respectively. In PWS area quantization, the mean value of the area error rate of this scheme was 2.61 ± 2.33. ConclusionsThe new 3D method developed in this study was able to achieve accurate quantification of PWS lesion area and has potentials for clinical applications.

A Cyber-physical-social systems approach to the semantic segmentation of pulmonary embolism

Cyber-Physical-Social Systems (CPSS) epitomize the modern era’s intelligent connectivity. They integrate physical devices, computer networks, and social networks, forming an innovative paradigm for intelligent systems. Utilizing CPSS to enhance intelligence, automation, and remote services in healthcare represents a primary research focus. Pulmonary embolism, a prevalent condition resulting from the blockage of the pulmonary artery and its branches by emboli, leads to a spectrum of clinical syndromes marked by impaired pulmonary circulation and right heart dysfunction, contributing to sudden and unpredictable fatalities. Nevertheless, the diagnosis of pulmonary embolism remains challenging due to non-specific clinical presentations, constrained diagnostic capabilities, delayed diagnoses, insufficient physician knowledge, and suboptimal diagnostic techniques. Consequently, we introduce the innovative LSCU-Net architecture within the CPSS framework, designed to develop an automated segmentation and intelligent assessment system for pulmonary embolism, facilitating its automated and intelligent detection. The experimental findings demonstrate that the model accurately segments pulmonary embolism, evidenced by a Jaccard index of 0.6958, a Dice coefficient of 0.8193, a Mean Pixel Accuracy (mPA) of 0.8519, and an accuracy of 0.9993. Empirical studies reveal that our proposed model substantially surpasses existing models in performance. Consequently, this model can aid physicians in the diagnosis of pulmonary embolism during clinical practice. The established pulmonary embolism automatic segmentation and assessment system also showcases the application successes of CPSS in intelligent remote healthcare. The system’s development and deployment not only streamline physicians’ diagnostic processes but also elevate public health standards and advance CPSS research within the medical domain.

Study on the Influence of Label Image Accuracy on the Performance of Concrete Crack Segmentation Network Models.

A high-quality dataset is a basic requirement to ensure the training quality and prediction accuracy of a deep learning network model (DLNM). To explore the influence of label image accuracy on the performance of a concrete crack segmentation network model in a semantic segmentation dataset, this study uses three labelling strategies, namely pixel-level fine labelling, outer contour widening labelling and topological structure widening labelling, respectively, to generate crack label images and construct three sets of crack semantic segmentation datasets with different accuracy. Four semantic segmentation network models (SSNMs), U-Net, High-Resolution Net (HRNet)V2, Pyramid Scene Parsing Network (PSPNet) and DeepLabV3+, were used for learning and training. The results show that the datasets constructed from the crack label images with pix-el-level fine labelling are more conducive to improving the accuracy of the network model for crack image segmentation. The U-Net had the best performance among the four SSNMs. The Mean Intersection over Union (MIoU), Mean Pixel Accuracy (MPA) and Accuracy reached 85.47%, 90.86% and 98.66%, respectively. The average difference between the quantized width of the crack image segmentation obtained by U-Net and the real crack width was 0.734 pixels, the maximum difference was 1.997 pixels, and the minimum difference was 0.141 pixels. Therefore, to improve the segmentation accuracy of crack images, the pixel-level fine labelling strategy and U-Net are the best choices.

Automatedly identify dryland threatened species at large scale by using deep learning

Dryland biodiversity is decreasing at an alarming rate. Advanced intelligent tools are urgently needed to rapidly, automatedly, and precisely detect dryland threatened species on a large scale for biological conservation. Here, we explored the performance of three deep convolutional neural networks (Deeplabv3+, Unet, and Pspnet models) on the intelligent recognition of rare species based on high-resolution (0.3 m) satellite images taken by an unmanned aerial vehicle (UAV). We focused on a threatened species, Populus euphratica, in the Tarim River Basin (China), where there has been a severe population decline in the 1970s and restoration has been carried out since 2000. The testing results showed that Unet outperforms Deeplabv3+ and Pspnet when the training samples are lower, while Deeplabv3+ performs best as the dataset increases. Overall, when training samples are 80, Deeplabv3+ had the best overall performance for Populus euphratica identification, with mean pixel accuracy (MPA) between 87.31 % and 90.2 %, which, on average is 3.74 % and 11.29 % higher than Unet and Pspnet, respectively. Deeplabv3+ can accurately detect the boundaries of Populus euphratica even in areas of dense vegetation, with lower identification uncertainty for each pixel than other models. This study developed a UAV imagery-based identification framework using deep learning with high resolution in large-scale regions. This approach can accurately capture the variation in dryland threatened species, especially those in inaccessible areas, thereby fostering rapid and efficient conservation actions.

Deep learning architectures for diagnosing the severity of apple frog-eye leaf spot disease in complex backgrounds.

In precision agriculture, accurately diagnosing apple frog-eye leaf spot disease is critical for effective disease management. Traditional methods, predominantly relying on labor-intensive and subjective visual evaluations, are often inefficient and unreliable. To tackle these challenges in complex orchard environments, we develop a specialized deep learning architecture. This architecture consists of a two-stage multi-network model. The first stage features an enhanced Pyramid Scene Parsing Network (L-DPNet) with deformable convolutions for improved apple leaf segmentation. The second stage utilizes an improved U-Net (D-UNet), optimized with bilinear upsampling and batch normalization, for precise disease spot segmentation. Our model sets new benchmarks in performance, achieving a mean Intersection over Union (mIoU) of 91.27% for segmentation of both apple leaves and disease spots, and a mean Pixel Accuracy (mPA) of 94.32%. It also excels in classifying disease severity across five levels, achieving an overall precision of 94.81%. This approach represents a significant advancement in automated disease quantification, enhancing disease management in precision agriculture through data-driven decision-making.

Semi-supervised segmentation for construction and demolition waste recognition in-the-wild: Adversarial dual-view networks

Precise, and automated segmentation of construction and demolition waste (CDW) is crucial for recognizing the composition of mixed waste streams and facilitating automatic waste sorting. Training a neural network for image segmentation is challenging due to the time and resource-intensive nature of annotating large-scale datasets, particularly for domain-specific waste recognition in cluttered environments. In this paper, we propose a semi-supervised multi-class segmentation approach to recognize CDW in real-world settings, utilizing an adversarial dual-view framework. In doing so, we utilize a critic network to enable mutual learning between views using high-confidence predictions. We collected and annotated images of CDW in-the-wild and experimented with various portions of unlabelled data. By minimizing a multi-task loss function, inclusive of supervised, unsupervised, and adversarial losses, our method achieves a frequency-weighted intersection over union of 0.62 and mean pixel accuracy of 0.76 across eight classes, with equal splits of labelled and unlabelled data. The findings realize the proposed method achieves competitive performance compared to fully supervised methods even with limited labelled data. This is useful in waste recognition practices by reducing the time and resources needed for data annotations. Furthermore, it paves the way for accurate waste sorting, facilitating efficient CDW recycling and resource recovery.

RTC_TongueNet: An improved tongue image segmentation model based on DeepLabV3.

Tongue segmentation as a basis for automated tongue recognition studies in Chinese medicine, which has defects such as network degradation and inability to obtain global features, which seriously affects the segmentation effect. This article proposes an improved model RTC_TongueNet based on DeepLabV3, which combines the improved residual structure and transformer and integrates the ECA (Efficient Channel Attention Module) attention mechanism of multiscale atrous convolution to improve the effect of tongue image segmentation. In this paper, we improve the backbone network based on DeepLabV3 by incorporating the transformer structure and an improved residual structure. The residual module is divided into two structures and uses different residual structures under different conditions to speed up the frequency of shallow information mapping to deep network, which can more effectively extract the underlying features of tongue image; introduces ECA attention mechanism after concat operation in ASPP (Atrous Spatial Pyramid Pooling) structure to strengthen information interaction and fusion, effectively extract local and global features, and enable the model to focus more on difficult-to-separate areas such as tongue edge, to obtain better segmentation effect. The RTC_TongueNet network model was compared with FCN (Fully Convolutional Networks), UNet, LRASPP (Lite Reduced ASPP), and DeepLabV3 models on two datasets. On the two datasets, the MIOU (Mean Intersection over Union) and MPA (Mean Pixel Accuracy) values of the classic model DeepLabV3 were higher than those of FCN, UNet, and LRASPP models, and the performance was better. Compared with the DeepLabV3 model, the RTC_TongueNet network model increased MIOU value by 0.9% and MPA value by 0.3% on the first dataset; MIOU increased by 1.0% and MPA increased by 1.1% on the second dataset. RTC_TongueNet model performed best on both datasets. In this study, based on DeepLabV3, we apply the improved residual structure and transformer as a backbone to fully extract image features locally and globally. The ECA attention module is combined to enhance channel attention, strengthen useful information, and weaken the interference of useless information. RTC_TongueNet model can effectively segment tongue images. This study has practical application value and reference value for tongue image segmentation.

Slender defect segmentation network of workpiece surface based on deep learning

As a key part of the workpiece production process, workpiece surface defect segmentation detection plays an important role in the quality assurance of the workpiece. Aiming at the problem of low accuracy of workpiece surface slender defect segmentation, we propose a slender defect segmentation network based on feature enhancement module (FEM) and feature fusion module (FFM). Firstly, the ConvNeXt is improved and used as the backbone network. Then, the FEM is designed to obtain rich defect features. Finally, the FFM is designed to expand the receptive field and fuse shallow features with deep features to enhance the features of slender defects. The proposed method improves the segmentation accuracy of slender defects. The mean pixel accuracy) of slender defect segmentation on the self-built diode glass shell dataset and public magnetic sheet dataset is 97.87% and 94.86%, respectively. The mean Intersection over Union of slender defect segmentation is 94.32% and 83.89%, respectively. The segmentation speed reaches 43FPS, and the parameter is 7.12 M, which balances the segmentation accuracy, speed and model size.

An improved DeepLabv3+ lightweight network for remote-sensing image semantic segmentation

To improve the accuracy of remote-sensing image semantic segmentation in complex scenario, an improved DeepLabv3+ lightweight neural network is proposed. Specifically, the lightweight network MobileNetv2 is used as the backbone network. In atrous spatial pyramid pooling (ASPP), to alleviate the gridding effect, the Dilated Convolution in original DeepLabv3+ network is replaced with the Hybrid Dilated Convolution (HDC) module. In addition, the traditional spatial mean pooling is replaced by the strip pooling module (SPN) to improve the local segmentation effect. In the decoder, to obtain the rich low-level target edge information, the ResNet50 residual network is added after the low-level feature fusion. To enhance the shallow semantic information, the efficient and lightweight Normalization-based Attention Module (NAM) is added to capture the feature information of small target objects. The results show that, under the INRIA Aerial Image Dataset and same parameter setting, the Mean Pixel Accuracy (MPA) and Mean Intersection over Union (MIoU) are generally best than DeepLabv3+ , U-Net, and PSP-Net, which are respectively improved by 1.22%, − 0.22%, and 2.22% and 2.17%, 1.35%, and 3.42%. Our proposed method has also a good performance on the small object segmentation and multi-object segmentation. What’s more, it significantly converges faster with fewer model parameters and stronger computing power while ensuring the segmentation effect. It is proved to be robust and can provide a methodological reference for high-precision remote-sensing image semantic segmentation.

Stainless steel cylindrical pot outer surface defect detection method based on cascade neural network

Addressing issues such as surface geometric distortion, high reflection, and the challenge of detecting minor defects on the external surfaces of stainless steel pots, this paper presents a novel approach for detecting defects on the external surfaces of cylindrical stainless steel pots based on high-resolution line-scan imaging. The method begins by creating a real-time inspection system that includes a line-scan camera and a strip light source. This system achieves distortion-free, high-resolution image acquisition of the outer surfaces of stainless steel cylindrical pots by establishing linear constraints on pot size, rotary table rotation speed, and line-scan frame rate. Subsequently, a You Only Look Once and fully convolutional network cascade neural network surface defect detection strategy is introduced, utilizing dual-channel images of the original and enhanced images as inputs. This approach enables the characterization of subtle defects in high-resolution image data. Finally, we conducted experiments using the proposed method on the provided dataset, and the results demonstrate the effectiveness of this approach in detecting various types of product defects. The mean pixel accuracy achieved a remarkable 91.69%, while the mean intersection over union score reached an impressive 83.59%. These findings provide an effective technical means for the qualitative detection of the types of defects on the surface of stainless steel pots and the quantitative measurement of the size of the defects.

A deep learning approach combining DeepLabV3+ and improved YOLOv5 to detect dairy cow mastitis

Dairy cow mastitis has a great impact on the productivity of dairy cows and the profits of livestock farms. Early detection is of great significance to improve the efficiency of mastitis treatment. However, due to the low resolution of thermal infrared images and the complexity of the living environment of dairy cows, it is difficult to detect the eyes and udders of cows, which reduces the detection accuracy of mastitis. To solve this problem, this paper proposes a two-stage model (DCYOLO) integrating the DeepLabV3 + semantic segmentation network and an improved YOLOv5 target recognition network, which is used to detect the eyes and udders of dairy cows under complex background and applied to the severity classification of dairy cow mastitis. In the first stage, the DeepLabV3 + model was used to segment the eyes and udders of dairy cows from thermal infrared images. In the second stage, the segmented image was input into the target recognition YOLOv5 network for key parts recognition. Finally, to further improve the detection accuracy of the model, a convolutional block attention module (CBAM) was added at the end of the main part of the YOLOv5 model. After comparing different semantic segmentation and target recognition networks, the DeepLabV3 + network and YOLOv5 network performed best. The mIoU and mean pixel accuracy (mPA) of the DeepLabV3 + network reached 86.98 % and 92.92 %, respectively. The mean average precision (mAP) and F1 scores of the YOLOv5 network for unsegmented thermal infrared images reached 93.4 % and 90.9 %, respectively. The CBAM-added YOLOv5 (CAYOLOv5) model was combined with the DeepLabV3 + model. Compared with the single YOLOv5 model, the mAP and F1 scores of DCYOLO increased by 5.4 % and 5.3 %, respectively. Therefore, the proposed model can achieve more accurate positioning of key parts of dairy cows. Based on this model, the eye and udder temperature differences of 50 dairy cows were extracted for mastitis detection, and the detection results were compared with the results of the somatic cell count (SCC) approach. The results showed that the classification accuracy of mastitis was 86 %, and the average sensitivity and specificity were 79.41 % and 92.49 %, respectively. The dairy cow mastitis detection method based on the two-stage model can accurately locate the key parts of dairy cows and realize the automatic detection and classification of dairy cow mastitis, and the accuracy is high.

Deep learning, geometric characterization and hydrodynamic modeling for assessing sewer defect impacts on urban flooding: A case study in Guangzhou, China

Deep learning techniques have offered innovative and efficient tools for accurate and automated detection of sewer defects by leveraging large-scale sewer data and advanced feature learning algorithms. However, there has been a lack of thorough characterization of the geometric properties of segmented defects, let alone systematically calculate the severity level of sewer defects and quantitatively evaluate their impacts on flood conditions in hydrodynamic models. This study proposed a comprehensive framework and related metrics to accurately and automatically detect, segment, characterize, and evaluate the impacts of sewer defects on flooded nodes and volumes by integrating a DeepLabv3+-based segmentation technique, an automated geometric characterization and severity quantification module, and a GIS and SWMM-based hydrodynamic modeling. The results clearly showed in details where and how much the urban flooding was affected by the different defect types. The segmentation model achieved satisfactory detection performance, with mean pixel accuracy (MPA), mean intersection over union (MIoU), and frequency weighted intersection over union (FWIoU) of 0.99, 0.74 and 0.95, respectively. In terms of severity level quantification, there were 98%, 90%, 90% and 83% of predictions consistent with real conditions for falling off, obstacle, disjoint and leakage. It was shown that the number of surcharging manholes and total flood volume (TFV) were greatly affected by sewer defects, with over 16% increase in TFVs under all investigated rainfall events. The results addressed the impacts of sewer defects on urban flooding and demonstrated the powerful tools provided by the proposed framework for decision-making on sewer defect detection and management.

Extracting Citrus in Southern China (Guangxi Region) Based on the Improved DeepLabV3+ Network

China is one of the countries with the largest citrus cultivation areas, and its citrus industry has received significant attention due to its substantial economic benefits. Traditional manual forestry surveys and remote sensing image classification tasks are labor-intensive and time-consuming, resulting in low efficiency. Remote sensing technology holds great potential for obtaining spatial information on citrus orchards on a large scale. This study proposes a lightweight model for citrus plantation extraction that combines the DeepLabV3+ model with the convolutional block attention module (CBAM) attention mechanism, with a focus on the phenological growth characteristics of citrus in the Guangxi region. The objective is to address issues such as inaccurate extraction of citrus edges in high-resolution images, misclassification and omissions caused by intra-class differences, as well as the large number of network parameters and long training time found in classical semantic segmentation models. To reduce parameter count and improve training speed, the MobileNetV2 lightweight network is used as a replacement for the Xception backbone network in DeepLabV3+. Additionally, the CBAM is introduced to extract citrus features more accurately and efficiently. Moreover, in consideration of the growth characteristics of citrus, this study augments the feature input with additional channels to better capture and utilize key phenological features of citrus, thereby enhancing the accuracy of citrus recognition. The results demonstrate that the improved DeepLabV3+ model exhibits high reliability in citrus recognition and extraction, achieving an overall accuracy (OA) of 96.23%, a mean pixel accuracy (mPA) of 83.79%, and a mean intersection over union (mIoU) of 85.40%. These metrics represent an improvement of 11.16%, 14.88%, and 14.98%, respectively, compared to the original DeepLabV3+ model. Furthermore, when compared to classical semantic segmentation models, such as UNet and PSPNet, the proposed model achieves higher recognition accuracy. Additionally, the improved DeepLabV3+ model demonstrates a significant reduction in both parameters and training time. Generalization experiments conducted in Nanning, Guangxi Province, further validate the model’s strong generalization capabilities. Overall, this study emphasizes extraction accuracy, reduction in parameter count, adherence to timeliness requirements, and facilitation of rapid and accurate extraction of citrus plantation areas, presenting promising application prospects.

An efficient approach to detect and track winter flush growth of litchi tree based on UAV remote sensing and semantic segmentation

The immature winter flush affects the flower bud differentiation, flowering and fruit of litchi, and then seriously reduces the yield of litchi. However, at present, the area estimation and growth process monitoring of winter flush still rely on manual judgment and operation, so it is impossible to accurately and effectively control flush. An efficient approach is proposed in this paper to detect the litchi flush from the unmanned aerial vehicle (UAV) remoting images of litchi crown and track winter flush growth of litchi tree. The proposed model is constructed based on U-Net network, of which the encoder is replaced by MobeilNetV3 backbone network to reduce model parameters and computation. Moreover, Convolutional Block Attention Module (CBAM) is integrated and convolutional layer is added to enhance feature extraction ability, and transfer learning is adopted to solve the problem of small data volume. As a result, the Mean Pixel Accuracy (MPA) and Mean Intersection over Union (MIoU) on the flush dataset are increased from 90.95% and 83.3% to 93.4% and 85%, respectively. Moreover, the size of the proposed model is reduced by 15% from the original model. In addition, the segmentation model is applied to the tracking of winter flushes on the canopy of litchi trees and investigating the two growth processes of litchi flushes (late-autumn shoots growing into flushes and flushes growing into mature leaves). It is revealed that the growth processes of flushes in a particular branch region can be quantitatively analysed based on the UAV images and the proposed semantic segmentation model. The results also demonstrate that a sudden drop in temperature can promote the rapid transformation of late-autumn shoots into flushes. The method proposed in this paper provide a new technique for accurate management of litchi flush and a possibility for the area estimation and growth process monitoring of winter flush, which can assist in the control operation and yield prediction of litchi orchards.

High-precision target ranging in complex orchard scenes by utilizing semantic segmentation results and binocular vision

The automation of orchard production is increasingly relying on robotics, driven by the advancements in artificial intelligence technology. However, accurately comprehending semantic information and precisely locating various targets within orchard environments remain challenges. Current research often relies on expensive multi-sensor fusion techniques or vision-only approaches that yield inadequate segmentation outcomes for perceiving orchard surroundings. To address these issues, this article proposes a novel approach for target ranging in complex orchard scenes, leveraging semantic segmentation results. The article introduces the MsFF-Segformer model, which employs multi-scale feature fusion to generate high-precision semantic segmentation images. The model incorporates the MiT-B0 encoder, which utilizes a pure attention mechanism, and the MsFF decoder, specifically designed for multi-scale feature fusion. The MsFF decoder includes the AFAM module to effectively align features of adjacent scales. Additionally, the channel attention module and depth separable convolution module are introduced to reduce model parameter size and obtain feature vectors with rich semantic levels, enhancing the segmentation performance of multi-scale targets in orchards. Based on the accurate semantic segmentation outcomes in orchard environments, this study introduces a novel approach named TPDMR that integrates binocular vision to estimate the distances of various objects within orchards. Firstly, the process involves matching the semantic category matrix with the depth information matrix. Subsequently, the depth information array that represents the target category is obtained, and any invalid depth information is filtered out. Finally, the average depth of the target is calculated. Evaluation of the MsFF-Segformer model on a self-made orchard dataset demonstrates superior performance compared to U-net and other models, achieving a Mean Intersection over Union (MIoU) of 86.52 % and a Mean Pixel Accuracy (MPA) of 94.05 %. The parameters and prediction time for a single frame are 15.1 M and 0.019 s, respectively. These values are significantly lower than those of U-net, Deeplabv3+, and Hrnet models, with reductions of 84.1 %, 32.5 %, 5.9 % and 69.4 %, 59.7 %, 64.2 % respectively. The TPDMR method demonstrates a high level of accuracy and stability in target ranging, with a ranging error of less than 6 % across all targets. Furthermore, the overall algorithm runtime is estimated to be approximately 0.8 s, indicating efficient performance.

A cutting width measurement method for the unmanned rice harvester based on RGB-D images

Currently, the unmanned rice harvester based on satellite navigation is affected by tracked chassis steering errors and navigation accuracy, which can result in harvesting omissions and undercutting. Therefore, we proposed a cutting width measurement method for unmanned rice harvesters based on RGB-D images. The proposed method can provide navigational aids for driving and may also provide a basis for feed quantity detection and adjustment. We designed a lightweight unharvested area segmentation model (UANet) with a mean pixel accuracy (mPA) and a mean intersection over union (mIoU) of 98.28% and 97.15%, respectively, which are both higher than the lightweight semantic segmentation models, such as DABNet, CFPNet-V2 and ENet. We implemented unharvested area segmentation and harvesting boundary points extraction on RGB images, combined with depth maps to convert the pixel coordinates of harvesting boundary points into 3D world coordinates for cutting width measurement. Through experiments on rice harvest field images under different working conditions, the results showed that the success rate of harvesting boundary points detection was 98.33%, the error of cutting width measurement was within 2.34%, and the speed of the algorithm was approximately 9.7 fps. The proposed method can be adapted to complex harvesting scenarios and has the advantages of low hardware cost, high measurement accuracy, and good robustness, which can be extended to wheat.

A tea bud segmentation, detection and picking point localization based on the MDY7-3PTB model.

The identification and localization of tea picking points is a prerequisite for achieving automatic picking of famous tea. However, due to the similarity in color between tea buds and young leaves and old leaves, it is difficult for the human eye to accurately identify them. To address the problem of segmentation, detection, and localization of tea picking points in the complex environment of mechanical picking of famous tea, this paper proposes a new model called the MDY7-3PTB model, which combines the high-precision segmentation capability of DeepLabv3+ and the rapid detection capability of YOLOv7. This model achieves the process of segmentation first, followed by detection and finally localization of tea buds, resulting in accurate identification of the tea bud picking point. This model replaced the DeepLabv3+ feature extraction network with the more lightweight MobileNetV2 network to improve the model computation speed. In addition, multiple attention mechanisms (CBAM) were fused into the feature extraction and ASPP modules to further optimize model performance. Moreover, to address the problem of class imbalance in the dataset, the Focal Loss function was used to correct data imbalance and improve segmentation, detection, and positioning accuracy. The MDY7-3PTB model achieved a mean intersection over union (mIoU) of 86.61%, a mean pixel accuracy (mPA) of 93.01%, and a mean recall (mRecall) of 91.78% on the tea bud segmentation dataset, which performed better than usual segmentation models such as PSPNet, Unet, and DeeplabV3+. In terms of tea bud picking point recognition and positioning, the model achieved a mean average precision (mAP) of 93.52%, a weighted average of precision and recall (F1 score) of 93.17%, a precision of 97.27%, and a recall of 89.41%. This model showed significant improvements in all aspects compared to existing mainstream YOLO series detection models, with strong versatility and robustness. This method eliminates the influence of the background and directly detects the tea bud picking points with almost no missed detections, providing accurate two-dimensional coordinates for the tea bud picking points, with a positioning precision of 96.41%. This provides a strong theoretical basis for future tea bud picking.

MSGNet: multi-source guidance network for fish segmentation in underwater videos

Fish segmentation in underwater videos provides basic data for fish measurements, which is vital information that supports fish habitat monitoring and fishery resources survey. However, because of water turbidity and insufficient lighting, fish segmentation in underwater videos has low accuracy and poor robustness. Most previous work has utilized static fish appearance information while ignoring fish motion in underwater videos. Considering that motion contains more detail, this paper proposes a method that simultaneously combines appearance and motion information to guide fish segmentation in underwater videos. First, underwater videos are preprocessed to highlight fish in motion, and obtain high-quality underwater optical flow. Then, a multi-source guidance network (MSGNet) is presented to segment fish in complex underwater videos with degraded visual features. To enhance both fish appearance and motion information, a non-local-based multiple co-attention guidance module (M-CAGM) is applied in the encoder stage, in which the appearance and motion features from the intra-frame salient fish and the moving fish in video sequences are reciprocally enhanced. In addition, a feature adaptive fusion module (FAFM) is introduced in the decoder stage to avoid errors accumulated in the video sequences due to blurred fish or inaccurate optical flow. Experiments based on three publicly available datasets were designed to test the performance of the proposed model. The mean pixel accuracy (mPA) and mean intersection over union (mIoU) of MSGNet were 91.89% and 88.91% respectively with the mixed dataset. Compared with those of the advanced underwater fish segmentation and video object segmentation models, the mPA and mIoU of the proposed model significantly improved. The results showed that MSGNet achieves excellent segmentation performance in complex underwater videos and can provide an effective segmentation solution for fisheries resource assessment and ocean observation. The proposed model and code are exposed via Github1.

A multilevel segmentation method of asymmetric semantics based on deep learning

AbstractAn asymmetric semantic multi‐level segmentation method based on depth learning is proposed in order to improve the precision and effect of semantic segmentation. A ‘content tree’ structure and an adjacency matrix are constructed to represent the parent‐child relationship between each image sub region in a complete image. Through multiple combinations of spatial attention mechanism and channel attention mechanism, the similarity semantic features of the target object can be selectively aggregated, so as to enhance its feature expression and avoid the impact of significant objects. The asymmetric semantic segmentation model asymmetric pyramid feature convolutional network (APFCN) is constructed, and the path feature extraction and parameter adjustment are realised through APFCN. On the basis of APFCN network, a full convolution network is introduced for end‐to‐end image semantic segmentation. Combining the advantages of convolution network in extracting image features and the advantages of short‐term and short‐term memory network in solving long‐term dependence, an end‐to‐end hybrid depth network is constructed for image semantic multi‐level segmentation. The experimental results show that the mean intersection over Union value and mean pixel accuracy value are higher than that of the literature method, both of which are increased by more than 3%, and the segmentation effect is good.

RA-UNet: an intelligent fish phenotype segmentation method based on ResNet50 and atrous spatial pyramid pooling

Introduction: Changes in fish phenotypes during aquaculture must be monitored to improve the quality of fishery resources. Therefore, a method for segmenting and measuring phenotypes rapidly and accurately without harming the fish is essential. This study proposes an intelligent fish phenotype segmentation method based on the residual network, ResNet50, and atrous spatial pyramid pooling (ASPP).Methods: A sufficient number of fish phenotypic segmentation datasets rich in experimental research was constructed, and diverse semantic segmentation datasets were developed. ResNet50 was then built as the backbone feature extraction network to prevent the loss of fish phenotypic feature information and improve the precision of fish phenotypic segmentation. Finally, an ASPP module was designed to improve the phenotypic segmentation accuracy of different parts of fish.Results: The test algorithm based on the collected fish phenotype segmentation datasets showed that the proposed algorithm (RA-UNet) yielded the best results among several advanced semantic segmentation models. The mean intersection over union (mIoU) and mean pixel accuracy (mPA) were 87.8% and 92.3%, respectively.Discussion: Compared with the benchmark UNet algorithm, RA-UNet demonstrated improvements in the mIoU and mPA by 5.0 and 1.8 percentage points, respectively. Additionally, RA-UNet exhibited superior fish phenotype segmentation performance, with a low false detection rate and clear and complete edge segmentation. Conclusively, the RA-UNet proposed in this study has high accuracy and edge segmentation ability and can, therefore, directly improve the efficiency of phenotypic monitoring in fish farming.