Pixel-level Annotations Research Articles

Beyond Pixel-Level Annotation: Exploring Self-Supervised Learning for Change Detection With Image-Level Supervision

Beyond Pixel-Level Annotation: Exploring Self-Supervised Learning for Change Detection With Image-Level Supervision

Dynamic video recognition for cell-encapsulating microfluidic droplets.

Droplet microfluidics is a highly sensitive and high-throughput technology extensively utilized in biomedical applications, such as single-cell sequencing and cell screening. However, its performance is highly influenced by the droplet size and single-cell encapsulation rate (following random distribution), thereby creating an urgent need for quality control. Machine learning has the potential to revolutionize droplet microfluidics, but it requires tedious pixel-level annotation for network training. This paper investigates the application software of the weakly supervised cell-counting network (WSCApp) for video recognition of microdroplets. We demonstrated its real-time performance in video processing of microfluidic droplets and further identified the locations of droplets and encapsulated cells. We verified our methods on droplets encapsulating six types of cells/beads, which were collected from various microfluidic structures. Quantitative experimental results showed that our approach can not only accurately distinguish droplet encapsulations (micro-F1 score > 0.94), but also locate each cell without any supervised location information. Furthermore, fine-tuning transfer learning on the pre-trained model also significantly reduced (>80%) annotation. This software provides a user-friendly and assistive annotation platform for the quantitative assessment of cell-encapsulating microfluidic droplets.

SKGC: A General Semantic-Level Knowledge Guided Classification Framework for Fetal Congenital Heart Disease.

Congenital heart disease (CHD) is the most common congenital disability affecting healthy development and growth, even resulting in pregnancy termination or fetal death. Recently, deep learning techniques have made remarkable progress to assist in diagnosing CHD. One very popular method is directly classifying fetal ultrasound images, recognized as abnormal and normal, which tends to focus more on global features and neglects semantic knowledge of anatomical structures. The other approach is segmentation-based diagnosis, which requires a large number of pixel-level annotation masks for training. However, the detailed pixel-level segmentation annotation is costly or even unavailable. Based on the above analysis, we propose SKGC, a universal framework to identify normal or abnormal four-chamber heart (4CH) images, guided by a few annotation masks, while improving accuracy remarkably. SKGC consists of a semantic-level knowledge extraction module (SKEM), a multi-knowledge fusion module (MFM), and a classification module (CM). SKEM is responsible for obtaining high-level semantic knowledge, serving as an abstract representation of the anatomical structures that obstetricians focus on. MFM is a lightweight but efficient module that fuses semantic-level knowledge with the original specific knowledge in ultrasound images. CM classifies the fused knowledge and can be replaced by any advanced classifier. Moreover, we design a new loss function that enhances the constraint between the foreground and background predictions, improving the quality of the semantic-level knowledge. Experimental results on the collected real-world NA-4CH and the publicly FEST datasets show that SKGC achieves impressive performance with the best accuracy of 99.68% and 95.40%, respectively. Notably, the accuracy improves from 74.68% to 88.14% using only 10 labeled masks.

ITER: Image-to-Pixel Representation for Weakly Supervised HSI Classification.

Recent years have witnessed the superiority of deep learning-based algorithms in the field of HSI classification. However, a prerequisite for the favorable performance of these methods is a large number of refined pixel-level annotations. Due to atmospheric changes, sensor differences, and complex land cover distribution, pixel-level labeling of high-dimensional hyperspectral image (HSI) is extremely difficult, time-consuming, and laborious. To overcome the above hurdle, an Image-To-pixEl Representation (ITER) approach is proposed in this paper. To the best of our knowledge, this is the first time that image-level annotation is introduced to predict pixel-level classification maps for HSI. The proposed model is along the lines of subject modeling to boundary refinement, corresponding to pseudo-label generation and pixel-level prediction. Concretely, in the pseudo-label generation part, the spectral/spatial activation, spectral-spatial alignment loss, and geographic element enhancement are sequentially designed to locate discriminate regions of each category, optimize multi-domain class activation map (CAM) collaborative training, and refine labels, respectively. For the pixel-level prediction portion, a high frequency-aware self-attention in a high-enhanced transformer is put forward to achieve detailed feature representation. With the two-stage pipeline, ITER explores weakly supervised HSI classification with image-level tags, bridging the gap between image-level annotation and dense prediction. Extensive experiments in three benchmark datasets with state-of-the-art (SOTA) works show the performance of the proposed approach.

Diabetic Macular Edema Optical Coherence Tomography Biomarkers Detected with EfficientNetV2B1 and ConvNeXt.

(1) Background: Diabetes mellitus (DM) is a growing challenge, both for patients and physicians, in order to control the impact on health and prevent complications. Millions of patients with diabetes require medical attention, which generates problems regarding the limited time for screening but also addressability difficulties for consultation and management. As a result, screening programs for vision-threatening complications due to DM have to be more efficient in the future in order to cope with such a great healthcare burden. Diabetic macular edema (DME) is a severe complication of DM that can be prevented if it is timely screened with the help of optical coherence tomography (OCT) devices. Newly developing state-of-the-art artificial intelligence (AI) algorithms can assist physicians in analyzing large datasets and flag potential risks. By using AI algorithms in order to process OCT images of large populations, the screening capacity and speed can be increased so that patients can be timely treated. This quick response gives the physicians a chance to intervene and prevent disability. (2) Methods: This study evaluated ConvNeXt and EfficientNet architectures in correctly identifying DME patterns on real-life OCT images for screening purposes. (3) Results: Firstly, we obtained models that differentiate between diabetic retinopathy (DR) and healthy scans with an accuracy of 0.98. Secondly, we obtained a model that can indicate the presence of edema, detachment of the subfoveolar neurosensory retina, and hyperreflective foci (HF) without using pixel level annotation. Lastly, we analyzed the extent to which the pretrained weights on natural images "understand" OCT scans. (4) Conclusions: Pretrained networks such as ConvNeXt or EfficientNet correctly identify features relevant to the differentiation between healthy retinas and DR, even though they were pretrained on natural images. Another important aspect of our research is that the differentiation between biomarkers and their localization can be obtained even without pixel-level annotation. The "three biomarkers model" is able to identify obvious subfoveal neurosensory detachments, retinal edema, and hyperreflective foci, as well as very small subfoveal detachments. In conclusion, our study points out the possible usefulness of AI-assisted diagnosis of DME for lowering healthcare costs, increasing the quality of life of patients with diabetes, and reducing the waiting time until an appropriate ophthalmological consultation and treatment can be performed.

DSST: A dual student model guided student–teacher framework for semi-supervised medical image segmentation

Numerous semi-supervised learning methods are emerging in medical image segmentation to reduce the dependency of deep learning models on pixel-level annotation data. Consistency regularization methods based on the Student–Teacher structure have achieved brilliant results in this domain. However, the current structures are unable to resolve the tight weight coupling satisfactorily between the teacher and student model, which leads to a decrease in the segmentation performance. In this paper, we propose a novel and practical semi-supervised learning framework, Dual-Student-Single-Teacher (DSST), to alleviate this problem. Particularly, the DSST framework consists of three segmentation models with the identical structure but different initial parameters, one serves as the teacher model and others as the student models, which employs an alternating manner to update the teacher model parameters. For the DSST framework, we present different supervised modes to sufficiently explore the enhancement of consistency regularization for model segmentation performance. Furthermore, we also introduce abundant and efficient input and feature perturbations for the proposed method. Finally, we evaluate our framework in three public medical image segmentation tasks, including Pancreas-CT, LA dataset, and cardiac segmentation on the ACDC dataset. Extensive experiments demonstrate that, compared with eight other superior semi-supervised methods, the DSST method obtains state-of-the-art segmentation performance and is an effective and generalizable framework. Code is available at https://github.com/LBL0704/DSST.

Deep Learning-Based Steel Bridge Corrosion Segmentation and Condition Rating Using Mask RCNN and YOLOv8

The application of deep learning (DL) algorithms has become of great interest in recent years due to their superior performance in structural damage identification, including the detection of corrosion. There has been growing interest in the application of convolutional neural networks (CNNs) for corrosion detection and classification. However, current approaches primarily involve detecting corrosion within bounding boxes, lacking the segmentation of corrosion with irregular boundary shapes. As a result, it becomes challenging to quantify corrosion areas and severity, which is crucial for engineers to rate the condition of structural elements and assess the performance of infrastructures. Furthermore, training an efficient deep learning model requires a large number of corrosion images and the manual labeling of every single image. This process can be tedious and labor-intensive. In this project, an open-source steel bridge corrosion dataset along with corresponding annotations was generated. This database contains 514 images with various corrosion severity levels, gathered from a variety of steel bridges. A pixel-level annotation was performed according to the Bridge Inspectors Reference Manual (BIRM) and the American Association of State Highway and Transportation Officials (AASHTO) regulations for corrosion condition rating (defect #1000). Two state-of-the-art semantic segmentation algorithms, Mask RCNN and YOLOv8, were trained and validated on the dataset. These trained models were then tested on a set of test images and the results were compared. The trained Mask RCNN and YOLOv8 models demonstrated satisfactory performance in segmenting and rating corrosion, making them suitable for practical applications.

RescueNet: A High Resolution UAV Semantic Segmentation Dataset for Natural Disaster Damage Assessment

Recent advancements in computer vision and deep learning techniques have facilitated notable progress in scene understanding, thereby assisting rescue teams in achieving precise damage assessment. In this paper, we present RescueNet, a meticulously curated high-resolution post-disaster dataset that includes detailed classification and semantic segmentation annotations. This dataset aims to facilitate comprehensive scene understanding in the aftermath of natural disasters. RescueNet comprises post-disaster images collected after Hurricane Michael, obtained using Unmanned Aerial Vehicles (UAVs) from multiple impacted regions. The uniqueness of RescueNet lies in its provision of high-resolution post-disaster imagery, accompanied by comprehensive annotations for each image. Unlike existing datasets that offer annotations limited to specific scene elements such as buildings, RescueNet provides pixel-level annotations for all classes, including buildings, roads, pools, trees, and more. Furthermore, we evaluate the utility of the dataset by implementing state-of-the-art segmentation models on RescueNet, demonstrating its value in enhancing existing methodologies for natural disaster damage assessment.

Detection and subtyping of basal cell carcinoma in whole-slide histopathology using weakly-supervised learning

The frequency of basal cell carcinoma (BCC) cases is putting an increasing strain on dermatopathologists. BCC is the most common type of skin cancer, and its incidence is increasing rapidly worldwide. AI can play a significant role in reducing the time and effort required for BCC diagnostics and thus improve the overall efficiency of the process. To train such an AI system in a fully-supervised fashion however, would require a large amount of pixel-level annotation by already strained dermatopathologists. Therefore, in this study, our primary objective was to develop a weakly-supervised for the identification of basal cell carcinoma (BCC) and the stratification of BCC into low-risk and high-risk categories within histopathology whole-slide images (WSI). We compared Clustering-constrained Attention Multiple instance learning (CLAM) with StreamingCLAM and hypothesized that the latter would be the superior approach. A total of 5147 images were used to train and validate the models, which were subsequently tested on an internal set of 949 images and an external set of 183 images. The labels for training were automatically extracted from free-text pathology reports using a rule-based approach. All data has been made available through the COBRA dataset. The results showed that both the CLAM and StreamingCLAM models achieved high performance for the detection of BCC, with an area under the ROC curve (AUC) of 0.994 and 0.997, respectively, on the internal test set and 0.983 and 0.993 on the external dataset. Furthermore, the models performed well on risk stratification, with AUC values of 0.912 and 0.931, respectively, on the internal set, and 0.851 and 0.883 on the external set. In every single metric the StreamingCLAM model outperformed the CLAM model or is on par. The performance of both models was comparable to that of two pathologists who scored 240 BCC positive slides. Additionally, in the public test set, StreamingCLAM demonstrated a comparable AUC of 0.958, markedly superior to CLAM’s 0.803. This difference was statistically significant and emphasized the strength and better adaptability of the StreamingCLAM approach.

Uncertainty-guided cross learning via CNN and transformer for semi-supervised honeycomb lung lesion segmentation

Objective. Deep learning networks such as convolutional neural networks (CNN) and Transformer have shown excellent performance on the task of medical image segmentation, however, the usual problem with medical images is the lack of large-scale, high-quality pixel-level annotations, which is a very time-consuming and laborious task, and its further leads to compromised the performance of medical image segmentation under limited annotation conditions. Approach. In this paper, we propose a new semi-supervised learning method, uncertainty-guided cross learning, which uses a limited number of annotated samples along with a large number of unlabeled images to train the network. Specifically, we use two networks with different learning paradigms, CNN and Transformer, for cross learning, and use the prediction of one of them as a pseudo label to supervise the other, so that they can learn from each other, fully extract the local and global features of the images, and combine explicit and implicit consistency regularization constraints with pseudo label methods. On the other hand, we use epistemic uncertainty as a guiding message to encourage the model to learn high-certainty pixel information in high-confidence regions, and minimize the impact of erroneous pseudo labels on the overall learning process to improve the performance of semi-supervised segmentation methods. Main results. We conducted honeycomb lung lesion segmentation experiments using a honeycomb lung CT image dataset, and designed several sets of comparison experiments and ablation experiments to validate the effectiveness of our method. The final experimental results show that the Dice coefficient of our proposed method reaches 88.49% on the test set, and our method achieves state-of-the-art performance in honeycomb lung lesion segmentation compared to other semi-supervised learning methods. Significance. Our proposed method can effectively improve the accuracy of segmentation of honeycomb lung lesions, which provides an important reference for physicians in the diagnosis and treatment of this disease.

A three-stage pavement image crack detection framework with positive sample augmentation

Most pavement crack detection methods based on deep learning rely too much on pixel-wise labels, and are facing with sample imbalance problem. This paper proposes a three-stage pavement crack detection framework with positive sample augmentation by an autoencoder-deep convolutional generative adversarial network (E-DCGAN). Positive samples are the images labeled as crack in the training datasets. Firstly, crack regions are located using graph based on visual saliency (GBVS). Secondly, a convolutional neural network (CNN) model is designed to recognize crack pixels in the located crack regions. The E-DCGAN is explored to generate crack images, which are then added as augmented positive samples in the training dataset of the CNN model. The trained CNN model takes the located crack regions as inputs, outputs the recognized crack pixels. Finally, fine crack detection is realized through region growing. Crack contours are completely depicted pixel-wise based on the recognized crack pixels and the Breadth-first search. Experiments are carried out on two benchmark datasets and in practical applications. The results showed that the framework could detect cracks at the pixel level. Its performance is improved compared with standard fully convolutional network (FCN)-based and U-shaped CNN (U-Net)-based crack detection methods. In addition, the standard FCN-based and U-Net-based methods require accurate pixel-wise annotations, while the proposed framework only requires image-wise annotations. It can be concluded that the proposed framework not only effectively avoids the dependence on pixel-level annotations, improves the sample imbalance issues, but also achieves competitive detection performance over the FCN and U-Net based methods.

Dual-Channel Semi-Supervised Adversarial Network for Building Segmentation from UAV-Captured Images

Accurate building extraction holds paramount importance in various applications such as urbanization rate calculations, urban planning, and resource allocation. In response to the escalating demand for precise low-altitude unmanned aerial vehicle (UAV) building segmentation in intricate scenarios, this study introduces a semi-supervised methodology to alleviate the labor-intensive process of procuring pixel-level annotations. Within the framework of adversarial networks, we employ a dual-channel parallel generator strategy that amalgamates the morphology-driven optical flow estimation channel with an enhanced multilayer sensing Deeplabv3+ module. This approach aims to comprehensively capture both the morphological attributes and textural intricacies of buildings while mitigating the dependency on annotated data. To further enhance the network’s capability to discern building features, we introduce an adaptive attention mechanism via a feature fusion module. Additionally, we implement a composite loss function to augment the model’s sensitivity to building structures. Across two distinct low-altitude UAV datasets within the domain of UAV-based building segmentation, our proposed method achieves average mean pixel intersection-over-union (mIoU) ratios of 82.69% and 79.37%, respectively, with unlabeled data constituting 70% of the overall dataset. These outcomes signify noteworthy advancements compared with contemporaneous networks, underscoring the robustness of our approach in tackling intricate building segmentation challenges in the domain of UAV-based architectural analysis.

A deep weakly semi-supervised framework for endoscopic lesion segmentation.

In the field of medical image analysis, accurate lesion segmentation is beneficial for the subsequent clinical diagnosis and treatment planning. Currently, various deep learning-based methods have been proposed to deal with the segmentation task. Albeit achieving some promising performances, the fully-supervised learning approaches require pixel-level annotations for model training, which is tedious and time-consuming for experienced radiologists to collect. In this paper, we propose a weakly semi-supervised segmentation framework, called Point Segmentation Transformer (Point SEGTR). Particularly, the framework utilizes a small amount of fully-supervised data with pixel-level segmentation masks and a large amount of weakly-supervised data with point-level annotations (i.e., annotating a point inside each object) for network training, which largely reduces the demand of pixel-level annotations significantly. To fully exploit the pixel-level and point-level annotations, we propose two regularization terms, i.e., multi-point consistency and symmetric consistency, to boost the quality of pseudo labels, which are then adopted to train a student model for inference. Extensive experiments are conducted on three endoscopy datasets with different lesion structures and several body sites (e.g., colorectal and nasopharynx). Comprehensive experimental results finely substantiate the effectiveness and the generality of our proposed method, as well as its potential to loosen the requirements of pixel-level annotations, which is valuable for clinical applications.

Deep Semi-Supervised Ultrasound Image Segmentation by Using a Shadow Aware Network With Boundary Refinement.

Accurate ultrasound (US) image segmentation is crucial for the screening and diagnosis of diseases. However, it faces two significant challenges: 1) pixel-level annotation is a time-consuming and laborious process; 2) the presence of shadow artifacts leads to missing anatomy and ambiguous boundaries, which negatively impact reliable segmentation results. To address these challenges, we propose a novel semi-supervised shadow aware network with boundary refinement (SABR-Net). Specifically, we add shadow imitation regions to the original US, and design shadow-masked transformer blocks to perceive missing anatomy of shadow regions. Shadow-masked transformer block contains an adaptive shadow attention mechanism that introduces an adaptive mask, which is updated automatically to promote the network training. Additionally, we utilize unlabeled US images to train a missing structure inpainting path with shadow-masked transformer, which further facilitates semi-supervised segmentation. Experiments on two public US datasets demonstrate the superior performance of the SABR-Net over other state-of-the-art semi-supervised segmentation methods. In addition, experiments on a private breast US dataset prove that our method has a good generalization to clinical small-scale US datasets.

Learning Deep Asymmetric Tolerant Part Representation

Categorization objects at a sub-ordinate level inevitably poses a significant challenge, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , inter-class difference is very subtle and only exists in a few key parts. Therefore, how to localize these key parts for discriminative visual categorization without requiring expensive pixel-level annotations becomes a core question. To that end, this paper introduces a novel asymmetry tolerant part segmentation network (ATP-Net). ATP-Net simultaneously learns to segment parts and identify objects in an end-to-end manner using only image-level category labels. Given the intrinsic asymmetry property of part alignment, a desirable learning of part segmentation should be capable of incorporating such property. Despite the efforts towards regularizing weakly supervised part segmentation, none of them consider this vital and intrinsic property, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , the spatial asymmetry of part alignment. Our work, for the first time, proposes to explicitly characterize the spatial asymmetry of part alignment for visual tasks. We propose a novel asymmetry loss function to guide the part segmentation by encoding the spatial asymmetry of part alignment, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , restricting the upper bound of how asymmetric those self-similar parts are to each other in the network learning. Via a comprehensive ablation study, we verify the effectiveness of the proposed ATP-Net in driving the network learning towards semantically meaningful part segmentation and discriminative visual categorization. Consistently superior/competitive performance are reported on 12 datasets covering crop cultivar classification, plant disease classification, bird/butterfly species classification, large-scale natural image classification, attribute recognition and landmark localization.

Weakly Supervised Semantic Segmentation via Box-driven Masking and Filling Rate Shifting.

Semantic segmentation has achieved huge progress via adopting deep Fully Convolutional Networks (FCN). However, the performance of FCN-based models severely rely on the amounts of pixel-level annotations which are expensive and time-consuming. Considering that bounding boxes also contain abundant semantic and objective information, an intuitive solution is to learn the segmentation with weak supervisions from the bounding boxes. How to make full use of the class-level and region-level supervisions from bounding boxes to estimate the uncertain regions is the critical challenge for the weakly supervised learning task. In this paper, we propose a mixture model to address this problem. First, we introduce a box-driven class-wise masking model (BCM) to remove irrelevant regions of each class. Moreover, based on the pixel-level segment proposal generated from the bounding box supervision, we calculate the mean filling rates of each class to serve as an important prior cue to guide the model ignoring the wrongly labeled pixels in proposals. To realize the more fine-grained supervision at instance-level, we further propose the anchor-based filling rate shifting module. Unlike previous methods that directly train models with the generated noisy proposals, our method can adjust the model learning dynamically with the adaptive segmentation loss. Thus it can help reduce the negative impacts from wrongly labeled proposals. Besides, based on the learned high-quality proposals with above pipeline, we explore to further boost the performance through two-stage learning. The proposed method is evaluated on the challenging PASCAL VOC 2012 benchmark and achieves 74.9% and 76.4% mean IoU accuracy under weakly and semi-supervised modes, respectively. Extensive experimental results show that the proposed method is effective and is on par with, or even better than current state-of-the-art methods. Code will be available at: https://github.com/developfeng/BCM.

WS-MTST: Weakly Supervised Multi-Label Brain Tumor Segmentation With Transformers.

Brain tumor segmentation is a key step in brain cancer diagnosis. Segmentation of brain tumor sub-regions, including necrotic, enhancing, and edematous regions, can provide more detailed guidance for clinical diagnosis. Weakly supervised brain tumor segmentation methods have received much attention because they do not require time-consuming pixel-level annotations. However, existing weakly supervised methods focus on the segmentation of the entire tumor region while ignoring the challenging task of multi-label segmentation for the tumor sub-regions. In this article, we propose a weakly supervised approach to solve the multi-label brain tumor segmentation problem. To the best of our knowledge, it's the first end-to-end multi-label weakly supervised segmentation model applied to brain tumor segmentation. With well-designed loss functions and a contrastive learning pre-training process, our proposed Transformer-based segmentation method (WS-MTST) has the ability to perform segmentation of brain tumor sub-regions. We conduct comprehensive experiments and demonstrate that our method reaches the state-of-the-art on the popular brain tumor dataset BraTS (from 2018 to 2020).

Semi-Supervised Semantic Segmentation-Based Remote Sensing Identification Method for Winter Wheat Planting Area Extraction

To address the cost issue associated with pixel-level image annotation in fully supervised semantic segmentation, a method based on semi-supervised semantic segmentation is proposed for extracting winter wheat planting areas. This approach utilizes self-training with pseudo-labels to learn from a small set of images with pixel-level annotations and a large set of unlabeled images, thereby achieving the extraction. In the constructed initial dataset, a random sampling strategy is employed to select 1/16, 1/8, 1/4, and 1/2 proportions of labeled data. Furthermore, in conjunction with the concept of consistency regularization, strong data augmentation techniques are applied to the unlabeled images, surpassing classical methods such as cropping and rotation to construct a semi-supervised model. This effectively alleviates overfitting caused by noisy labels. By comparing the prediction results of different proportions of labeled data using SegNet, DeepLabv3+, and U-Net, it is determined that the U-Net network model yields the best extraction performance. Moreover, the evaluation metrics MPA and MIoU demonstrate varying degrees of improvement for semi-supervised semantic segmentation compared to fully supervised semantic segmentation. Notably, the U-Net model trained with 1/16 labeled data outperforms the models trained with 1/8, 1/4, and 1/2 labeled data, achieving MPA and MIoU scores of 81.63%, 73.31%, 82.50%, and 76.01%, respectively. This method provides valuable insights for extracting winter wheat planting areas in scenarios with limited labeled data.

Multi-layered self-attention mechanism for weakly supervised semantic segmentation

Weakly Supervised Semantic Segmentation (WSSS) provides efficient solutions for semantic image segmentation using image-level annotations. WSSS requires no pixel-level labeling that Fully Supervised Semantic Segmentation (FSSS) does, which is time-consuming and label-intensive. Most WSSS approaches have leveraged Class Activation Maps (CAM) or Self-Attention (SA) to generate pseudo pixel-level annotations to perform semantic segmentation tasks coupled with fully supervised approaches (e.g., Fully Convolutional Network). However, those approaches often provides incomplete supervision that mainly includes discriminative regions from the last convolutional layer. They may fail to capture regions of low- or intermediate-level features that may not be present in the last convolutional layer. To address the issue, we proposed a novel Multi-layered Self-Attention (Multi-SA) method that applies a self-attention module to multiple convolutional layers, and then stack feature maps from the self-attention layers to generate pseudo pixel-level annotations. We demonstrated that integrated feature maps from multiple self-attention layers produce higher coverage in semantic segmentation than using only the last convolutional layer through intensive experiments using standard benchmark datasets.

Tobacco Leaves Maturity Classification Based on Deep Learning and Proximal Hyperspectral Imaging

Accurately measuring tobacco harvest maturity is crucial for optimizing quality and crop management. Traditional methods heavily rely on qualitative evaluation and chemical experiments, which introduce subjectivity and inherent limitations. This study proposes a novel method that combines deep learning with proximal hyperspectral imaging (HSI) to achieve precise recognition of tobacco leaf maturity. Unlike traditional techniques, HSI captures rich spectral and spatial data, enabling comprehensive analysis. It adopts a pixel-level annotation strategy to annotate each pixel based on its maturity level, thereby preserving spectral complexity. The dataset comprises 3000 randomly extracted cubes ( 5 × 5 × 176 ) from 150 original hyperspectral images, encompassing five maturity levels. Spectral and spatial features are extracted from hyperspectral data using a three-dimensional convolutional neural network (3D-CNN) architecture. This method effectively leverages complex spectral patterns for maturity recognition. During testing, the model demonstrated an impressive average accuracy of 99.93%. Visual predictions vividly illustrate the model’s proficiency in maturity recognition, affirming its practical utility. This study pioneers the integration of deep learning and hyperspectral near-end sensing technology in tobacco maturity assessment, mitigating the constraints of traditional methods, and establishing the groundwork for real-time monitoring and quality control of tobacco.