Precise Annotation Research Articles

How to Efficiently Annotate Images for Best-Performing Deep Learning-Based Segmentation Models: An Empirical Study with Weak and Noisy Annotations and Segment Anything Model.

Deep neural networks (DNNs) have demonstrated exceptional performance across various image segmentation tasks. However, the process of preparing datasets for training segmentation DNNs is both labor-intensive and costly, as it typically requires pixel-level annotations for each object of interest. To mitigate this challenge, alternative approaches such as using weak labels (e.g., bounding boxes or scribbles) or less precise (noisy) annotations can be employed. Noisy and weak labels are significantly quicker to generate, allowing for more annotated images within the same time frame. However, the potential decrease in annotation quality may adversely impact the segmentation performance of the resulting model. In this study, we conducted a comprehensive cost-effectiveness evaluation on six variants of annotation strategies (9 10 sub-variants in total) across 4 datasets and conclude that the common practice of precisely outlining objects of interest is virtually never the optimal approach when annotation budget is limited. Both noisy and weak annotations showed usage cases that yield similar performance to the perfectly annotated counterpart, yet had significantly better cost-effectiveness. We hope our findings will help researchers be aware of the different available options and use their annotation budgets more efficiently, especially in cases where accurately acquiring labels for target objects is particularly costly. Our code will be made available on https://github.com/yzluka/AnnotationEfficiency2D .

An image dataset for analyzing tea picking behavior in tea plantations

Tea is an important economic product in China, and tea picking is a key agricultural activity. As the practice of tea picking in China gradually shifts towards intelligent and mechanized methods, artificial intelligence recognition technology has become a crucial tool, showing great potential in recognizing large-scale tea picking operations and various picking behaviors. Constructing a comprehensive database is essential for these advancements. The newly developed Tea Garden Harvest Dataset offers several advantages that have a positive impact on tea garden management: 1) Enhanced image diversity: through advanced data augmentation techniques such as rotation, cropping, enhancement, and flipping, our dataset provides a rich variety of images. This diversity improves the model’s ability to accurately recognize tea picking behaviors under different environments and conditions. 2) Precise annotations: every image in our dataset is meticulously annotated with boundary box coordinates, object categories, and sizes. This detailed annotation helps to better understand the target features, enhancing the model’s learning process and overall performance. 3) Multi-Scale training capability: our dataset supports multi-scale training, allowing the model to adapt to targets of different sizes. This capability ensures versatility and accuracy in real-world applications, where objects may appear at varying distances and scales. This tea garden picking dataset not only fills the existing gap in the data related to tea picking in China but also makes a significant contribution to advancing intelligent tea picking practices. By leveraging its unique advantages, this dataset becomes a powerful resource for tea garden management, promoting increased efficiency, accuracy, and productivity in tea production.

A Semi-Supervised Fracture-Attention Model for Segmenting Tubular Objects with Improved Topological Connectivity.

Ensuring connectivity and preventing fractures in tubular object segmentation are critical for downstream analyses. Despite advancements in deep neural networks (DNNs) that have significantly improved tubular object segmentation, existing methods still face limitations. They often rely heavily on precise annotations, hindering their scalability to large-scale unlabeled image datasets. Additionally, current evaluation metrics are insufficient for effectively capturing segmentation fractures. To address these challenges, we propose a semi-supervised fracture-attention model (SSFA) for tubular object segmentation. SSFA enhances connectivity, reduces fractures, and maintains volumetric accuracy. It outperforms state-of-the-art models in topological performance. Extensive experiments on four public datasets validate the effectiveness of SSFA. Furthermore, we introduce a novel evaluation metric, the Fracture Rate (FR), which provides an intuitive and quantitative assessment of segmentation fractures. Our source code is available at http://github.com/Yanfeng-Zhou/SSFA. Supplementary data are available at Bioinformatics online.

TransCAE: Enhancing Cell Type Annotation in Single-cell RNA-seq Data with Transfer Learning and Convolutional Autoencoder.

Single-cell RNA sequencing (scRNA-seq) analysis offers tremendous potential for addressing various biological questions, with one key application being the annotation of query datasets with unknown cell types using well-annotated external reference datasets. However, the performance of existing supervised or semi-supervised methods largely depends on the quality of source data. Furthermore, these methods often struggle with the batch effects arising from different platforms when handling multiple reference or query datasets, making precise annotation challenging. We developed transCAE, a robust transfer learning-based algorithm for single-cell annotation that integrates unsupervised dimensionality reduction with supervised cell type classification. This approach fully leverages information from both reference and query datasets to achieve precise cell classification within the query data. Extensive evaluations show that transCAE significantly enhances classification accuracy and efficiently mitigates batch effects. Compared to other state-of-the-art methods, transCAE demonstrates superior performance in experiments involving multiple reference or query datasets. These strengths position transCAE as an optimal annotation method for scRNA-seq datasets.

Accurate and fast segmentation of filaments and membranes in micrographs and tomograms with TARDIS.

It is now possible to generate large volumes of high-quality images of biomolecules at near-atomic resolution and in near-native states using cryogenic electron microscopy/electron tomography (Cryo-EM/ET). However, the precise annotation of structures like filaments and membranes remains a major barrier towards applying these methods in high-throughput. To address this, we present TARDIS (Transformer-based Rapid Dimensionless Instance Segmentation), a machine-learning framework for fast and accurate annotation of micrographs and tomograms. TARDIS combines deep learning for semantic segmentation with a novel geometric model for precise instance segmentation of various macromolecules. We develop pre-trained models within TARDIS for segmenting microtubules and membranes, demonstrating high accuracy across multiple modalities and resolutions, enabling segmentation of over 13,000 tomograms from the CZI Cryo-Electron Tomography data portal. As a modular framework, TARDIS can be extended to new structures and imaging modalities with minimal modification. TARDIS is open-source and freely available at https://github.com/SMLC-NYSBC/TARDIS, and accelerates analysis of high-resolution biomolecular structural imaging data.

Sn-Position-Resolved Quantification of Aminophospholipids by Isotopic N,N-Dimethyl Leucine Labeling and High-Resolution Ion Mobility Mass Spectrometry.

Aminophospholipids (APLs), composed of phosphatidylethanolamines (PEs) and phosphatidylserines (PSs), are vital components of mammalian cell membranes and lipoproteins, participating in both homeostasis and cellular signaling. Their structural changes, including the permutation of fatty acid connectivity (sn-positions), due to dysfunctional metabolic processes have been linked to many diseases. However, the accurate quantification of APLs with unambiguous fatty acyl assignment through routine label-free LC-MS/MS lipidomic analysis remains a major challenge. In this study, we explore the functionalization of the free primary amine groups of APLs using amine-reactive isotopic N,N-dimethyl leucine (iDiLeu) and employ high-resolution ion mobility MS (IM-MS) to develop a novel method for sensitive discernment and accurate quantification of APL sn-isomers. With high-resolution demultiplexing (HRdm) providing IM resolving power >200, labeled sn-isomeric pairs of APLs (ΔCCS ≈ 1%) demonstrate excellent, near baseline separation. In addition to greatly enhanced sensitivity, 5-plex iDiLeu labeling enables the construction of an internal 4-point calibration curve and therefore absolute quantification of APL sn-isomers in a single run. This strategy enabled precise annotation and quantification of 239 APLs including 60 pairs of sn-isomers in the mouse cortex. Additionally, we were able to find ratio changes in multiple APL sn-isomer pairs between wild type and APP/PS1 Alzheimer's disease (AD) model mice at different ages, indicating their strong correlation to AD progression. This strategy could provide universal utility in unraveling the alteration of APL sn-isomers, which have long been considered as the "dark matter" of traditional lipidomic analyses, leading to more precise elucidation of molecular mechanisms of various diseases.

Diverse Dataset for Eyeglasses Detection: Extending the Flickr-Faces-HQ (FFHQ) Dataset.

Facial analysis is an important area of research in computer vision and machine learning, with applications spanning security, healthcare, and user interaction systems. The data-centric AI approach emphasizes the importance of high-quality, diverse, and well-annotated datasets in driving advancements in this field. However, current facial datasets, such as Flickr-Faces-HQ (FFHQ), lack detailed annotations for detecting facial accessories, particularly eyeglasses. This work addresses this limitation by extending the FFHQ dataset with precise bounding box annotations for eyeglasses detection, enhancing its utility for data-centric AI applications. The extended dataset comprises 70,000 images, including over 16,000 images containing eyewear, and it exceeds the CelebAMask-HQ dataset in size and diversity. A semi-automated protocol was employed to efficiently generate accurate bounding box annotations, minimizing the demand for extensive manual labeling. This enriched dataset serves as a valuable resource for training and benchmarking eyewear detection models. Additionally, the baseline benchmark results for eyeglasses detection were presented using deep learning methods, including YOLOv8 and MobileNetV3. The evaluation, conducted through cross-dataset validation, demonstrated the robustness of models trained on the extended FFHQ dataset with their superior performances over existing alternative CelebAMask-HQ. The extended dataset, which has been made publicly available, is expected to support future research and development in eyewear detection, contributing to advancements in facial analysis and related fields.

RIDGE: Reproducibility, Integrity, Dependability, Generalizability, and Efficiency Assessment of Medical Image Segmentation Models.

Deep learning techniques hold immense promise for advancing medical image analysis, particularly in tasks like image segmentation, where precise annotation of regions or volumes of interest within medical images is crucial but manually laborious and prone to interobserver and intraobserver biases. As such, deep learning approaches could provide automated solutions for such applications. However, the potential of these techniques is often undermined by challenges in reproducibility and generalizability, which are key barriers to their clinical adoption. This paper introduces the RIDGE checklist, a comprehensive framework designed to assess the Reproducibility, Integrity, Dependability, Generalizability, and Efficiency of deep learning-based medical image segmentation models. The RIDGE checklist is not just a tool for evaluation but also a guideline for researchers striving to improve the quality and transparency of their work. By adhering to the principles outlined in the RIDGE checklist, researchers can ensure that their developed segmentation models are robust, scientifically valid, and applicable in a clinical setting.

Missing microbial eukaryotes and misleading meta-omic conclusions

Meta-omics is commonly used for large-scale analyses of microbial eukaryotes, including species or taxonomic group distribution mapping, gene catalog construction, and inference on the functional roles and activities of microbial eukaryotes in situ. Here, we explore the potential pitfalls of common approaches to taxonomic annotation of protistan meta-omic datasets. We re-analyze three environmental datasets at three levels of taxonomic hierarchy in order to illustrate the crucial importance of database completeness and curation in enabling accurate environmental interpretation. We show that taxonomic membership of sequence clusters estimates community composition more accurately than returning exact sequence labels, and overlap between clusters can address database shortcomings. Clustering approaches can be applied to diverse environments while continuing to exploit the wealth of annotation data collated in databases, and selecting and evaluating these databases is a critical part of correctly annotating protistan taxonomy in environmental datasets. We argue that ongoing curation of genetic resources is crucial in accurately annotating protists in in situ meta-omic datasets. Moreover, we propose that precise taxonomic annotation of meta-omic data is a clustering problem rather than a feasible alignment problem.

Comparative transcriptomic analyses of thymocytes using 10x Genomics and Parse scRNA-seq technologies

BackgroundSingle-cell RNA sequencing experiments commonly use 10x Genomics (10x) kits due to their high-throughput capacity and standardized protocols. Recently, Parse Biosciences (Parse) introduced an alternative technology that uses multiple in-situ barcoding rounds within standard 96-well plates. Parse enables the analysis of more cells from multiple samples in a single run without the need for additional reagents or specialized microfluidics equipment. To evaluate the performance of both platforms, we conducted a benchmark study using biological and technical replicates of mouse thymus as a complex immune tissue.ResultsWe found that Parse detected nearly twice the number of genes compared to 10x, with each platform detecting a distinct set of genes. The comparison of multiplexed samples generated from 10x and Parse techniques showed 10x data to have lower technical variability and more precise annotation of biological states in the thymus compared to Parse.ConclusionOur results provide a comprehensive comparison of the suitability of both single-cell platforms for immunological studies.

Enhanced NSCLC subtyping and staging through attention-augmented multi-task deep learning: A novel diagnostic tool

ObjectivesThe objective of this study is to develop a novel multi-task learning approach with attention encoders for classifying histologic subtypes and clinical stages of non-small cell lung cancer (NSCLC), with superior performance compared to currently popular deep-learning models. Material and methodsData were collected from six publicly available datasets in The Cancer Imaging Archive (TCIA). Following the inclusion and exclusion criteria, a total of 4548 CT slices from 758 cases were allocated. We evaluated multiple multi-task learning models that integrate attention mechanisms to resolve challenges in NSCLC subtype classification and clinical staging. These models utilized convolution-based modules in their shared layers for feature extraction, while the task layers were dedicated to histological subtype classification and staging. Each branch sequentially processed features through convolution-based and attention-based modules prior to classification. ResultsOur study evaluated 758 NSCLC patients (mean age, 66.2 years ± 10.3; 473 men), spanning ADC and SCC cases. In the classification of histological subtypes and clinical staging of NSCLC, the MobileNet-based multi-task learning model enhanced with attention mechanisms (MN-MTL-A) demonstrated superior performance, achieving Area Under the Curve (AUC) scores of 0.963 (95 % CI: 0.943, 0.981) and 0.966 (95 % CI: 0.945, 0.982) for each task, respectively. The model significantly surpassed its counterparts lacking attention mechanisms and those configured for single-task learning, as evidenced by P-values of 0.01 or less for both tasks, according to DeLong’s test. ConclusionsThe integration of attention encoder blocks into our multi-task learning network significantly enhanced the accuracy of NSCLC histological subtyping and clinical staging. Given the reduced reliance on precise radiologist annotation, our proposed model shows promising potential for clinical application.

Employing the Artificial Intelligence Object Detection Tool YOLOv8 for Real-Time Pain Detection: A Feasibility Study.

Effective pain management is crucial for patient care, impacting comfort, recovery, and overall well-being. Traditional subjective pain assessment methods can be challenging, particularly in specific patient populations. This research explores an alternative approach using computer vision (CV) to detect pain through facial expressions. The study implements the YOLOv8 real-time object detection model to analyze facial expressions indicative of pain. Given four pain datasets, a dataset of pain-expressing faces was compiled, and each image was carefully labeled based on the presence of pain-associated Action Units (AUs). The labeling distinguished between two classes: pain and no pain. The pain category included specific AUs (AU4, AU6, AU7, AU9, AU10, and AU43) following the Prkachin and Solomon Pain Intensity (PSPI) scoring method. Images showing these AUs with a PSPI score above 2 were labeled as expressing pain. The manual labeling process utilized an open-source tool, makesense.ai, to ensure precise annotation. The dataset was then split into training and testing subsets, each containing a mix of pain and no-pain images. The YOLOv8 model underwent iterative training over 10 epochs. The model's performance was validated using precision, recall, and mean Average Precision (mAP) metrics, and F1 score. When considering all classes collectively, our model attained a mAP of 0.893 at a threshold of 0.5. The precision for "pain" and "nopain" detection was 0.868 and 0.919, respectively. F1 scores for the classes "pain", "nopain", and "all classes" reached a peak value of 0.80. Finally, the model was tested on the Delaware dataset and in a real-world scenario. Despite limitations, this study highlights the promise of using real-time computer vision models for pain detection, with potential applications in clinical settings. Future research will focus on evaluating the model's generalizability across diverse clinical scenarios and its integration into clinical workflows to improve patient care.

Seq-Scope: repurposing Illumina sequencing flow cells for high-resolution spatial transcriptomics.

Spatial transcriptomics technologies aim to advance gene expression studies by profiling the entire transcriptome with intact spatial information from a single histological slide. However, the application of spatial transcriptomics is limited by low resolution, limited transcript coverage, complex procedures, poor scalability and high costs of initial setup and/or individual experiments. Seq-Scope repurposes the Illumina sequencing platform for high-resolution, high-content spatial transcriptome analysis, overcoming these limitations. It offers submicrometer resolution, high capture efficiency, rapid turnaround time and precise annotation of histopathology at a much lower cost than commercial alternatives. This protocol details the implementation of Seq-Scope with an Illumina NovaSeq 6000 sequencing flow cell, allowing the profiling of multiple tissue sections in an area of 7 mm × 7 mm or larger. We describe the preparation of a fresh-frozen tissue section for both histological imaging and sequencing library preparation and provide a streamlined computational pipeline with comprehensive instructions to integrate histological and transcriptomic data for high-resolution spatial analysis. This includes the use of conventional software tools for single-cell and spatial analysis, as well as our recently developed segmentation-free method for analyzing spatial data at submicrometer resolution. Aside from array production and sequencing, which can be done in batches, tissue processing, library preparation and running the computational pipeline can be completed within 3 days by researchers with experience in molecular biology, histology and basic Unix skills. Given its adaptability across various biological tissues, Seq-Scope establishes itself as an invaluable tool for researchers in molecular biology and histology.

An automatic LC-MS/MS data analysis workflow for herbal compound annotation with AutoAnnotatoR: A case study of ten botanical origins of Fritillaria species

BackgroundDespite the widespread implementation of analytical hardware capable of recording large-scale datasets for botanical natural products, the data processing procedures for compound annotation remain a bothersome obstacle that demand a tremendous amount of time and expert knowledge. MethodsHerein, an automatic LC-MS/MS data analysis workflow with AutoAnnotatoR was introduced for the compound annotation of plant derived natural products, which has the merits of great efficiency, high accuracy, saving time and simplified process. This procedure enabled automatic matching of MS2 data with characteristic fragment ions, as well as MS1 data with compound libraries, which improves the accuracy of structural elucidation. Notably, the optimization of collision energy for each target ion was successfully performed for the first time, facilitating the acquisition of comprehensive fragmentation information. ResultsThe automatic analysis workflow with AutoAnnotatoR was successfully applied for the annotation of alkaloids from 10 botanical origins of Fritillaria species. Consequently, a total of 2684 chemical constituents were tentatively characterized, with 23 components being unambiguously validated by reference standards and 2434 being probable novel chemicals. ConclusionThe entire data analysis procedure takes only a few hours, vastly improving analysis speed while assuring high accuracy. This method provides a powerful tool for the rapid and precise annotation of complex natural products. The workflow is publicly accessible on Github as an open-source R package called AutoAnnotatoR (https://github.com/anyaling2022/AutoAnnotatoR).

Automated sample annotation for diabetes mellitus in healthcare integrated biobanking

Healthcare integrated biobanking describes the annotation and collection of residual samples from hospitalized patients for research purposes. The central idea of the current work is to establish an automated workflow for sample annotation, selection and storage for diabetes mellitus. This is challenging due to incomplete data at the time of sample selection. The study evaluates a machine learning (ML) and natural language processing (NLP) based two-step procedure for timely and precise sample annotation for diabetes mellitus. Electronic health record data of 785 persons were extracted from the hospital information system. In the first step, a conditional inference forest (CIF) model was trained and tested based on laboratory values from the first 72 h of the hospital stay using test- (n = 550) and training data sets (n = 235). Performance was compared with a simple laboratory cut-off classifier (LCC) and a logistic regression (LR) model. Algorithms based on laboratory values, ICD-10 codes or information from discharge summaries extracted by a natural language processing software (NLP-DS) were evaluated as a second (review) step designed to increase the precision of annotations. For the first step, recall/precision/F1-score/accuracy were 71 %/86 %/0.78/0.82 for CIF and 77 %/70 %/0.74/0.75 for LR compared to 73 %/68 %/0.70/0.72 for LCC. NLP-DS was the best-performing second (review) step (93 %/100 %/0.97/0.97). Combining first-step models with NLP-DS increased precision to 100 % for all procedures (66 %/100 %/0.80/0.85 for CIF&NLP-DS, 72 %/100 %/0.84/87.2 for LR&NLP-DS and 66 %/100 %/0.80/0.85 for LCC&NLP-DS). The number of samples removed by NLP-DS was higher for LR&NLP-DS and LCC&NLP-DS (removal rate 35 % and 38 % of initially selected samples) compared to CIF&NLP-DS (removal rate of 20 %). The developed two-step procedure is an efficient implementable method for timely and precise annotation of samples from diabetic hospitalized patients.

Query-aware multi-scale proposal network for weakly supervised temporal sentence grounding in videos

Recently, weakly supervised temporal sentence grounding in videos (TSGV) has attracted extensive attention because it does not require precise start-end time annotations during training, and it can quickly retrieve interesting segments according to user needs. In weakly supervised TSGV, query reconstruction (QR)-based methods are the current mainstream, and the quality of proposals determines their performance. QR-based methods have two problems in proposal quality. First, a multi-modal global token is usually mapped to proposals with limited duration diversity, making it difficult to capture relevant segments at varying durations in real scenarios. Additionally, Gaussian functions are typically used to generate relatively fixed weights for frames within proposals, which weigh the original video features to generate proposal-specific features. This results in query-irrelevant frames affecting the discrimination of the proposal features. In this study, we propose a query-aware multi-scale proposal network (QMN). Initially, pre-trained encoders are used to extract video and query features. Subsequently, a multi-scale proposal generation module is designed to refine video features guided by queries and diversify the duration of the proposal. This module performs multi-modal interaction and multi-scale modeling to obtain proposals of different durations. Furthermore, to extract discriminative proposal features and enhance the modeling of proposal frame correlation, a query-aware weight generator is constructed to learn frame weights to suppress query-irrelevant frame representations through contrastive learning. Finally, the masked query is reconstructed using the proposal features to select the best proposal. The effectiveness of the proposed QMN is verified through experiments on the Charades-STA and ActivityNet-Captions datasets.

PeptiHub: a curated repository of precisely annotated cancer-related peptides with advanced utilities for peptide exploration and discovery.

Peptihub (https://bioinformaticscollege.ir/peptihub/) is a meticulously curated repository of cancer-related peptides (CRPs) that have been documented in scientific literature. A diverse collection of CRPs is included in the PeptiHub, showcasing a spectrum of effects and activities. While some peptides demonstrated significant anticancer efficacy, others exhibited no discernible impact, and some even possessed alternative non-drug functionalities, including drug carrier or carcinogenic attributes. Presently, Peptihub houses 874 CRPs, subjected to evaluation across 10 distinct organism categories, 26 organs, and 438 cell lines. Each entry in the database is accompanied by easily accessible 3D conformations, obtained either experimentally or through predictive methodology. Users are provided with three search frameworks offering basic, advanced, and BLAST sequence search options. Furthermore, precise annotations of peptides enable users to explore CRPs based on their specific activities (anticancer, no effect, insignificant effect, carcinogen, and others) and their effectiveness (rate and IC50) under cancer conditions, specifically within individual organs. This unique property facilitates the construction of robust training and testing datasets. Additionally, PeptiHub offers 1141 features with the convenience of selecting the most pertinent features to address their specific research questions. Features include aaindex1 (in six main subcategories: alpha propensities, beta propensity, composition indices, hydrophobicity, physicochemical properties, and other properties), amino acid composition (Amino acid Composition and Dipeptide Composition), and Grouped Amino Acid Composition (Grouped amino acid composition, Grouped dipeptide composition, and Conjoint triad) categories. These utilities not only speed up machine learning-based peptide design but also facilitate peptide classification. Database URL: https://bioinformaticscollege.ir/peptihub/.

A natural language processing system for the efficient extraction of cell markers

Single-cell RNA sequencing (scRNA-seq) has emerged as a pivotal tool for exploring cellular landscapes across diverse species and tissues. Precise annotation of cell types is essential for understanding these landscapes, relying heavily on empirical knowledge and curated cell marker databases. In this study, we introduce MarkerGeneBERT, a natural language processing (NLP) system designed to extract critical information from the literature regarding species, tissues, cell types, and cell marker genes in the context of single-cell sequencing studies. Leveraging MarkerGeneBERT, we systematically parsed full-text articles from 3702 single-cell sequencing-related studies, yielding a comprehensive collection of 7901 cell markers representing 1606 cell types across 425 human tissues/subtissues, and 8223 cell markers representing 1674 cell types across 482 mouse tissues/subtissues. Comparative analysis against manually curated databases demonstrated that our approach achieved 76% completeness and 75% accuracy, while also unveiling 89 cell types and 183 marker genes absent from existing databases. Furthermore, we successfully applied the compiled brain tissue marker gene list from MarkerGeneBERT to annotate scRNA-seq data, yielding results consistent with original studies. Conclusions: Our findings underscore the efficacy of NLP-based methods in expediting and augmenting the annotation and interpretation of scRNA-seq data, providing a systematic demonstration of the transformative potential of this approach. The 27323 manual reviewed sentences for training MarkerGeneBERT and the source code are hosted at https://github.com/chengpeng1116/MarkerGeneBERT.

Retinal Vessel Segmentation by a Transformer-U-Net Hybrid Model With Dual-Path Decoder.

This paper introduces an effective and efficient framework for retinal vessel segmentation. First, we design a Transformer-CNN hybrid model in which a Transformer module is inserted inside the U-Net to capture long-range interactions. Second, we design a dual-path decoder in the U-Net framework, which contains two decoding paths for multi-task outputs. Specifically, we train the extra decoder to predict vessel skeletons as an auxiliary task which helps the model learn balanced features. The proposed framework, named as TSNet, not only achieves good performances in a fully supervised learning manner but also enables a rough skeleton annotation process. The annotators only need to roughly delineate vessel skeletons instead of giving precise pixel-wise vessel annotations. To learn with rough skeleton annotations plus a few precise vessel annotations, we propose a skeleton semi-supervised learning scheme. We adopt a mean teacher model to produce pseudo vessel annotations and conduct annotation correction for roughly labeled skeletons annotations. This learning scheme can achieve promising performance with fewer annotation efforts. We have evaluated TSNet through extensive experiments on five benchmarking datasets. Experimental results show that TSNet yields state-of-the-art performances on retinal vessel segmentation and provides an efficient training scheme in practice.

Dual-branch Transformer for semi-supervised medical image segmentation.

In recent years, the use of deep learning for medical image segmentation has become a popular trend, but its development also faces some challenges. Firstly, due to the specialized nature of medical data, precise annotation is time-consuming and labor-intensive. Training neural networks effectively with limited labeled data is a significant challenge in medical image analysis. Secondly, convolutional neural networks commonly used for medical image segmentation research often focus on local features in images. However, the recognition of complex anatomical structures or irregular lesions often requires the assistance of both local and global information, which has led to a bottleneck in its development. Addressing these two issues, in this paper, we propose a novel network architecture. We integrate a shift window mechanism to learn more comprehensive semantic information and employ a semi-supervised learning strategy by incorporating a flexible amount of unlabeled data. Specifically, a typical U-shaped encoder-decoder structure is applied to obtain rich feature maps. Each encoder is designed as a dual-branch structure, containing Swin modules equipped with windows of different size to capture features of multiple scales. To effectively utilize unlabeled data, a level set function is introduced to establish consistency between the function regression and pixel classification. We conducted experiments on the COVID-19 CT dataset and DRIVE dataset and compared our approach with various semi-supervised and fully supervised learning models. On the COVID-19 CT dataset, we achieved a segmentation accuracy of up to 74.56%. Our segmentation accuracy on the DRIVE dataset was 79.79%. The results demonstrate the outstanding performance of our method on several commonly used evaluation metrics. The high segmentation accuracy of our model demonstrates that utilizing Swin modules with different window sizes can enhance the feature extraction capability of the model, and the level set function can enable semi-supervised models to more effectively utilize unlabeled data. This provides meaningful insights for the application of deep learning in medical image segmentation. Our code will be released once the manuscript is accepted for publication.