Weak Annotations Research Articles

Toward high-quality pseudo masks from noisy or weak annotations for robust medical image segmentation

Deep learning networks excel in image segmentation with abundant accurately annotated training samples. However, in medical applications, acquiring large quantities of high-quality labeled images is prohibitively expensive. Thus, learning from imperfect annotations (e.g. noisy or weak annotations) has emerged as a prominent research area in medical image segmentation. This work aims to extract high-quality pseudo masks from imperfect annotations with the assistance of a small number of clean labels. Our core motivation is based on the understanding that different types of flawed imperfect annotations inherently exhibit unique noise patterns. Comparing clean annotations with corresponding imperfectly annotated labels can effectively identify potential noise patterns at minimal additional cost. To this end, we propose a two-phase framework including a noise identification network and a noise-robust segmentation network. The former network implicitly learns noise patterns and revises labels accordingly. It includes a three-branch network to identify different types of noises. The latter one further mitigates the negative influence of residual annotation noises based on parallel segmentation networks with different initializations and a label softening strategy. Extensive experimental results on two public datasets demonstrate that our method can effectively refine annotation flaws and achieve superior segmentation performance to the state-of-the-art methods.

Automated segment-level coronary artery calcium scoring on non-contrast CT: a multi-task deep-learning approach

ObjectivesTo develop and evaluate a multi-task deep-learning (DL) model for automated segment-level coronary artery calcium (CAC) scoring on non-contrast computed tomography (CT) for precise localization and quantification of calcifications in the coronary artery tree.MethodsThis study included 1514 patients (mean age, 60.0 ± 10.2 years; 56.0% female) with stable chest pain from 26 centers participating in the multicenter DISCHARGE trial (NCT02400229). The patients were randomly assigned to a training/validation set (1059) and a test set (455). We developed a multi-task neural network for performing the segmentation of calcifications on the segment level as the main task and the segmentation of coronary artery segment regions with weak annotations as an auxiliary task. Model performance was evaluated using (micro-average) sensitivity, specificity, F1-score, and weighted Cohen’s κ for segment-level agreement based on the Agatston score and performing interobserver variability analysis.ResultsIn the test set of 455 patients with 1797 calcifications, the model assigned 73.2% (1316/1797) to the correct coronary artery segment. The model achieved a micro-average sensitivity of 0.732 (95% CI: 0.710–0.754), a micro-average specificity of 0.978 (95% CI: 0.976–0.980), and a micro-average F1-score of 0.717 (95% CI: 0.695–0.739). The segment-level agreement was good with a weighted Cohen’s κ of 0.808 (95% CI: 0.790–0.824), which was only slightly lower than the agreement between the first and second observer (0.809 (95% CI: 0.798–0.845)).ConclusionAutomated segment-level CAC scoring using a multi-task neural network approach showed good agreement on the segment level, indicating that DL has the potential for automated coronary artery calcification classification.Critical relevance statementMulti-task deep learning can perform automated coronary calcium scoring on the segment level with good agreement and may contribute to the development of new and improved calcium scoring methods.Key PointsSegment-level coronary artery calcium scoring is a tedious and error-prone task.The proposed multi-task model achieved good agreement with a human observer on the segment level.Deep learning can contribute to the automation of segment-level coronary artery calcium scoring.Graphical

An anthropomorphic diagnosis system of pulmonary nodules using weak annotation-based deep learning

The accurate categorization of lung nodules in CT scans is an essential aspect in the prompt detection and diagnosis of lung cancer. The categorization of grade and texture for nodules is particularly significant since it can aid radiologists and clinicians to make better-informed decisions concerning the management of nodules. However, currently existing nodule classification techniques have a singular function of nodule classification and rely on an extensive amount of high-quality annotation data, which does not meet the requirements of clinical practice. To address this issue, we develop an anthropomorphic diagnosis system of pulmonary nodules (PN) based on deep learning (DL) that is trained by weak annotation data and has comparable performance to full-annotation based diagnosis systems. The proposed system uses DL models to classify PNs (benign vs. malignant) with weak annotations, which eliminates the need for time-consuming and labor-intensive manual annotations of PNs. Moreover, the PN classification networks, augmented with handcrafted shape features acquired through the ball-scale transform technique, demonstrate capability to differentiate PNs with diverse labels, including pure ground-glass opacities, part-solid nodules, and solid nodules. Through 5-fold cross-validation on two datasets, the system achieved the following results: (1) an Area Under Curve (AUC) of 0.938 for PN localization and an AUC of 0.912 for PN differential diagnosis on the LIDC-IDRI dataset of 814 testing cases, (2) an AUC of 0.943 for PN localization and an AUC of 0.815 for PN differential diagnosis on the in-house dataset of 822 testing cases. In summary, our system demonstrates efficient localization and differential diagnosis of PNs in a resource limited environment, and thus could be translated into clinical use in the future.

One point is all you need for weakly supervised object detection

Object detection with weak annotations has attracted much attention recently. Weakly supervised object detection(WSOD) methods which only use image-level labels to train a detector encounter some severe problems that it cannot cover the whole object and the region proposal methods waste a large amount of time. Meanwhile, point supervised object detection(PSOD) leverages point annotations that remarkably improves the performance. However, point annotation is still complex and increase the costs of annotation. To overcome these issues, we propose a novel method which only requires one point per category for a training image. Compared to point annotation, our method significantly reduces the annotation cost as the number of point annotations is largely reduced. We design a framework to train a detector with one point per category annotation. Firstly, a pseudo box generation module is introduced to generate the corresponding pseudo boxes of the annotated points. Then, inspired by the observation that the features of objects with the same class in an image are very similar, a dense instances mining module is proposed to make use of the similarity between the features of objects with the same class to discover unlabeled instances and generate pseudo category heatmaps. Finally, the pseudo boxes and pseudo category heatmaps are leveraged to train a detector. Experiments conducted on popular open-source datasets verify the effectiveness of our annotation method and framework. Our proposed method outperforms previous WSOD methods and achieves comparable performance with some PSOD methods in a more efficient way.

PPDistiller: Weakly-supervised 3D point cloud semantic segmentation framework via point-to-pixel distillation

Despite the significant growth in the availability of 3D light detection and ranging (LiDAR) point cloud data in recent years, annotation remains expensive and time-consuming. This has led to an increasing demand for weakly-supervised semantic segmentation (WSSS) methods in applications such as autonomous driving, mapping, and robotics. Existing approaches typically rely solely on LiDAR point cloud data for WSSS, which often results in lower segmentation accuracy due to the sparsity of point clouds. To address these challenges, we propose a novel architecture, PPDistiller, which employs multiple teacher networks from different modalities. Compared to other WSSS and multimodal approaches, PPDistiller achieves superior segmentation accuracy with fewer annotations. This is facilitated through the novel Mean Multi-Teacher Framework (MMT), which incorporates multiple modalities and teachers. To address the issue of lacking 2D labels, we propose the Distance-CAM Self-Training (DCAM-ST) module, which utilizes sparse 3D weak annotations to produce accurate 2D pixel-level annotations. To enable adaptive fusion of 2D and 3D data, we introduce the Attention Point to Pixel Fusion (APPF) module, facilitating bidirectional transfer of cross-modal knowledge. Additionally, to fully leverage the spatial semantic information in point cloud, we propose the Pyramid Semantic-context Neighbor Aggregation (PSNA) module, aiming to exploit spatial and semantic correlations to improve performance. Extensive experimentation on SemanticKITTI, ScribbleKITTI and nuScenes datasets demonstrates the superiority of our proposed method. Compared to state-of-the-art fusion and weakly-supervised methods, PPDistiller achieves the highest mean Intersection over Union (mIoU) scores under both fully-supervised and weakly-supervised settings.

A Segment Anything Model based weakly supervised learning method for crop mapping using Sentinel-2 time series images

Automatic crop mapping is essential for various agricultural applications. Although fully convolutional networks (FCNs) have shown effectiveness in crop mapping, they rely on labor-intensive pixel-level annotations. Weakly supervised semantic segmentation (WSSS) methods offer a solution by enabling pixel-level segmentation with less costly annotations, such as point, bounding box (bbox), and image-level annotations. However, these weak annotations often lack complete object information, leading to reduced accuracy. Moreover, there is limited exploration of WSSS methods in medium-resolution satellite imagery, such as Sentinel-2. Therefore, this study proposes SAMWS, a WSSS method based on the Segment Anything Model (SAM) for crop mapping using Sentinel-2 time series images. Leveraging SAM, our method can generate high-quality pseudo labels and accommodate various weak annotations without extensive adjustments. SAMWS comprises three stages: 1) finetuning SAM with adapters; 2) generating pseudo labels using weak annotations and finetuned SAM; and 3) training a segmentation network using pseudo labels for crop mapping. Experiments conducted on PASTIS and Munich datasets demonstrate the superiority of our approach. After SAM was finetuned with adapters, the F1-scores for parcel segmentation using point and bbox prompts increased by 75.0 % and 14.4 %, respectively, reaching 0.880 and 0.931. Additionally, our method achieves classification results closest to fully supervised learning when using point, bbox, and image-level annotations, with F1-scores of 0.810, 0.817, and 0.779, respectively. Our approach offers valuable insights into leveraging foundation models in the remote sensing domain and contains significant potential for crop monitoring. The relevant code will be made publicly available at https://github.com/Nick0317Sun/SAMWS.

Category-agnostic semantic edge detection by measuring neural representation randomness

Edge detection plays a fundamental role in computer vision tasks and gains wide applications. In particular, semantic edge detection recently draws more attention due to the high demand for a fine-grained understanding of visual scenes. However, detecting high-level semantic edges hidden in visual scenes is quite challenging. Existing semantic edge detection methods focus on category-aware semantic edges and require elaborate category annotations. Instead, we first propose the category-agnostic semantic edge detection task without additional semantic category annotations. To achieve this goal, we propose to utilize only edge position annotations and leverage the information randomness of semantic edges. Specifically, we align semantic edge positions to the ground truth by maximizing randomness on edge regions and minimizing randomness on non-edge regions in the training process. In the inference process, we first obtain neural representations by the trained network, and then generate semantic edges by measuring neural randomness. We evaluate our method by comparisons with alternative methods on two well-known datasets: Cityscapes (Cordts et al., 2016) and SBD (Hariharan et al., 2014). The results demonstrate our superiority over the alternatives, which is more significant under weak annotations. We also provide comprehensive mechanism studies to verify the generalizability, rationality, and validity of our working mechanism.

Weakly Supervised Detection of Baby Cry

Detection of baby cry is an important part of baby monitoring. Almost all existing methods use supervised SVM, CNN, or their varieties. In this work, we propose to use weakly supervised anomaly detection to detect baby cry. In this weak supervision framework, we only need weak annotation if there is a cry in an audiofile. We design a data mining technique using the pre-trained VGGish feature extractor and an anomaly detection network on long untrimmed audiofiles. The obtained datasets are used to train a delicately designed super lightweight CNN for cry/non-cry classification. This CNN is then used as a feature extractor in an anomaly detection framework to achieve better cry detection performance.

WSPolyp-SAM: Weakly Supervised and Self-Guided Fine-Tuning of SAM for Colonoscopy Polyp Segmentation

Ensuring precise segmentation of colorectal polyps holds critical importance in the early diagnosis and treatment of colorectal cancer. Nevertheless, existing deep learning-based segmentation methods are fully supervised, requiring extensive, precise, manual pixel-level annotation data, which leads to high annotation costs. Additionally, it remains challenging to train large-scale segmentation models when confronted with limited colonoscopy data. To address these issues, we introduce the general segmentation foundation model—the Segment Anything Model (SAM)—into the field of medical image segmentation. Fine-tuning the foundation model is an effective approach to tackle sample scarcity. However, current SAM fine-tuning techniques still rely on precise annotations. To overcome this limitation, we propose WSPolyp-SAM, a novel weakly supervised approach for colonoscopy polyp segmentation. WSPolyp-SAM utilizes weak annotations to guide SAM in generating segmentation masks, which are then treated as pseudo-labels to guide the fine-tuning of SAM, thereby reducing the dependence on precise annotation data. To improve the reliability and accuracy of pseudo-labels, we have designed a series of enhancement strategies to improve the quality of pseudo-labels and mitigate the negative impact of low-quality pseudo-labels. Experimental results on five medical image datasets demonstrate that WSPolyp-SAM outperforms current fully supervised mainstream polyp segmentation networks on the Kvasir-SEG, ColonDB, CVC-300, and ETIS datasets. Furthermore, by using different amounts of training data in weakly supervised and fully supervised experiments, it is found that weakly supervised fine-tuning can save 70% to 73% of annotation time costs compared to fully supervised fine-tuning. This study provides a new perspective on the combination of weakly supervised learning and SAM models, significantly reducing annotation time and offering insights for further development in the field of colonoscopy polyp segmentation.

Semantic-driven diffusion for sign language production with gloss-pose latent spaces alignment

Sign Language Production (SLP) aims to translate spoken language into visual sign language sequences. The most challenging process in SLP is the transformation of a sequence of sign glosses into corresponding sign poses (G2P). Existing approaches on G2P mainly focus on constructing mappings of sign language glosses to frame-level sign pose representations, while neglecting gloss is just a weak annotation of the sequence of sign poses. To address this problem, this paper proposes the semantic-driven diffusion model with gloss-pose latent spaces alignment (SDD-GPLA) for G2P. G2P is divided into two phases. In the first phase, we design the gloss-pose latent spaces alignment (GPLA) to model the sign pose latent representations with glosses dependency. In the second phase, we propose semantic-driven diffusion (SDD) with supervised pose reconstruction guidance as a mapping between the gloss and sign poses latent features. In addition, we propose the sign pose decoder (Decoderp) to progressively generate high-resolution sign poses from latent sign pose features and to guide the SDD training process. We evaluated SDD-GPLA on a self-collected dataset of Daily Chinese Sign Language (DCSL) and a public dataset called RWTH-Phoenix-Weather-2014T. Compared with the state-of-the-art G2P methods, we obtain at least 22.9% and 2.3% improvement in WER scores on the above two datasets, respectively.

Blood Cell Attribute Classification Algorithm Based on Partial Label Learning

Hematological morphology examinations, essential for diagnosing blood disorders, increasingly utilize deep learning. Blood cell classification, determined by combinations of cell attributes, is complicated by the complex relationships and subtle differences among the attributes, resulting in significant time and cost penalties. This study introduces the Partial Label Learning for Blood Cell Classification (P4BC) strategy, a method that trains neural networks using the blood cell attribute labeling data of weak annotations. Using morphological knowledge, we predefined candidate label sets for the blood cell attributes to blend this knowledge with deep learning. This improves the model’s prediction accuracy and interpretability in classifying attributes. This method effectively combines morphological knowledge with deep learning, an approach we refer to as knowledge alignment. It results in an 8.66% increase in attribute recognition accuracy and a 1.09% improvement in matching predictions to the candidate label sets, compared to the original method. These results confirm our method’s ability to grasp the characteristic information of blood cell attributes, enhancing the model interpretability and achieving knowledge alignment between hematological morphology and deep learning. Our algorithm ensures attribute classification accuracy and shows excellent cell category classification, highlighting its wide application potential and practical value in blood cell category classification.

Mirrored X-Net: Joint classification and contrastive learning for weakly supervised GA segmentation in SD-OCT

Deep learning achieves impressive performance in medical image segmentation, but the training phase requires a large amount of annotated data with precise clinical definitions. Weakly supervised lesion segmentation aims to produce pixel-level masks by learning discriminatory information from weak annotations. In this paper, Mirrored X-Net, a novel segmentation model only supervised by image-level category labels, is proposed to segment the Geographic Atrophy (GA) regions in en-face projection of Spectral-Domain Optical Coherence Tomography (SD-OCT) data. Characterized by the dimension-asymmetric information in SD-OCT images, a novel Anisotropic Downsampling (ADS) is proposed to augment feature shapes. To extract the regions of normal retina for both images with and without lesions, we propose a contrastive learning module to assimilate the deep representation of normal tissue in the SD-OCT images and improve the class-wise difference of deep representation. Based on the contrastive learning module, Anomalous Probability Map (APM) can be obtained to draw the distribution of difference with normal tissue in images. We jointly train the image classification and contrastive learning module, and the final GA segmentation is refined based on the en-face projection of APM. The experimental results on two independent GA datasets demonstrate that the proposed weakly supervised model can produce satisfactory results, and obtain even higher accuracy than fully supervised approaches. The source code is available at https://github.com/maxiao0234/Mirrored-X-Net-pytorch.

Meta-learners for few-shot weakly-supervised medical image segmentation

Most uses of Meta-Learning in visual recognition are very often applied to image classification, with a relative lack of work in other tasks such as segmentation and detection. We propose a new generic Meta-Learning framework for few-shot weakly supervised segmentation in medical imaging domains. The proposed approach includes a meta-training phase that uses a meta-dataset. It is deployed on an out-of-distribution few-shot target task, where a single highly generalizable model, trained via a selective supervised loss function, is used as a predictor. The model can be trained in several distinct ways, such as second-order optimization, metric learning, and late fusion. Some relevant improvements of existing methods that are part of the proposed approach are presented. We conduct a comparative analysis of meta-learners from distinct paradigms adapted to few-shot image segmentation in different sparsely annotated radiological tasks. The imaging modalities include 2D chest, mammographic, and dental X-rays, as well as 2D slices of volumetric tomography and resonance images. Our experiments consider in total 9 meta-learners, 4 backbones, and multiple target organ segmentation tasks. We explore small-data scenarios in radiology with varying weak annotation styles and densities. Our analysis shows that metric-based meta-learning approaches achieve better segmentation results in tasks with smaller domain shifts compared to the meta-training datasets, while some gradient- and fusion-based meta-learners are more generalizable to larger domain shifts. Guidelines learned from the comparative performance assessment of the analyzed methods are summarized to support those readers interested in the field.

Predicting clinical endpoints and visual changes with quality-weighted tissue-based renal histological features.

Two common obstacles limiting the performance of data-driven algorithms in digital histopathology classification tasks are the lack of expert annotations and the narrow diversity of datasets. Multi-instance learning (MIL) can address the former challenge for the analysis of whole slide images (WSI), but performance is often inferior to full supervision. We show that the inclusion of weak annotations can significantly enhance the effectiveness of MIL while keeping the approach scalable. An analysis framework was developed to process periodic acid-Schiff (PAS) and Sirius Red (SR) slides of renal biopsies. The workflow segments tissues into coarse tissue classes. Handcrafted and deep features were extracted from these tissues and combined using a soft attention model to predict several slide-level labels: delayed graft function (DGF), acute tubular injury (ATI), and Remuzzi grade components. A tissue segmentation quality metric was also developed to reduce the adverse impact of poorly segmented instances. The soft attention model was trained using 5-fold cross-validation on a mixed dataset and tested on the QUOD dataset containing PAS and SR biopsies. The average ROC-AUC over different prediction tasks was found to be , significantly higher than using only ResNet50 ( ), only handcrafted features ( ), and the baseline ( ) of state-of-the-art performance. In conjunction with soft attention, weighting tissues by segmentation quality has led to further improvement . Using an intuitive visualisation scheme, we show that our approach may also be used to support clinical decision making as it allows pinpointing individual tissues relevant to the predictions.

All-in-SAM: from Weak Annotation to Pixel-wise Nuclei Segmentation with Prompt-based Finetuning

Abstract The Segment Anything Model (SAM) is a recently proposed prompt-based segmentation model in a generic zero-shot segmentation approach. With the zero-shot segmentation capacity, SAM achieved impressive flexibility and precision on various segmentation tasks. However, the current pipeline requires manual prompts during the inference stage, which is still resource intensive for biomedical image segmentation. In this paper, instead of using prompts during the inference stage, we introduce a pipeline that utilizes the SAM, called all-in-SAM, through the entire AI development workflow (from annotation generation to model finetuning) without requiring manual prompts during the inference stage. Specifically, SAM is first employed to generate pixel-level annotations from weak prompts (e.g., points, bounding box). Then, the pixel-level annotations are used to finetune the SAM segmentation model rather than training from scratch. Our experimental results reveal two key findings: 1) the proposed pipeline surpasses the state-of-the-art methods in a nuclei segmentation task on the public Monuseg dataset, and 2) the utilization of weak and few annotations for SAM finetuning achieves competitive performance compared to using strong pixelwise annotated data.

Basal Cell Carcinoma Diagnosis with Fusion of Deep Learning and Telangiectasia Features.

In recent years, deep learning (DL) has been used extensively and successfully to diagnose different cancers in dermoscopic images. However, most approaches lack clinical inputs supported by dermatologists that could aid in higher accuracy and explainability. To dermatologists, the presence of telangiectasia, or narrow blood vessels that typically appear serpiginous or arborizing, is a critical indicator of basal cell carcinoma (BCC). Exploiting the feature information present in telangiectasia through a combination of DL-based techniques could create a pathway for both, improving DL results as well as aiding dermatologists in BCC diagnosis. This study demonstrates a novel "fusion" technique for BCC vs non-BCC classification using ensemble learning on a combination of (a) handcrafted features from semantically segmented telangiectasia (U-Net-based) and (b) deep learning features generated from whole lesion images (EfficientNet-B5-based). This fusion method achieves a binary classification accuracy of 97.2%, with a 1.3% improvement over the corresponding DL-only model, on a holdout test set of 395 images. An increase of 3.7% in sensitivity, 1.5% in specificity, and 1.5% in precision along with an AUC of 0.99 was also achieved. Metric improvements were demonstrated in three stages: (1) the addition of handcrafted telangiectasia features to deep learning features, (2) including areas near telangiectasia (surround areas), (3) discarding the noisy lower-importance features through feature importance. Another novel approach to feature finding with weak annotations through the examination of the surrounding areas of telangiectasia is offered in this study. The experimental results show state-of-the-art accuracy and precision in the diagnosis of BCC, compared to three benchmark techniques. Further exploration of deep learning techniques for individual dermoscopy feature detection is warranted.

Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Data

Acquiring Weak Annotations for Tumor Localization in Temporal and Volumetric Data

Progressive Frame-Proposal Mining for Weakly Supervised Video Object Detection.

In this paper, we focus on the weakly supervised video object detection problem, where each training video is only tagged with object labels, without any bounding box annotations of objects. To effectively train object detectors from such weakly-annotated videos, we propose a Progressive Frame-Proposal Mining (PFPM) framework by exploiting discriminative proposals in a coarse-to-fine manner. First, we design a flexible Multi-Level Selection (MLS) scheme, with explicit guidance of video tags. By selecting object-relevant frames and mining important proposals from these frames, the proposed MLS can effectively reduce frame redundancy as well as improve proposal effectiveness to boost weakly-supervised detectors. Moreover, we develop a novel Holistic-View Refinement (HVR) scheme, which can globally evaluate importance of proposals among frames, and thus correctly refine pseudo ground truth boxes for training video detectors in a self-supervised manner. Finally, we evaluate the proposed PFPM on a large-scale benchmark for video object detection, on ImageNet VID, under the setting of weak annotations. The experimental results demonstrate that our PFPM significantly outperforms the state-of-the-art weakly-supervised detectors.

An efficient urban flood mapping framework towards disaster response driven by weakly supervised semantic segmentation with decoupled training samples

Despite the proven effectiveness of data-driven deep learning techniques in urban flood mapping, the availability of annotation data remains a critical factor impeding their timeliness in real applications. Recent progress in weakly supervised semantic segmentation (WSSS) presents promising solutions for addressing this limitation. To accomplish prompt and accurate flood mapping in complex urban areas from high-resolution remote sensing images to support disaster management and response, this study makes contributions in three key aspects: weak training data generation, the improvement of WSSS algorithm, and the construction of benchmark datasets. Firstly, we present a novel yet efficient weak training data generation strategy by decoupling the acquisition of positive and negative samples. This strategy enables the rapid generation of block-level weak annotations assisted by pre-flood river data or the segment anything model (SAM) for zero-shot segmentation, thereby alleviating the burden of weak data labeling. Secondly, to enhance the flood detection results in complex urban landscapes based on low-cost weak labels, we design an end-to-end WSSS framework incorporating tree filtering-based structure constraints and a perturbed dual-branch cross self-distillation mechanism. Lastly, to evaluate the performance of the proposed approach, we constructed two large aerial imagery datasets, namely Calgary-Flood and Huston-Flood. These datasets encompass diverse urban land covers and include challenging scenarios with extensive shadows, providing a robust benchmark for evaluating our method against various urban environments. Experimental results demonstrate that our weak data annotation strategy substantially enhances efficiency. Additionally, the proposed WSSS framework exhibits superior performance in comparison to existing state-of-the-art methods, particularly in terms of edge delineation and algorithmic stability. The advancements in weak data annotation strategy, end-to-end model architecture, and benchmark dataset development in this study collectively exploit the potential of the weakly supervised paradigm for rapid flood mapping in urgent situations. The datasets and code will be publicly available at (https://github.com/YJ-He/Flood_Mapping_WSSS).

Predicting OCT biological marker localization from weak annotations

Recent developments in deep learning have shown success in accurately predicting the location of biological markers in Optical Coherence Tomography (OCT) volumes of patients with Age-Related Macular Degeneration (AMD) and Diabetic Retinopathy (DR). We propose a method that automatically locates biological markers to the Early Treatment Diabetic Retinopathy Study (ETDRS) rings, only requiring B-scan-level presence annotations. We trained a neural network using 22,723 OCT B-Scans of 460 eyes (433 patients) with AMD and DR, annotated with slice-level labels for Intraretinal Fluid (IRF) and Subretinal Fluid (SRF). The neural network outputs were mapped into the corresponding ETDRS rings. We incorporated the class annotations and domain knowledge into a loss function to constrain the output with biologically plausible solutions. The method was tested on a set of OCT volumes with 322 eyes (189 patients) with Diabetic Macular Edema, with slice-level SRF and IRF presence annotations for the ETDRS rings. Our method accurately predicted the presence of IRF and SRF in each ETDRS ring, outperforming previous baselines even in the most challenging scenarios. Our model was also successfully applied to en-face marker segmentation and showed consistency within C-scans, despite not incorporating volume information in the training process. We achieved a correlation coefficient of 0.946 for the prediction of the IRF area.