Weakly-supervised Learning Research Articles

Weakly-supervised learning for catheter segmentation in 3D frustum ultrasound.

Accurate and efficient catheter segmentation in 3D ultrasound (US) is essential for ultrasound-guided cardiac interventions. State-of-the-art segmentation algorithms, based on convolutional neural networks (CNNs), suffer from high computational cost and large 3D data size for GPU implementation, which are far from satisfactory for real-time applications. In this paper, we propose a novel approach for efficient catheter segmentation in 3D US. Instead of using Cartesian US, our approach performs catheter segmentation in Frustum US (i.e., the US data before scan conversion). Compared to Cartesian US, Frustum US has a much smaller volume size, therefore the catheter can be segmented more efficiently in Frustum US. However, annotating the irregular and deformed Frustum images is challenging, and it is laborious to obtain the voxel-level annotation. To address this, we propose a weakly supervised learning framework, which requires only bounding-box annotations. The labels of the voxels are generated by incorporating class activation maps with line filtering, which are iteratively updated during the training cycles. Our experimental results show that, compared to Cartesian US, the catheter can be segmented much more efficiently in Frustum US (i.e., 0.25s per volume) with better accuracy. Extensive experiments also validate the effectiveness of the proposed weakly supervised learning method.

Abstract PD11-03: Deep learning applied on resection specimen tissue slides of ‘pure’ ductal carcinoma in situ predicts ipsilateral invasive breast cancer recurrence

Abstract Clinical problem. Ductal Carcinoma in Situ (DCIS) is a potential precursor for invasive breast cancer (IBC). Therefore, DCIS is currently treated with surgical excision, often supplemented with radiotherapy to prevent progression to ipsilateral IBC (iIBC). However, many DCIS lesions will never do so. Estimating the risk of progression is a grand challenge, as neither the histopathological grade of the DCIS lesion nor other biological markers are conclusively associated with the disease outcome. Aim. We aimed to develop a deep-learning based pipeline for estimating the risk of iIBC recurrence following DCIS using a dataset of 235 H&E-stained whole-slide images (WSIs) of the primary DCIS lesions with corresponding 10-year follow up metadata of DCIS recurrence collected at the Netherlands Cancer Institute. The patients included in our dataset did not receive radiotherapy and experienced recurrence in 167 of the cases. Results. We developed a two-step pipeline that is able to find predictive features for 10-year iIBC prediction. First, tissue regions on WSIs were divided into equally sized tiles. Mammary ducts were detected in the tiles using a RetinaNet with ResNet101 backbone that was implemented in Detectron2 and pre-trained on ImageNet. Selecting only tiles containing ducts served to reduce the input dimensionality of typically giga-pixel WSIs for the second step. Here, DCIS recurrence was predicted by a weakly-supervised multi-instance learning (MIL) classification model where the label of the WSI was determined by average weighting of duct labels. The performance of this model was enhanced by pre-training it with SimCLR, a self-supervised learning method, on image data from the histopathology domain. Our proposed model achieved an AUC of .93 ± .005, with a sensitivity of .83 ± .27 and a specificity of .85± .09. These results show that the model was able to correctly distinguish patients with low subsequent IBC-risk from those with a substantially higher risk. An active research pursuit of our group is now to develop a model which is able to predict iIBC progression for patients treated with radiotherapy. This poses additional challenges to the model, as the effect of radiotherapy, as well as disease outcome must be predicted. Conclusion and impact. Our method opens up an avenue for identifying biologically relevant features for estimating DCIS progression risk into invasive breast cancer. This knowledge may be used for appropriately choosing a personalized DCIS management option for patients - which may be active surveillance rather than surgical removal of the lesion. JW and JT were equal senior co-authors on this project. This work was supported by Cancer Research UK and by KWF Dutch Cancer Society (ref.C38317/A24043) Citation Format: Shannon Doyle, Francesco Dal Canton, Timo Koostra, Maartje van Seijen, Emilie Groen, Efstratios Gavves, Hugo Horlings, Esther Lips, Jelle Wesseling, Jonas Teuwen, Grand Challenge PRECISION Consortium. Deep learning applied on resection specimen tissue slides of ‘pure’ ductal carcinoma in situ predicts ipsilateral invasive breast cancer recurrence [abstract]. In: Proceedings of the 2021 San Antonio Breast Cancer Symposium; 2021 Dec 7-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2022;82(4 Suppl):Abstract nr PD11-03.

Deep weakly-supervised breast tumor segmentation in ultrasound images with explicit anatomical constraints

Breast tumor segmentation is an important step in the diagnostic procedure of physicians and computer-aided diagnosis systems. We propose a two-step deep learning framework for breast tumor segmentation in breast ultrasound (BUS) images which requires only a few manual labels. The first step is breast anatomy decomposition handled by a semi-supervised semantic segmentation technique. The input BUS image is decomposed into four breast anatomical structures, namely fat, mammary gland, muscle and thorax layers. Fat and mammary gland layers are used as constrained region to reduce the search space for breast tumor segmentation. The second step is breast tumor segmentation performed in a weakly-supervised learning scenario where only image-level labels are available. Breast tumors are first recognized by a classification network and then segmented by the proposed class activation mapping and deep level set (CAM-DLS) method. For breast anatomy decomposition, the proposed framework achieves Dice similarity coefficient (DSC) of 83.0±11.8%, 84.3±10.0%, 80.7±15.4% and 91.0±11.4% for fat, mammary gland, muscle and thorax layers, respectively. For breast tumor recognition, the proposed framework achieves sensitivity of 95.8%, precision of 92.4%, specificity of 93.9%, accuracy of 94.8% and F1-score of 0.941. For breast tumor segmentation, the proposed framework achieves DSC of 77.3% and intersection-over-union (IoU) of 66.0%. In conclusion, the proposed framework could efficiently perform breast tumor recognition and segmentation simultaneously in a weakly-supervised setting with anatomical constraints.

Batch feature standardization network with triplet loss for weakly-supervised video anomaly detection

Video anomaly detection refers to detecting anomalies automatically without manual labor, which is of great significance to intelligent security. With the emergence of weakly-supervised learning, the performance of video anomaly detection has been greatly advanced. However, the abnormal frames and their adjacent normal frames often make slight differences, increasing the difficulty and complexity of video anomaly detection. To address this problem, we propose a batch feature standardization module using a special standardization approach to facilitate the identification of obscure abnormal events. Meanwhile, we propose a novel strategy to refine the anomaly degree to classify the anomalous videos into two categories, i.e., weak anomalies and strong anomalies. Then the triplet loss is utilized to further improve the discriminative power of the model. Extensive experiments results demonstrate that our method works well on two benchmark datasets, and obtains a frame-level AUC 97.65% on ShanghaiTech and 84.29% on UCF-Crime, achieving comparable performance with the recent state-of-the-art methods.

Weakly-Supervised Salient Object Detection on Light Fields.

Most existing salient object detection (SOD) methods are designed for RGB images and do not take advantage of the abundant information provided by light fields. Hence, they may fail to detect salient objects of complex structures and delineate their boundaries. Although some methods have explored multi-view information of light field images for saliency detection, they require tedious pixel-level manual annotations of ground truths. In this paper, we propose a novel weakly-supervised learning framework for salient object detection on light field images based on bounding box annotations. Our method has two major novelties. First, given an input light field image and a bounding-box annotation indicating the salient object, we propose a ground truth label hallucination method to generate a pixel-level pseudo saliency map, to avoid heavy cost of pixel-level annotations. This method generates high quality pseudo ground truth saliency maps to help supervise the training, by exploiting information obtained from the light field (including depths and RGB images). Second, to exploit the multi-view nature of the light field data in learning, we propose a fusion attention module to calibrate the spatial and channel-wise light field representations. It learns to focus on informative features and suppress redundant information from the multi-view inputs. Based on these two novelties, we are able to train a new salient object detector with two branches in a weakly-supervised manner. While the RGB branch focuses on modeling the color contrast in the all-in-focus image for locating the salient objects, the Focal branch exploits the depth and the background spatial redundancy of focal slices for eliminating background distractions. Extensive experiments show that our method outperforms existing weakly-supervised methods and most fully supervised methods.

Global-Guided Weakly-Supervised Learning for Multi-Label Image Classification

Global-Guided Weakly-Supervised Learning for Multi-Label Image Classification

Mixed-Supervised Scene Text Detection With Expectation-Maximization Algorithm.

Scene text detection is an important and challenging task in computer vision. For detecting arbitrarily-shaped texts, most existing methods require heavy data labeling efforts to produce polygon-level text region labels for supervised training. In order to reduce the cost in data labeling, we study mixed-supervised arbitrarily-shaped text detection by combining various weak supervision forms (e.g., image-level tags, coarse, loose and tight bounding boxes), which are far easier to annotate. Whereas the existing weakly-supervised learning methods (such as multiple instance learning) do not promote full object coverage, to approximate the performance of fully-supervised detection, we propose an Expectation-Maximization (EM) based mixed-supervised learning framework to train scene text detector using only a small amount of polygon-level annotated data combined with a large amount of weakly annotated data. The polygon-level labels are treated as latent variables and recovered from the weak labels by the EM algorithm. A new contour-based scene text detector is also proposed to facilitate the use of weak labels in our mixed-supervised learning framework. Extensive experiments on six scene text benchmarks show that (1) using only 10% strongly annotated data and 90% weakly annotated data, our method yields comparable performance to that of fully supervised methods, (2) with 100% strongly annotated data, our method achieves state-of-the-art performance on five scene text benchmarks (CTW1500, Total-Text, ICDAR-ArT, MSRA-TD500, and C-SVT), and competitive results on the ICDAR2015 Dataset. We will make our weakly annotated datasets publicly available.

Global-Guided Weakly-Supervised Learning for Multi-Label Image Classification

Global-Guided Weakly-Supervised Learning for Multi-Label Image Classification

Weakly Supervised Sensitive Heatmap framework to classify and localize diabetic retinopathy lesions

Vision loss happens due to diabetic retinopathy (DR) in severe stages. Thus, an automatic detection method applied to diagnose DR in an earlier phase may help medical doctors to make better decisions. DR is considered one of the main risks, leading to blindness. Computer-Aided Diagnosis systems play an essential role in detecting features in fundus images. Fundus images may include blood vessels, exudates, micro-aneurysm, hemorrhages, and neovascularization. In this paper, our model combines automatic detection for the diabetic retinopathy classification with localization methods depending on weakly-supervised learning. The model has four stages; in stage one, various preprocessing techniques are applied to smooth the data set. In stage two, the network had gotten deeply to the optic disk segment for eliminating any exudate's false prediction because the exudates had the same color pixel as the optic disk. In stage three, the network is fed through training data to classify each label. Finally, the layers of the convolution neural network are re-edited, and used to localize the impact of DR on the patient's eye. The framework tackles the matching technique between two essential concepts where the classification problem depends on the supervised learning method. While the localization problem was obtained by the weakly supervised method. An additional layer known as weakly supervised sensitive heat map (WSSH) was added to detect the ROI of the lesion at a test accuracy of 98.65%, while comparing with Class Activation Map that involved weakly supervised technology achieved 0.954. The main purpose is to learn a representation that collect the central localization of discriminative features in a retina image. CNN-WSSH model is able to highlight decisive features in a single forward pass for getting the best detection of lesions.

Federated learning for computational pathology on gigapixel whole slide images.

Deep Learning-based computational pathology algorithms have demonstrated profound ability to excel in a wide array of tasks that range from characterization of well known morphological phenotypes to predicting non human-identifiable features from histology such as molecular alterations. However, the development of robust, adaptable and accurate deep learning-based models often rely on the collection and time-costly curation large high-quality annotated training data that should ideally come from diverse sources and patient populations to cater for the heterogeneity that exists in such datasets. Multi-centric and collaborative integration of medical data across multiple institutions can naturally help overcome this challenge and boost the model performance but is limited by privacy concerns among other difficulties that may arise in the complex data sharing process as models scale towards using hundreds of thousands of gigapixel whole slide images. In this paper, we introduce privacy-preserving federated learning for gigapixel whole slide images in computational pathology using weakly-supervised attention multiple instance learning and differential privacy. We evaluated our approach on two different diagnostic problems using thousands of histology whole slide images with only slide-level labels. Additionally, we present a weakly-supervised learning framework for survival prediction and patient stratification from whole slide images and demonstrate its effectiveness in a federated setting. Our results show that using federated learning, we can effectively develop accurate weakly-supervised deep learning models from distributed data silos without direct data sharing and its associated complexities, while also preserving differential privacy using randomized noise generation. We also make available an easy-to-use federated learning for computational pathology software package: http://github.com/mahmoodlab/HistoFL.

Multi-complementary and unlabeled learning for arbitrary losses and models

A weakly-supervised learning framework named as complementary-label learning has been proposed recently, where each sample is equipped with a single complementary label that denotes one of the classes the sample does not belong to. However, the existing complementary-label learning methods cannot learn from the easily accessible unlabeled samples and samples with multiple complementary labels, which are more informative. In this paper, to remove these limitations, we propose the novel multi-complementary and unlabeled learning framework that allows unbiased estimation of classification risk from samples with any number of complementary labels and unlabeled samples, for arbitrary loss functions and models. We first give an unbiased estimator of the classification risk from samples with multiple complementary labels, and then further improve the estimator by incorporating unlabeled samples into the risk formulation. The estimation error bounds show that the proposed methods are in the optimal parametric convergence rate. We also propose a risk correction scheme for alleviating over-fitting caused by negative empirical risk. Finally, the experiments on both linear and deep models show the effectiveness of our proposed methods.

Breast Invasive Ductal Carcinoma Classification on Whole Slide Images with Weakly-Supervised and Transfer Learning.

Simple SummaryIn this study, we have trained deep learning models using transfer learning and weakly-supervised learning for the classification of breast invasive ductal carcinoma (IDC) in whole slide images (WSIs). We evaluated the models on four test sets: one biopsy (n = 522) and three surgical (n = 1129) achieving AUCs in the range 0.95 to 0.99. We have also compared the trained models to existing pre-trained models on different organs for adenocarcinoma classification and they have achieved lower AUC performances in the range 0.66 to 0.89 despite adenocarcinoma exhibiting some structural similarity to IDC. Therefore, performing fine-tuning on the breast IDC training set was beneficial for improving performance. The results demonstrate the potential use of such models to aid pathologists in clinical practice.Invasive ductal carcinoma (IDC) is the most common form of breast cancer. For the non-operative diagnosis of breast carcinoma, core needle biopsy has been widely used in recent years for the evaluation of histopathological features, as it can provide a definitive diagnosis between IDC and benign lesion (e.g., fibroadenoma), and it is cost effective. Due to its widespread use, it could potentially benefit from the use of AI-based tools to aid pathologists in their pathological diagnosis workflows. In this paper, we trained invasive ductal carcinoma (IDC) whole slide image (WSI) classification models using transfer learning and weakly-supervised learning. We evaluated the models on a core needle biopsy (n = 522) test set as well as three surgical test sets (n = 1129) obtaining ROC AUCs in the range of 0.95–0.98. The promising results demonstrate the potential of applying such models as diagnostic aid tools for pathologists in clinical practice.

教師なし・半教師あり・弱教師あり学習の最先端とバイオ医療画像応用

教師なし・半教師あり・弱教師あり学習の最先端とバイオ医療画像応用

Automatic Modulation Recognition Based on Adaptive Attention Mechanism and ResNeXt WSL Model

Automatic modulation recognition (AMR) plays an important role in modern wireless communication. In this letter, a novel framework for AMR is proposed. The ResNeXt network serves as the backbone, and four proposed adaptive attention mechanism modules are incorporated. The time-frequency representations of the received signals are utilized as the inputs of the proposed deep learning (DL) network, and a transfer learning strategy is adopted based on the pre-trained ResNeXt weakly supervised learning (WSL) model. The comparisons with several state-of-the-art techniques on the RadioML2016.10B and RadioML2018.01A datasets show that the proposed framework converges quickly and can achieve higher robustness and 2% to 8% higher accuracy than other state-of-the-art techniques.

AAWS-Net: Anatomy-aware weakly-supervised learning network for breast mass segmentation.

Accurate segmentation of breast masses is an essential step in computer aided diagnosis of breast cancer. The scarcity of annotated training data greatly hinders the model’s generalization ability, especially for the deep learning based methods. However, high-quality image-level annotations are time-consuming and cumbersome in medical image analysis scenarios. In addition, a large amount of weak annotations is under-utilized which comprise common anatomy features. To this end, inspired by teacher-student networks, we propose an Anatomy-Aware Weakly-Supervised learning Network (AAWS-Net) for extracting useful information from mammograms with weak annotations for efficient and accurate breast mass segmentation. Specifically, we adopt a weakly-supervised learning strategy in the Teacher to extract anatomy structure from mammograms with weak annotations by reconstructing the original image. Besides, knowledge distillation is used to suggest morphological differences between benign and malignant masses. Moreover, the prior knowledge learned from the Teacher is introduced to the Student in an end-to-end way, which improves the ability of the student network to locate and segment masses. Experiments on CBIS-DDSM have shown that our method yields promising performance compared with state-of-the-art alternative models for breast mass segmentation in terms of segmentation accuracy and IoU.

Weakly-supervised learning for community detection based on graph convolution in attributed networks

Weakly-supervised learning for community detection based on graph convolution in attributed networks

Weakly-supervised contrastive learning in path manifold for Monte Carlo image reconstruction

Weakly-supervised contrastive learning in path manifold for Monte Carlo image reconstruction

Weakly-supervised contrastive learning in path manifold for Monte Carlo image reconstruction

Image-space auxiliary features such as surface normal have significantly contributed to the recent success of Monte Carlo (MC) reconstruction networks. However, path-space features, another essential piece of light propagation, have not yet been sufficiently explored. Due to the curse of dimensionality, information flow between a regression loss and high-dimensional path-space features is sparse, leading to difficult training and inefficient usage of path-space features in a typical reconstruction framework. This paper introduces a contrastive manifold learning framework to utilize path-space features effectively. The proposed framework employs weakly-supervised learning that converts reference pixel colors to dense pseudo labels for light paths. A convolutional path-embedding network then induces a low-dimensional manifold of paths by iteratively clustering intra-class embeddings, while discriminating inter-class embeddings using gradient descent. The proposed framework facilitates path-space exploration of reconstruction networks by extracting low-dimensional yet meaningful embeddings within the features. We apply our framework to the recent image- and sample-space models and demonstrate considerable improvements, especially on the sample space. The source code is available at https://github.com/Mephisto405/WCMC.

Data augmentation for ML-driven data preparation and integration

In recent years, we have witnessed the development of novel data augmentation (DA) techniques for creating additional training data needed by machine learning based solutions. In this tutorial, we will provide a comprehensive overview of techniques developed by the data management community for data preparation and data integration. In addition to surveying task-specific DA operators that leverage rules, transformations, and external knowledge for creating additional training data, we also explore the advanced DA techniques such as interpolation, conditional generation, and DA policy learning. Finally, we describe the connection between DA and other machine learning paradigms such as active learning, pre-training, and weakly-supervised learning. We hope that this discussion can shed light on future research directions for a holistic data augmentation framework for high-quality dataset creation.

Weakly supervised change detection using guided anisotropic diffusion

Large scale datasets created from crowdsourced labels or openly available data have become crucial to provide training data for large scale learning algorithms. While these datasets are easier to acquire, the data are frequently noisy and unreliable, which is motivating research on weakly supervised learning techniques. In this paper we propose original ideas that help us to leverage such datasets in the context of change detection. First, we propose the guided anisotropic diffusion (GAD) algorithm, which improves semantic segmentation results using the input images as guides to perform edge preserving filtering. We then show its potential in two weakly-supervised learning strategies tailored for change detection. The first strategy is an iterative learning method that combines model optimisation and data cleansing using GAD to extract the useful information from a large scale change detection dataset generated from open vector data. The second one incorporates GAD within a novel spatial attention layer that increases the accuracy of weakly supervised networks trained to perform pixel-level predictions from image-level labels. Improvements with respect to state-of-the-art are demonstrated on 4 different public datasets.