Strong Labels Research Articles

Playing for 3D Human Recovery.

Image- and video-based 3D human recovery (i.e., pose and shape estimation) have achieved substantial progress. However, due to the prohibitive cost of motion capture, existing datasets are often limited in scale and diversity. In this work, we obtain massive human sequences by playing the video game with automatically annotated 3D ground truths. Specifically, we contribute GTA-Human, a large-scale 3D human dataset generated with the GTA-V game engine, featuring a highly diverse set of subjects, actions, and scenarios. More importantly, we study the use of game-playing data and obtain five major insights. First, game-playing data is surprisingly effective. A simple frame-based baseline trained on GTA-Human outperforms more sophisticated methods by a large margin. For video-based methods, GTA-Human is even on par with the in-domain training set. Second, we discover that synthetic data provides critical complements to the real data that is typically collected indoor. We highlight that our investigation into domain gap provides explanations for our data mixture strategies that are simple yet useful, which offers new insights to the research community. Third, the scale of the dataset matters. The performance boost is closely related to the additional data available. A systematic study on multiple key factors (such as camera angle and body pose) reveals that the model performance is sensitive to data density. Fourth, the effectiveness of GTA-Human is also attributed to the rich collection of strong supervision labels (SMPL parameters), which are otherwise expensive to acquire in real datasets. Fifth, the benefits of synthetic data extend to larger models such as deeper convolutional neural networks (CNNs) and Transformers, for which a significant impact is also observed. We hope our work could pave the way for scaling up 3D human recovery to the real world.

Achieving More with Less: Combining Strong and Weak Labels for Intracranial Hemorrhage Detection.

Achieving More with Less: Combining Strong and Weak Labels for Intracranial Hemorrhage Detection.

Beyond strong labels: Weakly-supervised learning based on Gaussian pseudo labels for the segmentation of ellipse-like vascular structures in non-contrast CTs

Deep learning-based automated segmentation of vascular structures in preoperative CT angiography (CTA) images contributes to computer-assisted diagnosis and interventions. While CTA is the common standard, non-contrast CT imaging has the advantage of avoiding complications associated with contrast agents. However, the challenges of labor-intensive labeling and high labeling variability due to the ambiguity of vascular boundaries hinder conventional strong-label-based, fully-supervised learning in non-contrast CTs. This paper introduces a novel weakly-supervised framework using the elliptical topology nature of vascular structures in CT slices. It includes an efficient annotation process based on our proposed standards, an approach of generating 2D Gaussian heatmaps serving as pseudo labels, and a training process through a combination of voxel reconstruction loss and distribution loss with the pseudo labels. We assess the effectiveness of the proposed method on one local and two public datasets comprising non-contrast CT scans, particularly focusing on the abdominal aorta. On the local dataset, our weakly-supervised learning approach based on pseudo labels outperforms strong-label-based fully-supervised learning (1.54% of Dice score on average), reducing labeling time by around 82.0%. The efficiency in generating pseudo labels allows the inclusion of label-agnostic external data in the training set, leading to an additional improvement in performance (2.74% of Dice score on average) with a reduction of 66.3% labeling time, where the labeling time remains considerably less than that of strong labels. On the public dataset, the pseudo labels achieve an overall improvement of 1.95% in Dice score for 2D models with a reduction of 68% of the Hausdorff distance for 3D model.

Predicting Perceived Gloss: Do Weak Labels Suffice?

AbstractEstimating perceptual attributes of materials directly from images is a challenging task due to their complex, not fully‐understood interactions with external factors, such as geometry and lighting. Supervised deep learning models have recently been shown to outperform traditional approaches, but rely on large datasets of human‐annotated images for accurate perception predictions. Obtaining reliable annotations is a costly endeavor, aggravated by the limited ability of these models to generalise to different aspects of appearance. In this work, we show how a much smaller set of human annotations (“strong labels”) can be effectively augmented with automatically derived “weak labels” in the context of learning a low‐dimensional image‐computable gloss metric. We evaluate three alternative weak labels for predicting human gloss perception from limited annotated data. Incorporating weak labels enhances our gloss prediction beyond the current state of the art. Moreover, it enables a substantial reduction in human annotation costs without sacrificing accuracy, whether working with rendered images or real photographs.

More than magnetic isolation: Dynabeads as strong Raman reporters toward simultaneous capture and identification of targets

AbstractDynabeads are superparamagnetic particles used for the immunomagnetic purification of cells and biomolecules. Post‐capture, however, target identification relies on tedious culturing, fluorescence staining, and/or target amplification. Raman spectroscopy presents a rapid detection alternative, but current implementations target cells themselves with weak Raman signals. We present antibody‐coated Dynabeads as strong Raman reporter labels whose effect can be considered a Raman parallel of immunofluorescent probes. Recent developments in techniques for separating target‐bound Dynabeads from unbound Dynabeads make such an implementation feasible with high specificity. We deploy Dynabeads anti‐Salmonella to bind and identify Salmonella enterica, a major foodborne pathogen. Dynabeads present major peaks around 1000 and 1600 cm−1 from aliphatic and aromatic C‐C stretching of the polystyrene coating and near 1350 cm−1 from the ɣ‐Fe2O3 and Fe3O4 core, confirmed with electron dispersive X‐ray (EDX) imaging. Minor to no contributions are made from the surface antibodies themselves as confirmed by Raman analysis of surface‐activated, antibody‐free beads. Dynabeads' Raman signature can be measured in dry and liquid samples even at single shot ~30 × 30 μm area imaging using 0.5 s, 7‐mW laser acquisition with single and clustered beads providing a 44‐ and 68‐fold larger Raman intensity compared with the signature from cells. Higher polystyrene and iron oxide content in clusters yields larger signal intensity, and conjugation to bacteria strengthens clustering as a bacterium can bind to more than one bead as observed via transmission electron microscopy (TEM). Our findings shed light on the intrinsic Raman reporter nature of Dynabeads. When combined with emerging techniques for the separation of target‐bound Dynabeads from unbound Dynabeads such as using centrifugation through a density media bilayer, they have the potential to demonstrate their dual function for target isolation and detection without tedious staining steps or unique plasmonic substrate engineering, advancing their applications in heterogeneous samples like food, water, and blood.

Weakly-Interactive-Mixed Learning: Less Labelling Cost for Better Medical Image Segmentation.

Common medical image segmentation tasks require large training datasets with pixel-level annotations which are very expensive and time-consuming to prepare. To overcome such limitation and achieve the desired segmentation accuracy, a novel Weakly-Interactive-Mixed Learning (WIML) framework is proposed by efficiently using weak labels. On one hand, utilize weak labels to reduce annotation time for high-quality strong labels by designing a Weakly-Interactive Annotation (WIA) part of the WIML which prudently introduces interactive learning into the weakly-supervised segmentation strategy. On the other hand, utilize weak labels and very few strong labels to achieve desired segmentation accuracy by designing a Mixed-Supervised Learning (MSL) part of the WIML which can boost the segmentation accuracy by providing strong prior knowledge during training. Besides, a multi-task Full-Parameter-Sharing Network (FPSNet) is proposed to better implement this framework. Specifically, to further reduce annotation time, attention modules (scSE) are integrated into FPSNet to improve the class activation map (CAM) performance for the first time. To further improve segmentation accuracy, a Full-Parameter-Sharing (FPS) strategy is designed in FPSNet to alleviate the overfitting of the segmentation task supervised by very few strong labels. The proposed method is validated on the BraTS 2019 and LiTS 2017 datasets, and experiments demonstrate that the proposed method WIML-FPSNet outperforms several state-of-the-art segmentation methods with minimal annotation efforts.

FedMix: Mixed Supervised Federated Learning for Medical Image Segmentation.

The purpose of federated learning is to enable multiple clients to jointly train a machine learning model without sharing data. However, the existing methods for training an image segmentation model have been based on an unrealistic assumption that the training set for each local client is annotated in a similar fashion and thus follows the same image supervision level. To relax this assumption, in this work, we propose a label-agnostic unified federated learning framework, named FedMix, for medical image segmentation based on mixed image labels. In FedMix, each client updates the federated model by integrating and effectively making use of all available labeled data ranging from strong pixel-level labels, weak bounding box labels, to weakest image-level class labels. Based on these local models, we further propose an adaptive weight assignment procedure across local clients, where each client learns an aggregation weight during the global model update. Compared to the existing methods, FedMix not only breaks through the constraint of a single level of image supervision but also can dynamically adjust the aggregation weight of each local client, achieving rich yet discriminative feature representations. Experimental results on multiple publicly-available datasets validate that the proposed FedMix outperforms the state-of-the-art methods by a large margin. In addition, we demonstrate through experiments that FedMix is extendable to multi-class medical image segmentation and much more feasible in clinical scenarios. The code is available at: https://github.com/Jwicaksana/FedMix.

Deep labeller: automatic bounding box generation for synthetic violence detection datasets

Manually labelling datasets for training violence detection systems is time-consuming, expensive, and labor-intensive. Mind wandering, boredom, and short attention span can also cause labelling errors. Moreover, collecting and distributing sensitive images containing violence has ethical implications. Automation is the future for labelling sensitive image datasets. Deep labeller is a two-stage Deep Learning (DL) method that uses pre-trained DL object detection methods on MS-COCO for automatic labelling. The Deep Labeller method labels violent and nonviolent images in WVD and USI. In stage 1, WVD generates weak labels using synthetic images. In stage 2, the Deep labeller method is retrained on weak labels. USI dataset is used to test our method on real-world violence. Deep labeller generated weak and strong labels with an IoU of 0.80036 in stage 1 and 0.95 in stage 2 on the WVD. Automatically generated labels. To test our method’s generalisation power, violent and nonviolent image labels on USI dataset had a mean IoU of 0.7450.

Plasmon-enhanced electroreduction activity of Au-AgPd Janus nanoparticles for ochratoxin a detection

Ochratoxin A (OTA) was a dangerous biological toxin, and would easily contaminate food and induced food safety problems. The development of electrochemical aptasensors by designing strong and anti-interfere electroactive labels could improve the sensitivity and accuracy of OTA detection. In this contribution, novel electroactive Au-AgPd Janus NPs were firstly synthesized and exhibited electroreduction signal at −0.4 V, owing to the reduction process of Pd2+. The electroreduction signal was amplified 1.5 times under local surface plasmon resonance (LSPR) excitation, which could improve the sensitivity of OTA detection. Plasmon-enhanced electroreduction principle of Au-AgPd Janus NPs was verified, which endowed electrochemical aptasensor with high accuracy and anti-interference ability for OTA detection. Au-AgPd Janus NPs served as electrochemical beacon achieved sensitive and accurate OTA detection with the limit of detection (LOD) of 0.98 pM. This work opens up new directions for the construction of electroactive heterostructures for the sensitive and accurate biotoxins electroanalytical applications.

Strong Labeling of Sound Events Using Crowdsourced Weak Labels and Annotator Competence Estimation

Crowdsourcing is a popular tool for collecting large amounts of annotated data, but the specific format of the strong labels necessary for sound event detection is not easily obtainable through crowdsourcing. In this work, we propose a novel annotation workflow that leverages the efficiency of crowdsourcing weak labels, and uses a high number of annotators to produce reliable and objective strong labels. The weak labels are collected in a highly redundant setup, to allow reconstruction of the temporal information. To obtain reliable labels, the annotators' competence is estimated using MACE (Multi-Annotator Competence Estimation) and incorporated into the strong labels estimation through weighing of individual opinions. We show that the proposed method produces consistently reliable strong annotations not only for synthetic audio mixtures, but also for audio recordings of real everyday environments. While only a maximum 80% coincidence with the complete and correct reference annotations was obtained for synthetic data, these results are explained by an extended study of how polyphony and SNR levels affect the identification rate of the sound events by the annotators. On real data, even though the estimated annotators' competence is significantly lower and the coincidence with reference labels is under 69%, the proposed majority opinion approach produces reliable aggregated strong labels in comparison with the more difficult task of crowdsourcing directly strong labels.

Reliable Sound Re-labeling Approach for Bird Sound Classification

Passive acoustic monitoring has become an effective and economically scalable solution for wildlife monitoring in recent years, especially for population monitoring of endangered birds in hardly accessible and high-elevation areas. As sound recorders are deployed on the field for extended periods (months), continuous sound streams are often complex in nature, with noises and intermittent signals. Generally, it is prohibitively expensive to label bird sounds with the exact onset and offset time, thus, during training, the data is often provided with only presence/absence labeling (weak labeling) that states which bird species are present in each long recording without temporal information. During test time, it is, however, desirable to provide fine-grained detection in short audio segments. To bridge the gap between the difference of weak labels used for training and strong labels used for testing, we propose a re-labeling approach with two stages: (1) we train a model with weak labeling; and (2) using the model obtained from stage 1, we generate labels for short audio segments and retrain the model on short audio segments with the newly generated labels. We applied our approach in both classification and sound event detection and achieved consistently good performance across multiple random seeds. In BirdCLEF 2022, our model ranked in the top 1.1%, the 9th best entry out of the total 807 entries in the competition.

Multilabel Appliance Classification With Weakly Labeled Data for Non-Intrusive Load Monitoring

Non-Intrusive Load Monitoring consists in estimating the power consumption or the states of the appliances using electrical parameters acquired from a single metering point. State-of-the-art approaches are based on deep neural networks, and for training, they require a significant amount of data annotated at the sample level, defined as strong labels. This paper presents an appliance classification method based on a Convolutional Recurrent Neural Network trained with weak supervision. Learning is formulated as a Multiple-Instance Learning problem, and the network is trained on labels provided for an entire segment of the aggregate power, defined as weak labels. Weak labels are coarser annotations that are intrinsically less costly to obtain compared to strong labels. An extensive experimental evaluation has been conducted on the UK-DALE and REFIT datasets comparing the proposed approach to three benchmark methods. The results obtained for different amounts of strongly and weakly labeled data and mixing UK-DALE and REFIT confirm the effectiveness of weak labels compared to fully supervised and semi-supervised benchmarks methods.

Superpixel Consistency Saliency Map Generation for Weakly Supervised Semantic Segmentation of Remote Sensing Images

The weakly supervised semantic segmentation (WSSS) method aims to assign semantic labels to each image pixel from weak (image-level) instead of strong (pixel-level) labels, which can greatly reduce human labor costs. However, there are some problems in WSSS of remote sensing images such as how to locate labels accurately, and how to get precise segmentation edges. To address these issues, we propose a novel framework directly transferring the scene classification model to perform semantic segmentation. We first train a multi-label scene classification network as the encoder to obtain the pre-trained model, then the feature learned by the model is transferred to the decoder. Different from other methods, we propose a saliency map generator instead of the Class Activation Map for more accurate location information by making pixels belonging to the same class lie close together while different classes are separated in feature space. Meanwhile, we take the superpixel patch as processing unit to provide precise boundary inhibition for the saliency map. To assign semantic labels for each patch, combined with extracted salient region, we propose a module responsible for exploiting the consistency of spatial and semantic similarity between different patches. Finally, we incorporate the above two modules to supervise the training process of the decoder without generating pseudo labels as most methods do, thus simplifying the training process. Experimental results show that our method outperforms other weakly supervised approaches on DLRSD and WHDLD datasets with at least a 3% improvement on mean intersection over union.

Learning to detect radio frequency interference in radio astronomy without seeing it

ABSTRACT Radio frequency interference (RFI) corrupts astronomical measurements, thus affecting the performance of radio telescopes. To address this problem, supervised-segmentation models have been proposed as candidate solutions to RFI detection. However, the unavailability of large labelled data sets, due to the prohibitive cost of annotating, makes these solutions unusable. To solve these shortcomings, we focus on the inverse problem: training models on only uncontaminated emissions, thereby learning to discriminate RFI from all known astronomical signals and system noise. We use nearest latent neighbours – an algorithm that utilizes both the reconstructions and latent distances to the nearest neighbours in the latent space of generative autoencoding models for novelty detection. The uncontaminated regions are selected using weak labels in the form of RFI flags (generated by classical RFI flagging methods) available from most radio astronomical data archives at no additional cost. We evaluate performance on two independent data sets, one simulated from the Hydrogen Epoch of Reionization Array (HERA) telescope and the other consisting of real observations from the Low-Frequency Array (LOFAR) telescope. Additionally, we provide a small expert-labelled LOFAR data set (i.e. strong labels) for evaluation of our and other methods. Performance is measured using the area under the receiver operating characteristic (AUROC), area under precision–recall curve (AUPRC), and the maximum F1-score for a fixed threshold. For the simulated HERA data set, we outperform the current state of the art across all metrics. For the LOFAR data set, our algorithm offers both a 4 per cent increase in AUROC and AUPRC at the cost of increasing the false negative rate, but without any manual labelling.

Active multitask learning with uncertainty-weighted loss for coronary calcium scoring.

The coronary artery calcification (CAC) score is an independent marker for the risk of cardiovascular events. Automatic methods for quantifying CAC could reduce workload and assist radiologists in clinical decision-making. However, large annotated datasets are needed for training to achieve very good model performance, which is an expensive process and requires expert knowledge. The number of training data required can be reduced in an active learning scenario, which requires only the most informative samples to be labeled. Multitask learning techniques can improve model performance by joint learning of multiple related tasks and extraction of shared informativefeatures. We propose an uncertainty-weighted multitask learning model for coronary calcium scoring in electrocardiogram-gated (ECG-gated), noncontrast-enhanced cardiac calcium scoring CT. The model was trained to solve the two tasks of coronary artery region segmentation (weak labels) and coronary artery calcification segmentation (strong labels) simultaneously in an active learning scenario to improve model performance and reduce the number of samples needed for training. We compared our model with a single-task U-Net and a sequential-task model as well as other state-of-the-art methods. The model was evaluated on 1275 individual patients in three different datasets (DISCHARGE, CADMAN, orCaScore), and the relationship between model performance and various influencing factors (image noise, metal artifacts, motion artifacts, image quality) wasanalyzed. Joint learning of multiclass coronary artery region segmentation and binary coronary calcium segmentation improved calcium scoring performance. Since shared information can be learned from both tasks for complementary purposes, the model reached optimal performance with only 12% of the training data and one-third of the labeling time in an active learning scenario. We identified image noise as one of the most important factors influencing model performance along with anatomical abnormalities and metalartifacts. Our multitask learning approach with uncertainty-weighted loss improves calcium scoring performance by joint learning of shared features and reduces labeling costs when trained in an active learningscenario.

Learning to detect RFI in radio astronomy without seeing it

Radio Frequency Interference (RFI) corrupts astronomical measurements, thus affecting the performance of radio telescopes. To address this problem, supervised segmentation models have been proposed as candidate solutions to RFI detection. However, the unavailability of large labelled datasets, due to the prohibitive cost of annotating, makes these solutions unusable. To solve these shortcomings, we focus on the inverse problem; training models on only uncontaminated emissions thereby learning to discriminate RFI from all known astronomical signals and system noise. We use Nearest-Latent-Neighbours (NLN) - an algorithm that utilises both the reconstructions and latent distances to the nearest-neighbours in the latent space of generative autoencoding models for novelty detection. The uncontaminated regions are selected using weak-labels in the form of RFI flags (generated by classical RFI flagging methods) available from most radio astronomical data archives at no additional cost. We evaluate performance on two independent datasets, one simulated from the HERA telescope and another consisting of real observations from LOFAR telescope. Additionally, we provide a small expert-labelled LOFAR dataset (i.e., strong labels) for evaluation of our and other methods. Performance is measured using AUROC, AUPRC and the maximum F1-score for a fixed threshold. For the simulated data we outperform the current state-of-the-art by approximately 1% in AUROC and 3% in AUPRC for the HERA dataset. Furthermore, our algorithm offers both a 4% increase in AUROC and AUPRC at a cost of a degradation in F1-score performance for the LOFAR dataset, without any manual labelling.

Label-Efficient Hybrid-Supervised Learning for Medical Image Segmentation

Due to the lack of expertise for medical image annotation, the investigation of label-efficient methodology for medical image segmentation becomes a heated topic. Recent progresses focus on the efficient utilization of weak annotations together with few strongly-annotated labels so as to achieve comparable segmentation performance in many unprofessional scenarios. However, these approaches only concentrate on the supervision inconsistency between strongly- and weakly-annotated instances but ignore the instance inconsistency inside the weakly-annotated instances, which inevitably leads to performance degradation. To address this problem, we propose a novel label-efficient hybrid-supervised framework, which considers each weakly-annotated instance individually and learns its weight guided by the gradient direction of the strongly-annotated instances, so that the high-quality prior in the strongly-annotated instances is better exploited and the weakly-annotated instances are depicted more precisely. Specially, our designed dynamic instance indicator (DII) realizes the above objectives, and is adapted to our dynamic co-regularization (DCR) framework further to alleviate the erroneous accumulation from distortions of weak annotations. Extensive experiments on two hybrid-supervised medical segmentation datasets demonstrate that with only 10% strong labels, the proposed framework can leverage the weak labels efficiently and achieve competitive performance against the 100% strong-label supervised scenario.

The intelligent decision-making of copper flotation backbone process based on CK-XGBoost

This study creatively puts forward XGBoosts with correlation-based and knowledge-based function (CK-XGBoost) to design flotation backbone process according to the natural properties of a copper mine. The decision-making of flotation backbone process of copper mine is a multi label problem with high feature dimension and small instance set. The proposed CK-XGBoost selects label-specific features and trains a binary XGBoost for each label, and the information of other labels is used to assist the classifier modeling through correlation-based function and knowledge-based function. The correlation-based function utilizes the Pearson correlation coefficient between labels in training set, so that the predicted values of two strong positive correlation labels are similar and the predicted values of two strong negative correlation labels are contrary. The knowledge-based function utilizes domain knowledge, which is independent of the distribution of training set. The experimental results demonstrate the significantly superiority of the proposed CK-XGBoost in the decision-making of copper flotation backbone process.

Data Cleansing for Salt Dome Dataset With Noise Robust Network on Segmentation Task

Noisy labels seriously degrade the performance of the deep learning models. Especially on segmentation tasks, labels are represented at pixel level, and therefore, it is easier to generate them as noisy labels. In this letter, we aim to cleanse the dataset which contains noisy labels using Kullback–Leibler (KL) divergence algorithm and noise robust loss function. Here, we regard the whole dataset as noisy labels and separate noisy labels into two different parts. One is the strong noisy labels where there are the most noisy labels, which is referred to as incorrect labels, and the other is The weak noisy labels where pixels are incorrect in some parts of the image. The KL algorithm is more efficient in removing the labels which contain the most noise. At the same time, we use noise robust model to remove the labels where pixels are inaccurate only in small parts of the image. Noise robust loss function is robust to noisy labels, and thus, we consider noisy labels when intersection over union (IoU) is lower than a constant threshold despite using noise robust loss function. The effectiveness of our proposed method is assessed using Kaggle’s TGS Salt Identification Challenge dataset. We have demonstrated that using our method, segmentation performance is increased without heavy noisy labels.

Weakly supervised multitask learning models to identify symptom onset time of unclear-onset intracerebral hemorrhage.

Approximately one-third of spontaneous intracerebral hemorrhage patients did not know the onset time and were excluded from studies about time-dependent treatments for hyperacute spontaneous intracerebral hemorrhage. To help clinicians explore the benefit of time-dependent treatments for unclear-onset patients, we presented artificial intelligence models to identify onset time using non-contrast computed tomography (NCCT) based on weakly supervised multitask learning (WS-MTL) structure. The patients with reliable symptom onset time (strong label) or repeat CT (weak label) were included and split into training set and test set (internal and external). The WS-MTL structure utilized strong and weak labels simultaneously to improve performance. The models included three binary classification models for classifying whether NCCT acquired within 6, 8 or 12 h for different treatments measured by area under curve, and a regression model for determining the exact onset time measured by mean absolute error. The generalizability of models was also explored in comprehensive analysis. A total of 4004 patients with 10,780 NCCT scans were included. The performance of WS-MTL classification model showed high accuracy, and that of regression model was satisfactory in ≤6 h subgroup. In comprehensive analysis, the WS-MTL showed better performance for larger hematomas and thinner scans. And the performance improved effectively as training amounts increasing and could be improved steadily through retraining. The WS-MTL models showed good performance and generalizability. Considering the large number of unclear-onset spontaneous intracerebral hemorrhage patients, it may be worth to integrate the WS-MTL model into clinical practice to identify the onset time.