Small Annotated Datasets Research Articles

Deep learning-based image segmentation for instantaneous flame front extraction

This paper delves into the methodology employed in examining lean premixed turbulent flame fronts extracted from Planar Laser Induced Fluorescence (PLIF) images at elevated pressures. In such flow regimes, the PLIF signal suffers from significant collisional quenching, typically resulting in image data with low signal-to-noise ratio (SNR). This poses severe difficulties for conventional flame front extraction algorithms based on intensity gradients and requires intense user intervention to yield acceptable results. In this work, we propose Convolutional Neural Network (CNN)-based Deep Learning (DL) models as an alternative to problem specific conventional methods. The pretrained DL models were fine-tuned, employing data augmentation, on a small annotated dataset including a variety of conditions between SNR ≈\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\approx$$\\end{document} 1.6 to 2.6 and subsequently evaluated. All DL models significantly outperformed the best-scoring conventional implementation both quantitatively and visually, while having similar inference times. IoU-scores and Recall values were found to be up to a factor ≈\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\approx$$\\end{document} 1.2 and ≈\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\approx$$\\end{document} 2.5 higher, respectively, with ≈\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\approx$$\\end{document} 1.15 times improved Precision. Small-scale structures were captured much better with fewer erroneous predictions, becoming particularly pronounced for the lower SNR data investigated. Moreover, by applying artificially modeled noise, it was shown that the range of image conditions in terms of SNR that can be reliably processed extends well beyond the images included in the training data, and satisfactory segmentation performances were found for SNR as low as ≈\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\approx$$\\end{document} 1.1. The presented DL-based flame front detection algorithm marks a methodology with significantly increased detection performance, while a similar computational effort for inference is achieved and the need for user-based parameter tuning is eliminated. It enables a very accurate extraction of instantaneous flame fronts in large image datasets where supervised processing is infeasible, unlocking unprecedented possibilities for the study of flame dynamics and instability mechanisms at industry-relevant conditions.

Image factory: A method for synthesizing novel CT images with anatomical guidance.

Deep learning in medical applications is limited due to the low availability of large labeled, annotated, or segmented training datasets. With the insufficient data available for model training comes the inability of these networks to learn the fine nuances of the space of possible images in a given medical domain, leading to the possible suppression of important diagnostic features hence making these deep learning systems suboptimal in their performance and vulnerable to adversarialattacks. We formulate a framework to address this lack of labeled data problem. We test this formulation in computed tomographic images domain and present an approach that can synthesize large sets of novel CT images at high resolution across the full Hounsfield (HU)range. Our method only requires a small annotated dataset of lung CT from 30 patients (available online at the TCIA) and a large nonannotated dataset with high resolution CT images from 14k patients (received from NIH, not publicly available). It then converts the small annotated dataset into a large annotated dataset, using a sequence of steps including texture learning via StyleGAN, label learning via U-Net and semi-supervised learning via CycleGAN/Pixel-to-Pixel (P2P) architectures. The large annotated dataset so generated can then be used for the training of deep learning networks for medical applications. It can also be put to use for the synthesis of CT images with varied anatomies that were nonexistent within either of the input datasets, enriching the dataset evenfurther. We demonstrate our framework via lung CT-Scan synthesis along with their novel generated annotations and compared it with other state of the art generative models that only produce images without annotations. We evaluate our framework effectiveness via a visual turing test with help of a few doctors and radiologists. We gain the capability of generating an unlimited amount of annotated CT images. Our approach works for all HU windows with minimal depreciation in anatomical plausibility and hence could be used as a general purpose framework for annotated data augmentation for deep learning applications in medical imaging.

Self-supervised pre-training with contrastive and masked autoencoder methods for dealing with small datasets in deep learning for medical imaging

Deep learning in medical imaging has the potential to minimize the risk of diagnostic errors, reduce radiologist workload, and accelerate diagnosis. Training such deep learning models requires large and accurate datasets, with annotations for all training samples. However, in the medical imaging domain, annotated datasets for specific tasks are often small due to the high complexity of annotations, limited access, or the rarity of diseases. To address this challenge, deep learning models can be pre-trained on large image datasets without annotations using methods from the field of self-supervised learning. After pre-training, small annotated datasets are sufficient to fine-tune the models for a specific task. The most popular self-supervised pre-training approaches in medical imaging are based on contrastive learning. However, recent studies in natural image processing indicate a strong potential for masked autoencoder approaches. Our work compares state-of-the-art contrastive learning methods with the recently introduced masked autoencoder approach “SparK” for convolutional neural networks (CNNs) on medical images. Therefore, we pre-train on a large unannotated CT image dataset and fine-tune on several CT classification tasks. Due to the challenge of obtaining sufficient annotated training data in medical imaging, it is of particular interest to evaluate how the self-supervised pre-training methods perform when fine-tuning on small datasets. By experimenting with gradually reducing the training dataset size for fine-tuning, we find that the reduction has different effects depending on the type of pre-training chosen. The SparK pre-training method is more robust to the training dataset size than the contrastive methods. Based on our results, we propose the SparK pre-training for medical imaging tasks with only small annotated datasets.

Topic Classification for Political Texts with Pretrained Language Models

AbstractSupervised topic classification requires labeled data. This often becomes a bottleneck as high-quality labeled data are expensive to acquire. To overcome the data scarcity problem, scholars have recently proposed to use cross-domain topic classification to take advantage of preexisting labeled datasets. Cross-domain topic classification only requires limited annotation in the target domain to verify its cross-domain accuracy. In this letter, we propose supervised topic classification with pretrained language models as an alternative. We show that language models fine-tuned with 70% of the small annotated dataset in the target corpus could outperform models trained using large cross-domain datasets by 27% and that models fine-tuned with 10% of the annotated dataset could already outperform the cross-domain classifiers. Our models are competitive in terms of training time and inference time. Researchers interested in supervised learning with limited labeled data should find our results useful. Our code and data are publicly available.1

Synthetic Generation of 3D Microscopy Images using Generative Adversarial Networks.

Fluorescence microscopy images of cell organelles enable the study of various complex biological processes. Recently, deep learning (DL) models are being used for the accurate automatic analysis of these images. DL models present state-of-the-art performance in many image analysis tasks such as object classification, segmentation and detection. However, to train a DL model a large manually annotated dataset is required. Manual annotation of 3D microscopy images is a time-consuming task and must be performed by specialists in the area. Thus, only a few images with annotations are typically available. Recent advances in generative adversarial networks (GANs) have allowed the translation of images with some conditions into realistic looking synthetic images. Therefore, in this work we explore approaches based on GANs to create synthetic 3D microscopy images. We compare four approaches that differ in the conditions of the input image. The quality of the generated images was assessed visually and using a quantitative objective GAN evaluation metric. The results showed that the GAN is able to generate synthetic images similar to the real ones. Hence, we have presented a method based on GANs to overcome the issue of small annotated datasets in the biomedical imaging field.

A comparison of few-shot and traditional named entity recognition models for medical text.

Many research problems involving medical texts have limited amounts of annotated data available (e.g., expressions of rare diseases). Traditional supervised machine learning algorithms, particularly those based on deep neural networks, require large volumes of annotated data, and they underperform when only small amounts of labeled data are available. Few-shot learning (FSL) is a category of machine learning models that are designed with the intent of solving problems that have small annotated datasets available. However, there is no current study that compares the performances of FSL models with traditional models (e.g., conditional random fields) for medical text at different training set sizes. In this paper, we attempted to fill this gap in research by comparing multiple FSL models with traditional models for the task of named entity recognition (NER) from medical texts. Using five health-related annotated NER datasets, we benchmarked three traditional NER models based on BERT-BERT-Linear Classifier (BLC), BERT-CRF (BC) and SANER; and three FSL NER models-StructShot & NNShot, Few-Shot Slot Tagging (FS-ST) and ProtoNER. Our benchmarking results show that almost all models, whether traditional or FSL, achieve significantly lower performances compared to the state-of-the-art with small amounts of training data. For the NER experiments we executed, the F1-scores were very low with small training sets, typically below 30%. FSL models that were reported to perform well on non-medical texts significantly underperformed, compared to their reported best, on medical texts. Our experiments also suggest that FSL methods tend to perform worse on data sets from noisy sources of medical texts, such as social media (which includes misspellings and colloquial expressions), compared to less noisy sources such as medical literature. Our experiments demonstrate that the current state-of-the-art FSL systems are not yet suitable for effective NER in medical natural language processing tasks, and further research needs to be carried out to improve their performances. Creation of specialized, standardized datasets replicating real-world scenarios may help to move this category of methods forward.

Self-Supervised Human Activity Representation for Embodied Cognition Assessment

Physical activities, according to the embodied cognition theory, are an important manifestation of cognitive functions. As a result, in this paper, the Activate Test of Embodied Cognition (ATEC) system is proposed to assess various cognitive measures. It consists of physical exercises with different variations and difficulty levels designed to provide assessment of executive and motor functions. This work focuses on obtaining human activity representation from recorded videos of ATEC tasks in order to automatically assess embodied cognition performance. A self-supervised approach is employed in this work that can exploit a small set of annotated data to obtain an effective human activity representation. The performance of different self-supervised approaches along with a supervised method are investigated for automated cognitive assessment of children performing ATEC tasks. The results show that the supervised learning approach performance decreases as the training set becomes smaller, whereas the self-supervised methods maintain their performance by taking advantage of unlabeled data.

Combining Self-supervised Learning and Active Learning for Disfluency Detection

Spoken language is fundamentally different from the written language in that it contains frequent disfluencies or parts of an utterance that are corrected by the speaker. Disfluency detection (removing these disfluencies) is desirable to clean the input for use in downstream NLP tasks. Most existing approaches to disfluency detection heavily rely on human-annotated data, which is scarce and expensive to obtain in practice. To tackle the training data bottleneck, in this work, we investigate methods for combining self-supervised learning and active learning for disfluency detection. First, we construct large-scale pseudo training data by randomly adding or deleting words from unlabeled data and propose two self-supervised pre-training tasks: (i) a tagging task to detect the added noisy words and (ii) sentence classification to distinguish original sentences from grammatically incorrect sentences. We then combine these two tasks to jointly pre-train a neural network. The pre-trained neural network is then fine-tuned using human-annotated disfluency detection training data. The self-supervised learning method can capture task-special knowledge for disfluency detection and achieve better performance when fine-tuning on a small annotated dataset compared to other supervised methods. However, limited in that the pseudo training data are generated based on simple heuristics and cannot fully cover all the disfluency patterns, there is still a performance gap compared to the supervised models trained on the full training dataset. We further explore how to bridge the performance gap by integrating active learning during the fine-tuning process. Active learning strives to reduce annotation costs by choosing the most critical examples to label and can address the weakness of self-supervised learning with a small annotated dataset. We show that by combining self-supervised learning with active learning, our model is able to match state-of-the-art performance with just about 10% of the original training data on both the commonly used English Switchboard test set and a set of in-house annotated Chinese data.

A self-supervised learning strategy for postoperative brain cavity segmentation simulating resections

PurposeAccurate segmentation of brain resection cavities (RCs) aids in postoperative analysis and determining follow-up treatment. Convolutional neural networks (CNNs) are the state-of-the-art image segmentation technique, but require large annotated datasets for training. Annotation of 3D medical images is time-consuming, requires highly trained raters and may suffer from high inter-rater variability. Self-supervised learning strategies can leverage unlabeled data for training.MethodsWe developed an algorithm to simulate resections from preoperative magnetic resonance images (MRIs). We performed self-supervised training of a 3D CNN for RC segmentation using our simulation method. We curated EPISURG, a dataset comprising 430 postoperative and 268 preoperative MRIs from 430 refractory epilepsy patients who underwent resective neurosurgery. We fine-tuned our model on three small annotated datasets from different institutions and on the annotated images in EPISURG, comprising 20, 33, 19 and 133 subjects.ResultsThe model trained on data with simulated resections obtained median (interquartile range) Dice score coefficients (DSCs) of 81.7 (16.4), 82.4 (36.4), 74.9 (24.2) and 80.5 (18.7) for each of the four datasets. After fine-tuning, DSCs were 89.2 (13.3), 84.1 (19.8), 80.2 (20.1) and 85.2 (10.8). For comparison, inter-rater agreement between human annotators from our previous study was 84.0 (9.9).ConclusionWe present a self-supervised learning strategy for 3D CNNs using simulated RCs to accurately segment real RCs on postoperative MRI. Our method generalizes well to data from different institutions, pathologies and modalities. Source code, segmentation models and the EPISURG dataset are available at https://github.com/fepegar/resseg-ijcars.

Automated Coding of Under-Studied Medical Concept Domains: Linking Physical Activity Reports to the International Classification of Functioning, Disability, and Health.

Linking clinical narratives to standardized vocabularies and coding systems is a key component of unlocking the information in medical text for analysis. However, many domains of medical concepts, such as functional outcomes and social determinants of health, lack well-developed terminologies that can support effective coding of medical text. We present a framework for developing natural language processing (NLP) technologies for automated coding of medical information in under-studied domains, and demonstrate its applicability through a case study on physical mobility function. Mobility function is a component of many health measures, from post-acute care and surgical outcomes to chronic frailty and disability, and is represented as one domain of human activity in the International Classification of Functioning, Disability, and Health (ICF). However, mobility and other types of functional activity remain under-studied in the medical informatics literature, and neither the ICF nor commonly-used medical terminologies capture functional status terminology in practice. We investigated two data-driven paradigms, classification and candidate selection, to link narrative observations of mobility status to standardized ICF codes, using a dataset of clinical narratives from physical therapy encounters. Recent advances in language modeling and word embedding were used as features for established machine learning models and a novel deep learning approach, achieving a macro-averaged F-1 score of 84% on linking mobility activity reports to ICF codes. Both classification and candidate selection approaches present distinct strengths for automated coding in under-studied domains, and we highlight that the combination of (i) a small annotated data set; (ii) expert definitions of codes of interest; and (iii) a representative text corpus is sufficient to produce high-performing automated coding systems. This research has implications for continued development of language technologies to analyze functional status information, and the ongoing growth of NLP tools for a variety of specialized applications in clinical care and research.

Mimicking the radiologists' workflow: Estimating pediatric hand bone age with stacked deep neural networks.

Pediatric endocrinologists regularly order radiographs of the left hand to estimate the degree of bone maturation in order to assess their patients for advanced or delayed growth, physical development, and to monitor consecutive therapeutic measures. The reading of such images is a labor-intensive task that requires a lot of experience and is normally performed by highly trained experts like pediatric radiologists. In this paper we build an automated system for pediatric bone age estimation that mimics and accelerates the workflow of the radiologist without breaking it. The complete system is based on two neural network based models: on the one hand a detector network, which identifies the ossification areas, on the other hand gender and region specific regression networks, which estimate the bone age from the detected areas. With a small annotated dataset an ossification area detection network can be trained, which is stable enough to work as part of a multi-stage approach. Furthermore, our system achieves competitive results on the RSNA Pediatric Bone Age Challenge test set with an average error of 4.56 months. In contrast to other approaches, especially purely encoder-based architectures, our two-stage approach provides self-explanatory results. By detecting and evaluating the individual ossification areas, thus simulating the workflow of the Tanner-Whitehouse procedure, the results are interpretable for a radiologist.

BSCI-14. SYNTHETIC METASTATIC BRAIN DISEASE MRI IMAGES CREATED USING A GENERATIVE ADVERSARY NETWORK TO OVERCOME DEEP MACHINE LEARNING CHALLENGES IN HEALTHCARE

Deep Machine Learning (DML) in commercial applications such as recognizing animal species in photographs occurred through analyzing large volumes of public data. To achieve similar success in brain tumor imaging, additional factors must be addressed such as the need to follow strict regulatory protocols, work with limited datasets, and protect patient privacy. Generative adversary network (GAN) restricted to intracranial disease is one possibility to overcome these challenges and enable training on small annotated datasets to synthesize new samples. Large fabricated brain metastases (BM) training datasets derived from patient MRI using GAN models may enable DML of BM MRI studies. METHOD: We randomly selected 82 glioma patient imaging studies from the MICCAI BraTS 2018 Challenge1. All patients underwent contouring of GD-enhancing tumor (C+), peritumoral T2 (pT2), necrotic and non-enhancing tumor core (NCR/NET). Images are co-registered to the anatomical template and skull-stripped. Our network consists of a GAN and a discriminative network. The generative model works to synthesize images from labels. Labels comprise the normal brain mask as well as the contoured C+, pT2 and NCR/NET. Normal brain mask is extracted from threshold segmentation on T2-weighted image (T2WI). A discriminative network compares the difference between synthetic and real patient image in both pixel and perceptual difference. The generative model is trained to minimize the difference from the discriminative network. This method was refined in the glioblastoma dataset and applied to BM MRI. RESULTS: Figure 1. Synthetic BM MRI images derived from human brain MRI studies using the GAN model with four modalities (T2, T2 FLAIR, T1 contrasted image, and T1 non-contrasted Image). CONCLUSION: Training DML in BM disease using GAN MRI models may overcome limitations in applying DML to healthcare, namely volume of high-quality data and patient privacy. GAN based modeling for BM needs to be further refined and validated.

A Dual Attention Network with Semantic Embedding for Few-Shot Learning

Despite recent success of deep neural networks, it remains challenging to efficiently learn new visual concepts from limited training data. To address this problem, a prevailing strategy is to build a meta-learner that learns prior knowledge on learning from a small set of annotated data. However, most of existing meta-learning approaches rely on a global representation of images and a meta-learner with complex model structures, which are sensitive to background clutter and difficult to interpret. We propose a novel meta-learning method for few-shot classification based on two simple attention mechanisms: one is a spatial attention to localize relevant object regions and the other is a task attention to select similar training data for label prediction. We implement our method via a dual-attention network and design a semantic-aware meta-learning loss to train the meta-learner network in an end-to-end manner. We validate our model on three few-shot image classification datasets with extensive ablative study, and our approach shows competitive performances over these datasets with fewer parameters. For facilitating the future research, code and data split are available: https://github.com/tonysy/STANet-PyTorch

Ossification area localization in pediatric hand radiographs using deep neural networks for object detection.

BackgroundDetection of ossification areas of hand bones in X-ray images is an important task, e.g. as a preprocessing step in automated bone age estimation. Deep neural networks have emerged recently as de facto standard detection methods, but their drawback is the need of large annotated datasets. Finetuning pre-trained networks is a viable alternative, but it is not clear a priori if training with small annotated datasets will be successful, as it depends on the problem at hand. In this paper, we show that pre-trained networks can be utilized to produce an effective detector of ossification areas in pediatric X-ray images of hands.Methods and findingsA publicly available Faster R-CNN network, pre-trained on the COCO dataset, was utilized and finetuned with 240 manually annotated radiographs from the RSNA Pediatric Bone Age Challenge, which comprises over 14.000 pediatric radiographs. The validation is done on another 89 radiographs from the dataset and the performance is measured by Intersection-over-Union (IoU). To understand the effect of the data size on the pre-trained network, subsampling was applied to the training data and the training was repeated. Additionally, the network was trained from scratch without any pre-trained weights. Finally, to understand whether the trained model could be useful, we compared the inference of the network to an annotation of an expert radiologist. The finetuned network was able to achieve an average precision (mAP@0.5IoU) of 92.92 ± 1.93. Apart from the wrist region, all ossification areas were able to benefit from more data. In contrast, the network trained from scratch was not able to produce any correct results. When compared to the annotations of the expert radiologist, the network was able to localize the regions quite well, as the F1-Score was on average 91.85 ± 1.06.ConclusionsBy finetuning a pre-trained deep neural network, with 240 annotated radiographs, we were able to successfully detect ossification areas in prediatric hand radiographs.

Deep bilinear features for Her2 scoring in digital pathology

Abstract We present an automated approach for rating HER2 over-expressions in given whole-slide images of breast cancer histology slides. The slides have a very high resolution and only a small part of it is relevant for the rating. Our approach is based on Convolutional Neural Networks (CNN), which are directly modelling the whole computer vision pipeline, from feature extraction to classification, with a single parameterized model. CNN models have led to a significant breakthrough in a lot of vision applications and showed promising results for medical tasks. However, the required size of training data is still an issue. Our CNN models are pre-trained on a large set of datasets of non-medical images, which prevents over-fitting to the small annotated dataset available in our case. We assume the selection of the probe in the data with just a single mouse click defining a point of interest. This is reasonable especially for slices acquired together with another sample. We sample image patches around the point of interest and obtain bilinear features by passing them through a CNN and encoding the output of the last convolutional layer with its second-order statistics. Our approach ranked second in the Her2 contest held by the University of Warwick achieving 345 points compared to 348 points of the winning team. In addition to pure classification, our approach would also allow for localization of parts of the slice relevant for visual detection of Her2 over-expression.

AnnoFin–A hybrid algorithm to annotate financial text

Abstract In this work, we study the problem of annotating a large volume of Financial text by learning from a small set of human-annotated training data. The training data is prepared by randomly selecting some text sentences from the large corpus of financial text. Conventionally, bootstrapping algorithm is used to annotate large volume of unlabeled data by learning from a small set of annotated data. However, the small set of annotated data have to be carefully chosen as seed data. Thus, our approach is a digress from the conventional approach of bootstrapping as we let the users randomly select the seed data. We show that our proposed algorithm has an accuracy of 73.56% in classifying the financial texts into the different categories (“Accounting”, “Cost”, “Employee”, “Financing”, “Sales”, “Investments”, “Operations”, “Profit”, “Regulations” and “Irrelevant”) even when the training data is just 30% of the total data set. Additionally, the accuracy improves by an approximate average of 2% for an increase of the training data by 10% and the accuracy of our system is 77.91% when the training data is about 50% of the total data set. As a dictionary of hand chosen keywords prepared by domain experts are often used for financial text extraction, we assumed the existence of almost linearly separable hyperplanes between the different classes and therefore, we have used Linear Support Vector Machine along with a modified version of Label Propagation Algorithm which exploits the notion of neighborhood (in Euclidean space) for classification. We believe that our proposed techniques will be of help to Early Warning Systems used in banks where large volumes of unstructured texts need to be processed for better insights about a company.