Domain-specific Data Augmentation Research Articles

Towards interactive AI-authoring with prototypical few-shot classifiers in histopathology

A vast multitude of tasks in histopathology could potentially benefit from the support of artificial intelligence (AI). Many examples have been shown in the literature and first commercial products with FDA or CE-IVDR clearance are available. However, two key challenges remain: (1) a scarcity of thoroughly annotated images, respectively the laboriousness of this task, and (2) the creation of robust models that can cope with the data heterogeneity in the field (domain generalization). In this work, we investigate how the combination of prototypical few-shot classification models and data augmentation can address both of these challenges. Based on annotated data sets that include multiple centers, multiple scanners, and two tumor entities, we examine the robustness and the adaptability of few-shot classifiers in multiple scenarios. We demonstrate that data from one scanner and one site are sufficient to train robust few-shot classification models by applying domain-specific data augmentation. The models achieved classification performance of around 90% on a multiscanner and multicenter database, which is on par with the accuracy achieved on the primary single-center single-scanner data. Various convolutional neural network (CNN) architectures can be used for feature extraction in the few-shot model. A comparison of nine state-of-the-art architectures yielded that EfficientNet B0 provides the best trade-off between accuracy and inference time. The classification of prototypical few-shot models directly relies on class prototypes derived from example images of each class. Therefore, we investigated the influence of prototypes originating from images from different scanners and evaluated their performance also on the multiscanner database. Again, our few-shot model showed a stable performance with an average absolute deviation in accuracy compared to the primary prototypes of 1.8% points. Finally, we examined the adaptability to a new tumor entity: classification of tissue sections containing urothelial carcinoma into normal, tumor, and necrotic regions. Only three annotations per subclass (e.g., muscle and adipose tissue are subclasses of normal tissue) were provided to adapt the few-shot model, which obtained an overall accuracy of 93.6%. These results demonstrate that prototypical few-shot classification is an ideal technology for realizing an interactive AI authoring system as it only requires few annotations and can be adapted to new tasks without involving retraining of the underlying feature extraction CNN, which would in turn require a selection of hyper-parameters based on data science expert knowledge. Similarly, it can be regarded as a guided annotation system. To this end, we realized a workflow and user interface that targets non-technical users.

Transfer Learning Strategies for Fine-Tuning Pretrained Convolutional Neural Networks in Medical Imaging

In the area of medical imaging, transfer learning has become a potent technique that uses pretrained Convolutional Neural Networks (CNNs) to improve the performance of particular tasks. An overview of several transfer learning techniques used for optimising pretrained CNNs in the context of medical image analysis is given in this abstract. The size limitations of medical imaging datasets make it difficult to train deep learning models from scratch. Pre-trained CNNs are a good place to start, such as those that have been trained on huge natural picture datasets like ImageNet. When these pre-trained models are applied to medical imaging applications, fine-tuning is frequently used. One common method is feature extraction, where the bottom layers of the pretrained CNN are frozen and operate as feature extractors. Then, for the specific medical task at hand, these features are loaded into a bespoke classifier. The ability of the pretrained network to recognise subtle picture patterns is advantageous in this method. Another strategy is to optimise the CNN architecture as a whole, which enables the model to adjust to the features of medical images. Small learning rates are frequently used in transfer learning techniques to avoid overfitting during fine-tuning. Additionally, to further enhance model generalisation, domain-specific data augmentation is essential. The use of ensemble approaches, which combine several pretrained CNNs, is also investigated. These models are capable of offering various feature representations and improving classification precision. In order to bridge the domain gap between natural photos and medical images, domain adaption techniques are also used. One approach to align feature distributions is by adversarial training, while another is through domain-specific batch normalisation. The feature extraction, network fine-tuning, ensemble approaches, and domain adaptation are all part of transfer learning methodologies for optimising pretrained CNNs in medical imaging. Researchers have made great progress using these techniques in a number of medical image processing tasks, proving the value of transfer learning in this important area.

Domain-guided data augmentation for deep learning on medical imaging.

While domain-specific data augmentation can be useful in training neural networks for medical imaging tasks, such techniques have not been widely used to date. Our objective was to test whether domain-specific data augmentation is useful for medical imaging using a well-benchmarked task: view classification on fetal ultrasound FETAL-125 and OB-125 datasets. We found that using a context-preserving cut-paste strategy, we could create valid training data as measured by performance of the resulting trained model on the benchmark test dataset. When used in an online fashion, models trained on this hybrid data performed similarly to those trained using traditional data augmentation (FETAL-125 F-score 85.33 ± 0.24 vs 86.89 ± 0.60, p-value 0.014; OB-125 F-score 74.60 ± 0.11 vs 72.43 ± 0.62, p-value 0.004). Furthermore, the ability to perform augmentations during training time, as well as the ability to apply chosen augmentations equally across data classes, are important considerations in designing a bespoke data augmentation. Finally, we provide open-source code to facilitate running bespoke data augmentations in an online fashion. Taken together, this work expands the ability to design and apply domain-guided data augmentations for medical imaging tasks.

ReFixMatch-LS: reusing pseudo-labels for semi-supervised skin lesion classification.

Recent research on semi-supervised learning (SSL) is mainly based on the method of consistency regularization, which relies on domain-specific data augmentation. Pseudo-labeling is a more general method that has no such restrictions but performs limited by noisy training. We combine both approaches and focus on generating pseudo-labels using domain-independent weak augmentation. In this article, we propose ReFixMatch-LS and apply it to the classification of medical images. First, we reduce the impact of noisy artificial labels by label smoothing and consistent regularization. Then, by recording high-confidence pseudo-labels generated from each epoch during training, we reuse the generated pseudo-labels to train the model in the subsequent epochs. ReFixMatch-LS effectively increases the number of pseudo-labels and improves the model performance. We validate the effectiveness of ReFixMatch-LS on skin lesion diagnosis in the ISIC 2018 and ISIC 2019 challenge datasets, obtaining AUCs of 91.54%, 93.68%, 94.55%, and 95.47% on the four proportions of labeled data from ISIC 2018.

A Machine Learning Enhanced Approach for Automated Sunquake Detection in Acoustic Emission Maps

Sunquakes are seismic emissions visible on the solar surface, associated with some solar flares. Although discovered in 1998, they have only recently become a more commonly detected phenomenon. Despite the availability of several manual detection guidelines, to our knowledge, the astrophysical data produced for sunquakes is new to the field of machine learning. Detecting sunquakes is a daunting task for human operators, and this work aims to ease and, if possible, to improve their detection. Thus, we introduce a dataset constructed from acoustic egression-power maps of solar active regions obtained for Solar Cycles 23 and 24 using the holography method. We then present a pedagogical approach to the application of machine-learning representation methods for sunquake detection using autoencoders, contrastive learning, object detection and recurrent techniques, which we enhance by introducing several custom, domain-specific data augmentation transformations. We address the main challenges of the automated sunquake-detection task, namely the very high noise patterns in and outside the active region shadow and the extreme class imbalance given by the limited number of frames that present sunquake signatures. With our trained models, we find temporal and spatial locations of peculiar acoustic emission and qualitatively associate them to eruptive and high energy emission. While noting that these models are still in a prototype stage, and there is much room for improvement in metrics and bias levels, we hypothesize that their agreement on example use cases has the potential to enable detection of weak solar acoustic manifestations.

Self-supervised learning for macromolecular structure classification based on cryo-electron tomograms.

Macromolecular structure classification from cryo-electron tomography (cryo-ET) data is important for understanding macro-molecular dynamics. It has a wide range of applications and is essential in enhancing our knowledge of the sub-cellular environment. However, a major limitation has been insufficient labelled cryo-ET data. In this work, we use Contrastive Self-supervised Learning (CSSL) to improve the previous approaches for macromolecular structure classification from cryo-ET data with limited labels. We first pretrain an encoder with unlabelled data using CSSL and then fine-tune the pretrained weights on the downstream classification task. To this end, we design a cryo-ET domain-specific data-augmentation pipeline. The benefit of augmenting cryo-ET datasets is most prominent when the original dataset is limited in size. Overall, extensive experiments performed on real and simulated cryo-ET data in the semi-supervised learning setting demonstrate the effectiveness of our approach in macromolecular labeling and classification.

BirdNET: A deep learning solution for avian diversity monitoring

Variation in avian diversity in space and time is commonly used as a metric to assess environmental changes. Conventionally, such data were collected by expert observers, but passively collected acoustic data is rapidly emerging as an alternative survey technique. However, efficiently extracting accurate species richness data from large audio datasets has proven challenging. Recent advances in deep artificial neural networks (DNNs) have transformed the field of machine learning, frequently outperforming traditional signal processing techniques in the domain of acoustic event detection and classification. We developed a DNN, called BirdNET, capable of identifying 984 North American and European bird species by sound. Our task-specific model architecture was derived from the family of residual networks (ResNets), consisted of 157 layers with more than 27 million parameters, and was trained using extensive data pre-processing, augmentation, and mixup. We tested the model against three independent datasets: (a) 22,960 single-species recordings; (b) 286 h of fully annotated soundscape data collected by an array of autonomous recording units in a design analogous to what researchers might use to measure avian diversity in a field setting; and (c) 33,670 h of soundscape data from a single high-quality omnidirectional microphone deployed near four eBird hotspots frequented by expert birders. We found that domain-specific data augmentation is key to build models that are robust against high ambient noise levels and can cope with overlapping vocalizations. Task-specific model designs and training regimes for audio event recognition perform on-par with very complex architectures used in other domains (e.g., object detection in images). We also found that high temporal resolution of input spectrograms (short FFT window length) improves the classification performance for bird sounds. In summary, BirdNET achieved a mean average precision of 0.791 for single-species recordings, a F0.5 score of 0.414 for annotated soundscapes, and an average correlation of 0.251 with hotspot observation across 121 species and 4 years of audio data. By enabling the efficient extraction of the vocalizations of many hundreds of bird species from potentially vast amounts of audio data, BirdNET and similar tools have the potential to add tremendous value to existing and future passively collected audio datasets and may transform the field of avian ecology and conservation.

Strategies to improve deep learning-based salivary gland segmentation

BackgroundDeep learning-based delineation of organs-at-risk for radiotherapy purposes has been investigated to reduce the time-intensiveness and inter-/intra-observer variability associated with manual delineation. We systematically evaluated ways to improve the performance and reliability of deep learning for organ-at-risk segmentation, with the salivary glands as the paradigm. Improving deep learning performance is clinically relevant with applications ranging from the initial contouring process, to on-line adaptive radiotherapy.MethodsVarious experiments were designed: increasing the amount of training data (1) with original images, (2) with traditional data augmentation and (3) with domain-specific data augmentation; (4) the influence of data quality was tested by comparing training/testing on clinical versus curated contours, (5) the effect of using several custom cost functions was explored, and (6) patient-specific Hounsfield unit windowing was applied during inference; lastly, (7) the effect of model ensembles was analyzed. Model performance was measured with geometric parameters and model reliability with those parameters’ variance.ResultsA positive effect was observed from increasing the (1) training set size, (2/3) data augmentation, (6) patient-specific Hounsfield unit windowing and (7) model ensembles. The effects of the strategies on performance diminished when the base model performance was already ‘high’. The effect of combining all beneficial strategies was an increase in average Sørensen–Dice coefficient of about 4% and 3% and a decrease in standard deviation of about 1% and 1% for the submandibular and parotid gland, respectively.ConclusionsA subset of the strategies that were investigated provided a positive effect on model performance and reliability. The clinical impact of such strategies would be an expected reduction in post-segmentation editing, which facilitates the adoption of deep learning for autonomous automated salivary gland segmentation.

On the impact of selected modern deep-learning techniques to the performance and celerity of classification models in an experimental high-energy physics use case

Beginning from a basic neural-network architecture, we test the potential benefits offered by a range of advanced techniques for machine learning, in particular deep learning, in the context of a typical classification problem encountered in the domain of high-energy physics, using a well-studied dataset: the 2014 Higgs ML Kaggle dataset. The advantages are evaluated in terms of both performance metrics and the time required to train and apply the resulting models. Techniques examined include domain-specific data-augmentation, learning rate and momentum scheduling, (advanced) ensembling in both model-space and weight-space, and alternative architectures and connection methods.Following the investigation, we arrive at a model which achieves equal performance to the winning solution of the original Kaggle challenge, whilst being significantly quicker to train and apply, and being suitable for use with both GPU and CPU hardware setups. These reductions in timing and hardware requirements potentially allow the use of more powerful algorithms in HEP analyses, where models must be retrained frequently, sometimes at short notice, by small groups of researchers with limited hardware resources. Additionally, a new wrapper library for PYTORCH called LUMIN is presented, which incorporates all of the techniques studied.

Recognizing Named Entities in Specific Domain

The paper presents the results of applying the BERT representation model in the named entity recognition task (NER) for the cybersecurity domain in Russian. We compare several approaches to domain-specific NER combining BERT fine-tuning on a domain-specific text collection, general labeled data, domain-specific data augmentation, and a domain-specific annotated dataset. We showed that using a BERT model fine-tuned on a domain text collection and pre-trained on the combination of a general dataset and augmented data achieves the best results of named entity recognition. We also studied computational performance of the BERT model in so-called mixed precision regime.