Data Augmentation Approach Research Articles

Data augmentation for deep visual recognition using superpixel based pairwise image fusion

Data augmentation is an important paradigm for boosting the generalization capability of deep learning in image classification tasks. Image augmentation using cut-and-paste strategies has shown very good performance improvement for deep learning. However, these existing methods often overlook the image’s discriminative local context and rely on ad hoc regions consisting of square or rectangular local regions, leading to the loss of complete semantic object parts. In this work, we attempt to overcome these limitations and propose a superpixel-wise local-context-aware efficient image fusion approach for data augmentation. Our approach requires only one forward propagation using a superpixel attention-based label fusion with less computational complexity. The model is trained using a combination of a global classification of the fused (augmented) image loss, a superpixel-wise weighted local classification loss, and a superpixel-based weighted contrastive learning loss. The last two losses are based on the superpixel-aware attentive embeddings. Thus, the resulting deep encoder can learn both local and global features of the images while capturing object-part local context and information. Experiments on diverse benchmark image datasets indicate that the proposed method out-performs many region-based augmentation methods for visual recognition. We have demonstrated its effectiveness not only on CNN models but also on transformer models. The codes are accessible at https://github.com/DanielaPlusPlus/SAFuse.

Augmentation of scarce data—A new approach for deep-learning modeling of composites

High-fidelity full-field micro-mechanical modeling of the non-linear path-dependent materials demands a substantial computational effort. Recent trends in the field incorporates data-driven Artificial Neural Networks (ANNs) as surrogate models. However, ANNs are inherently data-hungry, functioning as a bottleneck for the development of high-fidelity data-driven models. This study introduces a novel approach for data augmentation, expanding an original dataset without additional computational simulations. A Recurrent Neural Network (RNN) was trained and validated on high-fidelity micro-mechanical simulations of elasto-plastic short fiber reinforced composites. The obtained results showed a considerable improvement of the network predictions trained on expanded datasets using the proposed data augmentation approach. The proposed method for augmentation of scarce data may be used not only for other kind of composites, but also for other materials and at different length scales, and hence, opening avenues for innovative data-driven models in materials science and computational mechanics.

Subdomain-Alignment Data Augmentation for Pipeline Fault Diagnosis: An Adversarial Self-Attention Network

Data augmentation (DA) has the potential to address the issue of imbalanced and insufficient datasets (I&ID) in pipeline fault diagnosis. However, the majority of existing DA methods for time series are inspired by computer vision techniques, ignoring the temporal dynamic properties and fine-grained fault features, which leads to limited performance of the augmentation. To tackle this problem, we introduce a novel DA approach called the subdomain-alignment adversarial self-attention network (SA-ASN), which takes into account both temporal association and semantic correlation. Our approach features a novel temporal association learning (TAL) mechanism, which transfers temporal information from the discriminator to the generator via a customized knowledge-sharing structure, improving the reliability of synthetic long-range associations. Additionally, we introduce a prototype-assisted subdomain alignment (PASA) strategy that forms a hierarchical structure in the synthetic dataset by incorporating local semantic correlation into the model training. With the support of TAL and PASA, our SA-ASN algorithm enhances the authenticity of temporal structure at the instance level and improves the discriminability of fault features at the category level. Our experimental results show that the SA-ASN algorithm provides a more diverse and accurate augmentation of pipeline data. The effectiveness of our SA-ASN algorithm encourages the use of data-driven diagnostic models in complex real-world oilfield pipeline networks.

Optimizing Few-Shot Remote Sensing Scene Classification Based on an Improved Data Augmentation Approach

In the realm of few-shot classification learning, the judicious application of data augmentation methods has a significantly positive impact on classification performance. In the context of few-shot classification tasks for remote sensing images, the augmentation of features and the efficient utilization of limited features are of paramount importance. To address the performance degradation caused by challenges such as high interclass overlap and large intraclass variance in remote sensing image features, we present a data augmentation-based classification optimization method for few-shot remote sensing image scene classification. First, we construct a distortion magnitude space using different types of features, and we perform distortion adjustments on the support set samples while introducing an optimal search for the distortion magnitude (ODS) method. Then, the augmented support set offers a wide array of feature distortions in terms of types and degrees, significantly enhancing the generalization of intrasample features. Subsequently, we devise a dual-path classification (DC) decision strategy, effectively leveraging the discriminative information provided by the postdistortion features to further reduce the likelihood of classification errors. Finally, we evaluate the proposed method using a widely used remote sensing dataset. Our experimental results demonstrate that our approach outperforms benchmark methods, achieving improved classification accuracy.

Animal Density Estimation for Large Unmarked Populations Using a Spatially Explicit Model

AbstractObtaining abundance and density estimates is a particularly important aspect within wildlife conservation and management. To monitor wildlife populations, the use of motion-sensor camera traps is becoming increasing popular due to its non-invasive nature. However, animal identification is not always feasible in practice due to poor quality images and/or individuals not having uniquely identifiable physical characteristics. Spatially explicit models for unmarked individuals permit the estimation of animal density when individuals cannot be uniquely identified. Due to the structure of these models, a Bayesian super-population (data augmentation) approach is often used to fit the models to data, which involves specifying some reasonably large upper limit for the population. However, this approach presents substantial computational challenges for larger populations, as demonstrated by the motivating dataset relating to barking deer (Muntiacus muntjak) collected in Ujung Kulon National Park, Indonesia (with a population size in the low thousands). We develop a new and computationally efficient Bayesian algorithm for fitting the models to data that does not require specifying an upper population limit a priori. We apply the new algorithm to the large barking deer dataset, where the standard super-population approach is computationally expensive, and demonstrate a substantial improvement in computational efficiency.Supplementary material to this paper is provided online.

Interpretable EU ETS Phase 4 prices forecasting based on deep generative data augmentation approach

This paper proposes an interpretable deep learning method based on generative data augmentation for forecasting carbon allowance prices in the EU Emissions Trading System (ETS) Phase 4. Utilizing TimeGAN, we generate near-real expanded data to enhance the training sets. Temporal Fusion Transformer (TFT) is used to quantify the contribution of impact factors. The results show that the augmentation effectively improved the prediction accuracy. Interpretability analysis reveals that Brent crude oil, NBP natural gas, and Rotterdam coal are the top three contributors. Our findings offer a strong approach for the new phase price forecasting, helping market participants and policymakers in informed decision-making.

RETRACTED: DL-SCA: An deep learning based approach for intra-class CutMix data augmentation

CutMix data augmentation can provide a large amount of augmented data for DL-SCA (deep learning side channel attacks) by generating new power traces. However, traces generated by CutMix may lose dependency with the new label, which may reduce the accuracy of the training model. In light of this, we propose an improved intra-class CutMix data augmentation method. Firstly, the original power traces are classified by the label. Then, the original power traces are selected by the same label constraint to generate new power traces according to CutMix, which can ensure the dependency between the generated trace and its label. Furthermore, to maintain balance among different classified datasets, the traces are generated sequentially according to distinct labels. Finally, based on the augmented power traces, the MLP (Multilayer Perceptron) and CNN (Convolutional Neural Network) models can be constructed and trained to recover the key of AES. In order to verify the effectiveness of the proposed method, we conducted experimental evaluations using the MLP and CNN models based on DPA-contest v4 dataset and ASCAD dataset. The test results show that the generated traces based on the intra-class CutMix method can be very similar to the original power traces, and the MLP and CNN models can be effectively trained based on the generated traces to recover the key of AES. Compared with existing data augmentation methods, the proposed method can complete the key recovery with faster convergence and fewer power traces.

Hate Speech Detection in Limited Data Contexts Using Synthetic Data Generation

A growing body of work has focused on text classification methods for detecting the increasing amount of hate speech posted online. This progress has been limited to only a select number of highly resourced languages causing detection systems to either under-perform or not exist in limited data contexts. This is mostly caused by a lack of training data, which are expensive to collect and curate in these settings. In this work, we propose a data augmentation approach that addresses the problem of lack of data for online hate speech detection in limited data contexts using synthetic data generation techniques. Given a handful of hate speech examples in a high-resource language such as English, we present three methods to synthesize new examples of hate speech data in a target language that retains the hate sentiment in the original examples but transfers the hate targets. We apply our approach to generate training data for hate speech classification tasks in Hindi and Vietnamese. Our findings show that a model trained on synthetic data performs comparably to, and in some cases outperforms, a model trained only on the samples available in the target domain. This method can be adopted to bootstrap hate speech detection models from scratch in limited data contexts. As the growth of social media within these contexts continues to outstrip response efforts, this work furthers our capacities for detection, understanding, and response to hate speech. Disclaimer: This work contains terms that are offensive and hateful. These, however, cannot be avoided due to the nature of the work.

Developing children's ASR system under low-resource conditions using end-to-end architecture

The work presented in this paper aims at enhancing the performance of end-to-end (E2E) speech recognition task for children's speech under low resource conditions. For majority of the languages, there is hardly any speech data from child speakers. Furthermore, even the available children's speech corpora are limited in terms of the number of hours of data. On the other hand, large amounts of adults' speech data are freely available for research as well as commercial purposes. As a consequence, developing an effective E2E automatic speech recognition (ASR) system for children becomes a very challenging task. One may develop an ASR system using adults' speech and then use it to transcribe children's data, but this leads to very poor recognition rates due to the stark differences in the acoustic attributes of adults' and children's speech. In order to overcome these hurdles and to develop a robust children's ASR system employing E2E architecture, we have resorted to several out-of-domain and in-domain data augmentation techniques. For out-of-domain data augmentation, we have explicitly modified adults' speech to render it acoustically similar to that of children's speech before pooling into training. On the other hand, in the case of in-domain data augmentation, we have slightly modified the pitch and duration of children's speech in order to create more data capturing greater diversity. Data augmentation approaches helps in mitigating the ill-effects resulting from the scarcity of data from child domain to a certain extent. This, in turn, reduces the error rates by a large margin. In addition to data augmentation, we have also studied the efficacy of Gamma-tone frequency cepstral coefficients (GFCC) and frequency domain linear prediction (FDLP) technique along with the most commonly used Mel-frequency cepstral coefficients (MFCC) for front-end speech parameterization. Both MFCC as well as GFCC capture and model the spectral envelope of speech. On the other hand, application of linear prediction on the frequency domain representation of speech signal helps to effectively capture the temporal envelope during front-end feature extraction. Employing FDLP features that model the temporal envelope provides important cues for the perception and understanding of stop bursts and, at times, complete phonemes. This motivated us to perform a comparative experimental study of the effectiveness of the three aforementioned front-end acoustic features. In our experimental explorations, the use of proposed data augmentation in combination of FDLP features has shown a relative improvement in character error rate by 67.6% over the baseline system. The combination of data augmentation with MFCC or GFCC features is observed to result in lower recognition performances.

Data augmentation strategy for generating realistic samples on defect segmentation task

In the anomaly and defect detection tasks, the number of negative samples greatly exceeds the number of defective samples. As a result, a high-class imbalance exists among different classes in the detection task. In our work, we introduce a data-level solution for improving the generalization performance of the semantic segmentation of surface defects based on a data augmentation (DA) strategy. In particular, our DA approach comprised a generative stage to simulate synthetic defects and a validation stage to validate the synthetic image as close as possible to the real one. A Siamese network fully validates our synthetic samples to select only synthetic defects as close to the real ones. We demonstrated the effectiveness of our approach in a real-use case scenario to baseline DA approaches. Our DA approach allows balancing the minority classes while improving the overall generalization performance for semantic segmentation for defect detection.

Predictive Machine Learning Models for Assessing Lebanese University Students' Depression, Anxiety, and Stress During COVID-19.

University students are experiencing a mental health crisis. COVID-19 has exacerbated this situation. We have surveyed students in 2 universities in Lebanon to gauge their mental health challenges. We have constructed a machine learning (ML) approach to predict symptoms of depression, anxiety, and stress based on demographics and self-rated health measures. Our approach involved developing 8 ML predictive models, including Logistic Regression (LR), multi-layer perceptron (MLP) neural network, support vector machine (SVM), random forest (RF) and XGBoost, AdaBoost, Naïve Bayes (NB), and K-Nearest neighbors (KNN). Following their construction, we compared their respective performances. Our evaluation shows that RF (AUC = 78.27%), NB (AUC = 76.37%), and AdaBoost (AUC = 72.96%) have provided the highest-performing AUC scores for depression, anxiety, and stress, respectively. Self-rated health is found to be the top feature in predicting depression, while age was the top feature in predicting anxiety and stress, followed by self-rated health. Future work will focus on using data augmentation approaches and extending to multi-class anxiety predictions.

Local Patch AutoAugment With Multi-Agent Collaboration

Data augmentation (DA) plays a critical role in improving the generalization of deep learning models. Recent works on automatically searching for DA policies from data have achieved great success. However, existing automated DA methods generally perform the search at the image level, which limits the exploration of diversity in local regions. In this paper, we propose a more fine-grained automated DA approach, dubbed Patch AutoAugment, to divide an image into a grid of patches and search for the joint optimal augmentation policies for the patches. We formulate it as a multi-agent reinforcement learning (MARL) problem, where each agent learns an augmentation policy for each patch based on its content together with the semantics of the whole image. The agents cooperate with each other to achieve the optimal augmentation effect of the entire image by sharing a team reward. We show the effectiveness of our method on multiple benchmark datasets of image classification, fine-grained image recognition and object detection ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</i> . <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">g</i> ., CIFAR-10, CIFAR-100, ImageNet, CUB-200-2011, Stanford Cars, FGVC-Aircraft and Pascal VOC 2007). Extensive experiments demonstrate that our method outperforms the state-of-the-art DA methods while requiring fewer computational resources. Automatic Data Augmentation. Multi-Agent Reinforcement Learning.

Data Augmentation and Innovative Machine Learning Approaches for Classifying EEL Spectra of Transition Metals Oxides

Data Augmentation and Innovative Machine Learning Approaches for Classifying EEL Spectra of Transition Metals Oxides

Brain Tumor Detection Using a Deep CNN Model

The diagnosis of brain tumors through magnetic resonance imaging (MRI) has become highly significant in the field of medical science. Relying solely on MR imaging for the detection and categorization of brain tumors demands significant time, effort, and expertise from medical professionals. This underscores the need for an autonomous model for brain tumor diagnosis. Our study involves the application of a deep convolutional neural network (DCNN) to diagnose brain tumors from MR images. The application of these algorithms offers several benefits, including rapid brain tumor prediction, reduced errors, and enhanced precision. The proposed model is built upon the state‐of‐the‐art CNN architecture VGG16, employing a data augmentation approach. The dataset utilized in this paper consists of 3000 brain MR images sourced from Kaggle, with 1500 images reported to contain tumors. Through training and testing, the pretrained CNN model achieves a precision and classification accuracy rate of 96%, and the loss is 1%. Moreover, it achieves an average precision, recall, and F1‐score of 98.7%, 97.44%, and 98.06%, respectively. These evaluation metric values demonstrate the effectiveness of the proposed solution.

Fine-grained Recognition with Learnable Semantic Data Augmentation.

Fine-grained image recognition is a longstanding computer vision challenge that focuses on differentiating objects belonging to multiple subordinate categories within the same meta-category. Since images belonging to the same meta-category usually share similar visual appearances, mining discriminative visual cues is the key to distinguishing fine-grained categories. Although commonly used image-level data augmentation techniques have achieved great success in generic image classification problems, they are rarely applied in fine-grained scenarios, because their random editing-region behavior is prone to destroy the discriminative visual cues residing in the subtle regions. In this paper, we propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Specifically, we produce diversified augmented samples by translating image features along semantically meaningful directions. The semantic directions are estimated with a covariance prediction network, which predicts a sample-wise covariance matrix to adapt to the large intra-class variation inherent in fine-grained images. Furthermore, the covariance prediction network is jointly optimized with the classification network in a meta-learning manner to alleviate the degenerate solution problem. Experiments on four competitive fine-grained recognition benchmarks (CUB-200-2011, Stanford Cars, FGVC Aircrafts, NABirds) demonstrate that our method significantly improves the generalization performance on several popular classification networks (e.g., ResNets, DenseNets, EfficientNets, RegNets and ViT). Combined with a recently proposed method, our semantic data augmentation approach achieves state-of-the-art performance on the CUB-200-2011 dataset. The source code will be released.

Research Challenges for Augmenting Endoscopy Image Datasets using Image Combination Methodologies

Metaplasia detection in upper gastrointestinal endoscopy is crucial to identify patients at higher risk of gastric cancer. Deep learning algorithms can be useful for detecting and localising these lesions during an endoscopy exam. However, to train these types of models, a lot of annotated data is needed, which can be a problem in the medical field. To overcome this, data augmentation techniques are commonly applied to increase the dataset’s variability but need to be adapted to the specificities of the application scenario. In this study, we discuss the potential benefits and identify four key research challenges of a promising data augmentation approach, namely image combination methodologies, such as CutMix, for metaplasia detection and localisation in gastric endoscopy imaging modalities.

Guide for the application of the data augmentation approach on sets of texts in Spanish for sentiment and emotion analysis.

Over the last ten years, social media has become a crucial data source for businesses and researchers, providing a space where people can express their opinions and emotions. To analyze this data and classify emotions and their polarity in texts, natural language processing (NLP) techniques such as emotion analysis (EA) and sentiment analysis (SA) are employed. However, the effectiveness of these tasks using machine learning (ML) and deep learning (DL) methods depends on large labeled datasets, which are scarce in languages like Spanish. To address this challenge, researchers use data augmentation (DA) techniques to artificially expand small datasets. This study aims to investigate whether DA techniques can improve classification results using ML and DL algorithms for sentiment and emotion analysis of Spanish texts. Various text manipulation techniques were applied, including transformations, paraphrasing (back-translation), and text generation using generative adversarial networks, to small datasets such as song lyrics, social media comments, headlines from national newspapers in Chile, and survey responses from higher education students. The findings show that the Convolutional Neural Network (CNN) classifier achieved the most significant improvement, with an 18% increase using the Generative Adversarial Networks for Sentiment Text (SentiGan) on the Aggressiveness (Seriousness) dataset. Additionally, the same classifier model showed an 11% improvement using the Easy Data Augmentation (EDA) on the Gender-Based Violence dataset. The performance of the Bidirectional Encoder Representations from Transformers (BETO) also improved by 10% on the back-translation augmented version of the October 18 dataset, and by 4% on the EDA augmented version of the Teaching survey dataset. These results suggest that data augmentation techniques enhance performance by transforming text and adapting it to the specific characteristics of the dataset. Through experimentation with various augmentation techniques, this research provides valuable insights into the analysis of subjectivity in Spanish texts and offers guidance for selecting algorithms and techniques based on dataset features.

Few-Shot Continual Learning via Flat-to-Wide Approaches.

The existing approaches on continual learning (CL) call for a lot of samples in their training processes. Such approaches are impractical for many real-world problems having limited samples because of the overfitting problem. This article proposes a few-shot CL approach, termed flat-to-wide approach (FLOWER), where a flat-to-wide learning process finding the flat-wide minima is proposed to address the catastrophic forgetting (CF) problem. The issue of data scarcity is overcome with a data augmentation approach making use of a ball-generator concept to restrict the sampling space into the smallest enclosing ball. Our numerical studies demonstrate the advantage of FLOWER achieving significantly improved performances over prior arts notably in the small base tasks. For further study, source codes of FLOWER, competitor algorithms, and experimental logs are shared publicly in https://github.com/anwarmaxsum/FLOWER.

Improving Turkish Text Sentiment Classification Through Task-Specific and Universal Transformations: An Ensemble Data Augmentation Approach

Improving Turkish Text Sentiment Classification Through Task-Specific and Universal Transformations: An Ensemble Data Augmentation Approach

Neurophysiologically Meaningful Motor Imagery EEG Simulation With Applications to Data Augmentation.

Motor imagery-based Brain-Computer Interfaces (MI-BCIs) have gained a lot of attention due to their potential usability in neurorehabilitation and neuroprosthetics. However, the accurate recognition of MI patterns in electroencephalography signals (EEG) is hindered by several data-related limitations, which restrict the practical utilization of these systems. Moreover, leveraging deep learning (DL) models for MI decoding is challenged by the difficulty of accessing user-specific MI-EEG data on large scales. Simulated MI-EEG signals can be useful to address these issues, providing well-defined data for the validation of decoding models and serving as a data augmentation approach to improve the training of DL models. While substantial efforts have been dedicated to implementing effective data augmentation strategies and model-based EEG signal generation, the simulation of neurophysiologically plausible EEG-like signals has not yet been exploited in the context of data augmentation. Furthermore, none of the existing approaches have integrated user-specific neurophysiological information during the data generation process. Here, we present PySimMIBCI, a framework for generating realistic MI-EEG signals by integrating neurophysiologically meaningful activity into biophysical forward models. By means of PySimMIBCI, different user capabilities to control an MI-BCI can be simulated and fatigue effects can be included in the generated EEG. Results show that our simulated data closely resemble real data. Moreover, a proposed data augmentation strategy based on our simulated user-specific data significantly outperforms other state-of-the-art augmentation approaches, enhancing DL models performance by up to 15%.