Realistic Samples Research Articles

Data Augmentation of 3D Brain Environment Using Deep Convolutional Refined Auto-Encoding Alpha GAN

Learning-based methods represent the state of the art in path planning problems. Their performance, however, depend on the number of medical images available for the training. Generative Adversarial Networks (GANs) are unsupervised neural networks that can be exploited to synthesize realistic images avoiding the dependency from the original data. In this article, we propose an innovative type of GAN, Deep Convolutional Refined Auto-Encoding Alpha GAN, able to successfully generate 3D brain Magnetic Resonance Imaging (MRI) data from random vectors by learning the data distribution. We combined a Variational Auto-Encoder GAN with a Code Discriminator to solve the common mode collapse problem and reduce the image blurriness. Finally, we inserted a Refiner in series with the Generator Network in order to smooth the shapes of the images and generate more realistic samples. A qualitative comparison between the generated images and the real ones has been used to test our model's quality. With the use of three indexes, namely the Multi-Scale Structural Similarity Metric, the Maximum Mean Discrepancy and the Intersection over Union, we also performed a quantitative analysis. The final results suggest that our model can be a suitable solution to overcome the shortage of medical images needed for learning-based methods.

Deep Reinforcement Adversarial Learning Against Botnet Evasion Attacks

As cybersecurity detectors increasingly rely on machine learning mechanisms, attacks to these defenses escalate as well. Supervised classifiers are prone to adversarial evasion, and existing countermeasures suffer from many limitations. Most solutions degrade performance in the absence of adversarial perturbations; they are unable to face novel attack variants; they are applicable only to specific machine learning algorithms. We propose the first framework that can protect botnet detectors from adversarial attacks through deep reinforcement learning mechanisms. It automatically generates realistic attack samples that can evade detection, and it uses these samples to produce an augmented training set for producing hardened detectors. In such a way, we obtain more resilient detectors that can work even against unforeseen evasion attacks with the great merit of not penalizing their performance in the absence of specific attacks. We validate our proposal through an extensive experimental campaign that considers multiple machine learning algorithms and public datasets. The results highlight the improvements of the proposed solution over the state-of-the-art. Our method paves the way to novel and more robust cybersecurity detectors based on machine learning applied to network traffic analytics.

Bias alleviating generative adversarial network for generalized zero-shot classification

Generalized zero-shot classification is predicting the labels of the test images coming from seen or unseen classes. The task is difficult because of the bias problem, that is, unseen samples are easily to be misclassified to seen classes. Many methods have handled the problem by training a generative adversarial network (GAN) to generate fake samples. However, the GAN model trained with seen samples might not be appropriate for generating unseen samples. For dealing with this problem, we learn a bias alleviating generative adversarial network for generalized zero-shot classification by generating seen and unseen samples, simultaneously. We train the generator to generate more realistic unseen samples by adding semantic similarity and cluster center regularizations to alleviate the bias problem. The semantic similarity regularization is to restrict the relationships of the generated unseen visual prototypes and seen visual prototypes by their class prototypes to avoid the generated unseen samples similar to the seen samples. The cluster center regularization is to utilize the cluster property of target data to make the generated unseen visual prototypes near to the most similar cluster centers, generating realistic unseen samples. From the experiments, we can see the proposed method achieves promising results.

A Novel GAN-Based Synthesis Method for In-Air Handwritten Words.

In recent years, with the miniaturization and high energy efficiency of MEMS (micro-electro-mechanical systems), in-air handwriting technology based on inertial sensors has come to the fore. Most of the previous works have focused on character-level in-air handwriting recognition. In contrast, few works focus on word-level in-air handwriting tasks. In the field of word-level recognition, researchers have to face the problems of insufficient data and poor generalization performance of recognition methods. On one hand, the training of deep neural learning networks usually requires a particularly large dataset, but collecting data will take a lot of time and money. On the other hand, a deep recognition network trained on a small dataset can hardly recognize samples whose labels do not appear in the training set. To address these problems, we propose a two-stage synthesis method of in-air handwritten words. The proposed method includes a splicing module guided by an additional corpus and a generating module trained by adversarial learning. We carefully design the proposed network so that it can handle word sample inputs of arbitrary length and pay more attention to the details of the samples. We design multiple sets of experiments on a public dataset. The experimental results demonstrate the success of the proposed method. What is impressive is that with the help of the air-writing word synthesizer, the recognition model learns the context information (combination information of characters) of the word. In this way, it can recognize words that have never appeared in the training process. In this paper, the recognition model trained on synthetic data achieves a word-level recognition accuracy of 62.3% on the public dataset. Compared with the model trained using only the public dataset, the word-level accuracy is improved by 62%. Furthermore, the proposed method can synthesize realistic samples under the condition of limited of in-air handwritten character samples and word samples. It largely solves the problem of insufficient data. In the future, mathematically modeling the strokes between characters in words may help us find a better way to splice character samples. In addition, we will apply our method to various datasets and improve the splicing module and generating module for different tasks.

Data Sanitization to Reduce Private Information Leakage from Functional Genomics

The generation of functional genomics datasets is surging, because they provide insight into gene regulation and organismal phenotypes (e.g., genes upregulated in cancer). The intent behind functional genomics experiments is not necessarily to study genetic variants, yet they pose privacy concerns due to their use of next-generation sequencing. Moreover, there is a great incentive to broadly share raw reads for better statistical power and general research reproducibility. Thus, we need new modes of sharing beyond traditional controlled-access models. Here, we develop a data-sanitization procedure allowing raw functional genomics reads to be shared while minimizing privacy leakage, enabling principled privacy-utility trade-offs. Our protocol works with traditional Illumina-based assays and newer technologies such as 10x single-cell RNA sequencing. It involves quantifying the privacy leakage in reads by statistically linking study participants to known individuals. We carried out these linkages using data from highly accurate reference genomes and more realistic environmental samples.

AUGMENTING DATA USING GAUSSIAN MIXTURE EMBEDDING FOR IMPROVING LAND COVER SEGMENTATION

Abstract. The use of convolutional neural networks improved greatly data synthesis in the last years and have been widely used for data augmentation in scenarios where very imbalanced data is observed, such as land cover segmentation. Balancing the proportion of classes for training segmentation models can be very challenging considering that samples where all classes are reasonably represented might constitute a small portion of a training set, and techniques for augmenting this small amount of data such as rotation, scaling and translation might be not sufficient for efficient training. In this context, this paper proposes a methodology to perform data augmentation from few samples to improve the performance of CNN-based land cover semantic segmentation. First, we estimate the latent data representation of selected training samples by means of a mixture of Gaussians, using an encoder-decoder CNN. Then, we change the latent embedding used to generate the mixture parameters, at random and in training time, to generate new mixture models slightly different from the original. Finally, we compute the displacement maps between the original and the modified mixture models, and use them to elastically deform the original images, creating new realistic samples out of the original ones. Our disentangled approach allows the spatial modification of displacement maps to preserve objects where deformation is undesired, like buildings and cars, where geometry is highly discriminant. With this simple pipeline, we managed to augment samples in training time, and improve the overall performance of two basal semantic segmentation CNN architectures for land cover semantic segmentation.

Clustering Analysis via Deep Generative Models With Mixture Models.

Clustering is a fundamental problem that frequently arises in many fields, such as pattern recognition, data mining, and machine learning. Although various clustering algorithms have been developed in the past, traditional clustering algorithms with shallow structures cannot excavate the interdependence of complex data features in latent space. Recently, deep generative models, such as autoencoder (AE), variational AE (VAE), and generative adversarial network (GAN), have achieved remarkable success in many unsupervised applications thanks to their capabilities for learning promising latent representations from original data. In this work, first we propose a novel clustering approach based on both Wasserstein GAN with gradient penalty (WGAN-GP) and VAE with a Gaussian mixture prior. By combining the WGAN-GP with VAE, the generator of WGAN-GP is formulated by drawing samples from the probabilistic decoder of VAE. Moreover, to provide more robust clustering and generation performance when outliers are encountered in data, a variant of the proposed deep generative model is developed based on a Student's-t mixture prior. The effectiveness of our deep generative models is validated though experiments on both clustering analysis and samples generation. Through the comparison with other state-of-art clustering approaches based on deep generative models, the proposed approach can provide more stable training of the model, improve the accuracy of clustering, and generate realistic samples.

Generative adversarial network-based electromagnetic signal classification: A semi-supervised learning framework

Generative adversarial network (GAN) has achieved great success in many fields such as computer vision, speech processing, and natural language processing, because of its powerful capabilities for generating realistic samples. In this paper, we introduce GAN into the field of electromagnetic signal classification (ESC). ESC plays an important role in both military and civilian domains. However, in many specific scenarios, we can't obtain enough labeled data, which cause failure of deep learning methods because they are easy to fall into over-fitting. Fortunately, semi-supervised learning (SSL) can leverage the large amount of unlabeled data to enhance the classification performance of classifiers, especially in scenarios with limited amount of labeled data. We present an SSL framework by incorporating GAN, which can directly process the raw in-phase and quadrature (IQ) signal data. According to the characteristics of the electromagnetic signal, we propose a weighted loss function, leading to an effective classifier to realize the end-to-end classification of the electromagnetic signal. We validate the proposed method on both public RML2016.04c dataset and real-world Aircraft Communications Addressing and Reporting System (ACARS) signal dataset. Extensive experimental results show that the proposed framework obtains a significant increase in classification accuracy compared with the state-of-the-art studies.

Development of an MEMS ultrasonic microphone array system and its application to compressed wavefield imaging of concrete

Although contactless ultrasonic wavefield imaging shows potential for effective nondestructive inspection of various engineering materials, it has been rarely applied to concrete materials owing to technical challenges including low signal-to-noise ratio (SNR) caused by inherent heterogeneity of concrete. This paper presents development of a multi-channel MEMS ultrasonic microphone array system and its application to compressed wavefield imaging of concrete materials. The developed multi-channel MEMS ultrasonic microphone array system contains eight MEMS ultrasonic microphone elements and a signal conditioning circuit that enables measurements of ultrasonic signals with high SNR. A compressed sensing approach, based on the multiple measurement vector (MMV) concept, is applied to reconstruct a full dense ultrasonic wavefield data from sparsely sampled ultrasonic wavefield data. Experiments are carried out on a laboratory concrete sample to verify the performance of the developed MEMS microphone array system and proposed compressed sensing approach and then large-scale concrete samples to demonstrate practical application. The experimental results demonstrate that the developed MEMS microphone array system provides high-quality (SNR > 20 dB) ultrasonic data collected from concrete elements; furthermore, the proposed compressed sensing approach provides accurate reconstruction of dense wavefield data, as determined by peak signal-to-noise ratio (PSNR), from sparsely measured wavefield data with compression ratios up to 85% and PSNR above 25 dB in data collected form realistic large-scale concrete samples. By combining the MEMS array system and compressed sensing approach, the total ultrasonic data acquisition time needed to produce dense wavefield data can be significantly reduced.

Shale Digital Core Image Generation Based on Generative Adversarial Networks

Abstract Stochastic reconstruction of digital core images is a vital part of digital core physics analysis, aiming to generate representative microstructure samples for sampling and uncertainty quantification analysis. This paper proposes a novel reconstruction method of the digital core of shale based on generative adversarial networks (GANs) with powerful capabilities of the generation of samples. GANs are a series of unsupervised generative artificial intelligence models that take the noise vector as an input. In this paper, the GANs with a generative and a discriminative network are created respectively, and the shale image with 45 nm/pixel preprocessed by the three-value-segmentation method is used as training samples. The generative network is used to learn the distribution of real training samples, and the discriminative network is used to distinguish real samples from synthetic ones. Finally, realistic digital core samples of shale are successfully reconstructed through the adversarial training process. We used the Fréchet inception distance (FID) and Kernel inception distance (KID) to evaluate the ability of GANs to generate real digital core samples of shale. The comparison of the morphological characteristics between them, such as the ratio of organic matter and specific surface area of organic matter, indicates that real and reconstructed samples are highly close. The results show that deep convolutional generative adversarial networks with full convolution properties can reconstruct digital core samples of shale effectively. Therefore, compared with the classical methods of reconstruction, the new reconstruction method is more promising.

Handwritten Text Segmentation via End-to-End Learning of Convolutional Neural Networks

We present a method that separates handwritten and machine-printed components that are mixed and overlapped in documents. Many conventional methods addressed this problem by extracting connected components (CCs) and classifying the extracted CCs into two classes. They were based on the assumption that two types of components are not overlapping each other, while we are focusing on more challenging and realistic cases where the components are often overlapping each other. For this, we propose a new method that performs pixel-level classification with a convolutional neural network. Unlike conventional neural network methods, our method works in an end-to-end manner and does not require any preprocessing steps (e.g., foreground extraction, handcrafted feature extraction, and so on). For the training of our network, we develop a cross-entropy based loss function to alleviate the class imbalance problem. Regarding the training dataset, although there are some datasets of mixed printed characters and handwritten scripts, most of them do not have overlapping cases and do not provide pixel-level annotations. Hence, we also propose a data synthesis method that generates realistic pixel-level training samples having many overlappings of printed and handwritten components. Experimental results on synthetic and real images have shown the effectiveness of the proposed method. Although the proposed network has been trained only with synthetic images, it also improves the OCR rate of real documents. Specifically, the OCR rate for machine-printed texts is increased from 0.8087 to 0.9442 by removing the overlapped handwritten scribbles by our method.

Modelling Fuel Injector Spray Characteristics in Jet Engines by Using Vine Copulas

SummaryThe emission requirements for jet engines are becoming more stringent and the combustion process determines pollutant emissions. Therefore, we model the distribution of fuel drops generated by a fuel injector in a jet engine, which can be assumed to be a five-dimensional problem in terms of drop size, x-position, y-position, x-velocity and y-velocity. The data are generated by numerical simulations of the fuel atomization process for several jet engine operating conditions. In combustion simulations, the variables are usually assumed to be independent at the start of the simulation, which is clearly not so as our data show. The dependence between some of the variables is non-monotone and asymmetric, which makes the modelling task difficult. Our aim is to provide a realistic parametric model for the dependence structure. For this, we employ vine copulas which provide a flexible way to construct a multivariate distribution function. However, we need to use non-standard bivariate copulas as building blocks. Using this copula representation enables us to create realistic samples of fuel spray droplets which improve the prediction of the combustion process and the pollutant emissions. Moreover, this approach is significantly faster than solving the set of differential equations describing fuel disintegration.

Improved recurrent generative adversarial networks with regularization techniques and a controllable framework

Generative Adversarial Network (GAN), a deep learning framework to generate synthetic but realistic samples, has produced astonishing results for image synthesis. However, because GAN is routinely used for image datasets, regularization methods for GAN have been developed for convolutional layers. In this study, to expand these methods for time-series data, which are one of the most common data types in various real datasets, modified regularization methods are proposed for Long Short-Term Memory (LSTM)-based GANs. Specifically, the spectral normalization, hinge loss, orthogonal regularization, and the truncation trick are modified and assessed for LSTM-based GANs. Furthermore, a conditional GAN architecture called Controllable GAN (ControlGAN) is applied to LSTM-based GANs to produce the desired samples. The evaluations are conducted with sine wave data, air pollution datasets, and a medical time-series dataset obtained from intensive care units. As a result, ControlGAN with the spectral normalization on gates and cell states consistently outperforms the others, including the conventional model, called Recurrent Conditional GAN (RCGAN).

RADIO: Parameterized Generative Radar Data Augmentation for Small Datasets

We present a novel, parameterised radar data augmentation (RADIO) technique to generate realistic radar samples from small datasets for the development of radar-related deep learning models. RADIO leverages the physical properties of radar signals, such as attenuation, azimuthal beam divergence and speckle noise, for data generation and augmentation. Exemplary applications on radar-based classification and detection demonstrate that RADIO can generate meaningful radar samples that effectively boost the accuracy of classification and generalisability of deep models trained with a small dataset.

Modeling Feature Representations for Affective Speech Using Generative Adversarial Networks

Emotion recognition is a classic field of research with a typical setup extracting features and feeding them through a classifier for prediction. On the other hand, generative models jointly capture the distributional relationship between emotions and the feature profiles. Recently, Generative Adversarial Networks (GANs) have surfaced as a new class of generative models and have shown considerable success in modeling distributions in the fields of computer vision and natural language understanding. In this article, we experiment with variants of GAN architectures to generate feature vectors corresponding to an emotion in two ways: (i) A generator is trained with samples from a mixture prior. Each mixture component corresponds to an emotional class and can be sampled to generate features from the corresponding emotion. (ii) A one-hot vector corresponding to an emotion can be explicitly used to generate the features. We perform analysis on such models and also propose different metrics used to measure the performance of the GAN models in their ability to generate realistic synthetic samples. Apart from evaluation on a given dataset of interest, we perform a cross-corpus study where we study the utility of the synthetic samples as additional training data in low resource conditions.

Visualization of Light Elements using 4D STEM: The Layered‐to‐Rock Salt Phase Transition in LiNiO2 Cathode Material

AbstractThe layered oxide LiNiO2 (LNO) has been extensively investigated as a cathode active material for lithium‐ion batteries. Despite LNO's high gravimetric capacity, instability issues hinder its commercialization. It suffers from capacity loss during electrochemical cycling and is difficult to synthesize without defects. This is related to poor structural stability, leading to decomposition into the parent rock‐salt‐type oxide. In order to understand such phase transformations and to develop measures to inhibit them, the development of techniques able to image all atoms is crucial. In this study, the use of a fast, pixelated detector and 4D imaging in scanning transmission electron microscopy are explored to tackle this challenge. Selecting specific angular regions in the diffraction patterns and calculating virtual annular bright‐field images significantly enhances the contrast of the lithium atoms, such that all atoms are visible even in realistic samples. The developed technique is applied to image the layered‐to‐rock salt phase transition region. The data show that in this region, nickel atoms are in tetrahedral positions and the oxygen atoms are asymmetrically distributed. Taken together, the results shed light on the phase transformation mechanism at the atomic scale and can guide future research toward stabilizing LNO.

Voronoi volume function: a new probe of cosmology and galaxy evolution

ABSTRACT We study the Voronoi volume function (VVF) – the distribution of cell volumes (or inverse local number density) in the Voronoi tessellation of any set of cosmological tracers (galaxies/haloes). We show that the shape of the VVF of biased tracers responds sensitively to physical properties such as halo mass, large-scale environment, substructure, and redshift-space effects, making this a hitherto unexplored probe of both primordial cosmology and galaxy evolution. Using convenient summary statistics – the width, median, and a low percentile of the VVF as functions of average tracer number density – we explore these effects for tracer populations in a suite of N-body simulations of a range of dark matter models. Our summary statistics sensitively probe primordial features such as small-scale oscillations in the initial matter power spectrum (as arise in models involving collisional effects in the dark sector), while being largely insensitive to a truncation of initial power (as in warm dark matter models). For vanilla cold dark matter (CDM) cosmologies, the summary statistics display strong evolution and redshift-space effects, and are also sensitive to cosmological parameter values for realistic tracer samples. Comparing the VVF of galaxies in the Galaxies & Mass Assembly (GAMA) survey with that of abundance-matched CDM (sub)haloes tentatively reveals environmental effects in GAMA beyond halo mass (modulo unmodelled satellite properties). Our exploratory analysis thus paves the way for using the VVF as a new probe of galaxy evolution physics as well as the nature of dark matter and dark energy.

On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models

This study investigates the effects of Markov chain Monte Carlo (MCMC) sampling in unsupervised Maximum Likelihood (ML) learning. Our attention is restricted to the family of unnormalized probability densities for which the negative log density (or energy function) is a ConvNet. We find that many of the techniques used to stabilize training in previous studies are not necessary. ML learning with a ConvNet potential requires only a few hyper-parameters and no regularization. Using this minimal framework, we identify a variety of ML learning outcomes that depend solely on the implementation of MCMC sampling.On one hand, we show that it is easy to train an energy-based model which can sample realistic images with short-run Langevin. ML can be effective and stable even when MCMC samples have much higher energy than true steady-state samples throughout training. Based on this insight, we introduce an ML method with purely noise-initialized MCMC, high-quality short-run synthesis, and the same budget as ML with informative MCMC initialization such as CD or PCD. Unlike previous models, our energy model can obtain realistic high-diversity samples from a noise signal after training.On the other hand, ConvNet potentials learned with non-convergent MCMC do not have a valid steady-state and cannot be considered approximate unnormalized densities of the training data because long-run MCMC samples differ greatly from observed images. We show that it is much harder to train a ConvNet potential to learn a steady-state over realistic images. To our knowledge, long-run MCMC samples of all previous models lose the realism of short-run samples. With correct tuning of Langevin noise, we train the first ConvNet potentials for which long-run and steady-state MCMC samples are realistic images.

Can friction replace roughness in the numerical simulation of granular materials?

We perform numerical simulations of plane strain compression tests of Ottawa sand samples inspired by previous micro computed tomography experiments. The numerical code used for computations makes it possible to mimic in a realistic way the complex boundary conditions of the experimental test, and to introduce grains with complex morphologies based on Fourier descriptors measured on real sand grains. Since the introduction of a realistic roughness of the sand grains is very costly in terms of computation, we wish to determine if this roughness could be replaced by an artificial increase of the contact friction coefficient, without modifying the mechanical response of the sample and the localization patterns. Numerical results show that such a smoothing is actually possible to a certain extent, and guidelines are provided in order to choose with care the amount of roughness that can be compensated by friction. The resulting reduction of the computation time makes it possible to foresee future large scale simulations of realistic granular samples.

Data synthesis using dual discriminator conditional generative adversarial networks for imbalanced fault diagnosis of rolling bearings

Diagnosis of rolling bearings plays an important role in condition monitoring of industrial rotating machinery. In many actual applications, rolling bearings work in normal state at most time and faulty samples are difficult to be collected. Thus, it is easy to arise problem of imbalanced dataset which restricts accuracy and stability of fault diagnosis. Generative adversarial networks (GANs) have been proved to be effective to produce artificial data that are alike real data, and have been widely used in image fields. Data synthesis using deep generative model provide a promising methodology for imbalanced fault diagnosis of machinery. In this paper, we propose a novel framework named dual discriminator conditional generative adversarial networks (D2CGANs) to learn from sensor signals on multimodal fault samples and automatically synthesize realistic one-dimensional signals of each fault. The framework is designed to produce realistic multimodal samples with fault labels and dual-discriminator structure is benefit to enhance the quality and diversity of synthesized data without mode collapse. Then, synthesized data can be used for data augmentation to improve the accuracy of imbalanced fault diagnosis. In order to evaluate the performance of the generative model, we introduce a set of assessments to evaluate quality and diversity of synthesized data, including quantitative statistical metrics and qualitative visualization. Finally, experiments on rolling bearings datasets from Case Western Reserve University (CWRU) are implemented to verify the effectiveness of the proposed approach for imbalanced fault diagnosis. Results demonstrate our method outperforms other widely used synthesis techniques in terms of data synthesis quality and fault diagnosis accuracy, and timeliness analysis also denotes our method can meet requirement of online fault diagnosis.