Application Of Generative Adversarial Networks Research Articles

Demiurge: A Music of Babel

Abstract The authors explore the unfinished, attempting a fragmentary and blurred snapshot of an ongoing research project on the musical application of generative adversarial networks, entitled Demiurge. The project was initiated in July 2020 as a collaboration between an international team of interdisciplinary artists and researchers led by Marek Poliks (instrument design) and Roberto Alonso Trillo (violin). In keeping with its namesake, Demiurge asks questions about creation, specifically in the era of machine learning, and works to formalize a new creative process endemic to and natural within a world of machine collaborators.

Super-resolution application of generative adversarial network on brain time-of-flight MR angiography: image quality and diagnostic utility evaluation.

To develop a generative adversarial network (GAN) model to improve image resolution of brain time-of-flight MR angiography (TOF-MRA) and to evaluate the image quality and diagnostic utility of the reconstructed images. We included 180 patients who underwent 1-min low-resolution (LR) and 4-min high-resolution (routine) brain TOF-MRA scans. We used 50 patients' datasets for training, 12 for quantitative image quality evaluation, and the rest for diagnostic validation. We modified a pix2pix GAN to suit TOF-MRA datasets and fine-tuned GAN-related parameters, including loss functions. Maximum intensity projection images were generated and compared using multi-scale structural similarity (MS-SSIM) and information theoretic-based statistic similarity measure (ISSM) index. Two radiologists scored vessels' visibilities using a 5-point Likert scale. Finally, we evaluated sensitivities and specificities of GAN-MRA in depicting aneurysms, stenoses, and occlusions. The optimal model was achieved with a lambda of 1e5 and L1 + MS-SSIM loss. Image quality metrics for GAN-MRA were higher than those for LR-MRA (MS-SSIM, 0.87 vs. 0.73; ISSM, 0.60 vs. 0.35; p.adjusted < 0.001). Vessels' visibility of GAN-MRA was superior to LR-MRA (rater A, 4.18 vs. 2.53; rater B, 4.61 vs. 2.65; p.adjusted < 0.001). In depicting vascular abnormalities, GAN-MRA showed comparable sensitivities and specificities, with greater sensitivity for aneurysm detection by one rater (93% vs. 84%, p < 0.05). An optimized GAN could significantly improve the image quality and vessel visibility of low-resolution brain TOF-MRA with equivalent sensitivity and specificity in detecting aneurysms, stenoses, and occlusions. • GAN could significantly improve the image quality and vessel visualization of low-resolution brain MR angiography (MRA). • With optimally adjusted training parameters, the GAN model did not degrade diagnostic performance by generating substantial false positives or false negatives. • GAN could be a promising approach for obtaining higher resolution TOF-MRA from images scanned in a fraction of time.

Integrated Schematic Design Method for Shear Wall Structures: A Practical Application of Generative Adversarial Networks

The intelligent design method based on generative adversarial networks (GANs) represents an emerging structural design paradigm where design rules are not artificially defined but are directly learned from existing design data. GAN-based methods have exhibited promising potential compared to conventional methods in the schematic design phase of reinforced concrete (RC) shear wall structures. However, for the following reasons, it is challenging to apply GAN-based approaches in the industry and to integrate them into the structural design process. (1) The data form of GAN-based methods is heterogeneous from that of the widely used computer-aided design (CAD) methods, and (2) GAN-based methods have high requirements on the hardware and software environment of the user’s computer. As a result, this study proposes an integrated schematic design method for RC shear wall structures, providing a workable GAN application strategy. Specifically, (1) a preprocessing method of architectural CAD drawings is proposed to connect the GAN with the upstream architectural design; (2) a user-friendly cloud design platform is built to reduce the requirements of the user’s local computer environment; and (3) a heterogeneous data transformation method and a parametric modeling procedure are proposed to automatically establish a structural analysis model based on GAN’s design, facilitating downstream detailed design tasks. The proposed method makes it possible for the entire schematic design phase of RC shear wall structures to be intelligent and automated. A case study reveals that the proposed method has a heterogeneous data transformation accuracy of 97.3% and is capable of generating shear wall layout designs similar to the designs of a competent engineer, with 225 times higher efficiency.

Application of Generative Adversarial Network and Diverse Feature Extraction Methods to Enhance Classification Accuracy of Tool-Wear Status

The means of accurately determining tool-wear status has long been important to manufacturers. Tool-wear status classification enables factories to avoid the unnecessary costs incurred by replacing tools too early and to prevent product damage caused by overly worn tools. While researchers have examined this topic for over a decade, most existing studies have focused on model development but have neglected two fundamental issues in machine learning: data imbalance and feature extraction. In view of this, we propose two improvements: (1) using a generative adversarial network to generate realistic computer numerical control machine vibration data to overcome data imbalance and (2) extracting features in the time domain, the frequency domain, and the time–frequency domain simultaneously for modeling and integrating these in an ensemble model. The experiment results demonstrate how both proposed modifications are reasonable and valid.

BrainGAN: Brain MRI Image Generation and Classification Framework Using GAN Architectures and CNN Models.

Deep learning models have been used in several domains, however, adjusting is still required to be applied in sensitive areas such as medical imaging. As the use of technology in the medical domain is needed because of the time limit, the level of accuracy assures trustworthiness. Because of privacy concerns, machine learning applications in the medical field are unable to use medical data. For example, the lack of brain MRI images makes it difficult to classify brain tumors using image-based classification. The solution to this challenge was achieved through the application of Generative Adversarial Network (GAN)-based augmentation techniques. Deep Convolutional GAN (DCGAN) and Vanilla GAN are two examples of GAN architectures used for image generation. In this paper, a framework, denoted as BrainGAN, for generating and classifying brain MRI images using GAN architectures and deep learning models was proposed. Consequently, this study proposed an automatic way to check that generated images are satisfactory. It uses three models: CNN, MobileNetV2, and ResNet152V2. Training the deep transfer models with images made by Vanilla GAN and DCGAN, and then evaluating their performance on a test set composed of real brain MRI images. From the results of the experiment, it was found that the ResNet152V2 model outperformed the other two models. The ResNet152V2 achieved 99.09% accuracy, 99.12% precision, 99.08% recall, 99.51% area under the curve (AUC), and 0.196 loss based on the brain MRI images generated by DCGAN architecture.

Digital Forensics Investigation for Attacks on Artificial Intelligence

The new research approaches are needed to be adopted to deal with security threats in Artificial Intelligence (AI)-based systems. This research is aimed at investigating the AI attacks that are “malicious by design.” It also deals with conceptualization of the problem and strategies for attacks on AI using digital forensic tools. A specific class of problems in adversarial attacks are tampering of images for computational processing in applications of digital photography, computer vision, pattern recognition (facial capping algorithms). State-of-the-art developments in forensics, such as 1. Application of end-to-end Neural Network training pipeline for image rendering and provenance analysis. 2. Deep fake image analysis using frequency methods, wavelet analysis, and tools like Amped Authenticate. 3. Capsule networks for detecting forged images. 4. Information transformation for feature extraction via image forensic tools, such as EXIF-SC, Splice Radar, and Noiseprint. 5. Application of generative adversarial networks (GAN) based models as anti-image forensics [8], will be studied in great detail and a new research approach will be designed incorporating these advancements for utility of digital forensics.

Applications of Generative Adversarial Networks (GANs) in Positron Emission Tomography (PET) imaging: A review

This paper reviews recent applications of Generative Adversarial Networks (GANs) in Positron Emission Tomography (PET) imaging. Recent advances in Deep Learning (DL) and GANs catalysed the research of their applications in medical imaging modalities. As a result, several unique GAN topologies have emerged and been assessed in an experimental environment over the last two years. The present work extensively describes GAN architectures and their applications in PET imaging. The identification of relevant publications was performed via approved publication indexing websites and repositories. Web of Science, Scopus, and Google Scholar were the major sources of information. The research identified a hundred articles that address PET imaging applications such as attenuation correction, de-noising, scatter correction, removal of artefacts, image fusion, high-dose image estimation, super-resolution, segmentation, and cross-modality synthesis. These applications are presented and accompanied by the corresponding research works. GANs are rapidly employed in PET imaging tasks. However, specific limitations must be eliminated to reach their full potential and gain the medical community's trust in everyday clinical practice.

Application of Generative Adversarial Network to Optimize Vehicle Allocation at Dispatch Stations of Paratransit Services

As aging populations increase worldwide, many governments have introduced the concept of paratransit services to assist individuals with limited mobility with transportation. A successful paratransit service must be able to satisfy most requests to the system; this success is typically related to the allocation of vehicles to dispatch stations. A suitable configuration can reduce unnecessary travel time and thus serve more people. This resembles the classic Dial-a-Ride problem, which previous studies have solved using heuristic algorithms. Most of these algorithms, however, incur heavy computational costs and, therefore, cannot be operated online, especially when there are many conditions to consider, many configuration requirements, or many vehicles requested. Therefore, this paper proposes an approach based on the generative adversary network (GAN), which can reduce computation significantly. In online environments, this approach can be implemented in just a few seconds. Furthermore, the amount of computation is not affected by the number of conditions, configuration requirements, or vehicles requested. This approach is based on three important concepts: (1) designing a GAN to solve the target problem; (2) using an improved Voronoi diagram to divide the overall service area to generate the input of the GAN generator; (3) using well-known system simulation software Arena to swiftly generate many conditions for the target problem and their corresponding best solutions to train the GAN. The efficiency of the proposed approach was verified using a case study of paratransit services in Yunlin, Taiwan.

Application of Generative Adversarial Networks (GANs) for Generating Synthetic Data and in Cybersecurity

Application of Generative Adversarial Networks (GANs) for Generating Synthetic Data and in Cybersecurity

Research on the Application of Generative Adversarial Networks in the Generation of Stock Market Forecast Trend Images

Investors make capital investment by buying stocks and expect to get a certain income from the stock market. When buying stocks, they need to draw up investment plans based on various information such as stock market historical transaction data and related news data of listed companies and collect and analyze these data. The data are relatively cumbersome and require a lot of time and effort. If you only rely on subjective analysis, the reference factors are often not comprehensive enough. At the same time, Internet social media, such as the speech in stock forums, also affect the judgment and behavior of investors, and investor sentiment will have a positive or negative effect on the stock market. This has an impact on the trend of stock prices. Therefore, this article proposes a stock market prediction model that uses data preprocessing technology based on past stock market transaction data to establish a stock market prediction model, and secondly, an image description generation model based on a generative confrontation network is designed. The model includes a generator and a discriminator. A time-varying preattention mechanism is proposed in the generator. This mechanism allows each image feature to pay attention to the image features of other stock markets to predict stock market trends so that the decoder can better understand the relational information in the image. The discriminator is based on the recurrent neural network and considers the degree of matching between the input sentence and the 4 reference sentences and the image features. Experiments show that the accuracy of the model is higher than that of the stock pretrend forecast model based on historical data, which proves the effectiveness of the data used in this paper in the stock price trend forecast.

Real-time super-resolution mapping of locally anisotropic grain orientations for ultrasonic non-destructive evaluation of crystalline material

Estimating the spatially varying microstructures of heterogeneous and locally anisotropic media non-destructively is necessary for the accurate detection of flaws and reliable monitoring of manufacturing processes. Conventional algorithms used for solving this inverse problem come with significant computational cost, particularly in the case of high-dimensional, nonlinear tomographic problems, and are thus not suitable for near-real-time applications. In this paper, for the first time, we propose a framework which uses deep neural networks (DNNs) with full aperture, pitch-catch and pulse-echo transducer configurations, to reconstruct material maps of crystallographic orientation. We also present the first application of generative adversarial networks (GANs) to achieve super-resolution of ultrasonic tomographic images, providing a factor-four increase in image resolution and up to a 50% increase in structural similarity. The importance of including appropriate prior knowledge in the GAN training data set to increase inversion accuracy is demonstrated: known information about the material’s structure should be represented in the training data. We show that after a computationally expensive training process, the DNNs and GANs can be used in less than 1 second (0.9 s on a standard desktop computer) to provide a high-resolution map of the material’s grain orientations, addressing the challenge of significant computational cost faced by conventional tomography algorithms.

Multi-class Generative Adversarial Networks: Improving One-class Classification of Pneumonia Using Limited Labeled Data.

From generating never-before-seen images to domain adaptation, applications of Generative Adversarial Networks (GANs) spread wide in the domain of vision and graphics problems. With the remarkable ability of GANs in learning the distribution and generating images of a particular class, they have been used for semi-supervised disease detection in medical images such as COVID-19 and Pneumonia in X-rays. However, the challenge is that if two classes of images share similar characteristics, the GAN might learn to generalize and hinder the classification of the two classes. In this paper, first we use MNIST and Fashion-MNIST datasets that are easy to visually inspect, to illustrate how similar images cause the GAN to generalize, leading to the poor classification of images. We then show how this generalization can misclassify pneumonia X-rays as healthy cases when using GANs for semi-supervised detection of pneumonia. We propose a modification to the traditional training of GANs that, using small sets of labeled data, allows for improved classification in similar classes of images in a semi-supervised learning framework.

On the application of generative adversarial networks for nonlinear modal analysis

Linear modal analysis is a useful and effective tool for the design and analysis of structures. However, a comprehensive basis for nonlinear modal analysis remains to be developed. In the current work, a machine learning scheme is proposed with a view to performing nonlinear modal analysis. The scheme is focussed on defining a one-to-one mapping from a latent ‘modal’ space to the natural coordinate space, whilst also imposing orthogonality of the mode shapes. The mapping is achieved via the use of the recently-developed cycle-consistent generative adversarial network (cycle-GAN) and an assembly of neural networks targeted on maintaining the desired orthogonality. The method is tested on simulated data from structures with cubic nonlinearities and different numbers of degrees of freedom, and also on data from an experimental three-degree-of-freedom set-up with a column-bumper nonlinearity. The results reveal the method’s efficiency in separating the ‘modes’. The method also provides a nonlinear superposition function, which in most cases has very good accuracy.

Application of Video-to-Video Translation Networks to Computational Fluid Dynamics

In recent years, the evolution of artificial intelligence, especially deep learning, has been remarkable, and its application to various fields has been growing rapidly. In this paper, I report the results of the application of generative adversarial networks (GANs), specifically video-to-video translation networks, to computational fluid dynamics (CFD) simulations. The purpose of this research is to reduce the computational cost of CFD simulations with GANs. The architecture of GANs in this research is a combination of the image-to-image translation networks (the so-called “pix2pix”) and Long Short-Term Memory (LSTM). It is shown that the results of high-cost and high-accuracy simulations (with high-resolution computational grids) can be estimated from those of low-cost and low-accuracy simulations (with low-resolution grids). In particular, the time evolution of density distributions in the cases of a high-resolution grid is reproduced from that in the cases of a low-resolution grid through GANs, and the density inhomogeneity estimated from the image generated by GANs recovers the ground truth with good accuracy. Qualitative and quantitative comparisons of the results of the proposed method with those of several super-resolution algorithms are also presented.

Generative Adversarial Networks

Abstract: Deep learning's breakthrough in the field of artificial intelligence has resulted in the creation of a slew of deep learning models. One of these is the Generative Adversarial Network, which has only recently emerged. The goal of GAN is to use unsupervised learning to analyse the distribution of data and create more accurate results. The GAN allows the learning of deep representations in the absence of substantial labelled training information. Computer vision, language and video processing, and image synthesis are just a few of the applications that might benefit from these representations. The purpose of this research is to get the reader conversant with the GAN framework as well as to provide the background information on Generative Adversarial Networks, including the structure of both the generator and discriminator, as well as the various GAN variants along with their respective architectures. Applications of GANs are also discussed with examples. Keywords: Generative Adversarial Networks (GANs), Generator, Discriminator, Supervised and Unsupervised Learning, Discriminative and Generative Modelling, Backpropagation, Loss Functions, Machine Learning, Deep Learning, Neural Networks, Convolutional Neural Network (CNN), Deep Convolutional GAN (DCGAN), Conditional GAN (cGAN), Information Maximizing GAN (InfoGAN), Stacked GAN (StackGAN), Pix2Pix, Wasserstein GAN (WGAN), Progressive Growing GAN (ProGAN), BigGAN, StyleGAN, CycleGAN, Super-Resolution GAN (SRGAN), Image Synthesis, Image-to-Image Translation.

Deep Convolutional Neural Networks With Ensemble Learning and Generative Adversarial Networks for Alzheimer's Disease Image Data Classification.

Recent advancements in deep learning (DL) have made possible new methodologies for analyzing massive datasets with intriguing implications in healthcare. Convolutional neural networks (CNN), which have proven to be successful supervised algorithms for classifying imaging data, are of particular interest in the neuroscience community for their utility in the classification of Alzheimer’s disease (AD). AD is the leading cause of dementia in the aging population. There remains a critical unmet need for early detection of AD pathogenesis based on non-invasive neuroimaging techniques, such as magnetic resonance imaging (MRI) and positron emission tomography (PET). In this comprehensive review, we explore potential interdisciplinary approaches for early detection and provide insight into recent advances on AD classification using 3D CNN architectures for multi-modal PET/MRI data. We also consider the application of generative adversarial networks (GANs) to overcome pitfalls associated with limited data. Finally, we discuss increasing the robustness of CNNs by combining them with ensemble learning (EL).

Super-Resolved in-Operando Observation of SOFC Pattern Electrodes

Nickel (Ni) is widely used for solid oxide fuel cell (SOFC) anodes due to high catalytic activity and electron conductivity. However, Ni coarsening and depletion are the major causes of electrode degradation during long time operation. In order to investigate the microstructural changes of Ni under actual operating environment, in-operando observations of Ni-pattern electrodes were carried using confocal laser microscope [1,2]. However, low resolution and shallow depth of field limit detailed investigation of the microstructure. On the other hand, high resolution images can be obtained by scanning electron microscope (SEM) or environmental SEM (ESEM), but the sample must be kept inside a vacuum chamber, which makes it extremely difficult to operate the sample cell with electrochemical reaction. Therefore, an automatic method to super-resolve the confocal laser scanning image is desired, which will be a great help to understand the Ni degradation phenomena under operation.Recently, deep learning is applied to image processing in many applications. Convolutional neural network (CNN) is widely used for various vision tasks including image classification and object detection. In particular, an application of generative adversarial network (GAN) [3] based on CNN has been successfully incorporated in image synthesis. The GAN consists of two networks, i.e. the generator to fabricate images and the discriminator to identify the output image as real or fake. Implementation of GAN such as image to image translation (pix2pix) [4] is highly expected in SOFC research.In the present study, two machine learning algorithms for improving the laser microscope image quality are investigated. The algorithms are applied to the Ni pattern electrodes deposited on mirror-polished yttrium stabilized zirconia (YSZ) substrate. Sets of low resolution (LR) laser scanning microscope images and high resolution (HR) SEM images of identical positions were collected for the training data.Firstly, the semantic segmentation proxy is proposed for the automatic material detection from the LR laser microscope images. The incorporated algorithms are based on CNN deep neural network. U-net [5] with encoder-decoder network is also conducted for comparison. In addition to visual assessment, phase fractions and triple phase boundary (TPB) density are calculated. The automatically segmented LR laser microscope images showed good agreement with the manually segmented HR SEM images. The estimated TPB density was close to the real value. The images segmented with U-net were better than those with encoder-decoder CNN in fine details, which indicates that U-net is superior in transferring local information.Secondly, a new method for synthesizing SEM-quality image from laser image is proposed. The super-resolution network incorporates pix2pix GAN architecture [4] which enables image to image translation. Figure 1A shows LR laser microscope image, which is an input for the pix2pix GAN. High resolution image generated by GAN is shown in Fig. 1B. Real HR SEM image is presented in Fig. 1C for comparison. Realistic HR images can be synthesized from LR laser image, which reveals that the proposed method is effective for super-resolving patterned electrode microstructures. The semantic segmentation proxy and super-resolution algorithms will enable quantitative evaluation of dynamic changes of SOFC electrodes under in-operando observation by a laser microscope.[1] Z. Jiao and N. Shikazono, J. Power Sources, 396, pp. 119–123 (2018).[2] Y. Suzuki et al., 14th European SOFC and SOE Forum, B1513 (2020).[3] I. Goodfellow et al., Advances in neural information processing systems, 27, pp. 2672-2680 (2014).[4] P. Isola et al., Proceeding of the IEEE on CVPR, pp. 1125-1134 (2017).[5] O. Ronneberger et al., MICCAI, pp. 234-241 (2015). Figure 1

Unsupervised Generative Adversarial Network for 3-D Microstructure Synthesis from 2-D Image

Three-dimensional microstructure reconstruction is indispensable for understanding the performance and reliability of SOFC electrodes. Not only the microstructure parameters, but also SOFC electrochemical characteristics such as overpotentials can be calculated when combined with numerical simulations. The most common method to analyze 3-D structures with a feature size of 0.01 – 10 μm is a focused ion beam - scanning electron microscope (FIB-SEM). However, there’s a tradeoff between reconstruction volume size and resolution. Besides, in order to obtain high quality reconstruction, accumulation of know-hows for pre and post-processing is required. Therefore, fast screening of the microstructures is not easy for FIB-SEM due to the difficulty of measurement. On the other hand, a single SEM image can be obtained very easily. An arising question is if it is possible to extract properties of microstructures and numerically synthesize 3-D model directly from a single 2-D SEM image.In recent years, machine learning techniques have become a powerful tool for image processing. Nevertheless, applications of machine learning in the field of porous materials, in particular fuel cells and batteries are still limited. The previous works concentrated on relatively simple artificial neural networks (ANN), but the class of neural networks that can effectively analyze visual images is the convolutional neural network (CNN). Gayon-Lombardo et al. (1) demonstrated a successful application of generative adversarial network (GAN) based on CNN layers to fabricate synthetic 3-D SOFC microstructures. It is indicated that the GAN-based approach can be very effective, but the algorithm requires 3-D teaching data (GAN3Dto3D).In the scope of this research, a novel method for fabrication of synthetic 3-D microstructure directly from a 2-D SEM image is proposed (Fig. 1A). The GAN2Dto3D network consists of a 3-D microstructure generator and a discriminator which operate on 2-D data. Each microstructure fabricated by the generator is treated as a set of individual cross-sectional 2-D images. The discriminator checks the validity of each 2-D slice in the generated microstructure in all spatial directions (height, width and thickness). Patches extracted from a real 2-D cross-section image are used as training data. In each training iteration, new random patches are extracted to increase the number of available training data. Additionally, the training images are augmented by random horizontal and vertical flipping. As all of the cross-section of the newly generated microstructure has to be statistically equivalent to the real 2-D cross-section, the generated 3-D microstructure is smooth in all spatial directions.The GAN2Dto3D approach is tested on the dataset of real SOFC microstructures. For the real microstructure, a set of nickel (Ni) – gadolinium doped ceria (GDC) microstructures was fabricated. All composite anode samples were prepared with NiO (AGC Seimi, Japan) and GDC (Gd0.1Ce0.9O ShinEtsu, Japan) powders mixed in a designated proportion and sintered at 1350 oC. Differences in the structure are due to the initial anode slurry composition and packing density. The solid phase fractions were controlled by GDC share in the composite in the range of 30 – 70 vol.%, and their porosity was controlled by isostatic pressing (2). 3-D microstructures were reconstructed by FIB-SEM and used as the ground truth data for the validation of GAN2Dto3D model. The results of the GAN2Dto3D algorithm applied to generate two synthetic microstructures with different GDC share and the porosity are shown in Fig. 1B and 1C. The size of the generated microstructures is 64 × 64 × 64 voxels, which is equivalent to 6.4 × 6.4 × 6.4 µm. The real microstructures reconstructed with FIB-SEM are shown for comparison. The proposed GAN algorithm produces realistic microstructures, which indicates that GAN has a great potential to enable fast evaluation of various samples without carrying out complex FIB-SEM measurements.1. Gayon-Lombardo, L. Mosser, N. P. Brandon, and S. J. Cooper, npj Comput. Mater., 6, 1–11 (2020).2. Komatsu, A. Sciazko, and N. Shikazono, J. Power Sources, 485, 229317 (2021). Figure 1

Generative adversarial networks for transition state geometry prediction.

This work introduces a novel application of generative adversarial networks (GANs) for the prediction of starting geometries in transition state (TS) searches based on the geometries of reactants and products. The multi-dimensional potential energy space of a chemical reaction often complicates the location of a starting TS geometry, leading to the correct TS combining reactants and products in question. The proposed TS-GAN efficiently maps the space between reactants and products and generates reliable TS guess geometries, and it can be easily combined with any quantum chemical software package performing geometry optimizations. The TS-GAN was trained and applied to generate TS guess structures for typical chemical reactions, such as hydrogen migration, isomerization, and transition metal-catalyzed reactions. The performance of the TS-GAN was directly compared to that of classical approaches, proving its high accuracy and efficiency. The current TS-GAN can be extended to any dataset that contains sufficient chemical reactions for training. The software is freely available for training, experimentation, and prediction at https://github.com/ekraka/TS-GAN.

GAN acoustic model for Kazakh speech synthesis

Recent studies on the application of generative adversarial networks (GAN) for speech synthesis have shown improvements in the naturalness of synthesized speech, compared to the conventional approaches. In this article, we present a new framework of GAN to train an acoustic model for speech synthesis. The proposed GAN consists of a generator and a pair of agent discriminators, where the generator produces acoustic parameters taking into account linguistic parameters; and the pair of agent discriminators are introduced to improve the naturalness of the synthesized speech. We feed the agents with acoustic and linguistic parameters, thereby the agents do not only examine the acoustic distribution, but also the relationship between linguistic and acoustic parameters. Training and testing were conducted on the Kazakh speech corpus. According to the results of this research, the proposed framework of GAN improves the accuracy of the acoustic model for the Kazakh text-to-speech system.