Generative Network Research Articles

Transformer fault diagnosis based on adversarial generative networks and deep stacked autoencoder

Establishing a deep learning model for transformer fault diagnosis using transformer oil chromatogram data requires a large number of fault samples. The lack and imbalance of oil chromatogram data can lead to overfitting, lack of representativeness of the model, and unsatisfactory prediction results on test set data, making it difficult to accurately diagnose transformer faults. A conditional Wasserstein generative adversarial network with gradient penalty optimization (CWGAN-GP) is adopted in this paper, which based on gradient penalty optimization and expand the oil chromatography fault samples of 500 sets of transformer oil chromatography data with 5 types of faults. The proposed method is used to classify transformer faults using a deep autoencoder, and the sample quality of the neural network model proposed in this paper is compared with several other variants of generative adversarial neural network models. The research results show that after using the method proposed in this paper for sample expansion, the overall accuracy of fault diagnosis can reach 93.2 %, which is 4.98 % higher than the original imbalanced samples. Compared with other sample expansion methods, the accuracy of fault diagnosis of the algorithm in this paper is improved by 1.70 %–3.05 %.

3D-DGGAN: A Data-Guided Generative Adversarial Network for High Fidelity in Medical Image Generation.

Three-dimensional images are frequently used in medical imaging research for classification, segmentation, and detection. However, the limited availability of 3D images hinders research progress due to network training difficulties. Generative methods have been proposed to create medical images using AI techniques. Nevertheless, 2D approaches have difficulty dealing with 3D anatomical structures, which can result in discontinuities between slices. To mitigate these discontinuities, several 3D generative networks have been proposed. However, the scarcity of available 3D images makes training these networks with limited samples inadequate for producing high-fidelity 3D images. We propose a data-guided generative adversarial network to provide high fidelity in 3D image generation. The generator creates fake images with noise using reference code obtained by extracting features from real images. The generator also creates decoded images using reference code without noise. These decoded images are compared to the real images to evaluate fidelity in the reference code. This generation process can create high-fidelity 3D images from only a small amount of real training data. Additionally, our method employs three types of discriminator: volume (evaluates all the slices), slab (evaluates a set of consecutive slices), and slice (evaluates randomly selected slices). The proposed discriminator enhances fidelity by differentiating between real and fake images based on detailed characteristics. Results from our method are compared with existing methods by using quantitative analysis such as Fréchet inception distance and maximum mean discrepancy. The results demonstrate that our method produces more realistic 3D images than existing methods.

Identification of propagation path and root cause of faults based on generative adversarial networks in industrial systems

Fault diagnosis and detection (FDD) improves product quality, process safety, and profitability in modern industrial plants. Faults can propagate between different parts of plants due to connections between devices and control loops. As a result, control room operators may receive multiple alarms that distract their attention from addressing the main causes of abnormal situations. Therefore, investigating causal relationships between process variables is essential in determining the root cause of faults. This study proposes a new systematic approach for causality analysis to identify the propagation path and root cause of faults in industrial plants. The research's innovations are categorized into two main parts. Initially, the study utilizes machine learning algorithms to identify causal relationships using probabilistic concepts. Additionally, it introduces a novel architecture by merging the generative adversarial network (GAN) with the variational auto-encoder (VAE). The process variables are mapped to a latent space, where an adversarial generative network will investigate the causal relationship between variables. Finally, the Tennessee Eastman Process simulation and gas turbine industrial data are used to evaluate the proposed algorithm.

Transferable Sparse Adversarial Attack on Modulation Recognition With Generative Networks

Transferable Sparse Adversarial Attack on Modulation Recognition With Generative Networks

Text Feature Adversarial Learning for Text Generation With Knowledge Transfer From GPT2.

Text generation is a key component of many natural language tasks. Motivated by the success of generative adversarial networks (GANs) for image generation, many text-specific GANs have been proposed. However, due to the discrete nature of text, these text GANs often use reinforcement learning (RL) or continuous relaxations to calculate gradients during learning, leading to high-variance or biased estimation. Furthermore, the existing text GANs often suffer from mode collapse (i.e., they have limited generative diversity). To tackle these problems, we propose a new text GAN model named text feature GAN (TFGAN), where adversarial learning is performed in a continuous text feature space. In the adversarial game, GPT2 provides the "true" features, while the generator of TFGAN learns from them. TFGAN is trained by maximum likelihood estimation on text space and adversarial learning on text feature space, effectively combining them into a single objective, while alleviating mode collapse. TFGAN achieves appealing performance in text generation tasks, and it can also be used as a flexible framework for learning text representations.

Chemical fault diagnosis network based on single domain generalization

Recent developments in fault diagnosis have leveraged domain generalization to address the issue of domain shift. Most existing methods focus on learning domain-invariant representations from multiple source domains. However, collecting valuable fault samples from varying operational conditions is challenging, and it is common for available data to originate from a single operational condition. Thus, this paper introduces a Multi-scale generative and adversarial Metric networks (MGAMN) for Chemical Process Fault Diagnosis. To enhance model generalization, a domain generation module was developed to create augmented domains with significant distributional differences from the source domain. The diagnostic task module then extracts domain-invariant features from both the source and augmented domains. A multi-scale generation strategy is established, utilizing multi-scale deep separable convolutions (Dsc) to ensure that the generated samples contain rich state information. Additionally, an adversarial training and metric learning strategy is designed to learn generalized features capable of resisting unknown domain shifts. Extensive diagnostic experiments on the non-isothermal continuous stirred tank reactor (CSTR) and the Tennessee Eastman Process (TEP) chemical datasets validate the effectiveness of the proposed method. Moreover, ablation studies confirm the effectiveness of the proposed modular strategy, demonstrating significant potential for practical applications.

Image fusion using Y-net-based extractor and global-local discriminator

Although some deep learning-based image fusion approaches have realized promising results, how to extract information-rich features from different source images while preserving them in the fused image with less distortions remains challenging issue that needs to be addressed. Here, we propose a well worked-out GAN-based scheme with multi-scale feature extractor and global-local discriminator for infrared and visible image fusion. We use Y-Net as the backbone architecture to design the generator network, and introduce the residual dense block (RDblock) to yield more realistic fused images for infrared and visible images by learning discriminative multi-scale representations that are closer to the essence of different modal images. During feature reconstruction, the cross-modality shortcuts with contextual attention (CMSCA) are employed to selectively aggregate features at different scales and different levels to construct information-rich fused images with better visual effect. To ameliorate the information content of the fused image, we not only constrain the structure and contrast information using structural similarity index, but also evaluate the intensity and gradient similarities at both feature and image levels. Two global-local discriminators that combine global GAN with PatchGAN as a unified architecture help to dig for finer differences between the generated image and reference images, which force the generator to learn both the local radiation information and pervasive global details in two source images. It is worth mentioning that image fusion is achieved during confrontation without fusion rules. Lots of assessment tests demonstrate that the reported fusion scheme achieves superior performance against state-of-the-art works in meaningful information preservation.

A Digital Animation Generation Model Based on Cycle Adversarial Neural Network

This paper proposes a digital animation generation model based on Cycle Adversarial Neural Network (CycleGAN). Compared with the classical CycleGAN, this research presents a multi-attention approach to enhance the network’s generalization. Specifically, a style enhancement module and a style cross-attention mechanism are introduced into the generator network structure, which enables the model to better parse the structural information of the content image and realize the accurate matching of content features and style features. Furthermore, the introduction of a multi-scale discriminator with a fused attention mechanism enhances the preservation of content information from the source image in the output animated image. The experiments conducted in this study demonstrate that the model proposed in this paper exhibits superior performance in generating digital animation. This model not only enhances the realism and variety of the generated effects, but also achieves significant improvements in the coherence and stability of long time-series animation. The contribution of this paper lies in the introduction of the multi-attention mechanism, which enhances the generalization of the network and is of great theoretical and practical significance for the development of the field of digital animation generation.

Ontoloji-Based Generalized Zero-Shot Learning with Generative Networks

Ontoloji-Based Generalized Zero-Shot Learning with Generative Networks

Real-time masked face recognition using deep learning-based double generator network

Real-time masked face recognition using deep learning-based double generator network

Image Generation from Text

Abstract: This work diverts from the usual by using Generative Adversarial Networks (GANs) to intentionally create flawed yet captivating images. We introduce a controlled chaos system, where a stalling generator network and a disoriented discriminator network co-exist. This strange training method encourages the development of peculiar irregularities within semi-realistic images. We welcome the unpredictability that arises from this chaos, purposely avoiding issues like mode collapse. Moreover, the project delves into a new evaluation paradigm for evaluating these deliberately flawed images. We go beyond typical metrics, exploring the use of faulty metrics and questionable visual inspection techniques. In addition, we investigate the impact of clumsy dataset selection, potentially employing unconventional or tampered datasets to train the GAN for specific types of errors. This work explores the potential of GANs for jumbled up art creation, data manipulation, and image confusion, opening a new frontier in GAN applications. While potentially causing confusion in the field, this research also reveals the unimaginable artistic and data exploration capabilities of GANs.

Self-clustered GAN for precipitation nowcasting

This paper proposes a novel GAN framework with self-clustering approach for precipitation nowcasting (ClusterCast). Previous studies have primarily captured the motion vector using only a single latent space, making the models difficult to adapt to disparate space-time distribution of precipitation. Environmental factors (e.g., regional characteristics and precipitation scale) have an impact on precipitation systems and can cause non-stationary distribution. To tackle this problem, our key idea is to train a generator network to predict future radar frames by learning a sub-network that automatically labels precipitation types from a generative model. The training process consists of (i) clustering the hierarchical features derived from the generator stem using a sub-network and (ii) predicting future radar frames according to the self-supervised labels, enabling heterogeneous latent representation. Additionally, we attempt an ensemble forecast that prescribes random perturbations to improve performance. With the flexibility of representation learning, ClusterCast enables the model to learn precipitation distribution more accurately. Results indicate that our method generates non-blurry future frames by preventing mode collapse, and the proposed method demonstrates robustness across various precipitation scenarios. Extensive experiments demonstrate that our method outperforms four benchmarks on a 2-h prediction basis with a mean squared error (MSE) of 8.9% on unseen datasets.

Automated extraction of Tridacna shell growth patterns via machine learning for enhanced paleoclimate/paleoweather research

Tridacna spp. are valuable archives for paleoclimate and paleoweather research due to their distinct daily growth patterns and the sensitivity of the daily growth patterns to environment changes. However, manually identifying daily growth lines and measuring the daily growth increment width (DGIW) of Tridacna shells from Laser Scanning Confocal Microscopy (LSCM) images is a tedious task that has become a significant barrier to Tridacna studies. This paper addresses this challenge by integrating machine learning into Tridacna research for the first time to automate the calculation of the number of daily growth lines and DGIW of Tridacna shells. Specifically, we propose an unsupervised generative adversarial attention network called TriGAN to automatically recognize distinct daily growth lines of Tridacna shells from LSCM images. Utilizing modern Tridacna specimens collected from the South China Sea, our experimental results demonstrate that TriGAN can effectively reconstruct the ambiguous and blurred regions in LSCM images and produce higher quality images of daily growth patterns compared to existing image generation networks. Furthermore, the daily growth line number and DGIW of Tridacna shells can be counted automatically from the images recognized by TriGAN, which are in good agreement with the statistical results obtained manually from the original LSCM images (R = 0.7, p < 0.01 for the DGIW profile of T. gigas specimen MD1 and R = 0.6, p < 0.01 for T. derasa specimen XB10). This automated method provides an efficient solution for researching the laminar chronology of Tridacna.

Stress Detection with Natural Language Processing Techniques from Social Media Articles

Abstract— Stress have significant effects on health and well-being. Therefore, NLP's ability to identify stress situations is critical to providing early intervention and support. NLP can extract meaningful clues from people's online communications and written texts; This makes it a powerful tool for detecting signs of stress. Identifying signs of stress early can help individuals improve their ability to cope with stress by providing appropriate support and guidance; NLP can automate this process. Considering these situations, this study tried to detect the stress states of the users using the Stress Detection from Social Media Articles dataset. Generative Pre-trained Transformer (GPT), Logistic Regression (LR) and Artificial Neural Network (ANN) methods were used for classification. Doc2vec method was used to convert texts into vector data. In the classification processes, the highest classification success was obtained from the ANN model with 80.34%. It is thought that this success can be increased with more data and different models.

Efficient Generation of Conformer Ensembles Using Internal Coordinates and a Generative Directional Graph Convolution Neural Network.

We present a neural-network-based high-throughput molecular conformer-generation algorithm. A chemical graph-convolutional network is trained to predict low-energy conformers in internal coordinate representation (bond lengths, bond, and torsion angles), starting from two-dimensional (2D) chemical topology. Generative neural network (NN) architecture performs denoising from torsion space, producing conformer ensembles with populations that are well correlated with torsion energy profiles. Short force-field-based energy minimization is applied to refine final conformers. All computation-intensive stages of the algorithm are GPU-optimized. The procedure (termed GINGER) is benchmarked on a commonly used test set of bioactive three-dimensional (3D) conformers from the PDB. We demonstrate highly competitive results in conformer recovery and throughput rates suitable for giga-scale compound library processing. A web server that allows interactive conformer ensemble generation by GINGER and their viewing is made freely available at https://www.molsoft.com/gingerdemo.html.

Application of SOM-GAN based deep learning technology in the security protection of rural bank depositors' funds information

Since the reform and opening up, the status of China's banking industry has been significantly improved, and the traditional banking industry has gradually cross-integrated with Internet technology [1], which has become an important support for national economic development and a core component of the financial system. For example, in the original physical bank and online bank, new Internet financial models have been expanded, such as online financial management, bank electronic accounts, mobile banking, P2P models, third-party payment platforms, crowdfunding, etc. [2]. However, in the period of rapid development of the transformation and upgrading of traditional banks facilitated by information technology, many problems have emerged, such as the relaxation of supervision, the lag of policies and regulations, and the difficulty of reasonable protection of private data, etc. These problems may affect the interests of depositors and even the state. For example, in 2022, China's Henan rural bank 40 billion event, this event involved about 400,000 depositors, and platforms such as Du Xiaoman and Xiaomi Finance have related cooperation with these rural banks. Therefore, reasonable security protection of depositors' fund information and information processing and encryption under certain deep learning technology have long become the focus of experts' research in information technology. For example, Tian Gengwen has conducted an in-depth discussion on customer information security in the digital transformation of banks [3]. For example, Yang Wanrong conducted an in-depth study on the path and measures of data security governance of small and medium-sized banks [4]. For example, Zhang Wenli, Peng Xiaolei et al have made an in-depth elaboration on the bank data security solution [5]. Following hot issues closely, this paper provides a method of deep learning technology based on SOM-GAN in the security protection of rural bank depositors' funds information, aiming to combine the advantages of self-organizing mapping neural network and generative adantagonistic network unsupervised learning. At the same time, it can complement the competition mode of self-organized neural network, generate the discriminant characteristics of adversarial network and generate real data to process and review the relevant data of the fund information security of rural bank depositors, ensure the fund management security of rural bank depositors, and realize the monitoring and analysis of their funds and the surrounding environment when depositors are storing funds. Thus, the security, confidentiality, authenticity and integrity of depositors' funds information are guaranteed, and hidden dangers such as theft of depositors' funds, loss of depositors' personal information and threats to depositors' personal information security brought about by the 40bn deposit red code incident of Henan Bank in 2022 are eliminated.

Deep convolutional generative adversarial network for generation of computed tomography images of discontinuously carbon fiber reinforced polymer microstructures

Computed tomography images are of utmost importance when characterizing the heterogeneous and complex microstructure of discontinuously fiber reinforced polymers. However, the devices are expensive and the scans are time- and energy-intensive. Through recent advances in generative adversarial networks, the instantaneous generation of endless numbers of images that are representative of the input images and hold physical significance becomes possible. Hence, this work presents a deep convolutional generative adversarial network trained on approximately 30,000 input images from carbon fiber reinforced polyamide 6 computed tomography scans. The challenge lies in the low contrast between the two constituents caused by the close proximity of the density of polyamide 6 and carbon fibers as well as the small fiber diameter compared to the necessary resolution of the images. In addition, the stochastic, heterogeneous microstructure does not follow any logical or predictable rules exacerbating their generation. The quality of the images generated by the trained network of 256 pixel ×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ imes$$\\end{document} 256 pixel was investigated through the Fréchet inception distance and nearest neighbor considerations based on Euclidean distance and structural similarity index measure. Additional visual qualitative assessment ensured the realistic depiction of the complex mixed single fiber and fiber bundle structure alongside flow-related physically feasible positioning of the fibers in the polymer. The authors foresee additionally huge potential in creating three-dimensional representative volume elements typically used in composites homogenization.

W-DRAG: A joint framework of WGAN with data random augmentation optimized for generative networks for bone marrow edema detection in dual energy CT

Dual-energy computed tomography (CT) is an excellent substitute for identifying bone marrow edema in magnetic resonance imaging. However, it is rarely used in practice owing to its low contrast. To overcome this problem, we constructed a framework based on deep learning techniques to screen for diseases using axial bone images and to identify the local positions of bone lesions. To address the limited availability of labeled samples, we developed a new generative adversarial network (GAN) that extends expressions beyond conventional augmentation (CA) methods based on geometric transformations. We theoretically and experimentally determined that combining the concepts of data augmentation optimized for GAN training (DAG) and Wasserstein GAN yields a considerably stable generation of synthetic images and effectively aligns their distribution with that of real images, thereby achieving a high degree of similarity. The classification model was trained using real and synthetic samples. Consequently, the GAN technique used in the diagnostic test had an improved F1 score of approximately 7.8% compared with CA. The final F1 score was 80.24%, and the recall and precision were 84.3% and 88.7%, respectively. The results obtained using the augmented samples outperformed those obtained using pure real samples without augmentation. In addition, we adopted explainable AI techniques that leverage a class activation map (CAM) and principal component analysis to facilitate visual analysis of the network’s results. The framework was designed to suggest an attention map and scattering plot to visually explain the disease predictions of the network.

Deep Multimodal Data Fusion

Multimodal Artificial Intelligence (Multimodal AI), in general, involves various types of data (e.g., images, texts, or data collected from different sensors), feature engineering (e.g., extraction, combination/fusion), and decision-making (e.g., majority vote). As architectures become more and more sophisticated, multimodal neural networks can integrate feature extraction, feature fusion, and decision-making processes into one single model. The boundaries between those processes are increasingly blurred. The conventional multimodal data fusion taxonomy (e.g., early/late fusion), based on which the fusion occurs in, is no longer suitable for the modern deep learning era. Therefore, based on the main-stream techniques used, we propose a new fine-grained taxonomy grouping the state-of-the-art (SOTA) models into five classes: Encoder-Decoder methods, Attention Mechanism methods, Graph Neural Network methods, Generative Neural Network methods, and other Constraint-based methods. Most existing surveys on multimodal data fusion are only focused on one specific task with a combination of two specific modalities. Unlike those, this survey covers a broader combination of modalities, including Vision + Language (e.g., videos, texts), Vision + Sensors (e.g., images, LiDAR), and so on, and their corresponding tasks (e.g., video captioning, object detection). Moreover, a comparison among these methods is provided, as well as challenges and future directions in this area.

PCSAGAN: a physics-constrained generative network based on self-attention for high-fidelity flow field reconstruction

PCSAGAN: a physics-constrained generative network based on self-attention for high-fidelity flow field reconstruction