Generative Adversarial Network Framework Research Articles

TFSemantic: A Time–Frequency Semantic GAN Framework for Imbalanced Classification Using Radio Signals

Recently, wireless sensing techniques have been widely used for Internet of Things (IoT) applications. Unlike traditional device-based sensing, wireless sensing is contactless, pervasive, low cost, and non-invasive, making it highly suitable for relevant IoT applications. However, most existing methods are highly dependent on high-quality datasets, and the minority class will not achieve a satisfactory performance when suffering from a class imbalance problem. In this article, we propose a time–frequency semantic generative adversarial network framework (i.e., TFSemantic) to address the imbalanced classification problem in human activity recognition using radio frequency (RF) signals. Specifically, the TFSemantic framework can learn semantic features from the minority classes and then generate high-quality signals to restore data balance. It includes a data pre-processing module, a semantic extraction module, a semantic distribution module, and a data augmenter module. In the data pre-processing module, we process four different RF datasets (i.e., WiFi, RFID, UWB, and mmWave). We also develop Fourier semantic feature convolution and attention semantic feature embedding methods for the semantic extraction module. A discrete wavelet transform is utilized for reconstructed RF samples in the semantic distribution module. In data augmenter module, we design an associated loss function to achieve effective adversarial training. Finally, we validate the effectiveness of the proposed TFSemantic framework using different RF datasets, which outperforms several state-of-the-art methods.

Conditional generative adversarial network driven variable-duration single-lead to 12-lead electrocardiogram reconstruction

The standard 12-lead ECG can represent the cardiac status from twelve different perspectives, while the single-lead ECG just involves a single view, so it is challenging for the single-lead ECG to provide information as rich as the 12-lead ECG. Therefore, this study adopts the improved generative adversarial network (GAN) framework to model the potential mapping relation between single-lead and 12-lead ECG. Taking a heartbeat (1024 points) in lead I as input, in which the R-peak aligning strategy will be used for variable-duration reconstruction. The average correlation coefficients(CC) between the generated and the real signals are 0.742 at the heartbeat level and 0.744 at the segment level, and the CC in lead aVR could be up to 0.91 at the segment level. This study demonstrates that the generated 12-lead ECG signals effectively retain pathological features, thereby providing valuable multi-scale diagnostic information. To assess the efficacy of the diagnostic information derived from the generated signals, this study employs the arrhythmia classifiers for both single-lead and 12-lead ECG, achieving accuracy of 0.71 and 0.81, and macro-F1 scores of 0.66 and 0.77. For the generated 12-lead ECG, the accuracy and macro-F1 for arrhythmia classification could be 0.74 and 0.70, successfully bridging the gap between the single-lead ECG and the 12-lead ECG. In addition, the supplement experiments for other cases have been shown and compared. The source code is available at https://github.com/Zehui-Zhan/12-lead-reconstruction.git.

Physically‐Based Lighting for 3D Generative Models of Cars

AbstractRecent work has demonstrated that Generative Adversarial Networks (GANs) can be trained to generate 3D content from 2D image collections, by synthesizing features for neural radiance field rendering. However, most such solutions generate radiance, with lighting entangled with materials. This results in unrealistic appearance, since lighting cannot be changed and view‐dependent effects such as reflections do not move correctly with the viewpoint. In addition, many methods have difficulty for full, 360° rotations, since they are often designed for mainly front‐facing scenes such as faces. We introduce a new 3D GAN framework that addresses these shortcomings, allowing multi‐view coherent 360° viewing and at the same time relighting for objects with shiny reflections, which we exemplify using a car dataset. The success of our solution stems from three main contributions. First, we estimate initial camera poses for a dataset of car images, and then learn to refine the distribution of camera parameters while training the GAN. Second, we propose an efficient Image‐Based Lighting model, that we use in a 3D GAN to generate disentangled reflectance, as opposed to the radiance synthesized in most previous work. The material is used for physically‐based rendering with a dataset of environment maps. Third, we improve the 3D GAN architecture compared to previous work and design a careful training strategy that allows effective disentanglement. Our model is the first that generate a variety of 3D cars that are multi‐view consistent and that can be relit interactively with any environment map.

Management Analysis Method of Multivariate Time Series Anomaly Detection in Financial Risk Assessment

The significance of financial risk lies in its impact on economic stability and individual/institutional financial security. Effective risk management is crucial for market confidence and crisis prevention. Current methods for multivariate time series anomaly detection have limitations in adaptability and generalization. To address this, we propose an innovative approach integrating contrastive learning and Generative Adversarial Networks (GANs). We use geometric distribution masking for data augmentation to enhance dataset diversity. Within the GAN framework, we train a Transformer-based autoencoder to capture normal point distributions. We include contrastive loss in the discriminator to ensure robust generalization. Rigorous experiments on four real-world datasets show that our method effectively mitigates overfitting and outperforms state-of-the-art approaches. This enhances anomaly identification in risk management, paving the way for deep learning in finance, and offering insights for future research and practical use.

Functional Connectivity Prediction With Deep Learning for Graph Transformation.

Inferring resting-state functional connectivity (FC) from anatomical brain wiring, known as structural connectivity (SC), is of enormous significance in neuroscience for understanding biological neuronal networks and treating mental diseases. Both SC and FC are networks where the nodes are brain regions, and in SC, the edges are the physical fiber nerves among the nodes, while in FC, the edges are the nodes' coactivation relations. Despite the importance of SC and FC, until very recently, the rapidly growing research body on this topic has generally focused on either linear models or computational models that rely heavily on heuristics and simple assumptions regarding the mapping between FC and SC. However, the relationship between FC and SC is actually highly nonlinear and complex and contains considerable randomness; additional factors, such as the subject's age and health, can also significantly impact the SC-FC relationship and hence cannot be ignored. To address these challenges, here, we develop a novel SC-to-FC generative adversarial network (SF-GAN) framework for mapping SC to FC, along with additional metafeatures based on a newly proposed graph neural network-based generative model that is capable of learning the stochasticity. Specifically, a new graph-based conditional generative adversarial nets model is proposed, where edge convolution layers are leveraged to encode the graph patterns in the SC in the form of a graph representation. New edge deconvolution layers are then utilized to decode the representation back to FC. Additional metafeatures of subjects' profile information are integrated into the graph representation with newly designed sparse-regularized layers that can automatically select features that impact FC. Finally, we have also proposed new post hoc explainer of our SF-GAN, which can identify which subgraphs in SC strongly influence which subgraphs in FC by a new multilevel edge-correlation-guided graph clustering problem. The results of experiments conducted to test the new model confirm that it significantly outperforms existing state-of-the-art methods, with additional interpretability for identifying important metafeatures and subgraphs.

MADRAS-NET: A deep learning approach for detecting and classifying android malware using Linknet

Malware is an intentionally created malicious software that still poses a serious threat in cyberspace. Android malware has become one of the most significant online threats in recent years due to its increase in prevalence. Even though a lot of work has gone into identifying and categorizing Android malware, problems including data theft, difficulties identifying Android malware, and time-consuming detection remain unresolved. To overcome this issue, this paper proposed a Deep Learning (DL) Based Malware Attack DetectoR in Android Smartphones using LinkNET (MADRAS-NET) which effectively detects and mitigates the types of malwares in Android devices. The Max Abs Scaler receives a set of data as input first for pre-processing. Following pre-processing, the output is sent to LinkNET for classification. LinkNET uses the pre-processed data to identify malware, and then divides the output into three groups: real users, Penetho malware, and FakeAV malware. Lastly, the AndMal2020 dataset, which detects and classifies malware as well as malware families, is used to evaluate the suggested MADRAS-NET approach. The suggested LinkNET achieves a maximum accuracy of 99.81%, while other models, such as the Deep Belief Network (DBN) at 96.75%, the Generative Adversarial Network (GAN) at 94.42%, and the Long Short Term Memory Network (LSTM) at 93.11%, achieve similar results. The comparison of the MADRAS-NET model with the LSTM, GAN, and DBN frameworks indicates its better performance.

Generating 3D images of material microstructures from a single 2D image: a denoising diffusion approach

Three-dimensional (3D) images provide a comprehensive view of material microstructures, enabling numerical simulations unachievable with two-dimensional (2D) imaging alone. However, obtaining these 3D images can be costly and constrained by resolution limitations. We introduce a novel method capable of generating large-scale 3D images of material microstructures, such as metal or rock, from a single 2D image. Our approach circumvents the need for 3D image data while offering a cost-effective, high-resolution alternative to existing imaging techniques. Our method combines a denoising diffusion probabilistic model with a generative adversarial network framework. To compensate for the lack of 3D training data, we implement chain sampling, a technique that utilizes the 3D intermediate outputs obtained by reversing the diffusion process. During the training phase, these intermediate outputs are guided by a 2D discriminator. This technique facilitates our method’s ability to gradually generate 3D images that accurately capture the geometric properties and statistical characteristics of the original 2D input. This study features a comparative analysis of the 3D images generated by our method, SliceGAN (the current state-of-the-art method), and actual 3D micro-CT images, spanning a diverse set of rock and metal types. The results shown an improvement of up to three times in the Frechet inception distance score, a typical metric for evaluating the performance of image generative models, and enhanced accuracy in derived properties compared to SliceGAN. The potential of our method to produce high-resolution and statistically representative 3D images paves the way for new applications in material characterization and analysis domains.

Optimizing GANs for Cryptography: The Role and Impact of Activation Functions in Neural Layers Assessing the Cryptographic Strength

Generative Adversarial Networks (GANs) have surfaced as a transformative approach in the domain of cryptography, introducing a novel paradigm where two neural networks, the generator (akin to Alice) and the discriminator (akin to Bob), are pitted against each other in a cryptographic setting. A third network, representing Eve, attempts to decipher the encrypted information. The efficacy of this encryption–decryption process is deeply intertwined with the choice of activation functions employed within these networks. This study conducted a comparative analysis of four widely used activation functions within a standardized GAN framework. Our recent explorations underscore the superior performance achieved when utilizing the Rectified Linear Unit (ReLU) in the hidden layers combined with the Sigmoid activation function in the output layer. The non-linear nature introduced by the ReLU provides a sophisticated encryption pattern, rendering the deciphering process for Eve intricate. Simultaneously, the Sigmoid function in the output layer guarantees that the encrypted and decrypted messages are confined within a consistent range, facilitating a straightforward comparison with original messages. The amalgamation of these activation functions not only bolsters the encryption strength but also ensures the fidelity of the decrypted messages. These findings not only shed light on the optimal design considerations for GAN-based cryptographic systems but also underscore the potential of investigating hybrid activation functions for enhanced system optimization. In our exploration of cryptographic strength and training efficiency using various activation functions, we discovered that the “ReLU and Sigmoid” combination significantly outperforms the others, demonstrating superior security and a markedly efficient mean training time of 16.51 s per 2000 steps. This highlights the enduring effectiveness of established methodologies in cryptographic applications. This paper elucidates the implications of these choices, advocating for their adoption in GAN-based cryptographic models, given the superior results they yield in ensuring security and accuracy.

A physics-informed GAN framework based on model-free data-driven computational mechanics

Model-free data-driven computational mechanics, first proposed by Kirchdoerfer and Ortiz, replace phenomenological models with numerical simulations based on sample datasets in strain–stress space. In this study, we integrate this paradigm within physics-informed generative adversarial networks (GANs). We enhance the conventional physics-informed neural network framework by implementing the principles of data-driven computational mechanics into GANs. Specifically, the generator is informed by physical constraints, while the discriminator utilizes the closest strain–stress data to discern the authenticity of the generator’s output. This combined approach presents a new formalism to harness data-driven mechanics and deep learning to simulate and predict mechanical behaviors.

Employing texture loss to denoise OCT images using generative adversarial networks.

OCT is a widely used clinical ophthalmic imaging technique, but the presence of speckle noise can obscure important pathological features and hinder accurate segmentation. This paper presents a novel method for denoising optical coherence tomography (OCT) images using a combination of texture loss and generative adversarial networks (GANs). Previous approaches have integrated deep learning techniques, starting with denoising Convolutional Neural Networks (CNNs) that employed pixel-wise losses. While effective in reducing noise, these methods often introduced a blurring effect in the denoised OCT images. To address this, perceptual losses were introduced, improving denoising performance and overall image quality. Building on these advancements, our research focuses on designing an image reconstruction GAN that generates OCT images with textural similarity to the gold standard, the averaged OCT image. We utilize the PatchGAN discriminator approach as a texture loss to enhance the quality of the reconstructed OCT images. We also compare the performance of UNet and ResNet as generators in the conditional GAN (cGAN) setting, as well as compare PatchGAN with the Wasserstein GAN. Using real clinical foveal-centered OCT retinal scans of children with normal vision, our experiments demonstrate that the combination of PatchGAN and UNet achieves superior performance (PSNR = 32.50) compared to recently proposed methods such as SiameseGAN (PSNR = 31.02). Qualitative experiments involving six masked clinical ophthalmologists also favor the reconstructed OCT images with PatchGAN texture loss. In summary, this paper introduces a novel method for denoising OCT images by incorporating texture loss within a GAN framework. The proposed approach outperforms existing methods and is well-received by clinical experts, offering promising advancements in OCT image reconstruction and facilitating accurate clinical interpretation.

A GCN-based adaptive generative adversarial network model for short-term wind speed scenario prediction

Wind prediction is of great significance for wind energy utilization due to the stochastic nature of wind. To effectively facilitate various downstream decision-making tasks such as wind turbine control, predictive wind Scenario Generation (SG), which is capable of providing a set of deterministic instantiated wind prediction results, plays a critical role. In this paper, a novel Graph neural networks-based Adaptive Predictive Generative Adversarial Network (GAPGAN) model is proposed for accurate prediction of short-term future scenarios of a wind field. In GAPGAN, the original multivariate time series data are first reconstructed into the form of a graph, and spatiotemporal features are then extracted using Graph Convolutional Networks (GCNs). Next, a predictive generative adversarial network (PGAN) framework is proposed, which could generate different outputs corresponding to given historical observations as conditions. Finally, an adaptive PGAN training mechanism is introduced to stabilize the training process, and the best SG model is selected based on the proposed comprehensive evaluation system. Based on wind speed data collected from 11 wind turbines, computational experiments validate that the GAPGAN outperforms five benchmarking models in terms of point prediction accuracy, shape similarity, uncertainty prediction quality, and prediction scenario diversity.

Noise aware content-noise complementary GAN with local and global discrimination for low-dose CT denoising

In response to rising concerns over radiation exposure in computed tomography (CT) imaging, effective denoising methods for low-dose CT (LDCT) images are crucial. In recent years, the use of deep learning techniques especially generative adversarial networks (GANs) significantly enhanced the efficiency of LDCT denoising methods, surpassing traditional methods. However, GAN-based denoising methods often face challenges in preserving structural consistency and fine details. This study introduces a novel GAN framework with three accretions to enhance the effectiveness of LDCT denoising. Firstly, our generator is designed to leverage a complementary learning scheme between image noise and image content via two distinct paths. One path focuses on exploring the anatomical information of the image, while the second path is dedicated to learning the noise pattern. This complementary learning scheme provides stable noise cancellation while preserving the maximum structural information of the image. Subsequently, we propose a novel noise-conscious mean absolute error loss to address the challenge posed by the non-stationary characteristic of CT noise. In contrast to conventional MAE loss, this loss attentively prioritizes the different parts of the image based on the local distribution of noise in that region. We also incorporate a gradient-domain loss into the loss function, which inspires the generator to preserve precise image details through explicit guidance. Finally, this study adopted a U-Net-based design for the discriminator to better regularize the model by discriminating between the clean image and the denoised image at both global and local levels. The merit of this discriminator is that it can better adapt to the non-stationary environment of GAN training and guide the generator to produce denoised images that are locally and globally consistent. Our thorough experiments using abdominal CT and lung CT datasets demonstrate the superior performance of our approach compared to state-of-the-art methods.

A Physics-Informed Low-Shot Adversarial Learning for sEMG-Based Estimation of Muscle Force and Joint Kinematics.

Muscle force and joint kinematics estimation from surface electromyography (sEMG) are essential for real-time biomechanical analysis of the dynamic interplay among neural muscle stimulation, muscle dynamics, and kinetics. Recent advances in deep neural networks (DNNs) have shown the potential to improve biomechanical analysis in a fully automated and reproducible manner. However, the small sample nature and physical interpretability of biomechanical analysis limit the applications of DNNs. This paper presents a novel physics-informed low-shot adversarial learning method for sEMG-based estimation of muscle force and joint kinematics. This method seamlessly integrates Lagrange's equation of motion and inverse dynamic muscle model into the generative adversarial network (GAN) framework for structured feature decoding and extrapolated estimation from the small sample data. Specifically, Lagrange's equation of motion is introduced into the generative model to restrain the structured decoding of the high-level features following the laws of physics. A physics-informed policy gradient is designed to improve the adversarial learning efficiency by rewarding the consistent physical representation of the extrapolated estimations and the physical references. Experimental validations are conducted on two scenarios (i.e. the walking trials and wrist motion trials). Results indicate that the estimations of the muscle forces and joint kinematics are unbiased compared to the physics-based inverse dynamics, which outperforms the selected benchmark methods, including physics-informed convolution neural network (PI-CNN), vallina generative adversarial network (GAN), and multi-layer extreme learning machine (ML-ELM).

Stochastic representation and conditioning of process-based geological model by deep generative and recognition networks

Accurate and realistic geological modeling is the core of oil and gas development and production. In recent years, process-based methods are developed to produce highly realistic geological models by simulating the physical processes that reproduce the sedimentary events and develop the geometry. However, the complex dynamic processes are extremely expensive to simulate, making process-based models difficult to be conditioned to field data. In this work, we propose a comprehensive generative adversarial network framework as a machine-learning-assisted approach for mimicking the outputs of process-based geological models with fast generation. The main objective of our work is to obtain a continuous parametrization of the highly realistic process-based geological models which enables us to calibrate the models and condition the models to data. Numerical results are presented to illustrate the capability of our proposed methodology.

Stripe Extraction of Oceanic Internal Waves Using PCGAN with Small-Data Training

Playing a crucial role in ocean activities, internal solitary waves (ISWs) are of significant importance. Currently, the use of deep learning for detecting ISWs in synthetic aperture radar (SAR) imagery is gaining growing attention. However, these approaches often demand a considerable number of labeled images, which can be challenging to acquire in practice. In this study, we propose an innovative method employing a pyramidal conditional generative adversarial network (PCGAN). At each scale, it employs the framework of a conditional generative adversarial network (CGAN), comprising a generator and a discriminator. The generator works to produce internal wave patterns as authentically as possible, while the discriminator is designed to differentiate between images generated by the generator and reference images. The architecture based on pyramids adeptly captures the encompassing as well as localized characteristics of internal waves. The incorporation of upsampling further bolsters the model’s ability to recognize fine-scale internal wave stripes. These attributes endow the PCGAN with the capacity to learn from a limited amount of internal wave observation data. Experimental results affirm that the PCGAN, trained with just four internal wave images, can accurately detect internal wave stripes in the test set. Through comparative experiments with other segmentation models, we demonstrate the effectiveness and robustness of PCGAN.

Generative adversarial reduced order modelling

In this work, we present GAROM, a new approach for reduced order modeling (ROM) based on generative adversarial networks (GANs). GANs attempt to learn to generate data with the same statistics of the underlying distribution of a dataset, using two neural networks, namely discriminator and generator. While widely applied in many areas of deep learning, little research is done on their application for ROM, i.e. approximating a high-fidelity model with a simpler one. In this work, we combine the GAN and ROM framework, introducing a data-driven generative adversarial model able to learn solutions to parametric differential equations. In the presented methodology, the discriminator is modeled as an autoencoder, extracting relevant features of the input, and a conditioning mechanism is applied to the generator and discriminator networks specifying the differential equation parameters. We show how to apply our methodology for inference, provide experimental evidence of the model generalization, and perform a convergence study of the method.

Dilated convolution for enhanced extractive summarization: A GAN-based approach with BERT word embedding

Text summarization (TS) plays a crucial role in natural language processing (NLP) by automatically condensing and capturing key information from text documents. Its significance extends to diverse fields, including engineering, healthcare, and others, where it offers substantial time and resource savings. However, manual summarization is a laborious task, prompting the need for automated text summarization systems. In this paper, we propose a novel strategy for extractive summarization that leverages a generative adversarial network (GAN)-based method and Bidirectional Encoder Representations from Transformers (BERT) word embedding. BERT, a transformer-based architecture, processes sentence bidirectionally, considering both preceding and following words. This contextual understanding empowers BERT to generate word representations that carry a deeper meaning and accurately reflect their usage within specific contexts. Our method adopts a generator and discriminator within the GAN framework. The generator assesses the likelihood of each sentence in the summary while the discriminator evaluates the generated summary. To extract meaningful features in parallel, we introduce three dilated convolution layers in the generator and discriminator. Dilated convolution allows for capturing a larger context and incorporating long-range dependencies. By introducing gaps between filter weights, dilated convolution expands the receptive field, enabling the model to consider a broader context of words. To encourage the generator to explore diverse sentence combinations that lead to high-quality summaries, we introduce various noises to each document within our proposed GAN. This approach allows the generator to learn from a range of sentence permutations and select the most suitable ones. We evaluate the performance of our proposed model using the CNN/Daily Mail dataset. The results, measured using the ROUGE metric, demonstrate the superiority of our approach compared to other tested methods. This confirms the effectiveness of our GAN-based strategy, which integrates dilated convolution layers, BERT word embedding, and a generator-discriminator framework in achieving enhanced extractive summarization performance.

Single-Pixel Infrared Hyperspectral Imaging via Physics-Guided Generative Adversarial Networks

A physics-driven generative adversarial network (GAN) was utilized to demonstrate a single-pixel hyperspectral imaging (HSI) experiment in the infrared spectrum, eliminating the need for extensive dataset training in most data-driven deep neural networks. Within the GAN framework, the physical process of single-pixel imaging (SPI) was integrated into the generator, and its estimated one-dimensional (1D) bucket signals and the actual 1D bucket signals were employed as constraints in the objective function to update the network’s parameters and optimize the generator with the assistance of the discriminator. In comparison to single-pixel infrared HSI methods based on compressive sensing and physics-driven convolution neural networks, our physics-driven GAN-based single-pixel infrared HSI exhibits superior imaging performance. It requires fewer samples and achieves higher image quality. We believe that our physics-driven network will drive practical applications in computational imaging, including various SPI-based techniques.

GAN-Enhanced Nonlinear Fusion Model for Stock Price Prediction

Stock price prediction is a significant field of finance research for both academics and practitioners. Numerous studies have proved that the stock movement can be fully reflect various internal features of stock price including non-stationary behavior, high persistence in the conditional variance. The fusion of time-series prediction model such as Auto-Regressive Integrated Moving Average (ARIMA) and neural network is an availability but difficult approach for stock price prediction. Although the orientation has been studied through some methods in different research, there are still difficulties with the poor capture ability of time-series features and insufficient effectiveness of integrating temporal feature and frequency domain information. In this paper, we propose a Generative Adversarial Network (GAN) framework with the Convolution Neural Networks (CNN) as the discriminator and a hybrid model as the generator for forecasting the stock price. The hybrid model includes Attention-based Convolution Neural Networks (ACNN), Long Short-Term Memory (LSTM), and ARIMA model. Moreover, this proposed framework uses the Generative Adversarial patten and Attention Mechanism to achieve effective analysis and feature extraction for stock price movement. The extensive experiments in different history periods of dataset demonstrate an improvement in forecasting of stock price using our model as compared to the baseline models.

A Generative Model to Embed Human Expressivity into Robot Motions.

This paper presents a model for generating expressive robot motions based on human expressive movements. The proposed data-driven approach combines variational autoencoders and a generative adversarial network framework to extract the essential features of human expressive motion and generate expressive robot motion accordingly. The primary objective was to transfer the underlying expressive features from human to robot motion. The input to the model consists of the robot task defined by the robot's linear velocities and angular velocities and the expressive data defined by the movement of a human body part, represented by the acceleration and angular velocity. The experimental results show that the model can effectively recognize and transfer expressive cues to the robot, producing new movements that incorporate the expressive qualities derived from the human input. Furthermore, the generated motions exhibited variability with different human inputs, highlighting the ability of the model to produce diverse outputs.