Conditional Generative Adversarial Nets Research Articles

Functional Connectivity Prediction With Deep Learning for Graph Transformation.

Inferring resting-state functional connectivity (FC) from anatomical brain wiring, known as structural connectivity (SC), is of enormous significance in neuroscience for understanding biological neuronal networks and treating mental diseases. Both SC and FC are networks where the nodes are brain regions, and in SC, the edges are the physical fiber nerves among the nodes, while in FC, the edges are the nodes' coactivation relations. Despite the importance of SC and FC, until very recently, the rapidly growing research body on this topic has generally focused on either linear models or computational models that rely heavily on heuristics and simple assumptions regarding the mapping between FC and SC. However, the relationship between FC and SC is actually highly nonlinear and complex and contains considerable randomness; additional factors, such as the subject's age and health, can also significantly impact the SC-FC relationship and hence cannot be ignored. To address these challenges, here, we develop a novel SC-to-FC generative adversarial network (SF-GAN) framework for mapping SC to FC, along with additional metafeatures based on a newly proposed graph neural network-based generative model that is capable of learning the stochasticity. Specifically, a new graph-based conditional generative adversarial nets model is proposed, where edge convolution layers are leveraged to encode the graph patterns in the SC in the form of a graph representation. New edge deconvolution layers are then utilized to decode the representation back to FC. Additional metafeatures of subjects' profile information are integrated into the graph representation with newly designed sparse-regularized layers that can automatically select features that impact FC. Finally, we have also proposed new post hoc explainer of our SF-GAN, which can identify which subgraphs in SC strongly influence which subgraphs in FC by a new multilevel edge-correlation-guided graph clustering problem. The results of experiments conducted to test the new model confirm that it significantly outperforms existing state-of-the-art methods, with additional interpretability for identifying important metafeatures and subgraphs.

Downhole Track Detection via Multi-dimensional Conditional Generative Adversarial Nets

Frequent mine disasters cause a large number of casualties and property losses. Autonomous driving is a fundamental measure for solving this problem, and track detection is one of the key technologies for computer vision to achieve downhole automatic driving. The track detection result based on the existing models lacks the semantic detail of the tracks and relies too much on visual postprocessing technology. Therefore, this paper proposes a track detection model based on the multi-dimensional conditional generative adversarial network. First, the generator is decomposed into global and local parts using a multi-granularity structure. Second, a multi-scale shared convolution structure is introduced into the discriminator to further guide the generator. In addition, this paper proposes a penalty mechanism based on Monte Carlo search to enhance the semantic constraints in the image generation process. Compared with the state-of-the-art semantic segmentation algorithms, extensive experiments on the downhole scene dataset demonstrate proposed model achieved the best results in terms of pixel accuracy, intersection-over-union (IOU) and the track detection accuracy. This paper provides a new idea for track line detection. In the future, the model can also be applied to other segmentation problems as well. Code and data will be shared.

Multi-Disciplinary and Multi-Objective Optimization Method Based on Machine Learning

The optimization of aircraft is a typical multidisciplinary and multi-objective problem. To solve this problem, the difficulty lies not only in the high cost of discipline performance evaluation but also in the complex coupling relationship between different disciplines. To improve the optimization efficiency, a new optimization method is proposed, including two new algorithms: conditional generative adversarial nets with vector similarity (VS-CGAN) and distributed single-step deep reinforcement learning with transfer learning (TL-DSDRL). For low-cost disciplines, VS-CGAN learns the relationship between variables and objectives through presampling to compress the variable domains. The cosine function is used to evaluate the similarity between the random noise and generated variables to avoid mode collapse. For high-cost disciplines, TL-DSDRL improves optimization efficiency through pretraining. The newly designed reward function and multi-agent cooperation mechanism enhance the multi-objective search ability of reinforcement learning.

A spatial weather generator based on conditional deep convolution generative adversarial nets (cDCGAN)

A spatial weather generator based on conditional deep convolution generative adversarial nets (cDCGAN)

Synthetic Data as a Proxy for Real-World Electronic Health Records in the Patient Length of Stay Prediction

While generative artificial intelligence has gained popularity, e.g., for the creation of images, it can also be used for the creation of synthetic tabular data. This bears great potential, especially for the healthcare industry, where data are often scarce and underlie privacy restrictions. For instance, the creation of synthetic electronic health records (EHR) promises to improve the usage of machine learning algorithms, which usually work with large amounts of data. This also applies for the prediction of the patient length of stay (LOS), a key measure for hospitals. Thereby, the LOS represents one of the core tools for decision makers to plan the allocation of resources. Thus, this paper aims to add to the still-young research concerning the application of generative adversarial nets (GAN) on tabular EHR. It does that with the intention to leverage the advantages of synthetic data for the prediction of the LOS in order to contribute to the efficiency-enhancing and cost-saving aspirations of hospitals and insurance companies. Therefore, the applicability of synthetic data that is generated using GANs as a proxy for scarce real-world EHR for the patient LOS multi-class classification task is examined. In this context, the Conditional Tabular GAN (CTGAN) and the Copula GAN are selected as the underlying models as they are state-of-the-art GAN architectures designed for generating synthetic tabular data. The CTGAN is found to be the superior model for the underlying use case. Nevertheless, the paper shows that there is still room for improvement when applying state-of-the-art GAN architectures to clinical healthcare data.

Representation learning with CGAN for casual inference

Conditional Generative Adversarial Nets (CGAN) is often used to improve conditional image generation performance. However, there is little research on Representation learning with CGAN for causal inference. This paper proposes a new method for finding representation learning functions by adopting the adversarial idea. We apply the pattern of CGAN and theoretically demonstrate the feasibility of finding a suitable representation function in the context of two distributions being balanced. The theoretical result shows that when two distributions are balanced, the ideal representation function can be found and thus can be used to further research.

A joint model for hot-rolled strip quality in advance prediction

ABSTRACT Aiming at the problem that previous product quality prediction models cannot directly assess product quality before hot rolling, this paper presents a joint model for process parameters generation and quality prediction based on Conditional Generative Adversarial Nets and Artificial Neural Network. After generated actual process parameters by the generation module, this model would predict the quality of products ahead of schedule according to the generated parameters before hot rolling, which do not rely on inputting actual process parameter online anymore. Finally, the model has been trained and tested with the actual data from a certain hot rolling plant. The experimental results show that the generated process parameters agree well with actual production and quality prediction accuracy can meet the production requirements, which confirms the proposed model can be applied to simulate the actual rolling process and predict strip quality ahead of hot rolling production, providing a reference for the adjustment of planning and scheduling in the future.

Imbalanced Working States Recognition of Sucker Rod Well Dynamometer Cards Based on Data Generation and Diversity Augmentation

Summary During sucker rod pump production, there is a commonly seen problem of class imbalance, which refers to the differences in the amount of data accumulated under different working conditions. This problem in rod pump diagnosis can lead to unsatisfactory classification results of surface dynamometer cards under working conditions with fewer samples. Therefore, this study adopts the conditional generative adversarial nets (CGANs) improved by mini-batch method to address the problem of class imbalance. CGAN is an efficient method of multiclass data generation, which learns the properties of dynamometer cards by training the generator and discriminator networks. CGAN is modified by mini-batch strategy to avoid mode collapse and enable the interaction among input samples of discriminator, so that the generated dynamometer cards can be much more diverse. Results show that the shapes of generated dynamometer cards are basically consistent with those of real samples, and the structural similarity (SSIM) among the generated samples decreases, indicating that the generated dynamometer cards have more types of shape. Meanwhile, as the generated dynamometer cards become more diverse, their differences with real samples in data distribution are reduced, according to the calculation of sliced Wasserstein (SW) distance. Based on real and generated dynamometer cards, we developed the classifiers for working condition diagnosis of rod pump through convolutional neural network (CNN). The classification results of the validation set indicate that without the mini-batch method, the recall of generated categories for pump hitting down and leakage has increased by 12 and 5.3%, respectively; in contrast, with the mini-batch method, the recall has increased more obviously by 7, 24, and 2%, respectively, for gas lock, pump hitting down, and leakage. Our research results have demonstrated that the proposed method can effectively solve the problem of insufficient data accumulation in the oil field.

CADW: CGAN-Based Attack on Deep Robust Image Watermarking

Robust watermarking plays an essential role in copyright protection for digital images. Meanwhile, the studies of robust image watermarking and corresponding attack methods promote and complement each other. Because the traditional attack methods are weak to attack the deep watermarking, in this work, we thus design an effective watermarking attack method based on the conditional generative adversarial nets (CGAN). Also, according to the network structure of CGAN, a targeted and combined loss function is constructed, which can guarantee acceptable visual quality of attacked image and make the watermark extraction fail simultaneously. Experimental results demonstrate that our method is effective for some state-of-the-art deep robust image watermarking methods with the transferability. In addition, as an attack method, it can be regarded as an evaluation standard to measure the robustness of watermarking.

Visible-to-infrared Image Translation Based on an Improved Conditional Generative Adversarial Nets

Visible-to-infrared Image Translation Based on an Improved Conditional Generative Adversarial Nets

Learning to rank method combining multi-head self-attention with conditional generative adversarial nets

The existing methods of learning to rank often ignore the relationship between ranking features. If the relationship between them can be fully utilized, the performance of learning to rank methods can be improved. Aiming at this problem, an approach of learning to rank that combines a multi-head self-attention mechanism with Conditional Generative Adversarial Nets (CGAN) is proposed in this paper, named *GAN-LTR. The proposed approach improves some design ideas of Information Retrieval Generative Adversarial Networks (IRGAN) framework applied to web search, and a new network model is constructed by integrating convolution layer, multi-head self-attention layer, residual layer, fully connected layer, batch normalization, and dropout technologies into the generator and discriminator of Conditional Generative Adversarial Nets (CGAN). The convolutional neural network is used to extract the ranking feature representation of the hidden layer and capture the internal correlation and interactive information between features. The multi-head self-attention mechanism is used to fuse feature information in multiple vector subspaces and capture the attention weight of features, so as to assign appropriate weights to different features. The experimental results on the MQ2008-semi learning to rank dataset show that compared with IRGAN, our proposed learning to rank method *GAN-LTR has certain performance advantages in various performance indicators on the whole.

Generation Method for Shaded Relief Based on Conditional Generative Adversarial Nets

Relief shading is the primary method for effectively representing three-dimensional terrain on a two-dimensional plane. Despite its expressiveness, manual relief shading is difficult and time-consuming. In contrast, although analytical relief shading is fast and efficient, the visual effect is quite different from that of manual relief shading due to the low degree of terrain generalisation, inability to adjust local illumination, and difficulty in exaggerating and selective representation. We introduce deep learning technology to propose a generation method for shaded relief based on conditional generative adversarial nets. This method takes the set of manual relief shading-digital elevation model (DEM) slices as a priori knowledge, optimises network parameters through a continuous game of “generation-discrimination”, and produces a shaded relief map of any region based on the DEM. Test results indicate that the proposed method retains the advantages of manual relief shading and can quickly generate shaded relief with quality and artistic style similar to those of manual shading. Compared with other networks, the shaded relief generated by the proposed method not only depicts the terrain clearly but also achieves a good generalisation effect. Moreover, through the use of an adversarial structure, the network demonstrates stronger cross-scale generation ability.

UAV-Satellite View Synthesis for Cross-View Geo-Localization

The goal of cross-view image matching based on geo-localization is to determine the location of a given ground-view image (front view) by matching it with a group of satellite-view images (vertical view) with geographic tags. Due to the rapid development of unmanned aerial vehicle (UAV) technology in recent years, it has provided a real viewpoint close to 45 degrees (oblique view) to bridge the visual gap between views. However, existing methods ignore the direct geometric space correspondence of UAV-satellite views, and only use brute force for feature matching, leading to inferior performance. In this context, we propose an end-to-end cross-view matching method that integrates cross-view synthesis module and geo-localization module, which fully considers the spatial correspondence of UAV-satellite views and the surrounding area information. To be specific, the cross-view synthesis module includes two parts: the oblique view of UAV is first converted to the vertical view by perspective projection transformation (PPT), which makes the UAV image closer to the satellite image; then we use conditional generative adversarial nets (CGAN) to synthesize the UAV image with vertical view style, which is close to the real satellite image by learning the converted UAV as the input image and the real satellite image as the label. Geo-localization module refers to existing local pattern network (LPN), which explicitly considers the surrounding environment of the target building. These modules are integrated in a single architecture called PCL, which mutually reinforce each other. Our method is superior to the existing UAV-satellite cross-view methods, which improves by about 5%.

Exploring Relational Semantics for Inductive Knowledge Graph Completion

Knowledge graph completion (KGC) aims to infer missing information in incomplete knowledge graphs (KGs). Most previous works only consider the transductive scenario where entities are existing in KGs, which cannot work effectively for the inductive scenario containing emerging entities. Recently some graph neural network-based methods have been proposed for inductive KGC by aggregating neighborhood information to capture some uncertainty semantics from the neighboring auxiliary triples. But these methods ignore the more general relational semantics underlying all the known triples that can provide richer information to represent emerging entities so as to satisfy the inductive scenario. In this paper, we propose a novel model called CFAG, which utilizes two granularity levels of relational semantics in a coarse-grained aggregator (CG-AGG) and a fine-grained generative adversarial net (FG-GAN), for inductive KGC. The CG-AGG firstly generates entity representations with multiple semantics through a hypergraph neural network-based global aggregator and a graph neural network-based local aggregator, and the FG-GAN further enhances entity representations with specific semantics through conditional generative adversarial nets. Experimental results on benchmark datasets show that our model outperforms state-of-the-art models for inductive KGC.

Pix2Pix-Based Monocular Depth Estimation for Drones with Optical Flow on AirSim.

In this work, we propose a method for estimating depth for an image of a monocular camera in order to avoid a collision for the autonomous flight of a drone. The highest flight speed of a drone is generally approximate 22.2 m/s, and long-distant depth information is crucial for autonomous flights since if the long-distance information is not available, the drone flying at high speeds is prone to collisions. However, long-range, measurable depth cameras are too heavy to be equipped on a drone. This work applies Pix2Pix, which is a kind of Conditional Generative Adversarial Nets (CGAN). Pix2Pix generates depth images from a monocular camera. Additionally, this work applies optical flow to enhance the accuracy of depth estimation. In this work, we propose a highly accurate depth estimation method that effectively embeds an optical flow map into a monocular image. The models are trained with taking advantage of AirSim, which is one of the flight simulators. AirSim can take both monocular and depth images over a hundred meter in the virtual environment, and our model generates a depth image that provides the long-distance information than images captured by a common depth camera. We evaluate accuracy and error of our proposed method using test images in AirSim. In addition, the proposed method is utilized for flight simulation to evaluate the effectiveness to collision avoidance. As a result, our proposed method is higher accuracy and lower error than a state of work. Moreover, our proposed method is lower collision than a state of work.

Transmission Line Vibration Damper Detection Using Multi-Granularity Conditional Generative Adversarial Nets Based on UAV Inspection Images.

The vibration dampers can eliminate the galloping phenomenon of transmission lines caused by the wind. The detection of vibration dampers based on visual technology is an important issue. Current CNN-based methods struggle to meet the requirements of real-time detection. Therefore, the current vibration damper detection work has mainly been carried out manually. In view of the above situation, we propose a vibration damper detection-image generation model called DamperGAN based on multi-granularity Conditional Generative Adversarial Nets. DamperGAN first generates a low-resolution detection result image based on a coarse-grained module, then uses Monte Carlo search to mine the latent information in the low-resolution image, and finally injects this information into a fine-grained module through an attention mechanism to output high-resolution images and penalize poor intermediate information. At the same time, we propose a multi-level discriminator based on the multi-task learning mechanism to improve the discriminator’s discriminative ability and promote the generator to output better images. Finally, experiments on the self-built DamperGenSet dataset show that the images generated by our model are superior to the current mainstream baselines in both resolution and quality.

Declutter-GAN: GPR B-Scan Data Clutter Removal Using Conditional Generative Adversarial Nets

Clutter removal in ground-penetrating radar (GPR) B-scan data has been widely studied in recent years. In this letter, we propose a novel data-driven clutter suppression method in GPR data based on conditional generative adversarial nets (cGANs). The proposed method learns a function that maps the cluttered data to the clutter-free data from the training set. The training set consists of pairs of cluttered data and corresponding clutter-free data. Different from the traditional method that only uses the simulation training set, we simulate the clutter-free data and add the real collected non-target data to the simulated clutter-free data as cluttered data, so that the trained network can generalize well to the real GPR data. The proposed method is compared with the subspace method, sparse representation-based method, and low-rank and sparse matrix decomposition (LRSD) methods on both simulation data and real collected data. The results show that the proposed method has higher performance in terms of computational complexity, clutter suppression results, and applicability than those state-of-the-art methods.

Chinese Medicine Prescription Recommendation Using Generative Adversarial Network

The theory of traditional Chinese medicine (TCM) is an important part of Chinese culture. In the long history, there are a large number of excellent prescriptions, whose laws have been explored by many studies, but few works directly studied the generation of prescriptions. With the rapid development of deep learning, many applications of text generation using neural networks have emerged. Prescriptions are the doctors’ clinical experience and the results of neural networks also come from the accumulated experience. So, it is very feasible to apply deep learning techniques to the recommendation on TCM prescriptions. GAN and its variants have been applied in text generation recently. It has advantages in many aspects, such as the rapid speed of computation, the update of parameters by back propagation without Markov chain and more real data generation with two-players game. We attempted to know the important attributes of prescriptions and use these contents as the training data for variants of GAN to generate prescriptions. Specifically, we attempted to apply SeqGAN (Sequence Generative Adversarial Nets) and CGAN (Conditional Generative Adversarial Nets) to prescription generations. By underlying the knowledge of TCM, the prescriptions with different characteristics can be successfully generated. In the experiments, we conducted the comparative evaluations on the original data with other models. The results showed that applications in the innovation of prescription sequence generations have certain feasibility and significance, even can provide some reference values for the innovation of TCM.

Learning to Rank Method Combining Multi-Head Self-Attention with Conditional Generative Adversarial Nets

Learning to Rank Method Combining Multi-Head Self-Attention with Conditional Generative Adversarial Nets

Conditional Deep 3D-Convolutional Generative Adversarial Nets for RGB-D Generation

Generation of synthetic data is a challenging task. There are only a few significant works on RGB video generation and no pertinent works on RGB-D data generation. In the present work, we focus our attention on synthesizing RGB-D data which can further be used as dataset for various applications like object tracking, gesture recognition, and action recognition. This paper has put forward a proposal for a novel architecture that uses conditional deep 3D-convolutional generative adversarial networks to synthesize RGB-D data by exploiting 3D spatio-temporal convolutional framework. The proposed architecture can be used to generate virtually unlimited data. In this work, we have presented the architecture to generate RGB-D data conditioned on class labels. In the architecture, two parallel paths were used, one to generate RGB data and the second to synthesize depth map. The output from the two parallel paths is combined to generate RGB-D data. The proposed model is used for video generation at 30 fps (frames per second). The frame referred here is an RGB-D with the spatial resolution of 512 × 512.