Dual Discriminator Research Articles

Development of a GAN architecture based on integrating global and local information for paired and unpaired medical image translation

The purpose of this study is to develop and evaluate a novel end-to-end generative adversarial network for image-to-image translation of medical paired and unpaired data. The proposed deep framework is based on generative adversarial networks, in which blocks are designed and used to improve the estimated images and achieve more accurate results. A joint attention block is designed to capture and fuse the global and local information during the training process. A dual discriminator is employed to accurately determine whether the generated images are real or fake. A global guidance up-sampling block is proposed to reconstruct the local details of the estimated images with guidance of the low-level information of the shallow layers. Moreover, an edge detection network is utilized to make the tissue edges sharper. The loss function is composed of different criteria considering the geometric characteristics for obtaining more accurate results. Different translation tasks including PD/T2, CT/MR, and PET/CT are considered to evaluate the performance of the proposed deep framework. The evaluation results indicated averagely 28.28, 10.87, 31.30, 0.90, 0.92, 0.85, 24.80, and 0.042 for root-mean-squared error (RMSE), mean absolute error (MAE), peak signal to noise ratio (PSNR), dice similarity coefficient (DSC), normalized cross-correlation (NCC), structural similarity index measure (SSIM), learned perceptual image patch similarity (LPIPS) and frechet inception distance (FID) respectively. Evaluation of the results showed improvement of 106, 66, 12, 0.22, 0.17, 0.15, 58, and 0.39 for RMSE, MAE, PSNR, DSC, NCC, SSIM, LPIPS, and FID, respectively when the proposed blocks were used. Also, comparing the performance of the proposed network against other methods in the two modes of using paired and unpaired data showed the better performance of the proposed network.

A Dual Discriminator Adversarial Learning Approach for Dental Occlusal Surface Reconstruction.

Objective Restoring the correct masticatory function of partially edentulous patient is a challenging task primarily due to the complex tooth morphology between individuals. Although some deep learning-based approaches have been proposed for dental restorations, most of them do not consider the influence of dental biological characteristics for the occlusal surface reconstruction. Description. In this article, we propose a novel dual discriminator adversarial learning network to address these challenges. In particular, this network architecture integrates two models: a dilated convolutional-based generative model and a dual global-local discriminative model. While the generative model adopts dilated convolution layers to generate a feature representation that preserves clear tissue structure, the dual discriminative model makes use of two discriminators to jointly distinguish whether the input is real or fake. While the global discriminator focuses on the missing teeth and adjacent teeth to assess whether it is coherent as a whole, the local discriminator aims only at the defective teeth to ensure the local consistency of the generated dental crown. Results Experiments on 1000 real-world patient dental samples demonstrate the effectiveness of our method. For quantitative comparison, the image quality metrics are used to measure the similarity of the generated occlusal surface, and the root mean square between the generated result and the target crown calculated by our method is 0.114 mm. In qualitative analysis, the proposed approach can generate more reasonable dental biological morphology. Conclusion The results demonstrate that our method significantly outperforms the state-of-the-art methods in occlusal surface reconstruction. Importantly, the designed occlusal surface has enough anatomical morphology of natural teeth and superior clinical application value.

A Modified Generative Adversarial Network for Fault Diagnosis in High-Speed Train Components with Imbalanced and Heterogeneous Monitoring Data

Data-driven methods are widely considered for fault diagnosis in complex systems. However, in practice the between-class imbalance due to limited faulty samples may deteriorate their classification performance. To address this issue, synthetic minority methods for enhancing data have been proved to be effective in many applications. Generative Adversarial Networks (GANs), capable of automatic features extraction, can also be adopted for augmenting the faulty samples. However, the monitoring data of a complex system may include not only continuous signals, but also discrete/categorical signals. Since the current GAN methods still have some challenges in handling such heterogeneous monitoring data, a Mixed Dual Discriminator GAN (noted as M-D2GAN) is proposed in this work. In order to render the expanded fault samples more aligned with the real situation, and improve the accuracy and robustness of the fault diagnosis model, different types of variables are generated in different ways, including: floating-point, integer, categorical, and hierarchical. For effectively considering the class imbalance problem, proper modifications are made to the GAN model, where a normal class discriminator is added. A practical case study concerning the braking system of a high-speed train is carried out to verify the effectiveness of the proposed framework. Compared to the classic GAN, the proposed framework achieves better results with respect to F-measure and G-mean metrics.

Dual Discriminator GAN: Restoring Ancient Yi Characters

In China, the damage of ancient Yi books are serious. Due to the lack of ancient Yi experts, the repairation of ancient Yi books is progressing very slowly. The artificial intelligence is successful in the field of image and text, so it is feasible for the automatic restoration of ancient books. In this article, a generative adversarial networks with dual discriminator (DDGAN) is designed to restore incomplete characters in the ancient Yi literature. The DDGAN integrates the deep convolution generative adversarial network with an ancient Yi comparison discriminator. Through two training stages, it could iteratively optimizes the ancient Yi character generation networks to obtain the text generator According to the loss of comparison discriminator, DDGAN mode could be optimized. The DDGAN model can generate characters to restore the missing stroke in the ancient Yi. The experiment shows that the proposed method achieves a restoration rate of 77.3% when no more than one third of the characters are missing. This work is effective for the protection of Yi ancient books.

Dual Discriminator Weighted Mixture Generative Adversarial Network for image generation

Dual Discriminator Weighted Mixture Generative Adversarial Network for image generation

Edge-enhanced dual discriminator generative adversarial network for fast MRI with parallel imaging using multi-view information

In clinical medicine, magnetic resonance imaging (MRI) is one of the most important tools for diagnosis, triage, prognosis, and treatment planning. However, MRI suffers from an inherent slow data acquisition process because data is collected sequentially in k-space. In recent years, most MRI reconstruction methods proposed in the literature focus on holistic image reconstruction rather than enhancing the edge information. This work steps aside this general trend by elaborating on the enhancement of edge information. Specifically, we introduce a novel parallel imaging coupled dual discriminator generative adversarial network (PIDD-GAN) for fast multi-channel MRI reconstruction by incorporating multi-view information. The dual discriminator design aims to improve the edge information in MRI reconstruction. One discriminator is used for holistic image reconstruction, whereas the other one is responsible for enhancing edge information. An improved U-Net with local and global residual learning is proposed for the generator. Frequency channel attention blocks (FCA Blocks) are embedded in the generator for incorporating attention mechanisms. Content loss is introduced to train the generator for better reconstruction quality. We performed comprehensive experiments on Calgary-Campinas public brain MR dataset and compared our method with state-of-the-art MRI reconstruction methods. Ablation studies of residual learning were conducted on the MICCAI13 dataset to validate the proposed modules. Results show that our PIDD-GAN provides high-quality reconstructed MR images, with well-preserved edge information. The time of single-image reconstruction is below 5ms, which meets the demand of faster processing.

Research on Multimodal Image Fusion Target Detection Algorithm Based on Generative Adversarial Network

In this paper, we propose a target detection algorithm based on adversarial discriminative domain adaptation for infrared and visible image fusion using unsupervised learning methods to reduce the differences between multimodal image information. Firstly, this paper improves the fusion model based on generative adversarial network and uses the fusion algorithm based on the dual discriminator generative adversarial network to generate high-quality IR-visible fused images and then blends the IR and visible images into a ternary dataset and combines the triple angular loss function to do migration learning. Finally, the fused images are used as the input images of faster RCNN object detection algorithm for detection, and a new nonmaximum suppression algorithm is used to improve the faster RCNN target detection algorithm, which further improves the target detection accuracy. Experiments prove that the method can achieve mutual complementation of multimodal feature information and make up for the lack of information in single-modal scenes, and the algorithm achieves good detection results for information from both modalities (infrared and visible light).

A Dual-Discriminator Fourier Acquisitive GAN for Generating Retinal Optical Coherence Tomography Images

Optical coherence tomography (OCT) images are widely used for clinical examination of the retina. Automatic deep learning-based methods have been developed to classify normal and pathological OCT images. However, lack of the big enough training data reduces the performance of these models. Synthesis of data using generative adversarial networks (GANs) is already known as an efficient alternative to increase the amount of the training data. However, the recent works show that despite high structural similarity between synthetic data and the real images, a considerable distortion is observed in frequency domain. Here, we propose a dual discriminator Fourier acquisitive GAN (DDFA-GAN) to generate more realistic OCT images with considering the Fourier domain similarity in structural design of the GAN. By applying two discriminators, the proposed DDFA-GAN is jointly trained with the Fourier and spatial details of the images and is proven to be feasible with a limited number of training data. Results are compared with popular GANs, namely, DCGAN, WGAN-GP, and LS-GAN. In comparison, Fréchet inception distance (FID) score of 51.30, and Multi Scale Structural Similarity Index Measure (MS-SSIM) of 0.19 indicate superiority of the proposed method in producing images resembling the same quality, discriminative features, and diversity, as the real normal and Diabetic Macular Edema (DME) OCT images. The statistical comparison illustrates this similarity in the spatial and frequency domains, as well. Overall, DDFA-GAN generates realistic OCT images to meet requirements of the training data in automatic deep learning-based methods, used for clinical examination of the retina, and to improve the accuracy of the subsequent measurements.

Data Augmentation for EEG-Based Emotion Recognition Using Generative Adversarial Networks.

One of the greatest limitations in the field of EEG-based emotion recognition is the lack of training samples, which makes it difficult to establish effective models for emotion recognition. Inspired by the excellent achievements of generative models in image processing, we propose a data augmentation model named VAE-D2GAN for EEG-based emotion recognition using a generative adversarial network. EEG features representing different emotions are extracted as topological maps of differential entropy (DE) under five classical frequency bands. The proposed model is designed to learn the distributions of these features for real EEG signals and generate artificial samples for training. The variational auto-encoder (VAE) architecture can learn the spatial distribution of the actual data through a latent vector, and is introduced into the dual discriminator GAN to improve the diversity of the generated artificial samples. To evaluate the performance of this model, we conduct a systematic test on two public emotion EEG datasets, the SEED and the SEED-IV. The obtained recognition accuracy of the method using data augmentation shows as 92.5 and 82.3%, respectively, on the SEED and SEED-IV datasets, which is 1.5 and 3.5% higher than that of methods without using data augmentation. The experimental results show that the artificial samples generated by our model can effectively enhance the performance of the EEG-based emotion recognition.

A Partial Differential Equation-Based Image Restoration Method in Environmental Art Design

With the rapid development of networks and the emergence of various devices, images have become the main form of information transmission in real life. Image restoration, as an important branch of image processing, can be applied to real-life situations such as pixel loss in image transmission or network prone to packet loss. However, existing image restoration algorithms have disadvantages such as fuzzy restoration effect and slow speed; to solve such problems, this paper adopts a dual discriminator model based on generative adversarial networks, which effectively improves the restoration accuracy by adding local discriminators to track the information of local missing regions of images. However, the model is not optimistic in generating reasonable semantic information, and for this reason, a partial differential equation-based image restoration model is proposed. A classifier and a feature extraction network are added to the dual discriminator model to provide category, style, and content loss constraints to the generative network, respectively. To address the training instability problem of discriminator design, spectral normalization is introduced to the discriminator design. Extensive experiments are conducted on a data dataset of partial differential equations, and the results show that the partial differential equation-based image restoration model provides significant improvements in image restoration over previous methods and that image restoration techniques are exceptionally important in the application of environmental art design.

Dual discriminator adversarial distillation for data-free model compression

Knowledge distillation has been widely used to produce portable and efficient neural networks which can be well applied on edge devices for computer vision tasks. However, almost all top-performing knowledge distillation methods need to access the original training data, which usually has a huge size and is often unavailable. To tackle this problem, we propose a novel data-free approach in this paper, named Dual Discriminator Adversarial Distillation (DDAD) to distill a neural network without the need of any training data or meta-data. To be specific, we use a generator to create samples through dual discriminator adversarial distillation, which mimics the original training data. The generator not only uses the pre-trained teacher’s intrinsic statistics in existing batch normalization layers but also obtains the maximum discrepancy from the student model. Then the generated samples are used to train the compact student network under the supervision of the teacher. The proposed method obtains an efficient student network which closely approximates its teacher network, without using the original training data. Extensive experiments are conducted to demonstrate the effectiveness of the proposed approach on CIFAR, Caltech101 and ImageNet datasets for classification tasks. Moreover, we extend our method to semantic segmentation tasks on several public datasets such as CamVid, NYUv2, Cityscapes and VOC 2012. To the best of our knowledge, this is the first work on generative model based data-free knowledge distillation on large-scale datasets such as ImageNet, Cityscapes and VOC 2012. Experiments show that our method outperforms all baselines for data-free knowledge distillation.

Multiple agents’ spatiotemporal data generation based on recurrent regression dual discriminator GAN

Generative Adversarial Networks (GANs) have proven their capability of generating realistic-looking data and have been widely used in image related and time-series applications. However, generation of multiple agents’ spatiotemporal data remains an unexplored region. In this work, we propose a recurrent regression dual discriminator GAN named R2D2GAN. A novel generator is designed to learn mappings from prior stochastic process to multiple agents’ spatiotemporal data, which is conditioned on spatial configuration of multiple agents only. A classification discriminator and a regression discriminator are proposed to represent different features of spatiotemporal data. The classification discriminator learns to represent spatial and sequential features of each agent. The regression discriminator learns to represent inherent sequential dependency for target agent. To stabilize training of GAN, a min–max game is elaborately designed and new training losses are proposed for dual discriminators and the generator. To validate learning ability of R2D2GAN, we embed it in vehicle trajectory prediction application. Through qualitative and quantitative evaluation, we show that the R2D2GAN is capable of generating realistic-looking multiple agents’ spatiotemporal data with acceptable performance degradation in prediction task.

GAN and CNN for imbalanced partial discharge pattern recognition in GIS

The convolutional neural network (CNN) achieves excellent performance in pattern recognition owing to its powerful automatic feature extraction capability and outstanding classification performance. However, the actual samples obtained are unbalanced, and accurate diagnoses are difficult for the existing methods. A classification method for partial discharge (PD) pattern recognition in gas-insulated switchgear (GIS) that uses a generative adversarial network (GAN) and CNN on unbalanced samples is proposed. First, a novel Wasserstein dual discriminator GAN is used to generate data to equalise the unbalanced samples. Second, a decomposed hierarchical search space is used to automatically construct an optimal diagnostic CNN. Finally, PD pattern recognition classification in GIS of the unbalanced samples is realised by the GAN and CNN. The experimental results show that the GAN and CNN methods proposed in this study have a pattern recognition accuracy of 99.15% on unbalanced samples, which is significantly higher than that obtained by other methods. Therefore, the method proposed in this study is more suitable for industrial applications.

On the effectiveness of dual discriminator weighted generative adversarial network

Generative adversarial network (GAN) has made great progress in image generation and reconstruction, but it may cause the mode collapse problem in practice. We proposed the dual discriminator weighted generative adversarial network (D2WGAN), whose objective function weights the Kullback–Leibler divergence (KL divergence) and the reverse KL divergence and uses the complementary characteristics of these two divergences to make the generated models more diverse. Moreover, we proved the theoretical conditional optimality of the D2WGAN to show that the generator can learn the real data distribution. Finally, we conduct experiments on a large amount of synthetic data and real-world datasets (e.g., MNIST and CIFAR-10). The results show that, compared with the traditional dual discriminator generative adversarial network and GAN, the proposed D2WGAN can process multiple mode data and generate better sample diversities.

Hierarchical Human-Like Deep Neural Networks for Abstractive Text Summarization.

Developing an abstractive text summarization (ATS) system that is capable of generating concise, appropriate, and plausible summaries for the source documents is a long-term goal of artificial intelligence (AI). Recent advances in ATS are overwhelmingly contributed by deep learning techniques, which have taken the state-of-the-art of ATS to a new level. Despite the significant success of previous methods, generating high-quality and human-like abstractive summaries remains a challenge in practice. The human reading cognition, which is essential for reading comprehension and logical thinking, is still relatively new territory and underexplored in deep neural networks. In this article, we propose a novel Hierarchical Human-like deep neural network for ATS (HH-ATS), inspired by the process of how humans comprehend an article and write the corresponding summary. Specifically, HH-ATS is composed of three primary components (i.e., a knowledge-aware hierarchical attention module, a multitask learning module, and a dual discriminator generative adversarial network), which mimic the three stages of human reading cognition (i.e., rough reading, active reading, and postediting). Experimental results on two benchmark data sets (CNN/Daily Mail and Gigaword) demonstrate that HH-ATS consistently and substantially outperforms the compared methods.

Every minimal dual discriminator variety is minimal as a quasivariety

Let $$(\dagger )$$ denote the following property of a variety $$\mathcal V$$ : Every subquasivariety of $$\mathcal V$$ is a variety. In this paper, we prove that every idempotent dual discriminator variety has property $$(\dagger )$$ . Property $$(\dagger )$$ need not hold for nonidempotent dual discriminator varieties, but $$(\dagger )$$ does hold for minimal nonidempotent dual discriminator varieties. Combining the results for the idempotent and nonidempotent cases, we obtain that every minimal dual discriminator variety is minimal as a quasivariety

Triple discriminator generative adversarial network for zero-shot image classification

One key challenge in zero-shot classification (ZSC) is the exploration of knowledge hidden in unseen classes. Generative methods such as generative adversarial networks (GANs) are typically employed to generate the visual information of unseen classes. However, the majority of these methods exploit global semantic features while neglecting the discriminative differences of local semantic features when synthesizing images, which may lead to sub-optimal results. In fact, local semantic information can provide more discriminative knowledge than global information can. To this end, this paper presents a new triple discriminator GAN for ZSC called TDGAN, which incorporates a text-reconstruction network into a dual discriminator GAN (D2GAN), allowing to realize cross-modal mapping from text descriptions to their visual representations. The text-reconstruction network focuses on key text descriptions for aligning semantic relationships to enable synthetic visual features to effectively represent images. Sharma-Mittal entropy is exploited in the loss function to make the distribution of synthetic classes be as close as possible to the distribution of real classes. The results of extensive experiments over the Caltech-UCSD Birds-2011 and North America Birds datasets demonstrate that the proposed TDGAN method consistently yields competitive performance compared to several state-of-the-art ZSC methods.

High quality and fast compressed sensing MRI reconstruction via edge-enhanced dual discriminator generative adversarial network

Generative adversarial networks (GAN) are widely used for fast compressed sensing magnetic resonance imaging (CSMRI) reconstruction. However, most existing methods are difficult to make an effective trade-off between abstract global high-level features and edge features. It easily causes problems, such as significant remaining aliasing artifacts and clearly over-smoothed reconstruction details. To tackle these issues, we propose a novel edge-enhanced dual discriminator generative adversarial network architecture called EDDGAN for CSMRI reconstruction with high quality. In this model, we extract effective edge features by fusing edge information from different depths. Then, leveraging the relationship between abstract global high-level features and edge features, a three-player game is introduced to control the hallucination of details and stabilize the training process. The resulting EDDGAN can offer more focus on edge restoration and de-aliasing. Extensive experimental results demonstrate that our method consistently outperforms state-of-the-art methods and obtains reconstructed images with rich edge details. In addition, our method also shows remarkable generalization, and its time consumption for each 256 × 256 image reconstruction is approximately 8.39 ms.

A Novel Multitask Adversarial Network via Redundant Lifting for Multicomponent Intelligent Fault Detection Under Sharp Speed Variation

Playing a vital role in the safe operation of equipment, intelligent fault detection has been extensively applied in mechanical systems. However, the complicated mapping of signal symptoms and multicomponent failure modes under sharp speed variation makes it hard. The classification error is large due to insufficient samples in the fault condition. Aiming at these problems, a novel multitask redundant lifting adversarial network (MRLAN) is proposed. First, inspired by redundant second-generation wavelet transform, a reconstruction network is established to generate high-quality time-frequency representation of vibration signal, thus eliminating speed fluctuations and expanding training samples. The reconstruction network consists of transposed convolution layer, resume update layer, resume predict layer, and merge layer. Then, multitask category discriminator, which forms dual discriminator together with distance discriminator, is constructed to separate the features of multiple components, thereby realizing multiple tasks (multicomponent fault location and single-component health assessment). Finally, the model is alternately optimized through adversarial process between the generation network and the double discriminators. The proposed method is validated by two cases and other methods are also established to show its superiority. The results show satisfactory performance of MRLAN in multicomponent fault diagnosis under sharp speed fluctuation and little training data.

Light Field Image Coding Using VVC Standard and View Synthesis Based on Dual Discriminator GAN

Light field (LF) technology is considered as a promising way for providing a high-quality virtual reality (VR) content. However, such an imaging technology produces a large amount of data requiring efficient LF image compression solutions. In this paper, we propose a LF image coding method based on a view synthesis and view quality enhancement techniques. Instead of transmitting all the LF views, only a sparse set of reference views are encoded and transmitted, while the remaining views are synthesized at the decoder side. The transmitted views are encoded using the versatile video coding (VVC) standard and are used as reference views to synthesize the dropped views. The selection of non-reference dropped views is performed using a rate-distortion optimization based on the VVC temporal scalability. The dropped views are reconstructed using the LF dual discriminator GAN (LF-D2GAN) model. In addition, to ensure that the quality of the views is consistent, at the decoder, a quality enhancement procedure is performed on the reconstructed views allowing smooth navigation across views. Experimental results show that the proposed method provides high coding performance and overcomes the state-of-the-art LF image compression methods by –36.22% in terms of BD-BR and 1.35 dB in BD-PSNR. The web page of this work is available at https://naderbakir79.github.io/LFD2GAN.html .