FID Scores Research Articles

Improved Consistency Regularization for GANs

Recent work has increased the performance of Generative Adversarial Networks (GANs) by enforcing a consistency cost on the discriminator. We improve on this technique in several ways. We first show that consistency regularization can introduce artifacts into the GAN samples and explain how to fix this issue. We then propose several modifications to the consistency regularization procedure designed to improve its performance. We carry out extensive experiments quantifying the benefit of our improvements. For unconditional image synthesis on CIFAR-10 and CelebA, our modifications yield the best known FID scores on various GAN architectures. For conditional image synthesis on CIFAR-10, we improve the state-of-the-art FID score from 11.48 to 9.21. Finally, on ImageNet-2012, we apply our technique to the original BigGAN model and improve the FID from 6.66 to 5.38, which is the best score at that model size.

IB-GAN: Disentangled Representation Learning with Information Bottleneck Generative Adversarial Networks

We propose a new GAN-based unsupervised model for disentangled representation learning. The new model is discovered in an attempt to utilize the Information Bottleneck (IB) framework to the optimization of GAN, thereby named IB-GAN. The architecture of IB-GAN is partially similar to that of InfoGAN but has a critical difference; an intermediate layer of the generator is leveraged to constrain the mutual information between the input and the generated output. The intermediate stochastic layer can serve as a learnable latent distribution that is trained with the generator jointly in an end-to-end fashion. As a result, the generator of IB-GAN can harness the latent space in a disentangled and interpretable manner. With the experiments on dSprites and Color-dSprites dataset, we demonstrate that IB-GAN achieves competitive disentanglement scores to those of state-of-the-art β-VAEs and outperforms InfoGAN. Moreover, the visual quality and the diversity of samples generated by IB-GAN are often better than those by β-VAEs and Info-GAN in terms of FID score on CelebA and 3D Chairs dataset.

Towards a Neural Graphics Pipeline for Controllable Image Generation

AbstractIn this paper, we leverage advances in neural networks towards forming a neural rendering for controllable image generation, and thereby bypassing the need for detailed modeling in conventional graphics pipeline. To this end, we present Neural Graphics Pipeline (NGP), a hybrid generative model that brings together neural and traditional image formation models. NGP decomposes the image into a set of interpretable appearance feature maps, uncovering direct control handles for controllable image generation. To form an image, NGP generates coarse 3D models that are fed into neural rendering modules to produce view‐specific interpretable 2D maps, which are then composited into the final output image using a traditional image formation model. Our approach offers control over image generation by providing direct handles controlling illumination and camera parameters, in addition to control over shape and appearance variations. The key challenge is to learn these controls through unsupervised training that links generated coarse 3D models with unpaired real images via neural and traditional (e.g., Blinn‐Phong) rendering functions, without establishing an explicit correspondence between them. We demonstrate the effectiveness of our approach on controllable image generation of single‐object scenes. We evaluate our hybrid modeling framework, compare with neural‐only generation methods (namely, DCGAN, LSGAN, WGAN‐GP, VON, and SRNs), report improvement in FID scores against real images, and demonstrate that NGP supports direct controls common in traditional forward rendering. Code is available at http://geometry.cs.ucl.ac.uk/projects/2021/ngp.

Gated SwitchGAN for Multi-Domain Facial Image Translation

Recent studies on multi-domain facial image translation have achieved impressive results. The existing methods generally provide a discriminator with an auxiliary classifier to impose domain translation. However, these methods neglect important information regarding domain distribution matching. To solve this problem, we propose a switch generative adversarial network (SwitchGAN) with a more adaptive discriminator structure and a matched generator to perform delicate image translation among multiple domains. A feature-switching operation is proposed to achieve feature selection and fusion in our conditional modules. We demonstrate the effectiveness of our model. Furthermore, we also introduce a new capability of our generator that represents attribute intensity control and extracts content information without tailored training. Experiments on the Morph, RaFD and CelebA databases visually and quantitatively show that our extended SwitchGAN (i.e., Gated SwitchGAN) can achieve better translation results than StarGAN, AttGAN and STGAN. The attribute classification accuracy achieved using the trained ResNet-18 model and the FID score obtained using the ImageNet pretrained Inception-v3 model also quantitatively demonstrate the superior performance of our models.

Text to Image GANs with RoBERTa and Fine-grained Attention Networks

Synthesizing new images from textual descriptions requires understanding the context of the text. It is a very chal-lenging problem in Natural Language Processing and Computer vision. Existing systems use Generative Adversarial Network (GAN) to generate images using a simple text encoder from their captions. This paper consist synthesizing images from textual descriptions using Caltech-UCSD birds datasets by baselining the generative model using Attentional Generative Adversarial Networks (AttnGAN) and using RoBERTa pre-trained neural language model for word embeddings. The results obtained are compared with the baseline AttnGAN model and conduct various analyses on incorporating RoBERTa text encoder concerning simple encoder in the existing system. Various performance improvements were noted compared to baseline Attention Gen-erative networks. The FID score has decreased from 23.98 in AttnGAN to 20.77 with incorporation of RoBERTa model with AttnGAN.

Activation Maximization with a Prior in Speech Data

Recently, more and more studies regarding neural networks have been done. However, the learning process of neural networks is often elusive to human beings, which leads to the advent of feature visualization techniques. Activation Maximization (AM) is one of the feature visualization techniques, originally designed for image data. In AM, the input data is optimized to find the data that activates the selected neuron. In this paper, the emotion recognizer’s output is selected as the neuron, and the latent code of a generator (of Generative Adversarial Networks) is optimized instead of the input raw data. The aim of this study is to apply AM to different representations of audio data (waveform-based data and mel-spectrogram-based data) and different model structures (CNN, WaveNet, LSTM), and to find out the most suitable condition for AM in audio domain data. Additionally, we have also tried to visualize the essential features of being a certain class for emotion classification in speech data, using 2 datasets: the Toronto emotional speech set (TESS) and the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). The mel-spectrogram-based models were found to be superior to the others, showing the distinctive features of selected emotions. More specifically, the CNN-mel-spectrogram-based model was the best in both qualitative and quantitative (FID score) results. Moreover, as demonstrated in this study, AM can also be employed as an output enhancer for generative models.

Multi-Style Unsupervised Image Synthesis Using Generative Adversarial Nets

Unsupervised cross-domain image-to-image translation is a very active topic in computer vision and graphics. This task has two challenges: 1) lack of paired training data and 2) numerous possible outputs from a single image. The existing methods rely on either paired data or perform one-to-one translation. A novel Multi-Style Unsupervised image synthesis model using Generative Adversarial Nets (MSU-GAN) is proposed in this paper to overcome these disadvantages. Firstly, the encoder-decoder structure is used to map the image to domain-shared content features space and domain-specific style features space. Secondly, to translate an image into another domain, the content code and the style code are combined to synthesize the resulting image. Finally, the bidirectional cycle-consistency loss is used for the unpaired training data; the inter-domain adversarial loss and the reconstruction loss are used to ensure the output image's realism. Simultaneously, MSU-GAN is able to synthesize multi-style images due to disentangled representation. A Multi-Style Unsupervised Feature-Wise image synthesis model using Generative Adversarial Nets (MSU-FW-GAN) based on the MSU-GAN is proposed for the shape variation tasks. There are two different testing strategies, which include random style transfer and style guide transfer. For objective comparison, the proposed model performs well on all evaluation metrics. The random style transfer experiment results show that compared with CycleGAN on the photo2portraits dataset, MSU-FW-GAN FID, IS scores dropped by 12.77% and 8.06%. For the summer2winter dataset, MSU-GAN FID and IS scores increased by 24.51% and 3.64%. Qualitative results show that without paired training data, MSU-GAN and MSU-FW-GAN can synthesize multi-style and better realistic images on various tasks.

Scenery Image Extension via Inpainting With a Mirrored Input

Generative image extension has the advantage of extending the overall image size while preserving the target image because, unlike other image extensions using interpolation, it completes the surroundings of the target image. However, existing generative image extension methods tend to have poor quality in the generation of outer pixels. One method deals only with a limited number of scene classes because the extension repeats the same semantics. We propose a mirrored input, which sandwiches the extended region by mirroring a part of the target image. This replaces generative image extension with an image inpainting problem and thus helps to achieve higher quality pixel generation and can extend semantics with more complex shapes than horizontal repetition. Experimental results show that our proposed method achieves a scenery image extension that exceeds the state-of-the-art generative image extension methods in both visual quality and FID score for datasets containing diverse scenes.

주파수 분리 및 이중생성자 기반의 고해상도 애니메이션 스타일 선화 자동 채색

In this paper, we use Generative Adversarial Networks (GAN) to address the industrial needs of auto colorization of line arts which takes enormous amount of manual labor. Auto-colorization method used in Image-to-Image conversion based on GAN has received a lot of attention due to its promising results. In this paper, we present a solution to not only colorize the line art but also transform the low resolution out image to match the resolution of the input image through two generators and frequency separation method. A high frequency components are extracted from the line, then two generators are used to colorize the image in low resolution. The high frequency component is merged with low resolution image to produce the high resolution colorized image. The resolution of fi nal output image matches the resolution of original image while preserving the texture of the input image, whereas the other schemes reduce the output image to 512 pixels. We performed visual and qualitative evaluation using FID, PSNR, and SSIM. The FID Score of the proposed method is better than the base model by about 4 (proposed: 47.87 and base model 51.64). PNSR and SSIM of the high-resolution images are also better than the base model. PSNR and SSIM of base model is 13.01 and 0.72 whereas the proposed is 20.77 and 0.86, respectively.

Adversarial symmetric GANs: Bridging adversarial samples and adversarial networks

Generative adversarial networks have achieved remarkable performance on various tasks but suffer from training instability. Despite many training strategies proposed to improve training stability, this issue remains as a challenge. In this paper, we investigate the training instability from the perspective of adversarial samples and reveal that adversarial training on fake samples is implemented in vanilla GANs, but adversarial training on real samples has long been overlooked. Consequently, the discriminator is extremely vulnerable to adversarial perturbation and the gradient given by the discriminator contains non-informative adversarial noises, which hinders the generator from catching the pattern of real samples. Here, we develop adversarial symmetric GANs (AS-GANs) that incorporate adversarial training of the discriminator on real samples into vanilla GANs, making adversarial training symmetrical. The discriminator is therefore more robust and provides more informative gradient with less adversarial noise, thereby stabilizing training and accelerating convergence. The effectiveness of the AS-GANs is verified on image generation on CIFAR-10, CIFAR-100, CelebA, and LSUN with varied network architectures. Not only the training is more stabilized, but the FID scores of generated samples are consistently improved by a large margin compared to the baseline. Theoretical analysis is also conducted to explain why AS-GAN can improve training. The bridging of adversarial samples and adversarial networks provides a new approach to further develop adversarial networks.

Rhinology on the move despite Covid-19!

The October 2020 issue of Rhinology is a very interesting edition as it illustrates how world-wide colleagues pave the way for a better future of patients affected by nose and sinus diseases. After the successful launch of EPOS2020 in Spring 2020, the editorial team of Rhinology is proud to present to you the latest and most exciting data in Rhinology research. Getting insight into the complexity and relevance of proteomics in CRS, epithelial-mesenchymal contribution to CRS, zinc levels in nasal and systemic compartment of CRS, nasal biomarkers of CRSwNP that predict recurrence of disease after sinus surgery, and the odor identification test for children, called "U-Sniff", and FID scores (Frequency, Intensity and Duration) scores for epistaxis are all in the 2020 October issue and highly relevant for Rhinology practice. These studies build further on the solid grounds of previous Rhinology research meeting the unmets needs in the field.

FID Score: an effective tool in Hereditary Haemorrhagic Telangiectasia - related epistaxis.

Hereditary haemorrhagic telangiectasia (HHT) is a rare disease characterized by a multisystemic vascular dysplasia and epistaxis, that is the most common cause of disability and social impairment. Patient management strictly depends on the severity of this symptom; therefore, it is of paramount importance for the clinicians to effectively grade epistaxis severity. The aim of this report was to validate the Frequency, Intensity and Duration score (FID) for grading epistaxis severity in patients with HHT; we studied repeatability and external validity comparing FID score with Epistaxis Severity Score (ESS). This is a descriptive, observational study that included 264 adult HHT patients with epistaxis. Diagnosis of HHT was established with Curacao criteria or positivity at genetic testing. Nosebleed severity was evaluated according to the FID score and the ESS. The first 30 patients were included in the validation of the FID score, which was graded on days 0, 1, 3 and 7. In the remaining 234 patients, a comparison between the ESS and FID score was performed. The statistical analysis performed in order to validate the FID score showed very good agreement between scores calculated on different days; analysis comparing the FID score with the ESS revealed a high correlation between the two grading systems. The FID score is a quick, easy and precise tool for evaluating HHT-related epistaxis and could be a possible alternative to the ESS. The FID score meets the need for an intuitive and smart grading system that is easy to manage in clinicians’ hands.

Improving GAN with Neighbors Embedding and Gradient Matching

We propose two new techniques for training Generative Adversarial Networks (GANs) in the unsupervised setting. Our objectives are to alleviate mode collapse in GAN and improve the quality of the generated samples. First, we propose neighbor embedding, a manifold learning-based regularization to explicitly retain local structures of latent samples in the generated samples. This prevents generator from producing nearly identical data samples from different latent samples, and reduces mode collapse. We propose an inverse t-SNE regularizer to achieve this. Second, we propose a new technique, gradient matching, to align the distributions of the generated samples and the real samples. As it is challenging to work with high-dimensional sample distributions, we propose to align these distributions through the scalar discriminator scores. We constrain the difference between the discriminator scores of the real samples and generated ones. We further constrain the difference between the gradients of these discriminator scores. We derive these constraints from Taylor approximations of the discriminator function. We perform experiments to demonstrate that our proposed techniques are computationally simple and easy to be incorporated in existing systems. When Gradient matching and Neighbour embedding are applied together, our GN-GAN achieves outstanding results on 1D/2D synthetic, CIFAR-10 and STL-10 datasets, e.g. FID score of 30.80 for the STL-10 dataset. Our code is available at: https://github.com/tntrung/gan