The study for optimization strategies on the performance of DCGAN

Abstract

Since Deep Convolutional Generative Adversarial Networks (DCGAN) was proposed, it has been perceived as a model with difficulty in training due to several factors. To solve this problem, dozens of optimization strategies were presented, but none of them was compared with the others. In this paper, the author chose three representative methods, namely one-label smoothing, the two Time-Scale Update Rule (TTUR), and the Earth-Mover Distance (EMD) or Wasserstein-1 to make a comparison of the optimization effect on the DCGAN model. To be specific, these three approaches were adopted respectively while using MNIST and Fashion-MNIST as datasets. One-side label smoothing was designed to prevent overconfidence in the model by adding a penalty term in the discriminator. TTUR was a simpler update strategy that could help the model find the stationary local Nash equilibrium under mild assumptions. EMD was an alternative loss function that enabled the model to distinguish the difference while the real distribution and generated distribution were not overlapped. Contrast experiments were conducted both vertically and horizontally. The author applied these three methods with the same dataset and the same method with different datasets in order to compare the time of the model collapse, the trend of loss in line graphs, and the impact of different datasets on results. Experimental results indicated that both one-label smoothing and TTUR postponed the model collapse while EMD completely get rid of it. Furthermore, generated images may lose texture information when using more complicated datasets.