Optimal Transport of Deep Feature for Image Style Transfer

Abstract

Image style transfer is a classic image editing task which aims to transfer arbitrary visual styles to content images. In recent years, it has been revealed that a well-trained convolutional neural network with sufficient labeled data is powerful to deal with the style transfer problem. Thanks to the recent advances in the analysis of neural style transfer, the image style transfer can be cast as a problem of distribution alignment. In this paper, we propose to solve this issue by incorporating the theory of optimal transport in a simple and intuitive way. The main component of our style transfer method is an optimal transportation map, which is derived from the Monge-Kantorovicth theory of mass transportation, to perform the alignment process from the content image to the style image. We compare the generated stylized images with a number of representative algorithms to demonstrate the effectiveness of our approach. We also show that our results are visually more consistent and well-stylized simultaneously.