Accelerate neural style transfer with super-resolution

Abstract

Style transfer is a task of migrating a style from one image to another. Recently, Full Convolutional Network (FCN) is adopted to create stylized images and make it possible to perform style transfer in real-time on advanced GPUs. However, problems are still existing in memory usage and time-consumption when processing high-resolution images. In this work, we analyze the architecture of the style transfer network and divide it into three parts: feature extraction, style transfer, and image reconstruction. And a novel way is proposed to accelerate the style transfer operation and reduce the memory usage at run-time by conducting the super-resolution style transfer network (SRSTN), which can generate super-resolution stylized images. Compared with other style transfer networks, SRSTN can produce competitive quality resulting images with a faster speed as well as less memory usage.