Abstract
Fast Style Transfer is a series of Neural Style Transfer algorithms that use feed-forward neural networks to render input images. Because of the high dimension of the output layer, these networks require much memory for computation. Therefore, for high-resolution images, most mobile devices and personal computers cannot stylize them, which greatly limits the application scenarios of Fast Style Transfer. At present, the two existing solutions are purchasing more memory and using the feathering-based method, but the former requires additional cost, and the latter has poor image quality. To solve this problem, we propose a novel image synthesis method named \emph{block shuffle}, which converts a single task with high memory consumption to multiple subtasks with low memory consumption. This method can act as a plug-in for Fast Style Transfer without any modification to the network architecture. We use the most popular Fast Style Transfer repository on GitHub as the baseline. Experiments show that the quality of high-resolution images generated by our method is better than that of the feathering-based method. Although our method is an order of magnitude slower than the baseline, it can stylize high-resolution images with limited memory, which is impossible with the baseline. The code and models will be made available on \url{https://github.com/czczup/block-shuffle}.
Highlights
Fast Style Transfer [1], [2] uses feed-forward neural networks to learn artistic styles from paintings and uses the learned style information to render input images
Prisma [4] is a famous mobile application based on Fast Style Transfer
The memory usage of stylizing an image below 4000 × 4000 pixels is less than 1.0 GB. It means that our method can enable most mobile devices and personal computers to support high-resolution Fast Style Transfer, which will contribute to the industrialization of Fast Style Transfer
Summary
Fast Style Transfer [1], [2] uses feed-forward neural networks to learn artistic styles from paintings and uses the learned style information to render input images. Prisma [4] is a famous mobile application based on Fast Style Transfer It has set off the trend of using photos for artistic creation, and more and more people are enthusiastic about using this application to render their photos and share them on social networks. One is to buy more memory to meet computing needs, but this approach increases the cost and does not completely solve the problem Another one is to divide the input image into many overlapping sub-images, stylize them respectively, and use the feathering effect [5], [6] to stitch them (hereinafter referred to as feathering-based method). 3) This method is non-invasive, which only adds pre-processing and post-processing steps before and after the image transformation network, and does not need to retrain the model
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.