Visual Prediction Based on Photorealistic Style Transfer

Abstract

In this study, we explore recent advances in photorealistic style transfer methods to make visual predictions of outdoor scenes. These methods transfer the elements’ visual appearance from one photo (style image) to another (content image), maintaining the original composition of the elements in the original image. However, the search for reference images containing the same elements as the content image and presenting all the desired style characteristics makes the process challenging and time-consuming. To overcome this challenge, we propose a dynamic search method based on the transient scene attributes performed in a dataset developed especially for this task. Our team developed a set of 924 3D images divided into six scenario groups, with the main elements found in the outdoor scenes to be used as style images. Each group has stylizations of the four seasons, the time of day, the presence or absence of rain, snow, and cloudy skies. In the end, we measured the similarity of the results obtained with real images. The structural similarity index measure (SSIM) reaches an average score greater than 0.8.