Progressive complex illumination image appearance transfer based on CNN

Abstract

Image appearance transfer is the process of transferring the appearance features from the user-supplied reference image to the target image. Since traditional appearance transfer methods are based on low-level feature, it is difficult to obtain natural effect in the case of complex illumination between the target image and the reference image. This is mainly because that the appearance mapping relation with complex illumination is a highly nonlinear problem. Although traditional methods are good at dealing with linear transformations, they are less suitable for solving the highly nonlinear problems caused by such complex illumination. Moreover, convolutional neural network (CNN) can extract the hierarchical abstraction features at different levels of the network layer, so it can be conveniently used to realize progressive transfer. In this paper, we propose a novel method for progressive appearance transfer for images with complex illumination. Firstly, we convert the input images from the RGB color space to the HSV color space. The illumination transfer is carried out only in the illumination channel of the image, and for the other two channels, the color distribution of the reference image is transferred to the target image. Secondly, CNN is specially designed to extract the hierarchical feature maps. To achieve progressive transfer, the histogram reshaping method is carried out by using the hierarchical feature maps extracted from the CNN. The appearance transfer results are obtained after the illumination transfer and color transfer. To optimize the transfer results, we adopt the joint bilateral filter to smooth the noises. The experimental results show that our method can effectively solve the problem of progressive appearance transfer for images with complex illumination.