Complex object relighting via split-then-composition by semantics and materials

Abstract

Complex object illumination transfer is a special challenge in computer vision. In our paper, we put forward a method for complex object illumination transfer. Firstly, the input object image was divided into object components by semantic analysis, to find the reference object images consistent with the object component material in the physical world by material analysis. Material has a great influence on the illumination transfer of object image, so the use of material analysis can greatly reduce the influence of material on the illumination transfer in the later stage. Next, a block matching algorithm was used to deform each reference object image and made it match with each component shape of the input object image. Then, each component of the input object image and each warped reference object image were illuminated by local and global transfer model. Finally, semantic analysis was used to synthesize the re-illumination components of the input object image to obtain the re-illumination input object image. The experimental results prove that the method could make a good effect on the illumination transfer. Our main contribution is the use of semantic and material analysis to split complex objects into simple objects, and skillfully combine semantic and material parsing and composition, block matching algorithm, local and global light migration model to achieve the relighting of complex objects.