LEIFR-Net: light estimation for implicit face relight network.

Abstract

Relighting facial images based on estimated lighting distribution and intensity from image backgrounds and environments can lead to more natural and convincing effects across diverse settings. In this paper, we introduce the Light Estimation for Implicit Face Relight Network (LEIFR-Net), which we believe to be a novel approach that significantly improves upon current methodologies. Initially, we present a method to estimate global illumination from a single image. We then detail our approach for structurally disentangled relighting of faces using pixel-aligned implicit functions. Furthermore, we elaborate on constructing a paired synthetic dataset, which includes environments, maps of lighting distribution, albedo and relighted faces, utilizing a process we refer to as stable diffusion. Our experimental results, evaluated against specific benchmarks, demonstrate the effectiveness of LEIFR-Net in achieving more harmonious alignment of highlights and shadows with environmental lighting, surpassing the performance of other contemporary methods in this domain.