Face Hallucination Based on Modified Neighbor Embedding and Global Smoothness Constraint

Abstract

Based on the manifold assumption, some face hallucination methods have been developed. However, since the super-resolution (SR) is an ill-posed problem, the manifold assumption does not hold always. To solve this problem, we modify the assumption using Easy-Partial Least Squares (EZ-PLS) algorithm and present a new face hallucination scheme using the modified assumption. Firstly, the high-resolution (HR) and corresponding low-resolution (LR) images are divided into small patches. Secondly, EZ-PLS is employed to learn two projection matrices simultaneously, via which original HR and LR image patches are mapped onto a unified feature space. Through this method, we guarantee the consistency relationship between the HR representation manifold and corresponding LR representation manifold. Then, we hallucinate the preliminary HR result based on neighbor embedding algorithm using the unified feature space. Moreover, in order to improve the overall smoothness of the preliminary results, the high-frequency parts of the preliminary estimation are extracted and incorporated into the maximum a posteriori (MAP) formulation for SR problem so as to generate the final result. Experimental results show that the proposed method outperforms some state-of-the-art algorithms.