Locality constraint neighbor embedding via KPCA and optimized reference patch for face hallucination

Abstract

Given that the limitations of the manifold assumption that the low-resolution (LR) and high-resolution (HR) patch manifolds are locally isometric, the geometrical information of HR patch manifold, which is much more credible and discriminant than LR patch manifold, has been paid more attention to in the recent face super-resolution algorithms. In general, these algorithms first construct its initial HR patch using conventional face super-resolution methods and then update the K-nearest neighbors (K-NN) of the input patch as well as corresponding reconstruction weights based on the initial HR patch to generate the final HR patch. Whether or not we can effectively utilize the information of the HR manifold depends on the quality of the initial HR patch. In this paper, to capture the nonlinear similarity of face features, we apply kernel principal component analysis (KPCA) to the conventional face super-resolution method and achieve a better initial HR patch. Furthermore, we propose the concept “optimized reference patch” to deal with the variations in human facial features and find the best-matched neighbors of input patch. Experimental results show that the proposed method outperforms several state-of-the-art face super-resolution algorithms.