Data-Driven Facial Beauty Analysis: Prediction, Retrieval and Manipulation

Abstract

Facial beauty analysis becomes an emerging research area due to many potential applications, such as aesthetic surgery plan, cosmetic industry, photo retouching, and entertainment. In this paper, we propose a data-driven facial beauty analysis framework that contains three application modules: prediction, retrieval, and manipulation. A beauty model is the core of the framework. With carefully designed features, the model can be built for different purposes. For prediction, we combine several low-level face representations and high-level features to form a feature vector and perform feature selection to optimize the feature set. The model built with the optimized feature set outperforms state-of-the-art methods. Then, we discuss two scenarios of beauty-oriented face retrieval: for recommendation and for beautification. Finally, we propose two approaches for facial beauty manipulation. One is an exemplar-based approach that uses the retrieved results. The other is a model-based approach that modifies facial features along the gradient of the beauty model. In this case, the model is built with the shape or appearance feature. Experimental results show that the exemplar-based approach is better for shape beautification; the model-based approach is suitable for texture beautification; and the combination of them can increase the attractiveness of a query face robustly.