Facial Beauty Prediction Based on Vision Transformer

Abstract

Facial beauty analysis is a crucial subject in human culture among researchers across various applications. Recent studies have utilized multidisciplinary approaches to examine the relationship between facial traits, age, emotions, and other factors. Facial beauty prediction is a significant visual recognition challenge that evaluates facial attractiveness for human perception. This task demands considerable effort due to the novelty of the field and the limited resources available, including a small database for facial beauty prediction. In this context, a deep learning method has recently shown remarkable capabilities in predicting facial beauty. Additionally, vision Transformers have recently been introduced as novel deep learning approaches and have shown strong performance in various applications. The key issue is that the vision transformer performs significantly worse than ResNet when trained on a small ImageNet database. In this paper, we propose to address the challenges of predicting facial beauty by utilizing vision transformers instead of relying on feature extraction based on Convolutional Neural Networks, which are commonly used in traditional methods. Moreover, we define and optimize a set of hyperparameters according to the SCUT-FBP5500 benchmark dataset. The model achieves a Pearson coefficient of 0.9534. Experimental results indicated that using this proposed network leads to better predicting facial beauty closer to human evaluation than conventional technology that provides facial beauty assessment.