Development of Computer Vision System to Extract Kansei Information from Facial Expressions

Abstract

Kansei engineering is one field of engineering where the feelings of humans are applied to manufacturing. If Kansei can be extracted, the design, development, and evaluation of various things are possible. This study aims to build an automatic design and evaluation system for agricultural products and landscapes and then utilize the system for the evaluation and improvement of educational effects. In this presentation, we built a low price and near real-time system to extract Kansei information from the facial expressions of users and obtained knowledge about Kansei. Additionally, we created an algorithm to discriminate Kansei. We built a system to obtain facial expression information of a user from a camera image as our experimental equipment. The characteristic values calculated from them were the variation (or the normalized polygon area change ratio) of the four items; the inner ends of the eyebrows, upper parts of the eyes, lower parts of the eyes, and the corners of the mouth. The target Kansei was three items: positive Kansei (liking, fun, and happy) and negative Kansei (unpleasantness and hatred). The results are as follows:First, for the target three Kansei, changes could be captured through the four specified normalized polygon area change rates. As for the positive Kansei, the changes in the corners of the mouth were greater than those of the others in particular. As for the negative Kansei, the changes in the lower parts of the eyes were slightly greater than those of the others and, as for the Kansei of surprise, the changes in the corners of the mouth were slightly greater. The direction of the change showed a tendency where the positive Kansei increased at the corners of the mouth in particular. The negative Kansei showed a strong tendency to decrease at the inner ends of the eyebrows and the lower parts of the eyes in particular. The Kansei of surprise showed a tendency where the normalized polygon area change rates increased as a whole. Next, we selected one subject because the relationship between the facial expressions and Kansei varies among individuals, and then we built an algorithm to discriminate the three Kansei features. As for the Kansei of positive – negative and negative – surprise, it became possible to discriminate at a rate of nearly 100% from the information of the corners of the mouth. It became possible to discriminate the positive-surprise at a rate of about 80% from the information of the upper parts of the eyes. As described above, although it is necessary to prepare the template image of the seven points of the target person and adjust the parameters after measuring the Kansei facial expression in advance, a low price and near real-time computer vision system to extract facial expression Kansei could be built.