What is the best set of markers for facial movements recognition?

Abstract

Nowadays, motion capture with light and markers is valid tool for medical applications. However, in the face analysis domain, a framework for defining the optimal marker set layout does not exist yet. Therefore, the objective of this study is to propose an automatic approach to compute the optimized layout with the minimum number of facial marker. Hundred and twenty-two distinct face motion captures, acquired through 18 optoelectronic cameras T160 and two Bonita video cameras at a frequency of 100 Hz, have been analysed. Every capture contains the 3D coordinates of 109 markers (Ø 1.5 mm) fixed by a trained physiotherapist on the facial skin surface. The six facial movements acquired are the closure of the eyes, the forced closure of the eyes, the pronunciation of the sounds [o] and [pμ], a smile, and a spontaneous smile. These movements have been chosen due to their great importance in analysis of facial expression in healthy, pathological or rehabilitative subject. They take place in different zones of the face involving both the frontal and orbicular zones, and the zones of the lips and the chin. Then, the distances between each marker and its nearest neighbours are computed and used as input of a recursively two-step KNN classification process, performed on a subset of the previous computed features, selected through a Feature Selection Procedure. The subset that best of all allows an automatic identification of the types of the performed movement is recorded and the marker set layout is extracted from it. This systematic study has identified a set of 19 markers, shown in Fig. 1. The recognition rate obtained is 95%. Hence, the solution accurately records the motion information necessary to discriminate the six facial expressions.