Using Artificial Intelligence to Measure Facial Expression following Facial Reanimation Surgery

Abstract

Social interactions are largely dependent on the interpretation of information conveyed through facial expressions. Although facial reanimation seeks restoration of the facial expression of emotion, outcome measures have not addressed this directly. This study evaluates the use of a machine learning technology to directly measure facial expression before and after facial reanimation surgery. Fifteen study subjects with facial palsy were evaluated both before and after undergoing cross-facial nerve grafting and free gracilis muscle transfer. Eight healthy volunteers were assessed for control comparison. Video footage of subjects with their face in repose and with a posed, closed-lip smile was obtained. The video data were then analyzed using the Noldus FaceReader software application to measure the relative proportions of seven cardinal facial expressions detected within each clip. The facial expression recognition application detected a far greater happy signal in postoperative (42 percent) versus preoperative (13 percent) smile videos (p < 0.0001), compared to 53 percent in videos of control faces smiling. This increase in postoperative happy signal was achieved in exchange for a reduction in the sad signal (15 percent to 9 percent; p = 0.092) and the neutral signal (57 percent to 37 percent; p = 0.0012). For video clips of patients in repose, no significant difference in happy emotion was detected between preoperative (3.1 percent) and postoperative (1.4 percent) states (p = 0.5). This study provides the first proof of concept for the use of a machine learning software application to objectively quantify facial expression before and after surgical reanimation. CLINICAL QUESTION/LEVEL OF EVIDENCE:: Diagnostic, IV.