Using Artificial Intelligence to Measure Facial Expression following Facial Reanimation Surgery.

Abstract

In the article titled “Using Artificial Intelligence to Measure Facial Expression following Facial Reanimation Surgery,” Boonipat et al.1 proposed an innovative approach to assess facial expression using machine learning software. By recording and analyzing videos of facial expression preoperatively and postoperatively, the authors were able to assess functional outcomes following facial reanimation surgery. We are writing to offer some considerations on how facial expression tracking software could complement telemedicine applications in craniofacial surgery based on the facial transplantation literature. Beyond the first posttransplant year, facial transplantation patients are expected to return for in-person visits at least twice yearly for the rest of their lives. Recovery of facial expression in these patients has been tracked using clinical scales, optic movement tracking devices, and electromyography.2 In 2018, Bedeloglu et al.3 proposed using photograph analysis software in face transplant recipients to detect emotional expression, including anger, fear, and happiness. In 2019, Fischer et al.4 demonstrated the feasibility of tracking functional recovery in facial transplantation using video analysis software on recordings from a commercially available camera (Canon EOS 600D; Canon, Tokyo, Japan). Recipients were followed for a year after transplantation, and objective improvements in mouth opening, nose wrinkling, and smile excursion were detected relative to before transplantation. Interestingly, some of the analyzed videos were self-recorded by face transplant recipients while at home, which may pave the way for facial expression tracking via telemedicine.4 The paradigm shift from traditional face-to-face patient-provider interactions to social-distancing forms of health care delivery spurred on by the coronavirus disease of 2019 pandemic has undoubtedly accelerated broad acceptance of telemedicine.5 However, the inability to perform a comprehensive physical examination still limits integration of telemedicine in craniofacial surgery. Boonipat et al.1 proposed measuring the relative proportions of seven cardinal facial expressions using the Noldus FaceReader, a commercially available software (Noldus Information Technology BV, Wageningen, The Netherlands), and were able to objectively quantify facial expression among patients undergoing facial reanimation surgery.1 This software may also be used to analyze videos self-recorded by patients at home in a similar fashion to that noted by Fischer et al.4 in face transplant recipients. Such a development would have marked implications in craniofacial surgery care, by reducing the financial burden and infectious risks associated with long-distance travel during a pandemic while objectively quantifying facial surgical outcomes. In conclusion, facial expression is an important outcome following craniofacial surgery with significant functional and psychosocial implications, and there is potential for innovation in its measurement via telemedicine. The study by Boonipat et al.1 demonstrates that currently available software applications may be able to objectively measure functional outcomes in craniofacial procedures. Software-based video analysis of facial expression has been used to obtain important quantitative data on the postoperative functional recovery of face transplant recipients, even when patients self-record the videos. Given the ongoing disruption in health care delivery caused by COVID-19 and telemedicine’s speedy expansion to address some of the resultant challenges, facial expression analysis may enhance telemedicine applications tailored for patients undergoing craniofacial surgical procedures. DISCLOSURE The authors have no financial interest in this communication. Daniel Boczar, M.D.Ricardo Rodriguez Colon, B.S.Bachar F. Chaya, M.D.Jorge Trilles, B.S.Eduardo D. Rodriguez, M.D., D.D.S.Hansjörg Wyss Department of Plastic SurgeryNew York University Langone HealthNew York, N.Y.