Facial Landmark Tracking in Videos of Individuals with Neurological Impairments: Is There a Trade-off Between Smoothness and Accuracyƒ.

Abstract

Orofacial kinematics are valuable markers of function and progression in a variety of neurological disorders. Recent advances in facial landmark detection have been used to improve landmark tracking in video, for example by accounting for interframe optical flow. It has been demonstrated that finetuning (a type of transfer learning) can improve the performance of some facial landmark detection systems. Here, we asked whether a neural network model that is pretrained using video data (supervision by registration, SBR) can be finetuned to improve landmark detection and tracking, using data from the Toronto Neuroface Dataset (n=36), which comprises 3 different clinical populations. We finetuned the supervision by registration (SBR) model using data from 3 individuals from each of 3 clinical populations (n=9), with or without neurological impairments. The remaining individuals from our dataset (n=27) were used for evaluation. Finetuning SBR moderately improved the model's accuracy but substantially increased the smoothness of tracked landmarks. This suggests that finetuning on video-trained models, like SBR, could improve the estimation of orofacial kinematics in individuals with neurological impairments. This could be used to improve the detection and characterization of neurological diseases using video data.Clinical Relevance-This work demonstrated that transfer learning applied to video-trained facial landmark detectors could improve the measurement of orofacial kinematics in individuals with neurological impairments.