Automated warping procedure for facial thermal imaging based on features identification in the visible domain

Abstract

A software solution for an automated warping procedure of facial InfraRed Imaging (IRI) on a reference template is presented. Visible and IR videos of six subjects’ faces were simultaneously recorded during controlled movement of the head (i.e., yaw, pitch, frown, speaking). Sixty-eight facial landmarks were identified in each frame in the visible domain through the open source software OpenFace and transformed into the corresponding frame of IRI; the IRI landmarks were then used as basis for a warping procedure into a reference template relying on a Local Weighted Mean (LWM) transformation. The developed algorithm was accurate in both spatial (average Root Mean Square Error, RMSE = 0.66 pixel) and temporal (RMSE = 0.81 ms) co-registration ensuring reliable estimation of regional temperature (RMSE = 0.09 °C) within a defined range of head rotation (±24.23° for yaw and ±13.79° for pitch movements). The proposed algorithm represents a novel automated solution for warping of IRI video sequences of the face on a reference template, pursuing a strong automation of the processing stream, operator-independence, and objectivity in the extraction and analysis of functional thermal data. It permits to overcome current limitations of IRI data processing, mainly due to operator-dependent identification of the Regions of Interest (ROIs). The methodology may allow further development of contact-less evaluation of the psychophysiological state of the subject through IRI being robust to inter-subject geometrical differences of the face and intra-subject movement-related noise.