The Effects of Face Masks on the Performance of Modern MWIR Face Detectors

Abstract

AbstractSince the pandemic is still a challenge all over the world, wearing a mask has become the new norm. Even if we step out of this pandemic soon, another one may come in the future, and thus, wearing a mask will always be considered a step towards the right direction in preventing the spread of a virus. Wearing face masks poses a challenge for existing biometric systems that are passive and depend on the full facial information to perform well. This book chapter explains the impact of the usage of face masks on the performance of various face detectors trained using some of the most efficient object detection models to-date in terms of accuracy and computational complexity. The chapter presents an insight into the effects of face masks on face detection in one of the most challenging spectral bands, i.e. the Mid-Wave Infrared (MWIR; 3–5 μm) and addresses the problem by training new models using masked face image data. Initially, we train and test two of each model, one with masked and one with unmasked data, to establish a baseline of performance. Then, we test those model’s abilities to generalize the face boundaries on the opposite face data from what it was trained on. Subsequently, we determine a percent drop in performance, called a penalty, when comparing a model’s performance when tested on the same category of data it was trained on versus tested on the opposite category of data. For unmasked trained models tested on masked data, we noticed an average precision and recall penalty of 37.27% and 37.25%, respectively. Similarly, the computational time for the detection of a single face image increased by an average of 12.72%. Then, this process is reversed, where we test the models we trained on masked data with unmasked face data and determine the drop in performance when each model is asked to detect faces on this unfamiliar data. We notice a near identical drop or penalty in detection time at 11.81% as well as an average precision drop of 30.24% and a recall drop of 29.50%. Faster R-CNN Inception-ResNet V2 was determined to be superior in terms of performance. Across all experiments performed, it outperforms all other models in terms of precision and recall. The downside is the fact that it is slow compared to all other models, making it a less attractive solution when working in mobile or time sensitive operations. Our fastest model is the CenterNet RestNet50 V2, but performs poorly when the train and test data are different. This shows that unmasked train models are considerably disadvantaged in scenarios where masks are required and face detection algorithms are used. Models trained on solely masked face data perform marginally better, but largely cannot stand on their own. This highlights the need for using masked face data when training face detection models in today’s ever changing Covid-19 landscape.KeywordsFace detectionDeep learningMasked facesThermal imagingMWIRFace recognitionNight-time environmentsCOVID-19