Learning upper patch attention using dual-branch training strategy for masked face recognition

Abstract

In the context of pandemic, COVID-19, recognition of masked face images is a challenging problem, as most of the facial components become invisible. By utilizing prior information that mask-occlusion is located in the lower half of the face, we propose a dual-branch training strategy to guide the model to focus on the upper half of the face to extract robust features for Masked face recognition (MFR). During training, the features learned at the intermediate layers of the global branch are fed to our proposed attention module, named Upper Patch Attention (UPA), which acts as a local branch. Both branches are jointly optimized to enhance the feature extraction from non-occluded regions. We also propose a self-attention module, which integrates into the backbone network to enhance the interaction between the channels and spatial locations in the learning process. Extensive experiments on synthetic and real-masked face datasets demonstrate the effectiveness of our method.