Abstract Masked face recognition presents unique challenges especially due to occlusion caused by face masks. This paper introduces a novel approach called HOGE which integrates the visualisation of HOG images as input for a modified EfficientNetV2-S model to perform masked face recognition. This modified EfficientNetV2-S model employs an additional convolution layer for processing greyscale masked face images which differs from other pre-trained CNN-based models. It aims to address the challenge of recognising masked face images and investigate the effect of feature descriptor interaction with deep transfer learning technique in masked face recognition. Two benchmark datasets were used to evaluate the performance of proposed method. Experimental results demonstrate that HOGE achieves accuracies of 97.41% on LFW-SMFRD and 99.38% on RMFRD which show that the proposed method works well and effectively recognises masked face images from both datasets.