Robust residual convolutional neural network based pupil tracking for low-computational power applications

Abstract

Existing Deep Fully Convolutional Network (DFCN) based pupil segmentation models have been shown to perform robust and accurate pupil detection. However, they lack high computational resources when real-time predictions for low computational wearable devices such as Raspberry Pi are of interest. Simple FCN (Fully Convolutional Network) models, on the other hand, can provide real-time yet not as accurate predictions with low computational power devices. In this study, we address this dilemma by proposing a CNN (Convolutional Neural Network)-based model, Residual CNN, with several advanced operations such as residual connections, Squeeze and Excitation (SE) attention, and Atrous Spatial Pyramid Pooling (ASPP) to improve the prediction performance without complicating the model. Moreover, we leverage transfer learning by training synthetic images, then fine-tuning them with authentic eye images. We also fully quantize our model parameters to speed up the predictions and apply the Quantization Aware Training (QAT) strategy to obtain accurate predictions. Our experiments show that our quantized Res-CNN model trained by QAT strategy with 40x30 resolution images provides robust yet real-time predictions with an average of 1.351 Root Mean Square Error (RMSE) for the pupil center predictions and 8.317 ms response time per image on Raspberry Pi.