Robust Pupil Segmentation and Center Detection from Visible Light Images Using Convolutional Neural Network

Abstract

In this paper, we present a robust pupil detection method from visible light (VL) images using a convolutional neural network (CNN). In contrast to existing pupil detection algorithms, our method does not require infrared (IR) illuminations and cameras, and robustly works even for the images of dark-brown irises (black eyes) and/or images including strong corneal reflections such as outdoor scenes. Thus it has potential application scenarios. Our method first detects an eye region from an input image and applies CNN-based pupil segmentation which has a composition and decomposition structure. To learn the relationship between visible eye images and pupil segments, we construct two datasets and augmentation algorithms. One dataset is based on an existing eye image dataset (UBIRIS.v2), and the other consists of images taken by ourselves by using a corneal imaging camera that can take eye images in mobile environments. Applying color and corneal reflection augmentations to these image datasets and using them for learning the CNN, we built a robust pupil segmentation neural network. The performance is evaluated in three ways. First, we evaluate the segmentation accuracy. Second, we evaluate the pupil center detection accuracy using GI4E facial image sets. Third, we developed an eye gaze tracking (EGT) algorithm that uses the pupil detection and evaluated its accuracy. From the result, the proposed method detects pupil centers more accuracy than the state-of-the arts, and shows similar EGT accuracy to the commercial systems that use IR-active lighting setups.