Deep Learning Based Mobile Assistive Device for Visually Impaired People

Abstract

This paper presents a mobile assistive device for the visually impaired. Low vision people had difficulties in object recognition, text reading, and face-to-face communication. We developed a prototype of a portable smart mobile device that supports object detection, text reading, and facial expression recognition using recent deep learning technologies. After applying recent object detection technology and optical character recognition software(OCR), and facial expression recognition, we converted the networks for Android-based mobile devices. We developed a portable mobile device with enhanced battery power and a high-performance AP with a line-connected camera device. The system provides detected object names verbally when looking at the object by face direction and by indicating using a hand finger. The proposed device can assist the communication of the low vision with a normal person in addition to reduce threats or dangers due to unrecognized or mis-detected objects. The proposed system will support social interaction and a better life for visually impaired people.