ISeePlus : A cost effective smart assistance archetype based on deep learning model for visually impaired

Abstract

In this paper, a unique smart assistance aid, iSeePlus, for visually impaired people is presented. iSeePlus, as a prototype product, is equipped with smart glasses (iSee Glasses), a smart stick (iStick) and an android application which will be initially used to set up the device. The smart glasses are trained on proposed deep learning models to provide features like face detection and recognition, emotion detection and image captioning with a click of a specifically placed buttons on the device. The proposed prototype product also supports reading texts from images via Google Cloud Vision API and allows the user to send alert texts with his current location to the saved emergency contacts. The iStick is an integrated accessory that comes with iSeePlus which creates vibrations to alert the user of any incoming obstacle within the range of 25 cm. The android application will be used to set up the device at initial stages as it helps to add a database of known people and the emergency contacts. Presently, iSeePlus, has an accuracy of 98.03% on face detection and recognition model, FDREnet, 68.5% on emotion detection with ExpressNet and 65.7% on image captioning with CaptionMeNet BLUE-1: 65.7). With the help of iSeePlus, people suffering from visual impairment will now be able to do their day-to-day chores independently up to a significant degree.