Bionic Kinect device to assist visually impaired people by haptic and voice feedback

Abstract

This paper presents a wearable bionic Kinect device for blinds that helps visually challenging persons to move freely without any external assistance. This wearable consists of Kinect sensor, NUC (Next Unit of Computing) board and haptic feedback. Kinect sensor is used to get color data by RGB (Red Green Blue) camera and depth data by infrared projector which emits coded pattern received by receiver sensor that captures deformed pattern due to reflection by obstacles; algorithm decodes these variations into depth matrix. Image processing algorithms are applied on depth matrix to remove noise. Two dimensional Image processing algorithms of edge, contour detection and segmentation techniques are applied on multidimensional matrix at each level to extract important information about obstacle, humans and their characteristics. Extracted information of obstacles and humans are converted to haptic sequence and voice commands. Haptic feedback device receives the sequence, decodes sequence by using onboard microcontroller algorithm and creates vibration of different intensity at different regions. Text is recognized by optical character recognition system and faces by machine learning algorithms of device uses Kinect color data. All these tasks are performed by two processing units, a microcontroller and NUC which is excellent combination of wearable ubiquitous computing. Whole system is light weight, battery powered with intelligent power management circulatory to make it portable and easy to use.