Electronic Travel Aid for Visually Impaired People based on Computer Vision and Sensor Nodes using Raspberry Pi

Abstract

Objectives: To make robust, reliable and affordable Electronic Travel Aid (ETA) which is composed of ultrasonic sensor nodes, smart cane stick and object detection device used in synergy used by blind and visually impaired persons for navigation and object recognition in indoor and outdoor environments. Methods: For the navigation in indoor environment, four ultrasonic sensor nodes which measure proximity of surrounding obstacles and if any object is too close then vibration feedback is given to the user. Smart cane stick will detect wet floor and stairs and if detected then vibration feedback is given at cane handle. In object recognition device we have used Raspberry pi 3 and camera which will detect defined objects in front of the blind user and describe it using audio feedback via headphones in which we have used computer vision techniques from opencv libraries like SURF extraction and haar cascades used with neural networks for cognitive machine learning which makes object detection and recognition robust. Findings: Using ultrasonic sensor nodes at ankles, wrist and waist, yielded better navigation in indoor environment because the ultrasonic sensors are more accurate, fast and distortion less as compared to the other sensors. A simple cane stick gives less information about the environment while smart cane stick gives information about stairs, wet floor and unevenness of floor to the blind user to avoid collision and also better range of detection as compared to simple cane stick. For object recognition, SURF feature extraction and haar cascades are used which is best suited for object detection and recognition of defined objects. And using machine learning increases the accuracy of the object recognition for unknown objects so like this the blind user can understand what kind of object is there infront of him, which was not possible using distance sensors alone. Using the three devices in synergy the blind person can navigate and recognize the objects in indoor and outdoor environments more easily and makes our ETA more accurate, efficient and affordable. Application: For visualisation of the surroundings for blind and visually impaired people.