Communication Assistant Using IoT-Based Device for People with Vision, Speech, and Hearing Disability

Abstract

AbstractAccording to the census of 2011, about 10.6 million people are visually impaired, 1.2 million people have hearing disability, and 1.6 million people are speech impaired in India. In this paper, we propose an Internet of Thing (IoT)-based integrated device for these people in order to provide them a user-friendly and interactive environment through which they can easily communicate with others and interact with their surroundings. The device provides advanced features like facial recognition, emotion recognition, object recognition, sign language recognition, cloth pattern and color recognition, distance from an object, and optical character recognition. By giving input in the form of voice, text, and sign, one can use different functionalities of the device. Algorithms like image-to-text, text-to-speech, and speech-to-text conversion are implemented to provide intended functionality. Convolutional Neural Network (CNN) is used for feature extraction and classification. The proposed device considers various communications between speech, vision, and hearing impaired people with others.KeywordsFacial recognitionObject detection and recognitionEmotion recognitionOptical character recognitionSign language conversion