Abstract

Blind and Visually Impaired People (BVIP) are likely to experience difficulties with tasks that involve scene recognition. Wearable technology has played a significant role in researching and evaluating systems developed for and with the BVIP community. This paper presents a system based on Google Glass designed to assist BVIP with scene recognition tasks, thereby using it as a visual assistant. The camera embedded in the smart glasses is used to capture the image of the surroundings, which is analyzed using the Custom Vision Application Programming Interface (Vision API) from Azure Cognitive Services by Microsoft. The output of the Vision API is converted to speech, which is heard by the BVIP user wearing the Google Glass. A dataset of 5000 newly annotated images is created to improve the performance of the scene description task in Indian scenarios. The Vision API is trained and tested on this dataset, increasing the mean Average Precision (mAP) score from 63% to 84%, with an IoU > 0.5. The overall response time of the proposed application was measured to be less than 1 second, thereby providing accurate results in real-time. A Likert scale analysis was performed with the help of the BVIP teachers and students at the "Roman Catherine Lobo School for the Visually Impaired" at Mangalore, Karnataka, India. From their response, it can be concluded that the application helps the BVIP better recognize their surrounding environment in real-time, proving the device effective as a potential assistant for the BVIP.

Highlights

  • A CCORDING to the World Health Organization, it is estimated that there are at least 2.2 billion people globally who have vision impairment or blindness1

  • With the motivation of helping the Blind and Visually Impaired People (BVIP) community, this paper presents an application implemented on Google Glass2 that acts as a visual assistant to the BVIP

  • This paper presents a Google Glass-based application to solve some of the problems faced by the BVIP community by developing a scene descriptor using the Custom Vision API provided by Azure Cognitive Services3

Read more

Summary

Introduction

A CCORDING to the World Health Organization, it is estimated that there are at least 2.2 billion people globally who have vision impairment or blindness. Around 45 million are blind and in need of vocational and social support. This population faces many difficulties in perceiving and understanding their surroundings since more than 80% of the information entering the brain is visual [1]. The loss of sight represents a public health, social and economic issue in developing countries, where 9 out of 10 of the world’s blind live. It is estimated that more than 60% of the world’s blind reside in India, sub-Saharan Africa, and China. In terms of regional differences, the prevalence of vision impairment in low and middle-income regions is estimated

Methods
Results
Conclusion
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call