CNN-Based Object Recognition and Tracking System to Assist Visually Impaired People

Fahad Ashiq,Sadia Zafar,Muhammad Tariq Mahmood,Muhammad Asif,Ik Hyun Lee,Toqeer Mahmood,Maaz Bin Ahmad,Khalid Masood

doi:10.1109/access.2022.3148036

Abstract

Visually impaired persons (VIPs) comprise a significant portion of the population, and they are present around the globe and in every part of the world. In recent times, technology proved its presence in every domain, and innovative devices assist humans in their daily lives. In this work, a smart and intelligent system is designed for VIPs to assist mobility and ensure their safety. The proposed system provides navigation in real-time using an automated voice. Though VIPs wouldn’t be able to see objects in their surroundings, they can sense and visualize the roaming environment. Moreover, a web-based application is developed to ensure their safety. The user of this application can turn the on-demand function for sharing his/her location with the family while compromising privacy. Through this application, the family members of VIPs would be able to track their movement (get location and snapshots) while being at their homes. Hence, the device allows VIPs to visualize the environment and ensure their security. Such a comprehensive device was a missing link in the existing literature. The application uses MobileNet architecture due to its low computational complexity to run on low-power end devices. To assess the efficacy of the proposed system, six pilot studies have been performed that reflected satisfactory results. For object detection and recognition, a deep Convolution Neural Network (CNN) model is employed with an accuracy of 83.3%, whereas the dataset contains more than 1000 categories. Moreover, a score-based quantitative comparative analysis is performed using the supported features of devices. It is found that the proposed system has outperformed the existing devices having a total score of 9.1/10, which is 8% higher than the second-best.

Highlights

The World Health Organization (WHO) has reported that 285 million of the world's population is blind or visually impaired
For object detection and recognition, a deep Convolution Neural Network (CNN) model is employed with an accuracy of 83.3%, whereas the dataset contains more than 1000 categories
The main objective of this study is to develop a system for Visually impaired persons (VIPs) that contains the following features: It performs real-time object detection and recognition using a deep learning framework

Summary

INTRODUCTION

The World Health Organization (WHO) has reported that 285 million of the world's population is blind or visually impaired. Impaired persons (VIPs) face difficulties in performing activities of daily living (ADLs) e.g. opportunity of work and schooling, moving in their surroundings, ability to interact with the environment, and searching for common objects (indoor/outdoor) at their own or even with some assistance. Several approaches, systems, devices, and applications have been developed in the domain of assistive technology to facilitate VIPs to perform tasks that they were formerly unable to accomplish [4]. Such solutions generally comprise electronic devices equipped with cameras, sensors, and microprocessors capable of making decisions and providing tactile or auditory feedback to the user.

LITERATURE REVIEW

Findings

Conclusion