Outdoor mobility aid for people with visual impairment: Obstacle detection and responsive framework for the scene perception during the outdoor mobility of people with visual impairment

Abstract

Outdoor mobility of individuals with visual impairment is challenging particularly where collision with obstacles can have significant impact on both physical and mental health. A variety of technological mobility aids for visually impaired people (VIP) have been studied and proposed in the literature which mainly utilise machine intelligence and deep learning (DL) approaches for object detection. However, object detection via the existing approaches suffers from reliability challenge due to real-time dynamics or the lack of available domain knowledge for specific obstacles identified by the VIP as potential hazards. In the present study, an object detection model (ObDtM) based on deep transfer learning techniques was developed for a custom-built dataset comprising of specific obstacles identified by the VIP as potential hazards. A custom dataset was compiled and manually annotated from various publicly available sources to train the ObDtM. Experiments were conducted to evaluate the proposed ObDtM for unseen obstacles kept as the test set. Results showed that ObDtM outperformed the state-of-the-art with 97% mean Average Precision (mAP), indicating a robust and generalizable DL approach. The compiled dataset and the ObDtM is useful for several potential use cases, particularly highlighting the use of DL in IoT and smart city applications. Additionally, a smart synergetic outdoor mobility framework was proposed for VIP (SOMAVIP) allowing comprehensive and accurate semantic representation of the surroundings by utilising the proposed ObDtM, cloud services, internet of things (IoT), and digital environment in the context of emerging smart city infrastructure. The proposed SOMAVIP can be highly impactful for improving VIPs’ quality of life mainly for safer, cost-effective, and reliable independent outdoor mobility enriched with real-time perception and interpretations of the surroundings.