A Wearable Device for Indoor Imminent Danger Detection and Avoidance With Region-Based Ground Segmentation

Zhongen Li,Dustin R Grooms,Chang Liu,Brian C Clark,Fanghao Song

doi:10.1109/access.2020.3028527

Zhongen Li, Dustin R Grooms + Show 3 more

Open Access

https://doi.org/10.1109/access.2020.3028527

Copy DOI

Abstract

Avoiding objects independently in indoor environments for individuals with severe visual impairment is one of the significant challenges in daily life. This paper presents a wearable application to help visually impaired people quickly build situational awareness and traverse safely. The system utilizes Red, Green, Blue, and Depth (RGB-D) camera and an Inertial Measurement Unit (IMU) to detect objects and the collision-free path in real-time. A region proposal module is presented to decide where to identify the ground from 3D point clouds. The segmented ground area can act as the traversable path, and its corresponding region in the image is removed to prevent detecting painted objects. The system can provide information about the category, distance, and direction of the detected objects by fusing the depth image and the neural network results. A 3D acoustic feedback mechanism is designed to improve the situational awareness for visually impaired people, and guild them traverse safely. The advantage of this system is that our 3D region proposal module can robustly propose the potential ground region and greatly reduce the computation cost of the ground segmentation. Besides, a typical machine-learning-based approach may miss objects because they could not be recognized, though they still may pose a danger. Another advantage of our approach is that the imminent danger detector can detect such unrecognizable objects to help users avoid a collision. Finally, experimental results demonstrate that the proposed system can be a useful indoor assistant tool to help blind individuals with collision avoidance and wayfinding.

Highlights

According to the World Health Organization (WHO) report [1], there are no less than 2.2 billion people with visual impairment or blindness worldwide
This paper proposes a wearable device for indoor object detection and avoidance system with region-based ground segmentation to improve mobility for visually impaired people
EXPERIMENTS ON GROUND SEGMENTATION 1) ORIENTATION TRACKING Sensitivity, zero-offset, and bias are the intrinsic parameters of Inertial Measurement Unit (IMU), and they attempt to convert the raw sensor data into real measurements

Summary

INTRODUCTION

According to the World Health Organization (WHO) report [1], there are no less than 2.2 billion people with visual impairment or blindness worldwide. For the blind and visually impaired, accurate object detection and timely feedback are required to help them complete indoor activities. These challenges add to the classic problems of spatial navigation and obstacle avoidance. Z. Li et al.: Wearable Device for Indoor Imminent Danger Detection and Avoidance With Region-Based Ground Segmentation visual sensors [6], [7], and laser emitters [8]). Indoor assistive object detection still faces many challenges: low-cost and effective obstacle recognition and avoidance, real-time and rich feedback, and the complexity of a holistic system on compact portable devices for visually impaired users. This paper proposes a wearable device for indoor object detection and avoidance system with region-based ground segmentation to improve mobility for visually impaired people. The conclusions, along with future work, are discussed in the end

RELATED WORK

DEPTH PRE-PROCESSING

EXPERIMENTAL RESULTS AND DISCUSSIONS

CONCLUSION AND FUTURE WORK