Voice Controlled Object Grasping Robotic Arm for Visually Impaired Disabled Veterans

Abstract

Robotic arms have been growing in popularity for various applications like for the manufacturing industry, medical industry, and aerospace industry. This paper utilizes a low-cost robotic arm, DOBOT Magician, to help visually impaired individuals access essential items from a refrigerator. The system takes in audio input using natural language, i.e., without the need to learn special vocabulary from a user and converts it so the system can identify the object using a depth sensing camera and a robotic arm that retrieves the item based on the audio and visual input. This system can properly identify and retrieve the correct item using audio input through natural language processing, a modified version of the deep-learning object detection network YOLOv4 (You only look Once), visual input using Python Pyrealsense2 library for Intel Realsense D455, and a custom-made Inverse Kinematics and object retrieval Python program to control the DOBOT Magician.