Robotic Assistant for Object Recognition Using Convolutional Neural Network

Abstract

Visually impaired persons encounter certain challenges, which include access to information, environmental navigation, and obstacle detection. Navigating daily life becomes a big task with challenges relating to the search for misplaced personal items and being aware of objects in their environment to avoid collision. This necessitates the need for automated solutions to facilitate object recognition. While traditional methods like guide dogs, white canes, and Braille have offered valuable solutions, recent technological solutions, including smartphone-based recognition systems and portable cameras, have encountered limitations such as constraints relating to cultural-specific, device-specific, and lack of system autonomy. This study addressed and provided solutions to the limitations offered by recent solutions by introducing a Convolutional Neural Network (CNN) object recognition system integrated into a mobile robot designed to function as a robotic assistant for visually impaired persons. The robotic assistant is capable of moving around in a confined environment. It incorporates a Raspberry Pi with a camera programmed to recognize three objects: mobile phones, mice, and chairs. A Convolutional Neural Network model was trained for object recognition, with 30% of the images used for testing. The training was conducted using the Yolov3 model in Google Colab. Qualitative evaluation of the recognition system yielded a precision of 79%, recall of 96%, and accuracy of 80% for the Robotic Assistant. It also includes a Graphical User Interface where users can easily control the movement and speed of the robotic assistant. The developed robotic assistant significantly enhances autonomy and object recognition, promising substantial benefits in the daily navigation of visually impaired individuals.