Smart Robotic Surgical Assistant Using Voice Command and Image Processing

Abstract

This paper presents the development of a smart robotic surgical assistant that utilizes voice command and image processing to aid in surgical instrument handling. The system integrates a Dobot robotic arm controlled through verbal instructions to retrieve and position surgical tools, while an image recognition model, based on VGG16, identifies instruments in real-time from camera feeds. This automation enables hands-free operation, supporting sterile conditions and enhancing efficiency in the operating room (OR). A dataset comprising high-resolution surgical tool images was curated to fine-tune the VGG16 model, achieving over 95% accuracy in classification. Voice recognition, incorporated with OpenCV, reached a 92.5% accuracy rate in interpreting commands. This system addresses challenges in surgical tool management, offering an efficient and reliable alternative that reduces human error and improves workflow, signifying a major step toward integrated AI-robotics applications in healthcare.