A Deep Multimodal Network for Classification and Identification of Interventionists' Hand Motions during Cyborg Intravascular Catheterization

Abstract

Recent insights from human-robot intelligence and deep learning raise hope towards task-specific autonomy in robotic intravascular coronary interventions. However, lack of learning-based methods for characterizing the interventionists' kinesthetic data hinders the drive for shared control and robotic autonomy during cyborg catheterization. In this study, a deep multimodal network model is proposed for classification and recognition of interventionists' hand movements during cyborg intravascular catheterization. The model has two modules for extracting salient features in electromyography signal datasets, and classification of hand motions made during intravascular catheterization procedures. Network training and evaluation observed for in-vitro and in-vivo datasets obtained from trained novice subjects and expert with about 5 years of experience in percutaneous coronary interventions. Performance evaluation shows the learning model could classify interventionists' hand movements accurately in manual and robot-assisted navigations, respectively. This study is suggested to further stimulate the development of appropriate skill level assessments towards cyborg catheterization for cardiac interventions.