Enabling Real-time Gesture Recognition Data Delivery over ROS and OpenISS

Abstract

OpenISS is a motion capture data delivery frame-work for various types of applications ranging from computation arts to natural user interfaces and visual effects production. The core framework allows functionality for specific problems such as gesture recognition, person re-identification and facial analysis via respective specialization frameworks. In this paper, we present our solution to address an emerging problem, which is to broadcast gesture recognition data delivered using a gesture framework specialization over the network in real-time to further address our new requirements such as decoupling data source from the end user application. We leverage the Robot Operating System (ROS), an open source set of tools and libraries not only for its communication middleware to broadcast this data over its network stack but also because it allows us to interface and obtain relevant data from other depth-sensing devices that ROS can already interface via its community-driven open source packages. We evaluate our solution on various aspects and measure the related overhead or cost of such interoperability between our framework and the ROS middleware for which we employ metrics such as effective frame rate and network delay. We also provide our framework specialization as a ROS package which is the first known work to supply real-time gesture tracking data via ROS.