Robot Vision System for Real-Time Human Detection and Action Recognition

Abstract

Mobile robots equipped with camera sensors are required to perceive surrounding humans and their actions for safe autonomous navigation. These are so-called human detection and action recognition. In this paper, moving humans are target objects. Compared to computer vision, the real-time performance of robot vision is more important. For this challenge, we propose a robot vision system. In this system, images described by the optical flow are used as an input. For the classification of humans and actions in the input images, we use Convolutional Neural Network, CNN, rather than coding invariant features. Moreover, we present a novel detector, local search window, for clipping partial images around target objects. Through the experiment, finally, we show that the robot vision system is able to detect the moving human and recognize the action in real time.