A Robust Visual Human Detection Approach With UKF-Based Motion Tracking for a Mobile Robot

Abstract

Robust tracking of a human in a video sequence is an essential prerequisite to an increasing number of applications, where a robot needs to interact with a human user or operates in a human-inhabited environment. This paper presents a robust approach that enables a mobile robot to detect and track a human using an onboard RGB-D sensor. Such robots could be used for security, surveillance, and assistive robotics applications. The proposed approach has real-time computation power through a unique combination of new ideas and well-established techniques. In the proposed method, background subtraction is combined with depth segmentation detector and template matching method to initialize the human tracking automatically. A novel concept of head and hand creation based on depth of interest is introduced in this paper to track the human silhouette in a dynamic environment, when the robot is moving. To make the algorithm robust, a series of detectors (e.g., height, size, and shape) is utilized to distinguish target human from other objects. Because of the relatively high computation time of the silhouette-matching-based method, a confidence level is defined, which allows using the matching-based method only where it is imperative. An unscented Kalman filter is used to predict the human location in the image frame to maintain the continuity of the robot motion. The efficacy of the approach is demonstrated through a real experiment on a mobile robot navigating in an indoor environment.