People tracking in an environment with multiple depth cameras: A skeleton-based pairwise trajectory matching scheme

Abstract

This paper proposes a pairwise trajectory matching scheme from multiple cameras for people tracking, handling the mistracking situations caused by occlusion events occurred in one of the cameras. In a multiple cameras environment, a geometric calibration process is necessary for the co-plane of the overlapping field of views from different cameras as the initial step. Once the geometry is calibrated, according to the 2D positions of the analyzed foot joints from the depth cameras. Homography transformation is applied to project the detected foot points from different views into a synergistic virtual bird’s eye view for people tracking. At the virtual bird’s eye view, the people tracking results from each of the cameras based on Kalman filter are fused according to the proposed pairwise trajectory matching scheme. The contribution of this paper is trifold: (1) The proposed hand-gesture-triggered calibration process with temporally synchronization capability can effectively build and calibrate the geometry in a region of interest. (2) The proposed interleaving-based skeleton obtaining and moving average based valid skeleton determination can extend the skeleton tracking capability to track more people. (3) The proposed pairwise trajectory matching scheme effectively manages occlusion situations happened in one of the depth cameras. In addition, in the extensive experimental results, the proposed method can track up to six simultaneously freely moving persons in the field of view, with affordable complexity for real-time applications. Furthermore, the infrared-based depth cameras track people satisfactorily from bright to extremely dark environments.