Data‐driven recovery of hand depth using CRRF on stereo images

Abstract

Hand pose is emerging as an important interface for human–computer interaction. This study presents a data‐driven method to estimate a high‐quality depth map of a hand from a stereoscopic camera input by introducing a novel superpixel‐based regression framework that takes advantage of the smoothness of the depth surface of the hand. To this end, the authors introduce conditional regressive random forest (CRRF), a method that combines a conditional random field (CRF) and an RRF to model the mapping from a stereo red, green and blue image pair to a depth image. The RRF provides a unary term that adaptively selects different stereo‐matching measures as it implicitly determines matching pixels in a coarse‐to‐fine manner. While the RRF makes depth prediction for each superpixel independently, the CRF unifies the prediction of depth by modelling pairwise interactions between adjacent superpixels. Experimental results show that CRRF can generate a depth image more accurately than the leading contemporary techniques using an inexpensive stereo camera.