Depth Image Sequences Research Articles

ONLINE TRANSFER OF HUMAN MOTION TO HUMANOIDS

Transferring motion from a human demonstrator to a humanoid robot is an important step toward developing robots that are easily programmable and that can replicate or learn from observed human motion. The so called motion retargeting problem has been well studied and several off-line solutions exist based on optimization approaches that rely on pre-recorded human motion data collected from a marker-based motion capture system. From the perspective of human robot interaction, there is a growing interest in online motion transfer, particularly without using markers. Such requirements have placed stringent demands on retargeting algorithms and limited the potential use of off-line and pre-recorded methods. To address these limitations, we present an online task space control theoretic retargeting formulation to generate robot joint motions that adhere to the robot's joint limit constraints, joint velocity constraints and self-collision constraints. The inputs to the proposed method include low dimensional normalized human motion descriptors, detected and tracked using a vision based key-point detection and tracking algorithm. The proposed vision algorithm does not rely on markers placed on anatomical landmarks, nor does it require special instrumentation or calibration. The current implementation requires a depth image sequence, which is collected from a single time of flight imaging device. The feasibility of the proposed approach is shown by means of online experimental results on the Honda humanoid robot — ASIMO.

Joint source and channel coding for 3D video with depth image - based rendering

Following recent commercial availability of autostereoscopic 3D displays that allow 3D visual data to be viewed without the use of special headgear or glasses, it is anticipated that the applications of 3D video will increase rapidly in near future. In this paper we propose a joint source channel coding scheme for depth image-based rendering based 3D video coding. We considered different source and channel coding rates to find the optimum coding performance under a given channel bit rate for a WiMAX based communication channel. When the optimum bit allocation combination for color and depth image sequences are found, different protection levels have been considered for coding both image sequences. Finally, an optimum protection levels are proposed for the best video quality.

3D Video Player System with Haptic Interaction based on Depth Image-Based Representation

In this paper, we propose a three-dimensional (3D) video player system that provides haptic interaction with objects in a video scene based on a depth image-based representation. In order to represent dynamic 3D scenes, 3D video media combining general color video and synchronized depth video (depth image sequences) containing per-pixel depth information were exploited. With the proposed system, viewers can touch and explore the shape of objects visible in a scene via a three degree-of-freedom (DOF) haptic interface. To support haptic interaction in the 3D video player, a modified proxy graph algorithm was proposed which calculates the contact force between the 3D video scene and the viewers' finger. In addition, the proposed system architecture was implemented which encompassed 3D video contents generation, depth data enhancement, compression, and haptic rendering. The proposed 3D video player system showed stable, smooth, and immersive interaction in real-time.

Moving object detection by mobile Stereo Omni-directional System (SOS) using spherical depth image

Moving object detection with a mobile image sensor is an important task for mobile surveillance systems running in real environments. In this paper, we propose a novel method to effectively solve this problem by using a Stereo Omni-directional System (SOS), which can obtain both color and depth images of the environment in real time with a complete spherical field of view. Taking advantage of the SOS that the frame-out problem never occurs, we develop a method to detect the regions of moving objects stably under arbitrary movement and pose change of the SOS, by using the spherical depth image sequence obtained by the SOS. The method first predicts the depth image for the current time from that obtained at the previous time and the ego-motion of the SOS, and then detects moving objects by comparing the predicted depth image with the actual one obtained at the current time.

Object-based coding of stereo image sequences using joint 3-D motion/disparity compensation

An object-based coding scheme is proposed for the coding of a stereoscopic image sequence using motion and disparity information. A hierarchical block-based motion estimation approach is used for initialization, while disparity estimation is performed using a pixel-based hierarchical dynamic programming algorithm. A split-and-merge segmentation procedure based on three-dimensional (3-D) motion modeling is then used to determine regions with similar motion parameters. The segmentation part of the algorithm is interleaved with the estimation part in order to optimize the coding performance of the procedure. Furthermore, a technique is examined for propagating the segmentation information with time. A 3-D motion-compensated prediction technique is used for both intensity and depth image sequence coding. Error images and depth maps are encoded using discrete cosine transform (DCT) and Huffman methods. Alternately, an efficient wireframe depth modeling technique may be used to convey depth information to the receiver. Motion and wireframe model parameters are then quantized and transmitted to the decoder along with the segmentation information. As a straightforward application, the use of the depth map information for the generation of intermediate views at the receiver is also discussed. The performance of the proposed compression methods is evaluated experimentally and is compared to other stereoscopic image sequence coding schemes.