Single Kinect Research Articles

Projection Surfaces Detection and Image Correction for Mobile Robots in HRI

Projectors have become a widespread tool to share information in Human-Robot Interaction with large groups of people in a comfortable way. Finding a suitable vertical surface becomes a problem when the projector changes positions when a mobile robot is looking for suitable surfaces to project. Two problems must be addressed to achieve a correct undistorted image: (i) finding the biggest suitable surface free from obstacles and (ii) adapting the output image to correct the distortion due to the angle between the robot and a nonorthogonal surface. We propose a RANSAC-based method that detects a vertical plane inside a point cloud. Then, inside this plane, we apply a rectangle-fitting algorithm over the region in which the projector can work. Finally, the algorithm checks the surface looking for imperfections and occlusions and transforms the original image using a homography matrix to display it over the area detected. The proposed solution can detect projection areas in real-time using a single Kinect camera, which makes it suitable for applications where a robot interacts with other people in unknown environments. Our Projection Surfaces Detector and the Image Correction module allow a mobile robot to find the right surface and display images without deformation, improving its ability to interact with people.

Multiple Kinect Sensor Fusion for Human Skeleton Tracking Using Kalman Filtering

Kinect sensors are able to achieve considerable skeleton tracking performance in a convenient and low-cost manner. However, Kinect sensors often generate poor skeleton poses due to self-occlusion, which is a common problem among most vision-based sensing systems. A simple way to solve this problem is to use multiple Kinect sensors in a workspace and combine the measurements from the different sensors. However, this method creates a new issue known as the data fusion problem. In this research, we developed a human skeleton tracking system using the Kalman filter framework, in which multiple Kinect sensors are used to correct inaccurate tracking data from a single Kinect sensor. Our contribution is to propose a method to determine the reliability of each tracked 3D position of a joint and then combine multiple observations based on measurement confidence. We evaluate the proposed approach by comparison with the ground truth obtained using a commercial marker-based motion-capture system.

THE DEVELOPMENT OF A LOW COST MOTION ANALYSIS SYSTEM: CEKAK VISUAL 3D V1.0

To date, there is lack of biomechanical characterisation on defensive technique (martial arts) due to the unavailability of appropriate motion tracking system that can be used to characterise the technique in quasi-training environment. Therefore, this paper presents the development of a novel low cost motion analysis system, Cekak Visual 3D v1.0 which is capable to track dual martial art practitioner's skeleton motion in a single frame view using integration of Matlab GUIs and Microsoft Kinect. The accuracy and precision of the coordinate data recorded by the system was tested to ensure the quality of the system. The systems perform the tracking motion with a single Kinect which is a combination of various sensors (RGB and depth sensor) thus makes it capable in providing three dimensional coordinate data. The analysis reveals that Visual Cekak 3D v1.0 resulted lower percentage error with high internal consistency (Cronbach Alpha, α = 0.904). The proposed marker-less system is capable to track and store dual skeleton data in a single session tracking. This capability makes the systems suitable in providing quasi-training and natural setting environment for any martial arts biomechanics investigation. Therefore, we believe that the system provide new concept development for basic research in martial arts biomechanics.

3D 카메라 기반 직업성 근골격계 부담 작업 모니터링 장치

A 3D camera-based on-site work-related musculoskeletal disorder risk assessment(WMDs) tool has been developed. The device consists of Kinect a 3D camera manufactured by Microsoft, a servo-motor, and a mobile robot. To complement inherent narrow field of view(FOV) of Kinect, Kinect is rotated according to PID servo-control algorithm by a servo-motor attached underneath, to track movement of a subject, producing skeleton-based motion data. With servo-control, full 360 degrees tracking of a test subject is possible by single Kinect. It was found from experimental tests that the proposed device can be successfully employed for on-site WMDs risk assessing tool.

Parametric Human Body Reconstruction Based on Sparse Key Points.

We propose an automatic parametric human body reconstruction algorithm which can efficiently construct a model using a single Kinect sensor. A user needs to stand still in front of the sensor for a couple of seconds to measure the range data. The user's body shape and pose will then be automatically constructed in several seconds. Traditional methods optimize dense correspondences between range data and meshes. In contrast, our proposed scheme relies on sparse key points for the reconstruction. It employs regression to find the corresponding key points between the scanned range data and some annotated training data. We design two kinds of feature descriptors as well as corresponding regression stages to make the regression robust and accurate. Our scheme follows with dense refinement where a pre-factorization method is applied to improve the computational efficiency. Compared with other methods, our scheme achieves similar reconstruction accuracy but significantly reduces runtime.

Rapid 3D Human Body Modeling and Skinning Animation Based on Single Kinect

Dynamic geometries acquisition of human is one of the most popular topics in the fields of computer vision and computer graphics. This paper presents the related techniques on rapidly generation of human model sequences and makes skinning animation using single Kinect. In specified applied conditions, it has good animation effects and interaction functions. As the comparatively low request on resolution by animation, using the Kinect can greatly save human body animation production time and reduce production cost.

Reconstructing 3D human models with a Kinect

AbstractThree‐dimensional human model reconstruction has wide applications due to the rapid development of computer vision. The appearance of cheap depth camera, such as Kinect, opens up new horizons for home‐oriented 3D human reconstructions. However, the resolution of Kinect is relatively low, making it difficult to build accurate human models. In this paper, we improve the accuracy of human model reconstruction from two aspects. First, we improve the depth data quality by registering the depth images captured from multi‐views with a single Kinect. The part‐wise registration method and implicit‐surface‐based de‐noising method are proposed. Second, we utilize a statistical human model to iteratively augment and complete the human body information by fitting the statistical human model to the registered depth image. Experimental results and several applications demonstrate the applicability and quality of our system, which can be potentially used in virtual try‐on systems. Copyright © 2015 John Wiley & Sons, Ltd.

단일 키넥트를 이용한 골프 스윙 특징의 자동 추출

단일 키넥트를 이용한 골프 스윙 특징의 자동 추출

A Low Cost Scheme of Scene Depth Map for Stereoscopic Video Streaming Using Single Kinect Sensing

This study mainly described a low-cost stereoscopic sensing scheme using a single Kinect device of Microsoft. Basically, the integration of infrared (IR) depth map with visible light image of Kinect could support the virtual 3-dimensional view. However, the precision and unstable displays are problems practically that means the unstable variation of 3D displays and distorted projections of Kinect IR projector make the depth maps non-reliable so that the virtual view usually non-acceptable with defects in many cases especially in low lamination and corners of the sensing objects. The study proposed an algorithm to rebuild a good depth map includes: free-holes filling, extrapolation and prediction of filling-loss holes, rectification and trimming of depth-edge deformation, modification of the depths for no-IR-detectable object and the sparse IR-detectable depths, and finally applying the bilateral filtering for the edge smoothing. The simulation result demonstrated that the proposed algorithm can provide an acceptable virtual 3-dimensional images and the calculation loads are suitable for real-time stereoscopic displays.

Immersive 3D Telepresence

Cutting-edge work on 3D telepresence at a multinational research center provides insight into the technology's potential, as well as into its remaining challenges. The first Web extra at http://youtu.be/r4SqJdXkOjQ is a video describing FreeCam, a system capable of generating live free-viewpoint video by simulating the output of a virtual camera moving through a dynamic scene. The second Web extra at http://youtu.be/Dw1glKUKs9A is a video showing a system designed to capture the enhanced 3D structure of a room-sized dynamic scene with commodity depth cameras, such as Microsoft Kinects. The third Web extra at http://youtu.be/G_VPzXRrmIw is a video demonstrating a system that adapts to a wide variety of telepresence scenarios. By combining Kinect-based 3D scanning with optical see-through HMDs, the user can precisely control which parts of the scene are real and which are virtual or remote objects. The fourth Web extra at http://youtu.be/n45N5AHsoCI is a video demonstrating a method based on moving least squares surfaces that robustly and efficiently reconstructs dynamic scenes captured by a set of hybrid color+depth cameras. The reconstruction provides spatiotemporal consistency and seamlessly fuses color and geometric information. The video also illustrates the formulation on a variety of real sequences and demonstrates that it favorably compares to state-of-the-art methods. The fifth Web extra at http://youtu.be/OSl3f2qZzKs is a video demonstrating a 3D acquisition system capable of simultaneously capturing an entire room-sized volume with an array of commodity depth cameras and rendering it from a novel viewpoint in real time. The sixth Web extra at http://youtu.be/zKWByH7evo0 is a video demonstrating a gaze-correction approach based on a single Kinect sensor that preserves both the integrity and expressiveness of the face as well as the fidelity of the scene as a whole, producing nearly artifact-free imagery. The method is suitable for mainstream home video conferencing: it uses inexpensive consumer hardware, achieves real-time performance, and requires just a simple and short setup.

Accurate and real-time depth video acquisition using Kinect–stereo camera fusion

This paper presents a Kinect–stereo camera fusion system that significantly improves the accuracy of depth map acquisition. The typical Kinect depth map suffers from missing depth values and errors, resulting from a single Kinect input. To ameliorate such problems, the proposed system couples a Kinect with a stereo RGB camera to provide an additional disparity map. Kinect depth map and the disparity map are efficiently fused in real time by exploiting a spatiotemporal Markov random field framework on a graphics processing unit. An efficient temporal data cost is proposed to maintain the temporal coherency between frames. We demonstrate the performance of the proposed system on challenging real-world examples. Experimental results confirm that the proposed system is robust and accurate in depth video acquisition.

Method for User Interface of Large Displays Using Arm Pointing and Finger Counting Gesture Recognition

Although many three-dimensional pointing gesture recognition methods have been proposed, the problem of self-occlusion has not been considered. Furthermore, because almost all pointing gesture recognition methods use a wide-angle camera, additional sensors or cameras are required to concurrently perform finger gesture recognition. In this paper, we propose a method for performing both pointing gesture and finger gesture recognition for large display environments, using a single Kinect device and a skeleton tracking model. By considering self-occlusion, a compensation technique can be performed on the user's detected shoulder position when a hand occludes the shoulder. In addition, we propose a technique to facilitate finger counting gesture recognition, based on the depth image of the hand position. In this technique, the depth image is extracted from the end of the pointing vector. By using exception handling for self-occlusions, experimental results indicate that the pointing accuracy of a specific reference position was significantly improved. The average root mean square error was approximately 13 pixels for a 1920 × 1080 pixels screen resolution. Moreover, the finger counting gesture recognition accuracy was 98.3%.

3D self-portraits

We develop an automatic pipeline that allows ordinary users to capture complete and fully textured 3D models of themselves in minutes, using only a single Kinect sensor, in the uncontrolled lighting environment of their own home. Our method requires neither a turntable nor a second operator, and is robust to the small deformations and changes of pose that inevitably arise during scanning. After the users rotate themselves with the same pose for a few scans from different views, our system stitches together the captured scans using multi-view non-rigid registration, and produces watertight final models. To ensure consistent texturing, we recover the underlying albedo from each scanned texture and generate seamless global textures using Poisson blending. Despite the minimal requirements we place on the hardware and users, our method is suitable for full body capture of challenging scenes that cannot be handled well using previous methods, such as those involving loose clothing, complex poses, and props.

Fast 3D modeling in complex environments using a single Kinect sensor

Three-dimensional (3D) modeling technology has been widely used in inverse engineering, urban planning, robot navigation, and many other applications. How to build a dense model of the environment with limited processing resources is still a challenging topic. A fast 3D modeling algorithm that only uses a single Kinect sensor is proposed in this paper. For every color image captured by Kinect, corner feature extraction is carried out first. Then a spiral search strategy is utilized to select the region of interest (ROI) that contains enough feature corners. Next, the iterative closest point (ICP) method is applied to the points in the ROI to align consecutive data frames. Finally, the analysis of which areas can be walked through by human beings is presented. Comparative experiments with the well-known KinectFusion algorithm have been done and the results demonstrate that the accuracy of the proposed algorithm is the same as KinectFusion but the computing speed is nearly twice of KinectFusion. 3D modeling of two scenes of a public garden and traversable areas analysis in these regions further verified the feasibility of our algorithm.

Unobtrusive, Continuous, In-Home Gait Measurement Using the Microsoft Kinect

A system for capturing habitual, in-home gait measurements using an environmentally mounted depth camera, the Microsoft Kinect, is presented. Previous work evaluating the use of the Kinect sensor for in-home gait measurement in a lab setting has shown the potential of this approach. In this paper, a single Kinect sensor and computer were deployed in the apartments of older adults in an independent living facility for the purpose of continuous, in-home gait measurement. In addition, a monthly fall risk assessment protocol was conducted for each resident by a clinician, which included traditional tools such as the timed up a go and habitual gait speed tests. A probabilistic methodology for generating automated gait estimates over time for the residents of the apartments from the Kinect data is described, along with results from the apartments as compared to two of the traditionally measured fall risk assessment tools. Potential applications and future work are discussed.

Gaze correction for home video conferencing

Effective communication using current video conferencing systems is severely hindered by the lack of eye contact caused by the disparity between the locations of the subject and the camera. While this problem has been partially solved for high-end expensive video conferencing systems, it has not been convincingly solved for consumer-level setups. We present a gaze correction approach based on a single Kinect sensor that preserves both the integrity and expressiveness of the face as well as the fidelity of the scene as a whole, producing nearly artifact-free imagery. Our method is suitable for mainstream home video conferencing: it uses inexpensive consumer hardware, achieves real-time performance and requires just a simple and short setup. Our approach is based on the observation that for our application it is sufficient to synthesize only the corrected face. Thus we render a gaze-corrected 3D model of the scene and, with the aid of a face tracker, transfer the gaze-corrected facial portion in a seamless manner onto the original image.