Kinect System Research Articles

Validity of the Microsoft Kinect for providing lateral trunk lean feedback during gait retraining

Gait retraining programs are prescribed to assist in the rehabilitation process of many clinical conditions. Using lateral trunk lean modification as the model, the aim of this study was to assess the concurrent validity of kinematic data recorded using a marker-based 3D motion analysis (3DMA) system and a low-cost alternative, the Microsoft Kinect™ (Kinect), during a gait retraining session. Twenty healthy adults were trained to modify their gait to obtain a lateral trunk lean angle of 10°. Real-time biofeedback of the lateral trunk lean angle was provided on a computer screen in front of the subject using data extracted from the Kinect skeletal tracking algorithm. Marker coordinate data were concurrently recorded using the 3DMA system, and the similarity and equivalency of the trunk lean angle data from each system were compared. The lateral trunk lean angle data obtained from the Kinect system without any form of calibration resulted in errors of a high (>2°) magnitude (mean error=3.2±2.2°). Performing global and individualized calibration significantly (P<0.001) improved this error to 1.7±1.5° and 0.8±0.8° respectively. With the addition of a simple calibration the anatomical position coordinates of the Kinect can be used to create a real-time biofeedback system for gait retraining. Given that this system is low-cost, portable and does not require any sensors to be attached to the body, it could provide numerous advantages when compared to laboratory-based gait retraining systems.

A Robot Control System Based on Gesture Recognition Using Kinect

The Kinect camera is widely used for capturing human body images and human motion recognition in video game playing, and there are already some research works on gesture recognition. However, to achieve the anti-interference performance, the current recognition algorithms are often complex and tardiness, and most of the applications are based on the incomplete gesture library and not all hand gestures can be recognized. This paper explores a new method and algorithm which can describe all five fingertips for each hand in any time for hand gesture recognition with the Kinect system. The hand images are processed to build the hand models which are then compared with the gesture library for gesture recognition. After hand gestures are recognized with high accuracy and less computing, control commands corresponding to hand gestures are sent from the hand gesture recognition system to a hexagon robot controller wirelessly, the hexagon robot can then be controlled wirelessly and change its shape according to the hand gesture command. Thus the robot can interact with humans promptly through the gesture recognition system. DOI: http://dx.doi.org/10.11591/telkomnika.v11i5.2493

Photogrammetric Bundle Adjustment With Self-Calibration of the PrimeSense 3D Camera Technology: Microsoft Kinect

The Kinect system is arguably the most popular 3-D camera technology currently on the market. Its application domain is vast and has been deployed in scenarios where accurate geometric measurements are needed. Regarding the PrimeSense technology, a limited amount of work has been devoted to calibrating the Kinect, especially the depth data. The Kinect is, however, inevitably prone to distortions, as independently confirmed by numerous users. An effective method for improving the quality of the Kinect system is by modeling the sensor's systematic errors using bundle adjustment. In this paper, a method for modeling the intrinsic and extrinsic parameters of the infrared and colour cameras, and more importantly the distortions in the depth image, is presented. Through an integrated marker-and feature-based self-calibration, two Kinects were calibrated. A novel approach for modeling the depth systematic errors as a function of lens distortion and relative orientation parameters is shown to be effective. The results show improvements in geometric accuracy up to 53% compared with uncalibrated point clouds captured using the popular software RGBDemo. Systematic depth discontinuities were also reduced and in the check-plane analysis the noise of the Kinect point cloud was reduced by 17%.

Structure recovery by part assembly

This paper presents a technique that allows quick conversion of acquired low-quality data from consumer-level scanning devices to high-quality 3D models with labeled semantic parts and meanwhile their assembly reasonably close to the underlying geometry. This is achieved by a novel structure recovery approach that is essentially local to global and bottom up, enabling the creation of new structures by assembling existing labeled parts with respect to the acquired data. We demonstrate that using only a small-scale shape repository, our part assembly approach is able to faithfully recover a variety of high-level structures from only a single-view scan of man-made objects acquired by the Kinect system, containing a highly noisy, incomplete 3D point cloud and a corresponding RGB image.

S.P.11 Development of a virtual upper extremity assessment tool for individuals with DMD across the lifespan

Abstract Clinical trials are being conducted for DMD using timed walking as a primary outcome measure due to the ease of administration, ability to quantify distance, and established validity. Unfortunately, participation in clinical trials is not equally accessible to all individuals due to the lack of a reliable, valid and sensitive upper extremity measure for the non-ambulatory population. It is essential that this gap in knowledge be filled, as upper body movement is required for many activities of daily living impacting quality of life across the lifespan and abilities in individuals with DMD. The purpose of this project was to develop a reliable and engaging upper extremity measure to quantify workspace volume, reaching velocity, and rate of arm fatigue using the Kinect system to capture data during a video virtual reality game across the spectrum of abilities. The Microsoft Kinect Windows 7 used for this project is a controller-free gaming device that projects an infrared reference matrix that is reflected off of the person and generates a high-resolution depth map of each pixel allowing for tracking of movement. Software is available to track 20 joints and provide positional data which can be used to calculate measurements such as workspace volume, reaching velocity, and rate of arm fatigue. Virtual gaming was developed as a motivational tool to be used during assessment. Work space volume gaming uses bubble popping game while velocity and fatigue use a disappearing image to encourage rapid movement. A prototype assessment system is being validated against a marker based motion capture system and traditional upper extremity assessments such as the Jebsen Hand Function Test, 9-hole peg test, and the Brooke Upper Extremity Rating Scale. Preliminary data suggests that this may be an engaging, reliable and valid upper extremity assessment measure for individuals with DMD across the lifespan.

PERFORMANCE ANALYSIS OF A LOW-COST TRIANGULATION-BASED 3D CAMERA: MICROSOFT KINECT SYSTEM

Abstract. Recent technological advancements have made active imaging sensors popular for 3D modelling and motion tracking. The 3D coordinates of signalised targets are traditionally estimated by matching conjugate points in overlapping images. Current 3D cameras can acquire point clouds at video frame rates from a single exposure station. In the area of 3D cameras, Microsoft and PrimeSense have collaborated and developed an active 3D camera based on the triangulation principle, known as the Kinect system. This off-the-shelf system costs less than $150 USD and has drawn a lot of attention from the robotics, computer vision, and photogrammetry disciplines. In this paper, the prospect of using the Kinect system for precise engineering applications was evaluated. The geometric quality of the Kinect system as a function of the scene (i.e. variation of depth, ambient light conditions, incidence angle, and object reflectivity) and the sensor (i.e. warm-up time and distance averaging) were analysed quantitatively. This system's potential in human body measurements was tested against a laser scanner and 3D range camera. A new calibration model for simultaneously determining the exterior orientation parameters, interior orientation parameters, boresight angles, leverarm, and object space features parameters was developed and the effectiveness of this calibration approach was explored.

Automated Training and Maintenance through KINECT

In this paper, we have worked on reducing burden on mechanic involving complex automobile maintenance activities that are performed in centralised workshops. We have presented a system prototype that combines Augmented Reality with Kinect. With the use of Kinect, very high quality sensors are available at considerably low costs, thus reducing overall expenditure for system design. The system can be operated either in Speech mode or in Gesture mode. The system can be controlled by various audio commands if user opts for Speech mode. The same controlling can also be done by using a set of Gestures in Gesture mode. Gesture recognition is the task performed by Kinect system. This system, bundled with RGB and Depth camera, processes the skeletal data by keeping track of 20 different body joints. Recognizing Gestures is done by verifying user movements and checking them against predefined condition. Augmented Reality module captures real-time image data streams from high resolution camera. This module then generates 3D model that is superimposed on real time data.

SENSORY MOTOR INTERACTION IN VIRTUAL ENVIRONMENT TO PROMOTE TEACHING-LEARNING PROCESS

"So far the school has been structured on the book, on the laborious acquisition of knowledge formulated in verbal language. Nowadays, thanks to the computer and its ability to simulate reality, it is possible to learn more naturally and without effort using our faculties of perception" (Antinucci, 2011). The statement by Antinucci, as well as the studies on many scientific domains supporting the idea of the involvement of sensory-motor integration in the development of intelligence and learning (Piaget, 1952; Vereecken, 1961), suggest to investigate in the didactics methodologies and instruments able to foster this integration ability. These studies also justify the current full attention in the field of movement teaching for new technologies, therefore the present research aims at testing a “shapes game” module that uses Microsoft Kinect System as an input device for the acquisition of data related to the movement of the body segments of the user. The possibility to identify the movement of specific body segments allows to develop teaching methodologies designed to foster the achievement of visual-motor abilities useful to the process of learning in school, through immersive modes of interaction, typical of “exergames” based on a full body involvement (Coshott, 2009). The present study shows the results of the pre-test software, built on a sample of pupils (aged 8 to 10) attending the fifth year of Italian primary school. Key words: edutainment, exergame, kinect, sensory-motor coordination, videogame.

Scene Modeling and Autonomous Navigation for Robots Based on Kinect System

Monocular RGB camera and distance-limited RGB-D camera with Microsoft Kinect system are respectively utilized to solve the 6DoF navigation problem for indoor robots.At first,based on the traditional partial-DoF pose estimation algorithms,an incremental parameterized model is proposed for feature-points’ parameters,which is able to solve the scale ambiguity problem.This model differs from Euclid and inverse depth parameterized models with the sharply reduced statedimension and the consistent observability of system state.In addition,based on this proposed incremental parameterized model,the 6DoF motion is estimated by the observed image-sequence that comes from the RGB camera or infrared camera.At the end of this paper,both the RGB and depth image sequences that sampled from the Kinect system are utilized in Euclid parameterized model and the incremental parameterized model.And the corresponding results validate the effectiveness of the proposed autonomous navigation approach.

Didactics, movement and technology: new frontiers of the human-machine interaction

The current focus on the use of new technologies and media for teaching-learning purposes has led to an intensifying interest in the properties and peculiarities of educational videogames. Creating a learning environment within a video game might be an opportunity to capitalize and use in a constructive way the time that more and more teens spend playing video In light of this, it is particularly interesting the continuous evolution of computer videogames known as exergames, a term derived from the joining of the words and games. Exergames are video games which encourage the emotional involvement and the sense of presence through interactions based on devices that allow a greater involvement of the body respect to typical controllers such as joysticks, joy pads, keyboards and mousses. These games involve the player in some form of exercise and physical activity through video games and specific feedback and data collection devices. The aim of this study, which is currently under development, is therefore to design an active and engaging learning environment in order to assess whether it is suitable to stimulate and facilitate the learning processes, particularly focusing on the functions of coordination and sensorimotor integration, through the typical interaction of exergames and exploiting the properties and peculiarities of the Microsoft Kinect system as a device to acquire data on the body movements of the player.