Multiple Kinect Sensors Research Articles

Innovative reform strategy of artistic practice of university folk dance education based on information fusion technology

Abstract With the development of history, some folk dances are annihilated and lost in the inheritance. In order to protect and inherit national intangible cultural heritage and promote the construction of advanced socialist culture, this paper uses information fusion technology, combines national aesthetic personality, and refers to multi-factor theory, construction theory, and distillation theory to design and reform the art practice model of folk dance education. The data of dancers’ skeletal joint points are collected by using multiple Kinect sensors, and the Kalman filter algorithm is selected to establish the human skeletal motion model and fuse the data information of each joint point. Taking the hand_right and elbow_right joints as examples, it was found that the mean value of the error between the testers’ joint trajectory and the template movement was reduced from 10mm to 4mm, and the mean value of the error between the joint angle and the template movement was reduced from 90° to 35°. Comparing the dance movement training effects and scores of the experimental group and the control group, it was found that the experimental group kicked the front and side legs an average of 70 times/minute, the middle jump height was improved by 10cm, and the average score was 3.33 points higher. This indicates that the integration of information fusion technology in the art practice of dance education can realize the practical innovation of college folk dance education and improve the comprehensive level of folk dance.

3D Vision Using Multiple Structured Light-Based Kinect Depth Cameras

Real-time 3D scanning of a scene or object using multiple depth cameras is often required in many applications but is still a challenging task for the computer vision community, especially when the object or scene is partially occluded and dynamic. If active depth sensors are used in this case, their resulting depth map quality gets degraded due to interference between active radiations from each depth sensor. Passive 3D sensors like stereo cameras can avoid the issue of interference as they do not emit any active radiation, but they face correspondence problems. Since releasing the commodity depth sensor Microsoft Kinect, researchers are getting more interested in active depth-sensing. However, Kinect sensors have some easily noticeable limitations concerning 3D reconstruction such as: they can provide depth maps for a limited range, their field of view is restricted and holes are observed in the depth map due to occlusion. The above-mentioned limitations can be overcome if multiple Kinect sensors are used simultaneously instead of a single Kinect sensor. Still, the challenge here is to avoid interference between these sensors. We present a comprehensive review of possible solutions to avoid interference between multiple Kinect sensors. Furthermore, we introduce the Kinect technology in detail along with applications where multiple Kinect sensors are used in the literature. We expect that this paper will be helpful to the researchers who want to use multiple Kinect sensors in sharing the workplace in their research.

Automatic Diagnosis of Cerebral Palsy Gait Using Computational Intelligence Techniques: A Low-Cost Multi-Sensor Approach.

Automatic diagnosing of Cerebral Palsy (CP) gait is crucial in quantitative evaluation of a therapeutic intervention. Existing systems for such gait assessment are expensive and require user intervention. This study proposes a low-cost gait assessment system equipped with multiple Kinect sensors. Forty subjects (20 CP patients and 20 normal) were recruited for the experiment. To remove outlier frames from the combined gait signal of multiple sensors a data driven algorithm was proposed. Different supervised classifiers along with extreme learning machine were investigated to diagnose CP gait. In addition, a feature level analysis was also performed. Several spatio-temporal features (i.e. step length, stride length, stride time, etc.) were extracted. The strength of walking ratio, a speed invariant feature, to detect CP gait was thoroughly analyzed. The proposed system outperformed state-of-the-art with ≈98% of accuracy (sensitivity: 100%, and specificity: 96.87%). Results indicate a substantial improvement in abnormality detection performance after outlier removal. Based on ReliefF feature ranking algorithm, walking ratio ranked the best among other classical gait features. Performance of all classifiers increased substantially using walking ratio as a feature. Extreme learning machine demonstrated a competing performance in all cases. The higher classification accuracy of this low-cost system using only a single feature makes it attractive for CP gait detection.

Users behavior tracking for Education Assisting by using Multiple Kinect sensors

Users behavior tracking for Education Assisting by using Multiple Kinect sensors

Fusion of information from multiple Kinect sensors for 3D object reconstruction

In this paper, we estimate the accuracy of 3D object reconstruction using multiple Kinect sensors. First, we discuss the calibration of multiple Kinect sensors, and provide an analysis of the accuracy and resolution of the depth data. Next, the precision of coordinate mapping between sensors data for registration of depth and color images is evaluated. We test a proposed system for 3D object reconstruction with four Kinect V2 sensors and present reconstruction accuracy results. Experiments and computer simulation are carried out using Matlab and Kinect V2.

Validation of Foot Placement Locations from Ankle Data of a Kinect v2 Sensor.

The Kinect v2 sensor may be a cheap and easy to use sensor to quantify gait in clinical settings, especially when applied in set-ups integrating multiple Kinect sensors to increase the measurement volume. Reliable estimates of foot placement locations are required to quantify spatial gait parameters. This study aimed to systematically evaluate the effects of distance from the sensor, side and step length on estimates of foot placement locations based on Kinect’s ankle body points. Subjects (n = 12) performed stepping trials at imposed foot placement locations distanced 2 m or 3 m from the Kinect sensor (distance), for left and right foot placement locations (side), and for five imposed step lengths. Body points’ time series of the lower extremities were recorded with a Kinect v2 sensor, placed frontoparallelly on the left side, and a gold-standard motion-registration system. Foot placement locations, step lengths, and stepping accuracies were compared between systems using repeated-measures ANOVAs, agreement statistics and two one-sided t-tests to test equivalence. For the right side at the 2 m distance from the sensor we found significant between-systems differences in foot placement locations and step lengths, and evidence for nonequivalence. This distance by side effect was likely caused by differences in body orientation relative to the Kinect sensor. It can be reduced by using Kinect’s higher-dimensional depth data to estimate foot placement locations directly from the foot’s point cloud and/or by using smaller inter-sensor distances in the case of a multi-Kinect v2 set-up to estimate foot placement locations at greater distances from the sensor.

Implementation of a Virtual Training Simulator Based on 360° Multi-View Human Action Recognition

Virtual training has received a considerable amount of research attention in recent years due to its potential for use in a variety of applications, such as virtual military training, virtual emergency evacuation, and virtual firefighting. To provide a trainee with an interactive training environment, human action recognition methods have been introduced as a major component of virtual training simulators. Wearable motion capture suit-based human action recognition has been widely used for virtual training, although it may distract the trainee. In this paper, we present a virtual training simulator based on 360° multi-view human action recognition using multiple Kinect sensors that provides an immersive environment for the trainee without the need to wear devices. To this end, the proposed simulator contains coordinate system transformation, front-view Kinect sensor tracking, multi-skeleton fusion, skeleton normalization, orientation compensation, feature extraction, and classifier modules. Virtual military training is presented as a potential application of the proposed simulator. To train and test it, a database consisting of 25 military training actions was constructed. In the test, the proposed simulator provided an excellent, natural training environment in terms of frame-by-frame classification accuracy, action-by-action classification accuracy, and observational latency.

装着型アシストロボットの接触部位形状検討のための複数台のKinectを用いた体表面形状計測

装着型アシストロボットの接触部位形状検討のための複数台のKinectを用いた体表面形状計測

Data fusion of multiple kinect sensors for a rehabilitation system.

Kinect-like depth sensors have been widely used in rehabilitation systems. However, single depth sensor processes limb-blocking, data loss or data error poorly, making it less reliable. This paper focus on using two Kinect sensors and data fusion method to solve these problems. First, two Kinect sensors capture the motion data of the healthy arm of the hemiplegic patient; Second, merge the data using the method of Set-Membership-Filter (SMF); Then, mirror this motion data by the Middle-Plane; In the end, control the wearable robotic arm driving the patient's paralytic arm so that the patient can interactively and initiatively complete a variety of recovery actions prompted by computer with 3D animation games.

Real-time 3D human objects rendering based on multiple camera details

3D model construction techniques using RGB-D information have been gaining a great attention of the researchers around the world in recent decades. The RGB-D sensor, Microsoft Kinect is widely used in many research fields, such as in computer vision, computer graphics, and human computer interaction, due to its capacity of providing color and depth information. This paper presents our research finding on calibrating information from several Kinects in order to construct a 3D model of a human object and to render texture captured from RGB camera. We used multiple Kinect sensors, which are interconnected in a network. High bit rate streams captured at each Kinect are first sent to a centralized PC for the processing. This even can be extended to a remote PC in the Internet. Main contributions of this work include calibration of the multiple Kinects, properly aligning point clouds generated from multiple Kinects, and generation of the 3D shape of the human objects. Experimental results demonstrate that the proposed method provides a better 3D model of the human object being captured.

Multiple Kinect Sensor Fusion for Human Skeleton Tracking Using Kalman Filtering

Kinect sensors are able to achieve considerable skeleton tracking performance in a convenient and low-cost manner. However, Kinect sensors often generate poor skeleton poses due to self-occlusion, which is a common problem among most vision-based sensing systems. A simple way to solve this problem is to use multiple Kinect sensors in a workspace and combine the measurements from the different sensors. However, this method creates a new issue known as the data fusion problem. In this research, we developed a human skeleton tracking system using the Kalman filter framework, in which multiple Kinect sensors are used to correct inaccurate tracking data from a single Kinect sensor. Our contribution is to propose a method to determine the reliability of each tracked 3D position of a joint and then combine multiple observations based on measurement confidence. We evaluate the proposed approach by comparison with the ground truth obtained using a commercial marker-based motion-capture system.

View invariant real-time gesture recognition

The invention of several machine assisted systems has increased the scope of security in the society. Human activity recognition is one such very useful human computer interactive system that has gained the attention of the researchers for decades. By identifying and recognizing different actions performed by people, proper surveillance of different criminal activities or activities of elder people staying in the home alone can be performed. This paper as a whole contributes to this cause by developing a robust technique to identify the activities performed by person in a room. This technique provides a solution to most of the previous camera conscious approaches, by generating a system prototype that is view-invariant. For this purpose multiple Kinect sensors have been used. The proposed system is fast and accurate under different illumination conditions and the activity is recognized instead of the fact whether the person is looking at any camera or not.

Implementation of an Omnidirectional Human Motion Capture System Using Multiple Kinect Sensors

Implementation of an Omnidirectional Human Motion Capture System Using Multiple Kinect Sensors

Automatic Evaluation of Trainee Nurses' Patient Transfer Skills Using Multiple Kinect Sensors

To help student nurses learn to transfer patients from a bed to a wheelchair, this paper proposes a system for automatic skill evaluation in nurses' training for this task. Multiple Kinect sensors were employed, in conjunction with colored markers attached to the trainee's and patient's clothing and to the wheelchair, in order to measure both participants' postures as they interacted closely during the transfer and to assess the correctness of the trainee's movements and use of equipment. The measurement method involved identifying body joints, and features of the wheelchair, via the colors of the attached markers and calculating their 3D positions by combining color and depth data from two sensors. We first developed an automatic segmentation method to convert a continuous recording of the patient transfer process into discrete steps, by extracting from the raw sensor data the defining features of the movements of both participants during each stage of the transfer. Next, a checklist of 20 evaluation items was defined in order to evaluate the trainee nurses' skills in performing the patient transfer. The items were divided into two types, and two corresponding methods were proposed for classifying trainee performance as correct or incorrect. One method was based on whether the participants' relevant body parts were positioned in a predefined spatial range that was considered ‘correct’ in terms of safety and efficacy (e.g., feet placed appropriately for balance). The second method was based on quantitative indexes and thresholds for parameters describing the participants' postures and movements, as determined by a Bayesian minimum-error method. A prototype system was constructed and experiments were performed to assess the proposed approach. The evaluation of nurses' patient transfer skills was performed successfully and automatically. The automatic evaluation results were compared with evaluation by human teachers and achieved an accuracy exceeding 80%.

Analysis of Interference between Multiple Kinect Sensors and a Noise Reduction Method for Mobile Robots

Analysis of Interference between Multiple Kinect Sensors and a Noise Reduction Method for Mobile Robots

TH‐C‐BRC‐11: 3D Tracking of Interfraction and Intrafraction Head and Neck Anatomy during Radiotherapy Using Multiple Kinect Sensors

Purpose: The process of daily measurement and validation of tumors and normal structures with onboard imaging provides information useful for reducing patient setup uncertainty errors. However, the use of daily onboard CT imaging greatly increases the radiation dose to critical structures that lie within the CT volume. We now present a quantitative skin surface 3D imaging that when coupled with quantitative patient‐specific biomechanical models determine the tumor and normal organ deformation caused by routine patient head and neck misalignments. Incorporating such modeling and imaging could substantially decrease the number of cone‐ beam CT scans required for patient setup and ultimately the daily CT scan dose. Methods:The quantitative skin surface 3D imaging that monitors the patient anatomy are developed using multiple Kinect sensors. A set of 4 3D cameras are used for illustration purposes to track the patient anatomy externally. Of the 4 cameras, 3 of them are used to track the patientˈs anatomy contour (e.g. face, hands etc) using depth and intensity based contour tracking. Such an approach provides a set of 3D contours for each anatomical region from each camera. The 4th camera employs a marker less face recognition and tracking for delineating the region of the patientˈs face. The location of the face is then shared among the camera controllers in realtime and the anatomical contour that closely matches the face region is selected. Once selected, all the contours are then integrated to form a single 3D representation of the anatomy and overlapping contours are cleaned using Voronoi data re‐sampling. Results: The unified 3D representation presents a quantified 3D surface image within a precision range of 0.3 cm at an acquisition rate of 30 frame‐per‐second. Conclusions :Daily measurement of 3D skin surface with the proposed imaging system is feasible for reducing patient setup uncertainty errors while minimizing radiation exposure risk.