Human Motion Segmentation Research Articles

Unsupervised Human Motion Segmentation Based on Characteristic Force Signals of Contact Events

Unsupervised Human Motion Segmentation Based on Characteristic Force Signals of Contact Events

Human Motion Segmentation via Velocity-Sensitive Dual-Side Auto-Encoder.

Human motion segmentation (HMS) aims to segment a long human action video into a bunch of short and meaningful action clips. Existing supervised learning approaches need a large amount of training data which may be costly in real-world scenario, while most unsupervised clustering methods cannot fully explore the temporal correlations among human motions and hard to achieve promising performances. In our paper, we design a novel unsupervised framework, called Velocity-Sensitive Dual-Side Auto-Encoder (VSDA), for HMS task. Specifically, a multi-neighbor auto-encoder (MNA) is proposed to extract informative temporal features, which fully explores the local temporal patterns of human motions. In addition, a long-short distance encoding (LSE) strategy is designed. It constrains the encoded representations of close (short-distance) frames becoming similar while the representations of far-away (long-distance) frames becoming distinctive. Similarly, this strategy is also deployed on the decoded outputs as the long-short distance decoding (LSD) module. The LSE/LSD guides the learning process explicitly and implicitly to achieve the dual-side structure. Moreover, we consider the energy variations during the human motion to propose the velocity-sensitive (VS) guidance mechanism for further model improvement. VSDA leverages the temporal characteristics of human motion and derives promising HMS performance. Comprehensive experiments on six real-world human motion datasets illustrate the effectiveness of our proposed model.

Consistency and Diversity Induced Human Motion Segmentation.

Subspace clustering is a classical technique that has been widely used for human motion segmentation and other related tasks. However, existing segmentation methods often cluster data without guidance from prior knowledge, resulting in unsatisfactory segmentation results. To this end, we propose a novel Consistency and Diversity induced human Motion Segmentation (CDMS) algorithm. Specifically, our model factorizes the source and target data into distinct multi-layer feature spaces, in which transfer subspace learning is conducted on different layers to capture multi-level information. A multi-mutual consistency learning strategy is carried out to reduce the domain gap between the source and target data. In this way, the domain-specific knowledge and domain-invariant properties can be explored simultaneously. Besides, a novel constraint based on the Hilbert Schmidt Independence Criterion (HSIC) is introduced to ensure the diversity of multi-level subspace representations, which enables the complementarity of multi-level representations to be explored to boost the transfer learning performance. Moreover, to preserve the temporal correlations, an enhanced graph regularizer is imposed on the learned representation coefficients and the multi-level representations of the source data. The proposed model can be efficiently solved using the Alternating Direction Method of Multipliers (ADMM) algorithm. Extensive experimental results on public human motion datasets demonstrate the effectiveness of our method against several state-of-the-art approaches.

Human Motion Capture Data Retrieval and Segmentation Technology for Professional Sports Training

Human motion capturing is frequently used in sports research. The focus of this research is to help the user choose an appropriate motion capture system for their experimental setup for sports activities that addresses the challenges of linear size and acceptable fits with non-linear futures. In this paper, the eigenvalue combination was used to represent different motion postures, so as to construct an index space related to the motion sequence; and according to this index space, fast and accurate motion retrieval was done; at the same time, a motion data segmentation method based on MVU nonlinear dimensionality reduction was proposed. MVU can overcome the shortcomings that the linear size is reduced and difficult to deal with nonlinear features, and can better fit human motion data, and achieve higher accuracy.

Human motion segmentation based on structure constraint matrix factorization

Human motion segmentation based on structure constraint matrix factorization

Learning Event Representations for Temporal Segmentation of Image Sequences by Dynamic Graph Embedding.

Recently, self-supervised learning has proved to be effective to learn representations of events suitable for temporal segmentation in image sequences, where events are understood as sets of temporally adjacent images that are semantically perceived as a whole. However, although this approach does not require expensive manual annotations, it is data hungry and suffers from domain adaptation problems. As an alternative, in this work, we propose a novel approach for learning event representations named Dynamic Graph Embedding (DGE). The assumption underlying our model is that a sequence of images can be represented by a graph that encodes both semantic and temporal similarity. The key novelty of DGE is to learn jointly the graph and its graph embedding. At its core, DGE works by iterating over two steps: 1) updating the graph representing the semantic and temporal similarity of the data based on the current data representation, and 2) updating the data representation to take into account the current data graph structure. The main advantage of DGE over state-of-the-art self-supervised approaches is that it does not require any training set, but instead learns iteratively from the data itself a low-dimensional embedding that reflects their temporal and semantic similarity. Experimental results on two benchmark datasets of real image sequences captured at regular time intervals demonstrate that the proposed DGE leads to event representations effective for temporal segmentation. In particular, it achieves robust temporal segmentation on the EDUBSeg and EDUBSeg-Desc benchmark datasets, outperforming the state of the art. Additional experiments on two Human Motion Segmentation benchmark datasets demonstrate the generalization capabilities of the proposed DGE.

Reliability Analysis of Multi-Objective Spatio-Temporal Segmentation of Human Motion in Video Sequences

In view of the problem of uneven distribution of edge contour of multi-target human motion image in video sequence, which leads to the decline of target detection ability, an algorithm of multi-target spatial-temporal segmentation of human motion in video sequence based on edge contour feature detection and block fusion is proposed. Firstly, a multi-target spatial-temporal detection model of human motion in video sequence was constructed, extracting video image frame sequence, using discrete frame fusion method to segment and fuse moving target image, matching moving multi-target in video sequence, secondly segmenting motion features in moving target image, combining with SURF algorithm (speeded up robust features, accelerated robust features) to detect and extract human motion objects in video sequence. The experimental results show that the gray histogram of human motion multi-target space-time segmentation is close to the original image histogram, and the detection and recognition ability of human motion target is improved.

A Framework for Human Motion Segmentation Based on Multiple Information of Motion Data

A Framework for Human Motion Segmentation Based on Multiple Information of Motion Data

Articulated deformable structure approach to human motion segmentation and shape recovery from an image sequence

The aim of this study is to perform motion segmentation and three‐dimensional shape recovery of a dynamic human body from an image sequence. The authors note that human body motion generally consists of large articulations between different body parts and small local deformations within each body part. On the basis of this notion, they develop an integrated framework that combines articulated structure from motion and non‐rigid SFM to estimate human body motion and shape as an articulated deformable structure. Unlike existing approaches that apply a low‐rank subspace method for motion segmentation, they use a metric constraint for identifying rigid subsets, which is more robust and, therefore, allow a more relaxed error threshold to be set for fitting rigid subsets, catering for small deformations within individual rigid subsets. They provide an automated statistical procedure for setting the aforementioned error threshold. The rigid subsets are then linked into articulated kinematic chains by minimum spanning tree search in a graph of joint costs. Finally, the blend‐shape method is applied to model local deformations of each individual subset. Experimental results show that the proposed method provides better performance for human motion segmentation and shape recovery compared with existing methods.

Low-Rank Transfer Human Motion Segmentation.

Human motion segmentation has great potential in real world applications. Conventional segmentation approaches cluster data with no guidance from prior knowledge, which could easily cause unpredictable segmentation output and decrease the performance. To this end, we seek to improve the humanmotion segmentation performance by fully utilizing pre-existing well-labeled source data. Specifically, we design a new transfer subspace clustering method for motion segmentation with a weighted rank constraint. Specifically, our proposed model obtains representations of both source and target sequences by mitigating their distribution divergence, which allows for more effective knowledge transfer to the target. To guide new representation learning, we designed a novel sequential graph to preserve temporal information residing in both the source and target. Furthermore, a weighted low-rank constraint is added to enforce the graph regularizer and uncover clustering structures within data. Experiments are evaluated on four human motion databases, which prove the enhanced performance and increased stability of our model compared to state-of-the-art baselines.

Human motion segmentation using collaborative representations of 3D skeletal sequences

Currently, human motion analysis using three‐dimensional (3D) data creates closer awareness in computer vision with the introduction of cost‐effective Kinect or other depth cameras. This study attempts to segment a continuous 3D skeletal sequence into several disjointed sub‐sequences, each of which is corresponding to a complete action. To address this issue, the authors propose a supervised time‐series segmentation algorithm. A bidirectional propagation search scheme is employed to reach a solution. Specifically, a human skeleton is formulated as a point in multidimensional space, and a motion trajectory is further represented as a sequence. Each training action sequence serves as an atom in a dictionary, which is adopted by anl2‐regularised collaborative representation classifier. Based on the fact that the reconstruction error of the collaborative representation measures the similarity between a test sub‐sequence and training sequences, they utilise its variation over time to capture action transition. Cut point detection and sub‐sequence recognition are simultaneously achieved. Experiments on the authors’ recorded 3D skeletal sequences demonstrate that the proposed algorithm outperforms existing human motion segmentation techniques. Their algorithm is capable of extending to segment various dimensional sequences. This extensibility is validated by synthetic signal segmentation experiments.

Sensor Network Oriented Human Motion Segmentation With Motion Change Measurement

Smart Internet of Things has greatly improved the quality of human life with increasingly intelligent sensor networks. Efficient and accurate human motion time series segmentation is the key issue in human motion analysis and understanding. To realize human motion sequence segmentation, a comprehensive human motion description and an intelligent segmentation algorithm are required. Hence, this paper proposes a sensor network-based human motion sequence segmentation framework. With the facilitation of sensor network and sensor network-based feature fusion method, human motions can be comprehensively described. Based on the comprehensive description of motion data, a new motion change variation-based segmentation method is proposed to realize human motion sequence segmentation. Moreover, to satisfy the time efficiency demand in the applications of large scale sensor networks, a hashing algorithm is introduced to compress the original captured sensor data, which can effectively represent the human motions with short binary codes and facilitate the motion change measurement. Experiments on real-world human motion data sets validate the effectiveness of our proposed sensor network-based human motion sequence segmentation framework compared with other state-of-the-art human motion segmentation methods.

Clustering in pursuit of temporal correlation for human motion segmentation

Temporal correlation is an important property of the video sequence. However, most methods only accomplish the clustering of frames via the measurement of similarity between frame pair, and the temporal correlation among frames is rarely taken into account. In this paper, a method for clustering in pursuit of temporal correlation is proposed to address human motion segmentation problem. Aiming at the video sequence, a one-hot indicator vector is extracted from a frame as a frame-level feature. The description of the relationship between the features is formulated as a minimization problem with respect to a similarity graph. A temporal constraint in the form of a trace is imposed on the similarity graph to capture the temporal correlation. On the premise of the non-negative similarity graph, an optimal solution to the graph augments the relationship between the selected features and their adjacent features, while suppressing its relevance to the features that are far away from it in terms of the time span. Normalized cut is implemented on the graph so as to give clustering results. The experiments on human motion segmentation demonstrate the superior performance of the proposed method in tackling the motion data.

Segmentation and Feature Extraction of Human Gait Motion

<div><p>This paper presents segmentation and feature extraction of human gait motion. The methodology of this paper focuses on segmenting ‘XYZ’ position curves, in reference to time of gait motion based on the velocity or acceleration of the movement. The extracted features include amplitude, time, and equally spaced sample data, maximum and minimum for each segment. The results can be used for reconstruction of a viable dataset that is critical for simulation and validation of human gaits. We propose a method to enables the fitting of the same curve with limited data. Such data sets may prove valuable for studying impairments and improving simulations of rehabilitation tools, and statistical classification for researchers worldwide.</p></div>

Significant Change Spotting for Periodic Human Motion Segmentation of Cleaning Tasks Using Wearable Sensors.

The proportion of the aging population is rapidly increasing around the world, which will cause stress on society and healthcare systems. In recent years, advances in technology have created new opportunities for automatic activities of daily living (ADL) monitoring to improve the quality of life and provide adequate medical service for the elderly. Such automatic ADL monitoring requires reliable ADL information on a fine-grained level, especially for the status of interaction between body gestures and the environment in the real-world. In this work, we propose a significant change spotting mechanism for periodic human motion segmentation during cleaning task performance. A novel approach is proposed based on the search for a significant change of gestures, which can manage critical technical issues in activity recognition, such as continuous data segmentation, individual variance, and category ambiguity. Three typical machine learning classification algorithms are utilized for the identification of the significant change candidate, including a Support Vector Machine (SVM), k-Nearest Neighbors (kNN), and Naive Bayesian (NB) algorithm. Overall, the proposed approach achieves 96.41% in the F1-score by using the SVM classifier. The results show that the proposed approach can fulfill the requirement of fine-grained human motion segmentation for automatic ADL monitoring.

Human motion segmentation and recognition using machine vision for mechanical assembly operation.

The observation, decomposition and record of motion are usually accomplished through artificial means during the process of motion analysis. This method not only has a heavy workload, its efficiency is also very low. To solve this problem, this paper proposes a novel method to segment and recognize continuous human motion automatically based on machine vision for mechanical assembly operation. First, the content-based dynamic key frame extraction technology was utilized to extract key frames from video stream, and then automatic segmentation of action was implemented. Further, the SIFT feature points of the region of interest (ROIs) were extracted, on the basis of which the characteristic vector of the key frame was derived. The feature vector can be used not only to represent the characteristic of motion, but also to describe the connection between motion and environment. Finally, the classifier is constructed based on support vector machine (SVM) to classify feature vectors, and the type of therblig is identified according to the classification results. Our approach enables robust therblig recognition in challenging situations (such as changing of light intensity, dynamic backgrounds) and allows automatic segmentation of motion sequences. Experimental results demonstrate that our approach achieves recognition rates of 96.00 % on sample video which captured on the assembly line.

Movement Primitive Segmentation for Human Motion Modeling: A Framework for Analysis

Movement primitive segmentation enables long sequences of human movement observation data to be segmented into smaller components, termed movement primitives , to facilitate movement identification, modeling, and learning. It has been applied to exercise monitoring, gesture recognition, human–machine interaction, and robot imitation learning. This paper proposes a segmentation framework to categorize and compare different segmentation algorithms considering segment definitions, data sources, application-specific requirements, algorithm mechanics, and validation techniques. The framework is applied to human motion segmentation methods by grouping them into online, semionline, and offline approaches. Among the online approaches, distance-based methods provide the best performance, while stochastic dynamic models work best in the semionline and offline settings. However, most algorithms to date are tested with small datasets, and algorithm generalization across participants and to movement changes remains largely untested.

Graph-based representation learning for automatic human motion segmentation

3D human motion segmentation is a primary processing step for recognition and analysis of motion data recorded by motion capture system. We propose a novel graph based method to segment long motion...

Real-time Unsupervised Segmentation of human whole-body motion and its application to humanoid robot acquisition of motion symbols

An interactive loop between motion recognition and motion generation is a fundamental mechanism for humans and humanoid robots. We have been developing an intelligent framework for motion recognition and generation based on symbolizing motion primitives. The motion primitives are encoded into Hidden Markov Models (HMMs), which we call “motion symbols”. However, to determine the motion primitives to use as training data for the HMMs, this framework requires a manual segmentation of human motions. Essentially, a humanoid robot is expected to participate in daily life and must learn many motion symbols to adapt to various situations. For this use, manual segmentation is cumbersome and impractical for humanoid robots. In this study, we propose a novel approach to segmentation, the Real-time Unsupervised Segmentation (RUS) method, which comprises three phases. In the first phase, short human movements are encoded into feature HMMs. Seamless human motion can be converted to a sequence of these feature HMMs. In the second phase, the causality between the feature HMMs is extracted. The causality data make it possible to predict movement from observation. In the third phase, movements having a large prediction uncertainty are designated as the boundaries of motion primitives. In this way, human whole-body motion can be segmented into a sequence of motion primitives. This paper also describes an application of RUS to AUtonomous Symbolization of motion primitives (AUS). Each derived motion primitive is classified into an HMM for a motion symbol, and parameters of the HMMs are optimized by using the motion primitives as training data in competitive learning. The HMMs are gradually optimized in such a way that the HMMs can abstract similar motion primitives. We tested the RUS and AUS frameworks on captured human whole-body motions and demonstrated the validity of the proposed framework.

Human motion segmentation by data point classification.

Contemporary physiotherapy and rehabilitation practice uses subjective measures for motion evaluation and requires time-consuming supervision. Algorithms that can accurately segment patient movement would provide valuable data for progress tracking and on-line patient feedback. In this paper, we propose a two-class classifier approach to label each data point in the patient movement data as either a segment point or a non-segment point. The proposed technique was applied to 20 healthy subjects performing lower body rehabilitation exercises, and achieves a segmentation accuracy of 82%.