Motion Segmentation Algorithm Research Articles

Consistency and Diversity Induced Human Motion Segmentation.

Subspace clustering is a classical technique that has been widely used for human motion segmentation and other related tasks. However, existing segmentation methods often cluster data without guidance from prior knowledge, resulting in unsatisfactory segmentation results. To this end, we propose a novel Consistency and Diversity induced human Motion Segmentation (CDMS) algorithm. Specifically, our model factorizes the source and target data into distinct multi-layer feature spaces, in which transfer subspace learning is conducted on different layers to capture multi-level information. A multi-mutual consistency learning strategy is carried out to reduce the domain gap between the source and target data. In this way, the domain-specific knowledge and domain-invariant properties can be explored simultaneously. Besides, a novel constraint based on the Hilbert Schmidt Independence Criterion (HSIC) is introduced to ensure the diversity of multi-level subspace representations, which enables the complementarity of multi-level representations to be explored to boost the transfer learning performance. Moreover, to preserve the temporal correlations, an enhanced graph regularizer is imposed on the learned representation coefficients and the multi-level representations of the source data. The proposed model can be efficiently solved using the Alternating Direction Method of Multipliers (ADMM) algorithm. Extensive experimental results on public human motion datasets demonstrate the effectiveness of our method against several state-of-the-art approaches.

Multi-Motion Segmentation: Combining Geometric Model-Fitting and Optical Flow for RGB Sensors

This paper presents a novel multi-motion segmentation framework by combining the geometric model fitting and optical flow. More precisely, we use the spatial information extracted by the multi-model fitting and the motion information provided by the optical flow to improve the practicality of the motion segmentation algorithm. The real scenes contain challenges such as multiple motions, camera translation, large field of views, and occlusions. To improve the segmentation accuracy of the real scenes, we first propose a cluster selection method to generate several clusters that can cover all motion models in the data, then a cluster merging strategy is proposed to estimate the basic motion models and the number of motions accurately for the scene. Finally, motion assignment of the remaining points is carried out to complete the motion segmentation. Detailed experiments on two challenging real-world benchmark datasets (62-clip and KT3DMoSeg) show that the proposed framework is superior to the state-of-the-art multi-motion segmentation methods.

Real-Time Analysis of Athletes’ Physical Condition in Training Based on Video Monitoring Technology of Optical Flow Equation

This article is dedicated to the research of video motion segmentation algorithms based on optical flow equations. First, some mainstream segmentation algorithms are studied, and on this basis, a segmentation algorithm for spectral clustering analysis of athletes’ physical condition in training is proposed. After that, through the analysis of the existing methods, compared with some algorithms that only process a single frame in the video, this article analyzes the continuous multiple frames in the video and extracts the continuous multiple frames of the sampling points through the Lucas-Kanade optical flow method. We densely sampled feature points contain as much motion information as possible in the video and then express this motion information through trajectory description and finally achieve segmentation of moving targets through clustering of motion trajectories. At the same time, the basic concepts of image segmentation and video motion target segmentation are described, and the division standards of different video motion segmentation algorithms and their respective advantages and disadvantages are analyzed. The experiment determines the initial template by comparing the gray-scale variance of the image, uses the characteristic optical flow to estimate the search area of the initial template in the next frame, reduces the matching time, judges the template similarity according to the Hausdorff distance, and uses the adaptive weighted template update method for the templates with large deviations. The simulation results show that the algorithm can achieve long-term stable tracking of moving targets in the mine, and it can also achieve continuous tracking of partially occluded moving targets.

Wearable muscle movement information measuring device based on acceleration sensor

In this paper, a wearable device designed using acceleration sensors to obtain acceleration information generated by the user's muscles during exercise follows data collection, data pre-processing, feature extraction and feature selection, classification model training, and evaluation of this muscle information of human movement. Designed a relevant model verification system and wearable device, and carried out verification experiments on the model in real-time, accuracy, interactivity, and parallelism with other sensor data types. Transitional motion detection and segmentation algorithms can effectively segment out the transition actions included in the acceleration sequence, and using this method to classify and recognize nine kinds of human motion information, the average recognition rate reaches 98.56%.

Rapid Motion Segmentation of LiDAR Point Cloud Based on a Combination of Probabilistic and Evidential Approaches for Intelligent Vehicles.

Point clouds from light detecting and ranging (LiDAR) sensors represent increasingly important information for environmental object detection and classification of automated and intelligent vehicles. Objects in the driving environment can be classified as either or depending on their movement characteristics. A LiDAR point cloud is also segmented into and points based on the motion properties of the measured objects. The segmented motion information of a point cloud can be useful for various functions in automated and intelligent vehicles. This paper presents a fast motion segmentation algorithm that segments a LiDAR point cloud into and points in real-time. The segmentation algorithm classifies the motion of the latest point cloud based on the LiDAR’s laser beam characteristics and the geometrical relationship between consecutive LiDAR point clouds. To accurately and reliably estimate the motion state of each LiDAR point considering the measurement uncertainty, both probability theory and evidence theory are employed in the segmentation algorithm. The probabilistic and evidential algorithm segments the point cloud into three classes: , , and . Points are placed in the class when LiDAR point cloud is not sufficient for motion segmentation. The point motion segmentation algorithm was evaluated quantitatively and qualitatively through experimental comparisons with previous motion segmentation methods.

Randomized Voting-Based Rigid-Body Motion Segmentation

In this paper, we propose a novel rigid-body motion segmentation algorithm that uses randomized voting to assign high scores to correctly estimated models and low scores to wrongly estimated models. This algorithm is based on an epipolar geometrical representation of the camera motion, and computes scores using the distance between the feature point and the corresponding epipolar line. These scores are accumulated and utilized for motion segmentation. To evaluate the efficacy of our algorithm, we conduct a series of experiments using the Hopkins 155 data set and the UdG data set, which are representative test sets for rigid motion segmentation. Among several state-of-the-art data sets, our algorithm achieves the most accurate motion segmentation results and, in the presence of measurement noise, achieves comparable results to the other algorithms. Finally, we analyze why our motion segmentation algorithm works using probabilistic and theoretical analysis.

IGuard: A Real-Time Anti-Theft System for Smartphones

Smartphone theft is a non-negligible problem that causes serious concerns on personal property and privacy. The existing solutions to this problem either provide only functions like retrieving a phone, or require dedicated hardware to detect thefts. How to protect smartphones from being stolen at all times is still an open problem. In this paper, we propose iGuard, a real-time anti-theft system for smartphones. iGuard utilizes only the inertial sensing data from the smartphone. The basic idea behind iGuard is to distinguish different people holding a smartphone, by identifying the order of the motions during the ‘take-out’ behavior and how each motion is performed. For this purpose, we design a motion segmentation algorithm to detect the transition between two motions from the noisy sensing data. We then leverage the distinct feature contained in each sub-segment of a motion to estimate the probability that the motion is performed by the smartphone owner himself/herself. Based on such pre-processed data, we propose a Markov based model to track the behavior of a smartphone user. According to this model, iGuard instantly alarms once the tracked data deviate from the smartphone owner's usual habit. We implement iGuard on Android and evaluate its performance in real environments. The experimental results show that iGuard is accurate and robust in various scenarios.

Learning from Demonstration: Generalization via Task Segmentation

In this paper, a motion segmentation algorithm design is presented with the goal of segmenting a learned trajectory from demonstration such that each segment is locally maximally different from its neighbors. This segmentation is then exploited to appropriately scale (dilate/squeeze and/or rotate) a nominal trajectory learned from a few demonstrations on a fixed experimental setup such that it is applicable to different experimental settings without expanding the dataset and/or retraining the robot. The algorithm is computationally efficient in the sense that it allows facile transition between different environments. Experimental results using the Baxter robotic platform showcase the ability of the algorithm to accurately transfer a feeding task.

Prior-Free Dependent Motion Segmentation Using Helmholtz-Hodge Decomposition Based Object-Motion Oriented Map

Motion segmentation in moving camera videos is a very challenging task because of the motion dependence between the camera and moving objects. Camera motion compensation is recognized as an effective approach. However, existing work depends on prior-knowledge on the camera motion and scene structure for model selection. This is not always available in practice. Moreover, the image plane motion suffers from depth variations, which leads to depth-dependent motion segmentation in 3D scenes. To solve these problems, this paper develops a prior-free dependent motion segmentation algorithm by introducing a modified Helmholtz-Hodge decomposition (HHD) based object-motion oriented map (OOM). By decomposing the image motion (optical flow) into a curl-free and a divergence-free component, all kinds of camera-induced image motions can be represented by these two components in an invariant way. HHD identifies the camera-induced image motion as one segment irrespective of depth variations with the help of OOM. To segment object motions from the scene, we deploy a novel spatio-temporal constrained quadtree labeling. Extensive experimental results on benchmarks demonstrate that our method improves the performance of the state-of-the-art by 10%~20% even over challenging scenes with complex background.

A collection of challenging motion segmentation benchmark datasets

An in-depth analysis of computer vision methodologies is greatly dependent on the benchmarks they are tested upon. Any dataset is as good as the diversity of the true nature of the problem enclosed in it. Motion segmentation is a preprocessing step in computer vision whose publicly available datasets have certain limitations. Some databases are not up-to-date with modern requirements of frame length and number of motions, and others do not have ample ground truth in them. In this paper, we present a collection of diverse multifaceted motion segmentation benchmarks containing trajectory- and region-based ground truth. These datasets enclose real-life long and short sequences, with increased number of motions and frames per sequence, and also real distortions with missing data. The ground truth is provided on all the frames of all the sequences. A comprehensive benchmark evaluation of the state-of-the-art motion segmentation algorithms is provided to establish the difficulty of the problem and to also contribute a starting point. All the resources of the datasets have been made publicly available at http://dixie.udg.edu/udgms/.

Weakly supervised motion segmentation with particle matching

Motion segmentation refers to the task of segmenting moving objects subject to their motion in order to distinguish and track them in a video. This is a challenging task in situations where different objects share similar movement patterns, or in cases where one object is occluded by others in part of the scene. In such cases, unsupervised motion segmentation fails and additional information is needed to boost the performance. Based on a formulation of the clustering task as an optimization problem using a multi-labeled Markov Random Field, we develop a semi-supervised motion segmentation algorithm by setting up a framework for incorporating prior knowledge into the segmentation algorithm. Prior knowledge is given in the form of manually labelling trajectories that belong to the various objects in one or more frames of the video. Clearly, one wishes to limit the amount of manual labelling in order for the algorithm to be as autonomous as possible. Towards that end, we propose a particle matching procedure that extends the prior knowledge by automatically matching particles in frames over which fast motion or occlusion occur. The performance of the proposed method is studied through a variety of experiments on videos involving fast and complicated motion, occlusion and re-appearance, and low quality film. The qualitative and quantitative results confirm reliable performance on the types of applications our method is designed for.

Online Motion Segmentation Based on Sparse Subspace Clustering

We consider the problem of online motion segmentation for video streams. Most existing motion segmentation algorithms based on subspace clustering operate in a batch fashion. The main di‐culty of applying these algorithms to real-world applications is that their e‐ciencies can hardly meet the speed requirement when dealing with video streams. In this paper, we propose an online motion segmentation method based on Sparse Subspace Clustering (SSC). Two strategies are adopted in our approach, namely the incremental Principal Component Analysis (PCA) and a warm start from previously obtained Sparse Representation (SR), to accelerate the dimension reduction and SSC in each trail. Through extensive experiments on both synthetic and real-world data sets, we conclude that our algorithm can achieve a signiflcant acceleration under a comparable misclassiflcation rate with respect to other state-of-the-art algorithms.

Using In-frame Shear Constraints for Monocular Motion Segmentation of Rigid Bodies

It is a well known result in the vision literature that the motion of independently moving objects viewed by an affine camera lie on affine subspaces of dimension four or less. As a result a large number of the recently proposed motion segmentation algorithms model the problem as one of clustering the trajectory data to its corresponding affine subspace. While these algorithms are elegant in formulation and achieve near perfect results on benchmark datasets, they fail to address certain very key real-world challenges, including perspective effects and motion degeneracies. Within a robotics and autonomous vehicle setting, the relative configuration of the robot and moving object will frequently be degenerate leading to a failure of subspace clustering algorithms. On the other hand, while gestalt-inspired motion similarity algorithms have been used for motion segmentation, in the moving camera case, they tend to over-segment or under-segment the scene based on their parameter values. In this paper we present a principled approach that incorporates the strengths of both approaches into a cohesive motion segmentation algorithm capable of dealing with the degenerate cases, where camera motion follows that of the moving object. We first generate a set of prospective motion models for the various moving and stationary objects in the video sequence by a RANSAC-like procedure. Then, we incorporate affine and long-term gestalt-inspired motion similarity constraints, into a multi-label Markov Random Field (MRF). Its inference leads to an over-segmentation, where each label belongs to a particular moving object or the background. This is followed by a model selection step where we merge clusters based on a novel motion coherence constraint, we call in-frame shear, that tracks the in-frame change in orientation and distance between the clusters, leading to the final segmentation. This oversegmentation is deliberate and necessary, allowing us to assess the relative motion between the motion models which we believe to be essential in dealing with degenerate motion scenarios.We present results on the Hopkins-155 benchmark motion segmentation dataset [27], as well as several on-road scenes where camera and object motion are near identical. We show that our algorithm is competitive with the state-of-the-art algorithms on [27] and exceeds them substantially on the more realistic on-road sequences.

An unsupervised feature selection dynamic mixture model for motion segmentation.

The automatic clustering of time-varying characteristics and phenomena in natural scenes has recently received great attention. While there exist many algorithms for motion segmentation, an important issue arising from these studies concerns that for which attributes of the data should be used to cluster phenomena with a certain repetitiveness in both space and time. It is difficult because there is no knowledge about the labels of the phenomena to guide the search. In this paper, we present a feature selection dynamic mixture model for motion segmentation. The advantage of our method is that it is intuitively appealing, avoiding any combinatorial search, and allowing us to prune the feature set. Numerical experiments on various phenomena are conducted. The performance of the proposed model is compared with that of other motion segmentation algorithms, demonstrating the robustness and accuracy of our method.

Learning Multi-Boosted HMMs for Lip-Password Based Speaker Verification

This paper proposes a concept of lip motion password (simply called lip-password hereinafter), which is composed of a password embedded in the lip movement and the underlying characteristic of lip motion. It provides a double security to a visual speaker verification system, where the speaker is verified by both of the private password information and the underlying behavioral biometrics of lip motions simultaneously. Accordingly, the target speaker saying the wrong password or an impostor who knows the correct password will be detected and rejected. To this end, we shall present a multi-boosted Hidden Markov model (HMM) learning approach to such a system. Initially, we extract a group of representative visual features to characterize each lip frame. Then, an effective lip motion segmentation algorithm is addressed to segment the lip-password sequence into a small set of distinguishable subunits. Subsequently, we integrate HMMs with boosting learning framework associated with a random subspace method and data sharing scheme to formulate a precise decision boundary for these subunits verification, featuring on high discrimination power. Finally, the lip-password, whether spoken by the target speaker with the pre-registered password or not, is identified based on all the subunit verification results learned from multi-boosted HMMs. The experimental results show that the proposed approach performs favorably compared with the state-of-the-art methods.

3D human motion analysis framework for shape similarity and retrieval

3D shape similarity from video is a challenging problem lying at the heart of many primary research areas in computer graphics and computer vision applications. In this paper, we address within a new framework the problem of 3D shape representation and shape similarity in human video sequences. Our shape representation is formulated using extremal human curve (EHC) descriptor extracted from the body surface. It allows taking benefits from Riemannian geometry in the open curve shape space and therefore computing statistics on it. It also allows subject pose comparison regardless of geometrical transformations and elastic surface change. Shape similarity is performed by an efficient method which takes advantage of a compact EHC representation in open curve shape space and an elastic distance measure. Thanks to these main assets, several important exploitations of the human action analysis are performed: shape similarity computation, video sequence comparison, video segmentation, video clustering, summarization and motion retrieval.Experiments on both synthetic and real 3D human video sequences show that our approach provides an accurate static and temporal shape similarity for pose retrieval in video, compared with the state-of-the-art approaches. Moreover, local 3D video retrieval is performed using motion segmentation and dynamic time warping (DTW) algorithm in the feature vector space. The obtained results are promising and show the potential of this approach.

Multi-object reconstruction from dynamic scenes: An object-centered approach

In this paper, we present a new framework for three-dimensional (3D) reconstruction of multiple rigid objects from dynamic scenes. Conventional 3D reconstruction from multiple views is applicable to static scenes, in which the configuration of objects is fixed while the images are taken. In our framework, we aim to reconstruct the 3D models of multiple objects in a more general setting where the configuration of the objects varies among views. We solve this problem by object-centered decomposition of the dynamic scenes using unsupervised co-recognition approach. Unlike conventional motion segmentation algorithms that require small motion assumption between consecutive views, co-recognition method provides reliable accurate correspondences of a same object among unordered and wide-baseline views. In order to segment each object region, we benefit from the 3D sparse points obtained from the structure-from-motion. These points are reliable and serve as automatic seed points for a seeded-segmentation algorithm. Experiments on various real challenging image sequences demonstrate the effectiveness of our approach, especially in the presence of abrupt independent motions of objects.

Motion-based segmentation of objects using overlapping temporal windows

Motion segmentation refers to the problem of separating the objects in a video sequence according to their motion. It is a fundamental problem of computer vision, since various systems focusing on the analysis of dynamic scenes include motion segmentation algorithms. In this paper we present a novel approach, where a video shot is temporally divided in successive and overlapping windows and motion segmentation is performed on each window respectively. This attribute renders the algorithm suitable even for long video sequences. In the last stage of the algorithm the segmentation results for every window are aggregated into a final segmentation. The presented algorithm can handle effectively asynchronous trajectories on each window even when they have no temporal intersection. The evaluation of the proposed algorithm on the Berkeley motion segmentation benchmark demonstrates its scalability and accuracy compared to the state of the art.

Development of head detection and tracking systems for visual surveillance

This paper proposes a technique for the detection of head nod and shake gestures based on eye tracking and head motion decision. The eye tracking step is divided into face detection and eye location. Here, we apply a motion segmentation algorithm that examines differences in moving people's faces. This system utilizes a Hidden Markov Model-based head detection module that carries out complete detection in the input images, followed by the eye tracking module that refines the search based on a candidate list provided by the preprocessing module. The novelty of this paper is derived from differences in real-time input images, preprocessing to remove noises (morphological operators and so on), detecting edge lines and restoration, finding the face area, and cutting the head candidate. Moreover, we adopt a K-means algorithm for finding the head region. Real-time eye tracking extracts the location of eyes from the detected face region and is performed at close to a pair of eyes. After eye tracking, the coordinates of the detected eyes are transformed into a normalized vector of x-coordinate and y-coordinate. Head nod and shake detector uses three hidden Markov models (HMMs). HMM representation of the head detection can estimate the underlying HMM states from a sequence of face images. Head nod and shake can be detected by three HMMs that are adapted by a directional vector. The directional vector represents the direction of the head movement. The vector is HMMs for determining neutral as well as head nod and shake. These techniques are implemented on images, and notable success is notified.

Identification and tracking of robots in an intelligent space using static cameras and an XPFCP

This paper tackles the problem of identification and tracking of multiple robots in an intelligent space using an array of cameras placed in fixed positions within the environment. Several types of agent can be found in an intelligent space: controlled agents (mobile robots) and uncontrolled ones (users and obstacles). The information transferred between the controlled agents and the intelligent space is limited to the control commands sent to the robots and the measurements of the odometers received from the robots. The proposed solution allows the localization of mobile agents, even if they are not robots; however, we have focused on the controlled agents. The proposal does not require prior knowledge or invasive landmarks on board the robots. It starts from the segmentation of different agents in motion that allows obtaining the boundaries of all robots and an estimation of all 3D points that define those boundaries. Then, the identification of the robots is obtained by comparing the components of the linear velocity estimated by the motion segmentation algorithm and received from the odometers. In order to track the robots, an eXtended Particle Filter with Classification Process (XPFCP) is employed. Several experimental tests have been carried out, and the results obtained validate the proposal.