3D Motion Segmentation Research Articles

Guided Sampling for Multistructure Data via Neighborhood Consensus and Residual Sorting

Robust model fitting is a critical technique for artificial intelligence. The performance of most robust model fitting techniques heavily depends on the use of sampling algorithms. In this paper, we propose an efficient guided sampling algorithm for multi-structure data by using the neighborhood consensus and the residual sorting. Specifically, a Neighborhood Consensus based Strategy (NCS) is first proposed to select the first datum (i.e., seed datum) of a minimal subset, and then a Residual Sorting based Strategy (RSS) samples the rest data of the minimal subset based on the seed datum. This strategy effectively combines the benefits of neighborhood consensus and residual sorting, where neighborhood consensus can judge whether a selected data point is an inlier, and residual sorting encourages this strategy to select data points from the same structure of the first selected data point. Moreover, to achieve better fitting performance, the Markov Chain Monte Carlo process is used to combine NCS with the random selection strategy to select the seed datum, and an appropriate size is set to the initial block of randomly sampled hypotheses for RSS. Experimental results on three vision tasks (e.g., two-view motion segmentation and 3D motion segmentation) demonstrate that the proposed algorithm achieves superior performance to several state-of-the-art sampling algorithms.

Efficient sampling using feature matching and variable minimal structure size

Greedy search-based guided sampling is a promising research field in model fitting to data with multiple structures in the presence of a large number of outliers. However, these greedy search-based guided sampling algorithms are sensitive to the fixed minimal (acceptable) structure size and the initial model hypothesis: when the fixed minimal structure size is too small, data subsets sampled by these algorithms are not representative. In contrast, when it is too large, data subsets might be contaminated by outliers. Furthermore, these algorithms may fail to obtain an accurate model hypothesis, if the initial model hypothesis is far from the true model. In this paper, we address the above-mentioned two issues by proposing two greedy search-based strategies: one aims to adaptively estimate minimal structure sizes and the other aims to generate effective initial model hypotheses. Specifically, on one hand, to avoid using the fixed minimal structure size, a strategy is proposed to adaptively estimate minimal structure sizes by using previously obtained ones. On the other hand, to reduce the impact of outliers, a strategy is proposed to filter out outliers to obtain a reduced data subset by using a feature matching algorithm. Then, this strategy generates promising initial model hypotheses by using a proximity sampling on the reduced data subset. Finally, an efficient sampling algorithm based on the two proposed greedy search-based strategies is applied to three vision tasks, i.e., fundamental matrix estimation, homography plane detection and 3D motion segmentation. Extensive experimental results demonstrate the effectiveness of the proposed sampling algorithm.

3D Rigid Motion Segmentation with Mixed and Unknown Number of Models.

Many real-world video sequences cannot be conveniently categorized as general or degenerate; in such cases, imposing a false dichotomy in using the fundamental matrix or homography model for motion segmentation on video sequences would lead to difficulty. Even when we are confronted with a general scene-motion, the fundamental matrix approach as a model for motion segmentation still suffers from several defects, which we discuss in this paper. The full potential of the fundamental matrix approach could only be realized if we judiciously harness information from the simpler homography model. From these considerations, we propose a multi-model spectral clustering framework that synergistically combines multiple models (homography and fundamental matrix) together. We show that the performance can be substantially improved in this way. For general motion segmentation tasks, the number of independently moving objects is often unknown a priori and needs to be estimated from the observations. This is referred to as model selection and it is essentially still an open research problem. In this work, we propose a set of model selection criteria balancing data fidelity and model complexity. We perform extensive testing on existing motion segmentation datasets with both segmentation and model selection tasks, achieving state-of-the-art performance on all of them; we also put forth a more realistic and challenging dataset adapted from the KITTI benchmark, containing real-world effects such as strong perspectives and strong forward translations not seen in the traditional datasets.

Static object imaging features recognition algorithm in dynamic scene mapping

In dynamic scene monocular visual SLAM, it is important to recognize static object imaging features for mapping. The optical flow of noise and the object moving in the same orientations would have the same orientations of static feature for two frames image, that would disturb static feature point recognition. Because the translational motion optical flow orientations method judges with optical flow of single feature point and translational motion orientations, it is not reliable for static feature point recognition. Therefore, a 3D motion segmentation combined for static object imaging features recognition is proposed, which clusters the same optical flow of feature points in 3D motion subspace, including orientations and amplitude of translation and rotation. The simulation results show that the proposed static object imaging features recognition method improves the recognition correct rate and reliability of static features. So, it is effective for static object imaging features recognition in dynamic scene mapping.

Graph-based representation learning for automatic human motion segmentation

3D human motion segmentation is a primary processing step for recognition and analysis of motion data recorded by motion capture system. We propose a novel graph based method to segment long motion...

Modified balanced iterative reducing and clustering using hierarchies (m-BIRCH) for visual clustering

In modern visual clustering applications where datasets are large and updates with new data may be ongoing, methods of online clustering are extremely important. Online clustering algorithms incrementally cluster the data points, use a fraction of the dataset memory, and update the clustering decisions when new data comes in. In this paper we adapt a classic online clustering algorithm called balanced iterative reducing and clustering using hierarchies (BIRCH) to incrementally cluster large datasets of features commonly used in visual clustering, e.g., 840 K color SIFT descriptors, 1.09 million color patches, 60 K outlier corrupted grayscale patches, and 700 K grayscale SIFT descriptors. We use the algorithm to cluster datasets consisting of non-convex clusters, e.g., Hopkins 155 3D motion segmentation dataset. We call the adapted version modified-BIRCH (m-BIRCH). BIRCH was originally developed by the database management community, but has not been used in computer vision. Modifications made in m-BIRCH enable data-driven parameter selection and effectively handle varying density regions in the feature space. Data-driven parameter selection automatically controls the level of coarseness of the data summarization. Effective handling of varying density regions is necessary to well represent the different density regions in the data summarization. Our implementation of the algorithm provides a useful clustering tool and is made publicly available.

Rank estimation in 3D multibody motion segmentation

A novel technique for rank estimation in 3D multibody motion segmentation is proposed. It is based on the study of the frequency spectra of moving rigid objects and does not use or assume a prior knowledge of the objects contained in the scene (i.e. number of objects and motion). The significance of rank estimation on multibody motion segmentation results is shown by using two motion segmentation algorithms over both synthetic and real data.

Optical Flow 3D Segmentation and Interpretation: A Variational Method with Active Curve Evolution and Level Sets

This study investigates a variational, active curve evolution method for dense three-dimentional (3D) segmentation and interpretation of optical flow in an image sequence of a scene containing moving rigid objects viewed by a possibly moving camera. This method jointly performs 3D motion segmentation, 3D interpretation (recovery of 3D structure and motion), and optical flow estimation. The objective functional contains two data terms for each segmentation region, one based on the motion-only equation which relates the essential parameters of 3D rigid body motion to optical flow, and the other on the Horn and Schunck optical flow constraint. It also contains two regularization terms for each region, one for optical flow, the other for the region boundary. The necessary conditions for a minimum of the functional result in concurrent 3D-motion segmentation, by active curve evolution via level sets, and linear estimation of each region essential parameters and optical flow. Subsequently, the screw of 3D motion and regularized relative depth are recovered analytically for each region from the estimated essential parameters and optical flow. Examples are provided which verify the method and its implementation.

Generalized principal component analysis (GPCA)

This paper presents an algebro-geometric solution to the problem of segmenting an unknown number of subspaces of unknown and varying dimensions from sample data points. We represent the subspaces with a set of homogeneous polynomials whose degree is the number of subspaces and whose derivatives at a data point give normal vectors to the subspace passing through the point. When the number of subspaces is known, we show that these polynomials can be estimated linearly from data; hence, subspace segmentation is reduced to classifying one point per subspace. We select these points optimally from the data set by minimizing certain distance function, thus dealing automatically with moderate noise in the data. A basis for the complement of each subspace is then recovered by applying standard PCA to the collection of derivatives (normal vectors). Extensions of GPCA that deal with data in a high-dimensional space and with an unknown number of subspaces are also presented. Our experiments on low-dimensional data show that GPCA outperforms existing algebraic algorithms based on polynomial factorization and provides a good initialization to iterative techniques such as K-subspaces and Expectation Maximization. We also present applications of GPCA to computer vision problems such as face clustering, temporal video segmentation, and 3D motion segmentation from point correspondences in multiple affine views.

The Ouchi illusion as an artifact of biased flow estimation

A pattern by Ouchi has the surprising property that small motions can cause illusory relative motion between the inset and background regions. The effect can be attained with small retinal motions or a slight jiggling of the paper and is robust over large changes in the patterns, frequencies and boundary shapes. In this paper, we explain that the cause of the illusion lies in the statistical difficulty of integrating local one-dimensional motion signals into two-dimensional image velocity measurements. The estimation of image velocity generally is biased, and for the particular spatial gradient distributions of the Ouchi pattern the bias is highly pronounced, giving rise to a large difference in the velocity estimates in the two regions. The computational model introduced to describe the statistical estimation of image velocity also accounts for the findings of psychophysical studies with variations of the Ouchi pattern and for various findings on the perception of moving plaids. The insight gained from this computational study challenges the current models used to explain biological vision systems and to construct robotic vision systems. Considering the statistical difficulties in image velocity estimation in conjunction with the problem of discontinuity detection in motion fields suggests that theoretically the process of optical flow computations should not be carried out in isolation but in conjunction with the higher level processes of 3D motion estimation, segmentation and shape computation.