Uncalibrated Video Sequences Research Articles

Robust Structure and Motion Recovery Based on Augmented Factorization

This paper proposes a new strategy to promote the robustness of structure from motion algorithm from uncalibrated video sequences. First, an augmented affine factorization algorithm is formulated to circumvent the difficulty in image registration with noise and outliers contaminated data. Then, an alternative weighted factorization scheme is designed to handle the missing data and measurement uncertainties in the tracking matrix. Finally, a robust strategy for structure and motion recovery is proposed to deal with outliers and large measurement noise. This paper makes the following main contributions: 1) An augmented factorization algorithm is proposed to circumvent the difficult image registration problem of previous affine factorization, and the approach is applicable to both rigid and nonrigid scenarios; 2) by employing the fact that image reprojection residuals are largely proportional to the error magnitude in the tracking data, a simple outliers detection approach is proposed; and 3) a robust factorization strategy is developed based on the distribution of the reprojection residuals. Furthermore, the proposed approach can be easily extended to nonrigid scenarios. Experiments using synthetic and real image data demonstrate the robustness and efficiency of the proposed approach over previous algorithms.

Model-based human gait tracking, 3D reconstruction and recognition in uncalibrated monocular video

Automatic analysis of human motion includes initialisation, tracking, pose recovery and activity recognition. In this paper, a computing framework is developed to automatically analyse human motions through uncalibrated monocular video sequences. A model-based kinematics approach is proposed for human gait tracking. Based on the tracking results, 3D human poses and gait features are recovered and extracted. The recognition performance is evaluated by using different classifiers. The proposed method is advantageous in its capability of recognising human subjects walking non-parallel to the image plane.

Bayesian object extraction from uncalibrated image pairs

This paper proposes a region-based algorithm for the extraction of foreground objects in uncalibrated images or video sequences. At the initialization step, a pair of images are overly segmented based on color and texture. Then, several dominant transforms that represent the relationship between two images are found, and each pixel is transformed by these transformations. By considering transform parameters and using the proposed area-ratio test, each pixel is assigned to a background or an object. We consider the resulting binary image as a binary random field, from which the likelihood of being foreground is computed for each initial segment. Using the likelihood model for each segment and the a priori assumption on the smoothness of labels, Bayesian approach is applied to label the segments. Experiments on various images and videos show promising results that the objects are clearly extracted.

Tri-focal tensor-based multiple video synchronization with subframe optimization

In this paper, we present a novel method for synchronizing multiple (more than two) uncalibrated video sequences recording the same event by free-moving full-perspective cameras. Unlike previous synchronization methods, our method takes advantage of tri-view geometry constraints instead of the commonly used two-view one for their better performance in measuring geometric alignment when video frames are synchronized. In particular, the tri-ocular geometric constraint of point/line features, which is evaluated by tri-focal transfer, is enforced when building the timeline maps for sequences to be synchronized. A hierarchical approach is used to reduce the computational complexity. To achieve subframe synchronization accuracy, the Levenberg-Marquardt method-based optimization is performed. The experimental results on several synthetic and real video datasets demonstrate the effectiveness and robustness of our method over previous methods in synchronizing full-perspective videos.

A model-based image-matching technique for three-dimensional reconstruction of human motion from uncalibrated video sequences

In many situations, e.g. sports injuries, three-dimensional kinematics cannot be obtained with traditional lab methods. However, if methods for reconstructing motion patterns from video sequences were available, our understanding of injury mechanisms could be improved. The aim of this study was to assess the accuracy of a new model-based image-matching technique for human motion reconstruction from one or more uncalibrated video sequences, using traditional motion analysis as a gold standard. A laboratory trial was conducted with one test subject performing jogging and side step cutting, while being filmed with three ordinary video cameras. This provided three single camera matchings, three double camera matchings and one triple camera matching for each of the motions. The test subject wore 33 reflective skin markers and was filmed with a seven-camera, 240 Hz motion analysis system. Root mean square (RMS) hip and knee flexion/extension angle differences were less than 12° for all the matchings. Estimates for ad-/abduction (<15°) and internal/external rotation (<16°) were less precise. RMS velocity differences up to 0.62 m/s were found for the single camera matchings, but for the triple camera matching the RMS differences were less than 0.13 m/s for each direction. In conclusion, a new model-based image-matching technique has been developed, that can be used to estimate temporal joint angle histories, velocities and accelerations from uncalibrated video recordings. The kinematic estimates, in particular for center of mass velocity and acceleration, are clearly better when two or more camera views are available. This method can potentially be used to arrive at more precise descriptions of the mechanisms of sports injuries than what has been possible without elaborate methods for three-dimensional reconstruction from uncalibrated video sequences, e.g. for knee injuries.

Integrating scene parallelism in camera auto-calibration

This paper presents an approach for camera auto-calibration from uncalibrated video sequences taken by a hand-held camera. The novelty of this approach lies in that the line parallelism is transformed to the constraints on the absolute quadric during camera autocalibration. This makes some critical cases solvable and the reconstruction more Euclidean. The approach is implemented and validated using simulated data and real image data. The experimental results show the effectiveness of the approach.

A PRACTICAL APPROACH FOR 3D BUILDING MODELING FROM UNCALIBRATED VIDEO SEQUENCES

This paper presents an approach for reconstructing a realistic 3D model of a building from its uncalibrated video sequences taken by a hand-held camera. The novelty of this approach lies in the integration of some prior scene knowledge in the different stages of the Structure From Motion problem (SFM). First, the coplanarity of buildings is considered in the calculation of the fundamental matrices to deal with the critical configurations. Second, the line parallelism and plane orthogonality are transformed to the constraints on the absolute quadric during camera auto-calibration. This makes some critical cases solvable and the reconstruction more Euclidean. The approach is implemented and validated using simulated data and real image data. The experimental results at the end of the paper show the effectiveness of our approach.

Looking to build a model world: automatic construction of static object models using computer vision

Interest in virtual reality and multimedia has provided a great impetus to the development of automatic techniques for building graphical CAD/CAM models of objects and environments by sensing reality itself. The learning of models in this way is essential, particularly in terms of production times and attaining the required high fidelity needed for many applications. Research has produced techniques for extracting full 3D shape models using a variety of sensors and a spectrum of techniques. These include the use of static video cameras, mobile video cameras (e.g. walk through video), multiple camera platforms and/or specialist active range sensors (typically based on laser striping or sonar). This paper introduces the principles and methodologies underlying several of these methods and presents algorithms and examples from systems representative of three major approaches: models from silhouettes, models from active range sensors and, finally, models from passive uncalibrated video sequences.