Multiple Uncalibrated Cameras Research Articles

Recovering absolute scale for Structure from Motion using the law of free fall

Reconstructions of objects (or scenes) using Structure from Motion (SfM) based on monocular camera or multiple uncalibrated cameras are only possible up to an unknown scale. Most of existing approaches for recovering absolute scale for SfM rely on additional constraints in absolute scale quantified by either metric data of the object (or scene) or motion data of the camera(s). This paper presented a novel approach for recovering absolute scale for SfM based on motion constraints defined by a relative free fall motion between the camera and the object. The proposed approach is strict because the adopted relative free fall motion is directly reconstructed by SfM and is therefore not complicated by the intrinsic problem that the motion of the camera recorded by external sensors cannot truly represent the motion of the center of perspective of the camera. Experiments of the proposed approach based on a widely used SfM-MVS pipeline had derived scale factors with relative errors smaller than 5%, yet returned camera motions that did not approximate with a reasonable degree of accuracy to what the law of free fall predicted. It is suggested that this deviation is owing to that SfM processes are not constrained by physical laws, and might be moderated by fusing kinematic constraints of free fall motion into bundle adjustment. In addition, uncertainties related to camera properties and experiment designs should also be reduced so that the proposed closed-form analytical solution for recovering absolute scale can work properly in practice. To our knowledge, this paper for the first time shows the potential of recovering absolute scale for geometric SfM reconstruction using a universal physical law.

Correction to: Pose estimation of soccer players using multiple uncalibrated cameras

In the pdf version of this article, the captured corresponding author was incorrect when it is correctly presented in the online version. The correct corresponding author should be Hadi Seyedarabi. It is corrected in this correction article.

Pose estimation of soccer players using multiple uncalibrated cameras

Fully automatic algorithm for estimating the 3D human pose from multiple uncalibrated cameras is presented. Unlike the state-of-the-art methods which use the estimated pose of previous frames to re...

Collaborative detection of repetitive behavior by multiple uncalibrated cameras

In smart environments, the embedded sensing systems should intelligently adapt to the behavior of the users. Many interesting types of behavior are characterized by repetition of actions such as certain activities or movements. A generic methodology to detect and classify repetitions that may occur at different scales is introduced in this paper. The proposed method is called Action History Matrices (AHM). The properties of AHM for detecting repetitive movement behavior are demonstrated in analyzing four customer behavior classes in a shop environment observed by multiple uncalibrated cameras.Two different datasets, video recordings in the shop environment and motion path simulations, are created and used in the experiments. The AHM-based system achieves an accuracy of 97% with most suitable scale and naive Bayesian classifier on the single-view simulated movement data. In addition, the performance of two fusion levels and three fusion methods are compared with AHM method on the multi-view recordings. In our results, fusion at the decision-level offers consistently better accuracy than feature-level, and the coverage-based view-selection fusion method (51%) marginally outperforms the majority method. The upper limit with the recorded data for accuracy by view-selection is found to be 75%.

A flexible architecture for multi-view 3DTV based on uncalibrated cameras

This paper presents a novel flexible architecture for 3DTV based on multiple uncalibrated cameras. The proposed signal representation improves the interactivity of dense point-based methods, making them appropriate for modeling the scene semantics and free-viewpoint 3DTV applications. The main concern is to address the shortcomings of depth image-based 3D video systems for free-viewpoint visualization, and to provide an efficient implementation of the rendering part which is computationally intensive as well potentially determine the view quality. Novel rendering algorithms are added that specifically aim at solving the rendering artifacts, and sampling issues encountered in wide baseline extensions and arbitrary camera movements. To optimize the process, a “selective” warping technique is proposed that takes the advantage of temporal coherence to reduce the computational overhead. Performance is illustrated on challenging videos to prove the suitability and flexibility of the architecture for advanced 3DTV systems.

Multi-View ML Object Tracking With Online Learning on Riemannian Manifolds by Combining Geometric Constraints

This paper addresses issues in object tracking with occlusion scenarios, where multiple uncalibrated cameras with overlapping fields of view are exploited. We propose a novel method where tracking is first done independently in each individual view and then tracking results are mapped from different views to improve the tracking jointly. The proposed tracker uses the assumptions that objects are visible in at least one view and move uprightly on a common planar ground that may induce a homography relation between views. A method for online learning of object appearances on Riemannian manifolds is also introduced. The main novelties of the paper include: 1) define a similarity measure, based on geodesics between a candidate object and a set of mapped references from multiple views on a Riemannian manifold; 2) propose multi-view maximum likelihood estimation of object bounding box parameters, based on Gaussian-distributed geodesics on the manifold; 3) introduce online learning of object appearances on the manifold, taking into account of possible occlusions; 4) utilize projective transformations for objects between views, where parameters are estimated from warped vertical axis by combining planar homography, epipolar geometry, and vertical vanishing point; 5) embed single-view trackers in a three-layer multi-view tracking scheme. Experiments have been conducted on videos from multiple uncalibrated cameras, where objects contain long-term partial/full occlusions, or frequent intersections. Comparisons have been made with three existing methods, where the performance is evaluated both qualitatively and quantitatively. Results have shown the effectiveness of the proposed method in terms of robustness against tracking drift caused by occlusions.

Audiovisual Probabilistic Tracking of Multiple Speakers in Meetings

Tracking speakers in multiparty conversations constitutes a fundamental task for automatic meeting analysis. In this paper, we present a novel probabilistic approach to jointly track the location and speaking activity of multiple speakers in a multisensor meeting room, equipped with a small microphone array and multiple uncalibrated cameras. Our framework is based on a mixed-state dynamic graphical model defined on a multiperson state-space, which includes the explicit definition of a proximity-based interaction model. The model integrates audiovisual (AV) data through a novel observation model. Audio observations are derived from a source localization algorithm. Visual observations are based on models of the shape and spatial structure of human heads. Approximate inference in our model, needed given its complexity, is performed with a Markov Chain Monte Carlo particle filter (MCMC-PF), which results in high sampling efficiency. We present results-based on an objective evaluation procedure-that show that our framework 1) is capable of locating and tracking the position and speaking activity of multiple meeting participants engaged in real conversations with good accuracy, 2) can deal with cases of visual clutter and occlusion, and 3) significantly outperforms a traditional sampling-based approach

Simultaneous pose recovery and camera registration from multiple views of a walking person

We present an algorithm to estimate the body pose of a walking person given synchronized video input from multiple uncalibrated cameras. We construct an appearance model of human walking motion by generating examples from the space of body poses and camera locations, and clustering them using expectation–maximization. Given a segmented input video sequence, we find the closest matching appearance cluster for each silhouette and use the sequence of matched clusters to extrapolate the position of the camera with respect to the person's direction of motion. For each frame, the matching cluster also provides an estimate of the walking phase. We combine these estimates from all views and find the most likely sequence of walking poses using a cyclical, feed-forward hidden Markov model. Our algorithm requires no manual initialization and no prior knowledge about the locations of the cameras.

Consistent labeling of tracked objects in multiple cameras with overlapping fields of view

We address the issue of tracking moving objects in an environment covered by multiple uncalibrated cameras with overlapping fields of view, typical of most surveillance setups. In such a scenario, it is essential to establish correspondence between tracks of the same object, seen in different cameras, to recover complete information about the object. We call this the problem of consistent labeling of objects when seen in multiple cameras. We employ a novel approach of finding the limits of field of view (FOV) of each camera as visible in the other cameras. We show that, if the FOV lines are known, it is possible to disambiguate between multiple possibilities for correspondence. We present a method to automatically recover these lines by observing motion in the environment, Furthermore, once these lines are initialized, the homography between the views can also be recovered. We present results on indoor and outdoor sequences containing persons and vehicles.