Ego-motion Estimation Problem Research Articles

LS-VO: Learning Dense Optical Subspace for Robust Visual Odometry Estimation

This work proposes a novel deep network architecture to solve the camera Ego-Motion estimation problem. A motion estimation network generally learns features similar to Optical Flow (OF) fields starting from sequences of images. This OF can be described by a lower dimensional latent space. Previous research has shown how to find linear approximations of this space. We propose to use an Auto-Encoder network to find a non-linear representation of the OF manifold. In addition, we propose to learn the latent space jointly with the estimation task, so that the learned OF features become a more robust description of the OF input. We call this novel architecture LS-VO. The experiments show that LS-VO achieves a considerable increase in performances in respect to baselines, while the number of parameters of the estimation network only slightly increases.

Ego-Motion Estimation of a Quadrotor Based on Nonlinear Observer

In this paper, an ego-motion estimation problem of a quadrotor unmanned aerial vehicle is studied. A nonlinear observer is proposed to simultaneously estimate the depth of the point features and the translational velocity of the quadrotor using visual feedback from an on-board monocular camera. Image moments defined on a rotated image plane are chosen as image features, and a decoupled image feature dynamics is achieved to facilitate the observer design. Based on the image feature dynamics and the dynamic model of the quadrotor, an observer is proposed to estimate the ego-motion of the quadrotor. Only several point features need to be tracked in consecutive acquired images, which makes the observer computationally efficient and robust. The stability of the observer is proved by means of Lyapunov analysis. Finally, computer simulations and experimental tests, including closed loop image-based visual servoing control and comparison with extended-Kalman-filter based observer, are conducted to verify the validity of the observer.

Monocular Vision- and IMU-Based System for Prosthesis Pose Estimation During Total Hip Replacement Surgery.

The average age of population increases worldwide, so does the number of total hip replacement surgeries. Total hip replacement, however, often involves a risk of dislocation and prosthetic impingement. To minimize the risk after surgery, we propose an instrumented hip prosthesis that estimates the relative pose between prostheses intraoperatively and ensures the placement of prostheses within a safe zone. We create a model of the hip prosthesis as a ball and socket joint, which has four degrees of freedom (DOFs), including 3-DOF rotation and 1-DOF translation. We mount a camera and an inertial measurement unit (IMU) inside the hollow ball, or "femoral head prosthesis," while printing customized patterns on the internal surface of the socket, or "acetabular cup." Since the sensors were rigidly fixed to the femoral head prosthesis, measuring its motions poses a sensor ego-motion estimation problem. By matching feature points in images of the reference patterns, we propose a monocular vision based method with a relative error of less than 7% in the 3-DOF rotation and 8% in the 1-DOF translation. Further, to reduce system power consumption, we apply the IMU with its data fused by an extended Kalman filter to replace the camera in the 3-DOF rotation estimation, which yields a less than 4.8% relative error and a 21.6% decrease in power consumption. Experimental results show that the best approach to prosthesis pose estimation is a combination of monocular vision-based translation estimation and IMU-based rotation estimation, and we have verified the feasibility and validity of this system in prosthesis pose estimation.

Planar-Based Visual Inertial Navigation: Observability Analysis and Motion Estimation

In this paper, we address the problem of ego-motion estimation by fusing visual and inertial information. The hardware consists of an inertial measurement unit (IMU) and a monocular camera. The camera provides visual observations in the form of features on a horizontal plane. Exploiting the geometric constraint of features on the plane into visual and inertial data, we propose a novel closed form measurement model for this system. Our first contribution in this paper is an observability analysis of the proposed planar-based visual inertial navigation system (VINS). In particular, we prove that the system has only three unobservable states corresponding to global translations parallel to the plane, and rotation around the gravity vector. Hence, compared to general VINS, an advantage of using features on the horizontal plane is that the vertical translation along the normal of the plane becomes observable. As the second contribution, we present a state-space formulation for the pose estimation in the analyzed system and solve it via a modified unscented Kalman filter (UKF). Finally, the findings of the theoretical analysis and 6-DoF motion estimation are validated by simulations as well as using experimental data.

Simultaneous egomotion estimation, segmentation, and moving object detection

AbstractRobust egomotion estimation is a key prerequisite for making a robot truly autonomous. In previous work, a multimodel extension of random sample consensus (RANSAC) was introduced to deal with environments with rapid changes by incorporating moving object information. A multiscale matching algorithm was also proposed to resolve the issue of imperfect segmentation. In this paper, we present a novel specialization of RANSAC that extends the previous work. A unified framework is introduced to achieve simultaneously egomotion estimation, multiscale segmentation, and moving object detection in the RANSAC paradigm. The motivation of this work is to provide a robust real‐time solution to the problem of egomotion estimation, segmentation, and moving object detection in highly dynamic environments. The idea is to augment the discriminative power of spatial and temporal appearances of objects by the spatiotemporal consistency. The objective is twofold. First, split mismerged segments and distinguish nonstationary objects from stationary objects by the spatial consistency. Second, merge oversegmented segments and differentiate moving objects from outlying objects by the temporal consistency. Moving objects of considerably different sizes, from pedestrians to trucks, can be properly segmented and correctly detected. We also show that the performance of egomotion estimation can be further improved by taking into account both stationary and moving object information. Our approach is extensively evaluated on challenging data sets and compared to the state of the art. The experiments also show that our approach serves as a general framework that works well with various planar range data. © 2011 Wiley Periodicals, Inc.

Error Aware Monocular Visual Odometry using Vertical Line Pairs for Small Robots in Urban Areas

We report a new error-aware monocular visual odometry method that only uses vertical lines, such as vertical edges of buildings and poles in urban areas as landmarks. Since vertical lines are easy to extract, insensitive to lighting conditions/ shadows, and sensitive to robot movements on the ground plane, they are robust features if compared with regular point features or line features. We derive a recursive visual odometry method based on the vertical line pairs. We analyze how errors are propagated and introduced in the continuous odometry process by deriving the closed form representation of covariance matrix. We formulate the minimum variance ego-motion estimation problem and present a method that outputs weights for different vertical line pairs. The resulting visual odometry method is tested in physical experiments and compared with two existing methods that are based on point features and line features, respectively. The experiment results show that our method outperforms its two counterparts in robustness, accuracy, and speed. The relative errors of our method are less than 2% in experiments.

On an Optical Inertial Navigation System—Part II

In part I, we developed the optical transfer function of the lens-fiber system for quasi-monochromatic, incoherent excitation, and studied the properties of the kernel function. We also studied the cross-talk between the fibers of the lens-fiber system for a worker bee and an artificial eye, and showed that it is not significant. This allows us in this paper, to consider a mathematical idealization of a corneal surface as a continuum of lens-fiber systems. We consider this surface to be a regular immersion of class rges 2 that is the image in R3 of a simply connected, open set in R2. We study the change in the power propagated in the fiber due to virtual motions of the corneal surface and show that for motion along the axis, the power propagated is invariant. Finally, we show that the ego-motion estimation problem is well-posed for sufficiently rich quasi-monochromatic, incoherent excitation on an allowable, regular corneal surface, and further show that the solution does not depend on the parameterization of the surfaces or the parameters of the aircraft (such as mass and inertia matrix) on which the ONS is mounted.

Camera cluster in motion

The first comparison of different types of cameras (as opposed to different algorithms) for the ego-motion estimation problem is presented. As technology and computational power increase, the effectiveness of visual algorithms is limited only by inherent statistical uncertainties in the problems they are solving. The Fisher information matrix is a powerful analysis technique that can apply to any problem that involves searching for a parameter set that minimizes an error function. This includes problems such as pose-estimation, object recognition, or classification. Designing camera systems optimized for particular tasks may significantly improve the success of visual algorithms.

Robust egomotion estimation from the normal flow using search subspaces

We address the problem of egomotion estimation for a monocular observer moving under arbitrary translation and rotation, in an unknown environment. The method we propose is uniquely based on the spatio-temporal image derivatives, or the normal flow. We introduce a search paradigm which is based on geometric properties of the normal flow field, and consists in considering a family of search subspaces to estimate the egomotion parameters. Various algorithms are proposed within this framework. In order to decrease the noise sensitivity of the estimation methods, we use statistical tools, based on robust regression theory. Finally, we present and discuss a wide variety of experiments with synthetic and real images, for various kinds of camera motion.