Joint Estimation of Camera Orientation and Vanishing Points from an Image Sequence in a Non-Manhattan World

A widely-used approach for estimating camera orientation is to use points at infinity, i.e., vanishing points (VPs). By enforcing the orthogonal constraint between the VPs, called the Manhattan world constraint, a drift-free camera orientation estimation can be achieved. However, in practical applications this approach suffers from many spurious parallel line segments or does not perform in non-Manhattan world scenes. To overcome these limitations, we propose a novel method that jointly estimates the VPs and camera orientation based on sequential Bayesian filtering. The proposed method does not require the Manhattan world assumption, and can perform a highly accurate estimation of camera orientation in real time. In addition, in order to enhance the robustness of the joint estimation, we propose a feature management technique that removes false positives of line clusters and classifies newly detected lines. We demonstrate the superiority of the proposed method through an extensive evaluation using synthetic and real datasets and comparison with other state-of-the-art methods.

This paper presents a camera orientation estimation method based on 3-line RANSAC using motion vector in a driving straight ahead vehicle. The proposed method consists of three steps: i) motion vector and z-axis vanishing point estimation using feature extraction and matching, ii) line detection and classification using 3-line RANSAC algorithm, and iii) camera orientation estimation with vanishing points using classified lines. The experimental result shows proposed method effectively estimate camera orientation parameter. Therefore, the proposed method can be applied in vehicle systems for automatic driving assistance system.

Real time vanishing points detection on smartphones under Manhattan world assumption

This paper proposes new techniques to generate high quality textures for urban building models by automatic camera calibration and pose recovery. The camera pose is decomposed into an orientation and a translation, an edge error model and knowledge-based filters are used to estimate correct vanishing points with heavy trees occlusion, and the vanishing points are used for the camera calibration and orientation estimation. We propose new techniques to estimate the camera orientation with infinite vanishing points and translation with under-constraints. The final textures are generated using color calibration and blending with the recovered pose. A number of textures for outdoor buildings are automatically generated, which shows the effectiveness of our algorithms.

In this paper, we propose a novel camera orientation estimation method based on the computation of the vanishing point of water drops in leaking indoor environment. Camera orientation estimation is an important component of robots as it allows them to perform complex tasks such as three-dimensional (3D) reconstruction of different environments. Camera estimation usually involves sensors, such as cameras or encoders and sophisticated processing algorithms. In recent years, computer vision techniques have been widely used to estimate the camera orientation in robotics-related research as visual sensing can improve the autonomy of the systems. Although most of these methods perform well in outdoor environments, they are problematic in the environments of indoor disasters, where common visual features may be missing due to collapse and erosion. To solve these problems, we developed a novel technique that employs particular characteristics of leaking indoor environment. Our method uses the vanishing point generated form the trajectories of water drops, to estimate the rotation of the camera. The proposed technique can potentially be applied for inspecting nuclear power plants. Computer-simulated and real data experiments have been performed to evaluate the accuracy of the proposed method. The results of these experiments demonstrate that our method can detect the vanishing point of water drops and estimate the rotation angle accurately.

Image lines projected from parallel 3D lines intersect at a common point called the vanishing point (VP). Manhattan world holds for the scenes with three orthogonal VPs. In Manhattan world, given several lines in a calibrated image, we aim to cluster them by three unknown-but-sought VPs. The VP estimation can be reformulated as computing the rotation between the Manhattan frame and camera frame. To estimate three degrees of freedom (DOF) of this rotation, state-of-the-art methods are based on either data sampling or parameter search. However, they fail to guarantee high accuracy and efficiency simultaneously. In contrast, we propose a set of approaches that hybridize these two strategies. We first constrain two or one DOF of the rotation by two or one sampled image line. Then we search for the remaining one or two DOF based on branch and bound. Our sampling accelerates our search by reducing the search space and simplifying the bound computation. Our search achieves quasi-global optimality. Specifically, it guarantees to retrieve the maximum number of inliers on the condition that two or one DOF is constrained. Our hybridization of two-line sampling and one-DOF search can estimate VPs in real time. Our hybridization of one-line sampling and two-DOF search can estimate VPs in near real time. Experiments on both synthetic and real-world datasets demonstrated that our approaches outperform state-of-the-art methods in terms of accuracy and/or efficiency.

Purpose – This paper aims to present a novel scheme of multiple vanishing points (VPs) estimation and corresponding lanes identification. Design/methodology/approach – The scheme proposed here includes two main stages: VPs estimation and lane identification. VPs estimation based on vanishing direction hypothesis and Bayesian posterior probability estimation in the image Hough space is a foremost contribution, and then VPs are estimated through an optimal objective function. In lane identification stage, the selected linear samples supervised by estimated VPs are clustered based on the gradient direction of linear features to separate lanes, and finally all the lanes are identified through an identification function. Findings – The scheme and algorithms are tested on real data sets collected from an intelligent vehicle. It is more efficient and more accurate than recent similar methods for structured road, and especially multiple VPs identification and estimation of branch road can be achieved and lanes of branch road can be identified for complex scenarios based on Bayesian posterior probability verification framework. Experimental results demonstrate VPs, and lanes are practical for challenging structured and semi-structured complex road scenarios. Originality/value – A Bayesian posterior probability verification framework is proposed to estimate multiple VPs and corresponding lanes for road scene understanding of structured or semi-structured road monocular images on intelligent vehicles.

Vanishing points and vanishing lines are classical geometrical concepts in perspective cameras that have a lineage dating back to 3 centuries. A vanishing point is a point on the image plane where parallel lines in 3D space appear to converge, whereas a vanishing line passes through 2 or more vanishing points. While such concepts are simple and intuitive in perspective cameras, their counterparts in catadioptric cameras (obtained using mirrors and lenses) are more involved. For example, lines in the 3D space map to higher degree curves in catadioptric cameras. The projection of a set of 3D parallel lines converges on a single point in perspective images, whereas they converge to more than one point in catadioptric cameras. To the best of our knowledge, we are not aware of any systematic development of analytical models for vanishing points and vanishing curves in different types of catadioptric cameras. In this paper, we derive parametric equations for vanishing points and vanishing curves using the calibration parameters, mirror shape coefficients, and direction vectors of parallel lines in 3D space. We show compelling experimental results on vanishing point estimation and absolute pose estimation for a wide range of catadioptric cameras in both simulations and real experiments.

The image lines projected from parallel 3D lines intersect at a common point called the vanishing point (VP). Manhattan world holds for the scenes with three orthogonal VPs. In Manhattan world, given several lines in a calibrated image, we aim at clustering them by three unknown-but-sought VPs. The VP estimation can be reformulated as computing the rotation between the Manhattan frame and the camera frame. To compute this rotation, state-of-the-art methods are based on either data sampling or parameter search, and they fail to guarantee the accuracy and efficiency simultaneously. In contrast, we propose to hybridize these two strategies. We first compute two degrees of freedom (DOF) of the above rotation by two sampled image lines, and then search for the optimal third DOF based on the branch-and-bound. Our sampling accelerates our search by reducing the search space and simplifying the bound computation. Our search is not sensitive to noise and achieves quasi-global optimality in terms of maximizing the number of inliers. Experiments on synthetic and real-world images showed that our method outperforms state-of-the-art approaches in terms of accuracy and/or efficiency.

Calibration of a vehicle camera is a key technology for advanced driver assistance systems (ADAS). This paper presents a novel estimation method to measure the orientation of a camera that is mounted on a driving vehicle. By considering the characteristics of vehicle cameras and driving environment, we detect three orthogonal vanishing points as a basis of the imaging geometry. The proposed method consists of three steps: i) detection of lines projected to the Gaussian sphere and extraction of the plane normal, ii) estimation of the vanishing point about the optical axis using linear Hough transform, and iii) voting for the rest two vanishing points using circular histogram. The proposed method increases both accuracy and stability by considering the practical driving situation using sequentially estimated three vanishing points. In addition, we can rapidly estimate the orientation by converting the voting space into a 2D plane at each stage. As a result, the proposed method can quickly and accurately estimate the orientation of the vehicle camera in a normal driving situation.

A framework is presented for refining GPS location and estimate the camera orientation using a single urban building image, a 2D city map with building outlines, given a noisy GPS location. We propose to use tilt-invariant vertical building corner edges extracted from the building image. A location-orientation hypothesis, which we call an LOH, is a proposed map location from which an image of building corners would occur at the observed positions of corner edges in the photo. The noisy GPS location is refined and orientation is estimated using the computed LOHs. Experiments show the framework improves GPS accuracy significantly, generally produces reliable orientation estimation, and is computationally efficient.

Vanishing points are often used as constraints in lane detection or road following systems of intelligent vehicles. This paper proposes a new method for vanishing point estimation in consecutive frames based on computer vision. Parallel lines in the real world converge to vanishing points on an image plane, caused by the perspective projection. According to the duality between points and lines, estimation of vanishing points can be converted to a problem of line parameter estimation in a parameter space. Firstly, straight lines are detected from an extracted edge map of a road image by the Progressive Probability Hough Transform (PPHT) incorporated with gradient orientation constraints. Then, vanishing points are estimated via the Maximum A Posteriori (MAP) estimate, integrating information at the current frame and the vanishing point estimated at the previous frame into a probabilistic framework. For the detected lines are noisy, a weight is put on each line to indicate the probability ofbeing an inlier. But the weights are unknown, which are regarded as hidden variables here. Thus the Expectation Maximum (EM) algorithm is adopted to solve the MAP problem with hidden variables. Experimental results show the efficiency and robustness ofthe proposed method.

This paper proposes a real-time pipeline for estimating the camera orientation based on vanishing points for indoor navigation assistance on a Smartphone. The orientation of embedded camera relies on the ability to find a reliable triplet of orthogonal vanishing points. The proposed pipeline introduces a novel sampling strategy among finite and infinite vanishing points with a random sample consensus-based line clustering and a tracking along a video sequence to enforce the accuracy and the robustness by extracting the three most pertinent orthogonal directions while preserving a short processing time for real-time application. Experiments on real images and video sequences acquired with a Smartphone show that the proposed strategy for selecting orthogonal vanishing points is pertinent as our algorithm gives better results than the recently published RNS optimal method, in particular for the yaw angle, which is actually essential for the navigation task.

With the development of road detection related applications, higher performance is required on the accuracy, robustness and time efficiency. Road detection algorithm based on vanishing point estimation has been paid lots attention on for its adaptability when processing complicated situations. However, the existing algorithms based on vanishing point estimation have two shortcomings: 1) unable to handling images with no internal vanishing point, 2) high computational complexity. This paper proposes an improved road detection algorithm based on principal orientation and generalized vanishing point estimation. Proposed method reduce amount of calculation by select effective voting pixels with the principal orientation constraint, and improved the algorithm robustness by handling cases that vanishing point located external of images with the concept of generalized vanishing point. Quantitative and qualitative experiment results donate that the proposed method is both accurate and efficient in road detection. KeywordsRoad Detection; Vanishing Point Estimation; Generalized Vanishing Point; Principal Orientation; Multidimensional Voting Strategy

The problem of estimating vanishing points for visual scenes under the Manhattan world assumption [1, 2] has been addressed for more than a decade. Surprisingly, the special characteristic of the Manhattan world that lines should be orthogonal or parallel to each other is seldom well utilized. In this paper, we present an algorithm that accurately and efficiently estimates vanishing points and classifies lines by thoroughly taking advantage of this simple fact in the Manhattan world with a calibrated camera. We first present a one-unknown-parameter representation of the 3D line direction in the camera frame. Then derive a quadratic which is employed to solve three orthogonal vanishing points formed by a line triplet. Finally, we develop a RANSAC-based approach to fulfill the task. The performance of proposed approach is demonstrated on the York Urban Database[3] and compared to the state-of-the-art method.