Camera Orientation Estimation Research Articles

Multilayer Perceptron-Based Error Compensation for Automatic On-the-Fly Camera Orientation Estimation Using a Single Vanishing Point from Road Lane

This study introduces a multilayer perceptron (MLP) error compensation method for real-time camera orientation estimation, leveraging a single vanishing point and road lane lines within a steady-state framework. The research emphasizes cameras with a roll angle of 0°, predominant in autonomous vehicle contexts. The methodology estimates pitch and yaw angles using a single image and integrates two Kalman filter models with inputs from image points (u, v) and derived angles (pitch, yaw). Performance metrics, including avgE, minE, maxE, ssE, and Stdev, were utilized, testing the system in both simulator and real-vehicle environments. The outcomes indicate that our method notably enhances the accuracy of camera orientation estimations, consistently outpacing competing techniques across varied scenarios. This potency of the method is evident in its adaptability and precision, holding promise for advanced vehicle systems and real-world applications.

Globally Optimal Camera Orientation Estimation from Line Correspondences by BnB algorithm

This letter is concerned with the problem of estimating camera orientation from a set of 2D/3D line correspondences, which is a major part of the Perspective-n-Line (PnL) problem. There are some cases that usually occur in real applications for PnL: the input line correspondences are corrupted by mismatches (a.k.a. outlier correspondences). The RANdom SAmple Consensus (RANSAC) algorithm is the de facto standard for solving outlier-contaminated PnL problems. However, RANSAC is a non-deterministic algorithm, which means that it produces a reasonable result only with a certain probability. Therefore, a PnL algorithm that could obtain a certifiably optimal solution from outlier-contaminated data is a matter of priority for some safety-critical applications. In this letter, we take a big step towards this goal by investigating globally optimal camera orientation estimation algorithms. Firstly, we decouple the rotation and translation estimation of a PnL problem by considering the geometrical property of the PnL problem. The Branch-and-Bound (BnB) algorithm is applied and it globally searches the entire rotation space to obtain the optimal camera orientation. To investigate the performance of our method, we tested the proposed algorithm on both synthetic and real data, and the results show that our algorithms can obtain the optimal camera orientation and are more robust than several state-of-the-art PnL methods.

Camera parameters estimation from pose detections

Some computer vision tasks become easier with known camera calibration. We propose a method for camera focal length, location and orientation estimation by observing human poses in the scene. Weak requirements to the observed scene make the method applicable to a wide range of scenarios. Our evaluation shows that even being trained only on synthetic dataset, the proposed method outperforms known solution. Our experiments show that using only human poses as the input also allows the proposed method to calibrate dynamic visual sensors.

Camera Orientation Estimation Using Motion-Based Vanishing Point Detection for Advanced Driver-Assistance Systems

Advanced driver-assistance systems need a camera calibration algorithm for various vision applications including surround-view monitoring (SVM) and lane departure warning (LDW). Although cameras mounted on a vehicle are calibrated in the manufacturing process, their orientation angles are subject to tilting because of continuing vibration and external impact. To solve the problem, this paper presents an online calibration algorithm for camera orientation estimation using motion vectors and three-dimensional geometry. The proposed algorithm consists of three steps: i) driving direction estimation by calculating an intersection of motion vectors, ii) camera orientation estimation based on 3-line random sample consensus (RANSAC) using the estimated intersection, and iii) final orientation decision using extended Kalman filter from the result of each frame. Experimental results demonstrate that the proposed algorithm stably estimates camera orientation angles from motion vectors and lines under the parallelism and orthogonality assumptions.

Camera orientation estimation using voting approach on the Gaussian sphere for in-vehicle camera.

Calibration of a vehicle camera is a key technology for advanced driver assistance systems (ADAS). This paper presents a novel estimation method to measure the orientation of a camera that is mounted on a driving vehicle. By considering the characteristics of vehicle cameras and driving environment, we detect three orthogonal vanishing points as a basis of the imaging geometry. The proposed method consists of three steps: i) detection of lines projected to the Gaussian sphere and extraction of the plane normal, ii) estimation of the vanishing point about the optical axis using linear Hough transform, and iii) voting for the rest two vanishing points using circular histogram. The proposed method increases both accuracy and stability by considering the practical driving situation using sequentially estimated three vanishing points. In addition, we can rapidly estimate the orientation by converting the voting space into a 2D plane at each stage. As a result, the proposed method can quickly and accurately estimate the orientation of the vehicle camera in a normal driving situation.

Joint Estimation of Camera Orientation and Vanishing Points from an Image Sequence in a Non-Manhattan World

A widely used approach for estimating camera orientation is to use the points at infinity, i.e., the vanishing points (VPs). Enforcement of the orthogonal constraint between the VPs, known as the Manhattan world constraint, enables an estimation of the drift-free camera orientation to be achieved. However, in practical applications, this approach is neither effective (because of noisy parallel line segments) nor performable in non-Manhattan world scenes. To overcome these limitations, we propose a novel method that jointly estimates the VPs and camera orientation based on sequential Bayesian filtering. The proposed method does not require the Manhattan world assumption, and can perform a highly accurate estimation of camera orientation. In order to enhance the robustness of the joint estimation, we propose a keyframe-based feature management technique that removes false positives from parallel line clusters and detects new parallel line sets using geometric properties such as the orthogonality and rotational dependence for a VP, a line, and the camera rotation. In addition, we propose a 3-line camera rotation estimation method that does not require the Manhattan world assumption. The 3-line method is applied to the RANSAC-based outlier rejection technique to eliminate outlier measurements; therefore, the proposed method achieves accurate and robust estimation of the camera orientation and VPs in general scenes with non-orthogonal parallel lines. We demonstrate the superiority of the proposed method by conducting an extensive evaluation using synthetic and real datasets and by comparison with other state-of-the-art methods.

Real time vanishing points detection on smartphones under Manhattan world assumption

Vanishing points (VPs) detection is of high computation complexity with hundreds of line segments from the image. Experimentally, line segments detection alone, as the foundation of VPs detection, is of heavy load for smartphones. Motivated by fusion of vision and inertial sensors (Lobo and Dias, 2003)[17], we simplify VPs detection by only detecting potential vanishing lines and guarantee the accuracy based on three constraints, the attitude from sensors, the Manhattan world assumption and the constant camera motion mode. A prototype is built on iPhone 5S and tested in a typical office building at 15 frames per second. Compared with the state of the art (Elloumi et al., 2014)[12], only 50% CPU usage and 70% energy consumption are required for the proposed method. For camera orientation estimated from VPs, the average error is less than two degrees. The proposed method makes it possible for numerous real time applications on mobile devices, e.g., accurate camera orientation estimation and 3D reconstruction, which will further benefit indoor navigation and augmented reality.

Stable Camera Motion Estimation Using Convex Programming

We study the inverse problem of estimating $n$ locations $\mathbf{t}_1, \mathbf{t}_2, \ldots, \mathbf{t}_n$ (up to global scale, translation, and negation) in $\mathbb{R}^d$ from noisy measurements of a subset of the (unsigned) pairwise lines that connect them, that is, from noisy measurements of $\pm \frac{\mathbf{t}_i - \mathbf{t}_j}{\|\mathbf{t}_i - \mathbf{t}_j \|_2}$ for some pairs $(i,j)$ (where the signs are unknown). This problem is at the core of the structure from motion (SfM) problem in computer vision, where the $\mathbf{t}_i$ represent camera locations in $\mathbb{R}^3$. The noiseless version of the problem, with exact line measurements, has been considered previously under the general title of parallel rigidity theory, mainly in order to characterize the conditions for unique realization of locations. For noisy pairwise line measurements, current methods tend to produce spurious solutions that are clustered around a few locations. This sensitivity of the location estimates is a well-known problem in SfM, especially for large, irregular collections of images. In this paper we introduce a semidefinite programming (SDP) formulation, specially tailored to overcome the clustering phenomenon. We further identify the implications of parallel rigidity theory for the location estimation problem to be well-posed, and prove exact (in the noiseless case) and stable location recovery results. We also formulate an alternating direction method to solve the resulting semidefinite program, and provide a distributed version of our formulation for large numbers of locations. Specifically for the camera location estimation problem, we formulate a pairwise line estimation method based on robust camera orientation and subspace estimation. Finally, we demonstrate the utility of our algorithm through experiments on real images.

Real-time camera orientation estimation based on vanishing point tracking under Manhattan World assumption

This paper proposes a real-time pipeline for estimating the camera orientation based on vanishing points for indoor navigation assistance on a Smartphone. The orientation of embedded camera relies on the ability to find a reliable triplet of orthogonal vanishing points. The proposed pipeline introduces a novel sampling strategy among finite and infinite vanishing points with a random sample consensus-based line clustering and a tracking along a video sequence to enforce the accuracy and the robustness by extracting the three most pertinent orthogonal directions while preserving a short processing time for real-time application. Experiments on real images and video sequences acquired with a Smartphone show that the proposed strategy for selecting orthogonal vanishing points is pertinent as our algorithm gives better results than the recently published RNS optimal method, in particular for the yaw angle, which is actually essential for the navigation task.

Corisco: Robust edgel-based orientation estimation for generic camera models

The estimation of camera orientation from image lines using the anthropic environment restriction is a well-known problem, but traditional methods to solve it depend on line extraction, a relatively complex procedure that is also incompatible with distorted images. We propose Corisco, a monocular orientation estimation method based on edgels instead of lines. Edgels are points sampled from image edges with their tangential directions, extracted in Corisco using a grid mask. The estimation aligns the measured edgel directions with the predicted directions calculated from the orientation, using a known camera model. Corisco uses the M-estimation technique to define an objective function that is optimized by two algorithms in sequence: RANSAC, which gives robustness and flexibility to Corisco, and FilterSQP, which performs a continuous optimization to refine the initial estimate, using closed formulas for the function derivatives. Corisco is the first edgel-based method able to analyze images with any camera model, and it also allows for a compromise between speed and accuracy, so that its performance can be tuned according to the application requirements. Our experiments demonstrate the effectiveness of Corisco with various camera models, and its performance surpasses similar edgel-based methods. The accuracy displayed a mean error below 2° for execution times above 8s in a conventional computer, and above 3° for less than 2s.