Accurate Camera Pose Estimation Research Articles

Efficient bundle optimization for accurate camera pose estimation in mobile augmented reality systems

Augmented reality has a long research history in computer vision and computer graphics communities. It aims to enhance the user experience for real scenes via overlapping virtual objects. Nowadays, mobile augmented reality has attracted much attention from researchers and developers due to the development of hardware techniques. Modern mobile devices such as mobile phones have a powerful computational ability for augmented reality applications. As a result, many researchers have paid attention to mobile augmented reality. From the technical viewpoint of augmented reality, mobile augmented reality largely depends on camera pose estimation. However, existing methods make it difficult to achieve the best balance between accuracy and efficiency, according to our investigation, and this may handicap the performance of mobile augmented reality systems. To overcome the problem, in this paper, we propose a novel approach to camera pose estimation based on bundle optimization. Our proposed method is evaluated on real-world datasets and is also tested in the mobile augmented reality system. Both experiments demonstrate that our proposed method has fast speed and high accuracy.

Camera Pose Estimation Using a 3D Gaussian Splatting Radiance Field

Introduction. Accurate camera pose estimation is crucial for many applications ranging from robotics to virtual and augmented reality. The process of determining agents pose from a set of observations is called odometry. This work focuses on visual odometry, which utilizes only images from camera as the input data. The purpose of the paper is to demonstrate an approach for small-scale camera pose estimation using 3D Gaussians as the environment representation. Methods. Given the rise of neural volumetric representations for the environment reconstruction, this work relies on Gaussian Splatting algorithm for high-fidelity volumetric representation. Results. For a trained Gaussian Splatting model and the target image, unseen during training, we estimate its camera pose using differentiable rendering and gradient-based optimization methods. Gradients with respect to camera pose are computed directly from image-space per-pixel loss function via backpropagation. The choice of Gaussian Splatting as representation is particularly appealing because it allows for end-to-end estimation and removes several stages that are common for more classical algorithms. And differentiable rasterization as the image formation algorithm provides real-time performance which facilitates its use in real-world applications. Conclusions. This end-to-end approach greatly simplifies camera pose estimation, avoiding compounding errors that are common for multi-stage algorithms and provides a high-quality camera pose estimation. Keywords: radiance fields, scientific computing, odometry, slam, pose estimation, Gaussian Splatting, differentiable rendering.

Vision-based algorithm for structural response measurement using movable camera and damage localization

Vibration-based Structural Health Monitoring (SHM) is widely recognized as a prominent approach for evaluating the safety and integrity of civil infrastructure. This research presents a novel algorithm for vibration measurement using a movable camera, effectively overcoming the issue of signal drift caused by camera motion. The proposed algorithm comprises three modules: optical flow computation, camera pose estimation, and motion compensation. Validation experiments by virtual images demonstrate accurate camera pose estimation and effective mitigation of drifting signals. The dynamic characteristics of the frame structure are extracted using a movable camera and the proposed algorithm. Several challenges are addressed, including accurately estimating the scale factor to compensate for the drifting signals. Experimental validation is conducted to verify the proposed approach. The results demonstrate the efficacy and promising potential of the proposed approach. Finally, this study demonstrates the practical application through damage localization experiments.

IMPROVING CAMERA POSE ESTIMATION USING SWARM PARTICLE ALGORITHMS

Abstract. Most computer vision and photogrammetry applications rely on accurately estimating the camera pose, such as visual navigation, motion tracking, stereo photogrammetry, and structure from motion. The Essential matrix is a well-known model in computer vision that provides information about the relative orientation between two images, including the rotation and translation, for calibrated cameras with a known camera matrix. To estimate the Essential matrix, the camera calibration matrices, which include focal length and principal point location must be known, and the estimation process typically requires at least five matching points and the use of robust algorithms, such as RANSAC to fit a model to the data as a robust estimator. From the usually large number of matched points, choosing five points, the Essential matrix can be determined based on a simple solution, which could be good or bad. Obtaining a globally optimal and accurate camera pose estimation, however, requires additional steps, such as using evolutionary algorithms (EA) or swarm algorithms (SA), to prevent getting trapped in local optima by searching for solutions within a potentially huge solution space.This paper aims to introduce an improved method for estimating the Essential matrix using swarm particle algorithms that are known to efficiently solve complex problems. Various optimization techniques, including EAs and SAs, such as Particle Swarm Optimization (PSO), Gray Wolf Optimization (GWO), Improved Gray Wolf Optimization (IGWO), Genetic Algorithm (GA), Salp Swarm Algorithm (SSA) and Whale Optimization Algorithm (WOA), are explored to obtain the global minimum of the reprojection error for the five-point Essential matrix estimation based on using symmetric geometric error cost function. The experimental results on a dataset with known camera orientation demonstrate that the IGWO method has achieved the best score compared to other techniques and significantly speeds up the camera pose estimation for larger number of point pairs in contrast to traditional methods that use the collinearity equations in an iterative adjustment.

Improving Worst Case Visual Localization Coverage via Place-Specific Sub-Selection in Multi-Camera Systems

6-DoF visual localization systems utilize principled approaches rooted in 3D geometry to perform accurate camera pose estimation of images to a map. Current techniques use hierarchical pipelines and learned 2D feature extractors to improve scalability and increase performance. However, despite gains in typical recall <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$@0.25\,{\rm{m}}$</tex-math></inline-formula> type metrics, these systems still have limited utility for real-world applications like autonomous vehicles because of their <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">worst</i> areas of performance - the locations where they provide insufficient recall at a certain required error tolerance. Here we investigate the utility of using <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">place specific configurations</i> , where a map is segmented into a number of places, each with its own configuration for modulating the pose estimation step, in this case selecting a camera within a multi-camera system. On the Ford AV benchmark dataset, we demonstrate substantially improved worst-case localization performance compared to using off-the-shelf pipelines - minimizing the percentage of the dataset which has low recall at a certain error tolerance, as well as improved overall localization performance. Our proposed approach is particularly applicable to the crowdsharing model of autonomous vehicle deployment, where a fleet of AVs are regularly traversing a known route.

Accurate visual localization with semantic masking and attention

Visual localization is the task of accurate camera pose estimation within a scene and is a crucial technique for computer vision and robotics. Among the various approaches, relative pose estimation has gained increasing interest because it can generalize to new scenes. This approach learns to regress relative pose between image pairs. However, unreliable regions that contain objects such as the sky, persons, or moving cars are often present in real images, causing noise and interference to localization. In this paper, we propose a novel relative pose estimation pipeline to address the problem. The pipeline features a semantic masking module and an attention module. The two modules help suppress interfering information from unreliable regions, while at the same time emphasizing important features with an attention mechanism. Experiment results show that our framework outperforms alternative methods in the accuracy of camera pose prediction in all scenes.

3D registration based on V-SLAM and application in augmented reality

Augmented reality is currently a research hotspot in the field of computer vision and computer graphics, and its applications are becoming more and more extensive. One of its key technologies is the three-dimensional registration of virtual objects. Three-dimensional registration requires accurate camera pose estimation and scene three-dimensional reconstruction. Therefore, this paper studies the 3D registration based on visual SLAM, and mainly contributes the following three aspects. (1) A dense matching method based on a depth camera is proposed, which can be well applied to scenes where the camera moves fast or rotates strongly, such as augmented reality. (2) A dense reconstruction method based on Voxel Hashing is designed, which alleviates the low computational efficiency and low precision of the existing RGB-D SLAM method. (3) A simple augmented reality system was designed to verify the effects of the registration and fusion of virtual objects. Experiments show that compared with the state-of-the-art methods, the algorithm proposed in this paper generates a more refined and smooth reconstructed model, and the virtual-real fusion effect based on this model is also more realistic.

Stereo visual odometry based on dynamic and static features division

&lt;p style='text-indent:20px;'&gt;Accurate camera pose estimation in dynamic scenes is an important challenge for visual simultaneous localization and mapping, and it is critical to reduce the effects of moving objects on pose estimation. To tackle this problem, a robust visual odometry approach in dynamic scenes is proposed, which can precisely distinguish between dynamic and static features. The key to the proposed method is combining the scene flow and the static features relative spatial distance invariance principle. Moreover, a new threshold is proposed to distinguish dynamic features.Then the dynamic features are eliminated after matching with the virtual map points. In addition, a new similarity calculation function is proposed to improve the performance of loop-closure detection. Finally, the camera pose is optimized after obtaining a closed loop. Experiments have been conducted on TUM datasets and actual scenes, which shows that the proposed method reduces tracking errors significantly and estimates the camera pose precisely in dynamic scenes.&lt;/p&gt;

Accurate Dynamic SLAM Using CRF-Based Long-Term Consistency.

Accurate camera pose estimation is essential and challenging for real world dynamic 3D reconstruction and augmented reality applications. In this article, we present a novel RGB-D SLAM approach for accurate camera pose tracking in dynamic environments. Previous methods detect dynamic components only across a short time-span of consecutive frames. Instead, we provide a more accurate dynamic 3D landmark detection method, followed by the use of long-term consistency via conditional random fields, which leverages long-term observations from multiple frames. Specifically, we first introduce an efficient initial camera pose estimation method based on distinguishing dynamic from static points using graph-cut RANSAC. These static/dynamic labels are used as priors for the unary potential in the conditional random fields, which further improves the accuracy of dynamic 3D landmark detection. Evaluation using the TUM and Bonn RGB-D dynamic datasets shows that our approach significantly outperforms state-of-the-art methods, providing much more accurate camera trajectory estimation in a variety of highly dynamic environments. We also show that dynamic 3D reconstruction can benefit from the camera poses estimated by our RGB-D SLAM approach.

Vid2Curve

Thin structures, such as wire-frame sculptures, fences, cables, power lines, and tree branches, are common in the real world. It is extremely challenging to acquire their 3D digital models using traditional image-based or depth-based reconstruction methods, because thin structures often lack distinct point features and have severe self-occlusion. We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera. Specifically, we present a new curve-based approach to estimate accurate camera poses by establishing correspondences between featureless thin objects in the foreground in consecutive video frames, without requiring visual texture in the background scene to lock on. Enabled by this effective curve-based camera pose estimation strategy, we develop an iterative optimization method with tailored measures on geometry, topology as well as self-occlusion handling for reconstructing 3D thin structures. Extensive validations on a variety of thin structures show that our method achieves accurate camera pose estimation and faithful reconstruction of 3D thin structures with complex shape and topology at a level that has not been attained by other existing reconstruction methods.

Dynamic-DSO: Direct Sparse Odometry Using Objects Semantic Information for Dynamic Environments

Traditional Simultaneous Localization and Mapping (SLAM) (with loop closure detection), or Visual Odometry (VO) (without loop closure detection), are based on the static environment assumption. When working in dynamic environments, they perform poorly whether using direct methods or indirect methods (feature points methods). In this paper, Dynamic-DSO which is a semantic monocular direct visual odometry based on DSO (Direct Sparse Odometry) is proposed. The proposed system is completely implemented with the direct method, which is different from the most current dynamic systems combining the indirect method with deep learning. Firstly, convolutional neural networks (CNNs) are applied to the original RGB image to generate the pixel-wise semantic information of dynamic objects. Then, based on the semantic information of the dynamic objects, dynamic candidate points are filtered out in keyframes candidate points extraction; only static candidate points are reserved in the tracking and optimization module, to achieve accurate camera pose estimation in dynamic environments. The photometric error calculated by the projection points in dynamic region of subsequent frames are removed from the whole photometric error in pyramid motion tracking model. Finally, the sliding window optimization which neglects the photometric error calculated in the dynamic region of each keyframe is applied to obtain the precise camera pose. Experiments on the public TUM dynamic dataset and the modified Euroc dataset show that the positioning accuracy and robustness of the proposed Dynamic-DSO is significantly higher than the state-of-the-art direct method in dynamic environments, and the semi-dense cloud map constructed by Dynamic-DSO is clearer and more detailed.

MGC-VSLAM: A Meshing-Based and Geometric Constraint VSLAM for Dynamic Indoor Environments

Visual Simultaneous Localization and Mapping (VSLAM) system is considered to be a fundamental capability for autonomous mobile robots. However, most of the existing VSLAM algorithms adopt a strong scene rigidity assumption for analysis convenience, which ignored the influence of independently moving objects in the real environment on the accuracy of the SLAM system. Hence, this paper proposes MGC-VSLAM: Meshing-based and Geometric constraint VSLAM, a novel VSLAM algorithm for dynamic indoor environments, built on RGB-D mode of ORB-SLAM2, which relates to the problem of the ORB feature uniform distribution and the dynamic feature filtering. In detail, aiming at the problem of the over-uniform distribution of feature points extracted by the quadtree-based algorithm in ORB-SLAM2, a meshing-based feature uniform distribution algorithm is proposed. Meshes are divided at each layer of the image pyramid, and then a specific number of features in the meshes are reserved according to their Harris response value. In addition, aiming at the impact of features extracted from dynamic targets on the SLAM system, a dynamic feature filtering method is proposed. First, a stable matching relationship is established through a feature matching constraint method. Then a novel geometric constraint method is used to filter out the dynamic feature points in the scene. Only the remaining static features are reserved to achieve accurate camera pose estimation in dynamic environments. Experiments on the Oxford dataset and public TUM RGB-D dataset are conducted to evaluate the proposed approach. It revealed that the proposed MGC-VSLAM can effectively improve the positioning accuracy of ORB-SLAM2 in high-dynamic scenarios.

Fast and accurate visual odometry from a monocular camera

This paper aims at a semi-dense visual odometry system that is accurate, robust, and able to run realtime on mobile devices, such as smartphones, AR glasses and small drones. The key contributions of our system include: 1) the modified pyramidal Lucas-Kanade algorithm which incorporates spatial and depth constraints for fast and accurate camera pose estimation; 2) adaptive image resizing based on inertial sensors for greatly accelerating tracking speed with little accuracy degradation; and 3) an ultrafast binary feature description based directly on intensities of a resized and smoothed image patch around each pixel that is sufficiently effective for relocalization. A quantitative evaluation on public datasets demonstrates that our system achieves better tracking accuracy and up to about 2X faster tracking speed comparing to the state-of-the-art monocular SLAM system: LSD-SLAM. For the relocalization task, our system is 2.0X ~ 4.6X faster than DBoW2 and achieves a similar accuracy.

SOF-SLAM: A Semantic Visual SLAM for Dynamic Environments

Simultaneous Localization and Mapping (SLAM) plays an important role in the computer vision and robotics field. The traditional SLAM framework adopts a strong static world assumption for analysis convenience. How to cope with dynamic environments is of vital importance and attracts more attentions. Existing SLAM systems toward dynamic scenes either solely utilize semantic information, solely utilize geometry information, or naively combine the results from them in a loosely coupled way. In this paper, we present SOF-SLAM: Semantic Optical Flow SLAM, a visual semantic SLAM system toward dynamic environments, which is built on RGB-D mode of ORB-SLAM2. A new dynamic features detection approach called semantic optical flow is proposed, which is a kind of tightly coupled way and can fully take advantage of feature's dynamic characteristic hidden in semantic and geometry information to remove dynamic features effectively and reasonably. The pixel-wise semantic segmentation results generated by SegNet serve as mask in the proposed semantic optical flow to get a reliable fundamental matrix, which is then used to filter out the truly dynamic features. Only the remaining static features are reserved in the tracking and optimization module to achieve accurate camera pose estimation in dynamic environments. Experiments on public TUM RGB-D dataset and in real-world environment are conducted. Compared with ORB-SLAM2, the proposed SOF-SLAM achieves averagely 96.73% improvements in high-dynamic scenarios. It also outperforms the other four state-of-the-art SLAM systems which cope with the dynamic environments.