Estimation Of Pose Parameters Research Articles

Multi-modal image matching to colorize a SLAM based point cloud with arbitrary data from a thermal camera

Thermal mapping of buildings can be one approach to assess the insulation, which is important in regard to upgrade buildings to increase energy efficiency and for climate change adaptation. Personal laser scanning (PLS) is a fast and flexible option that has become increasingly popular to efficiently map building facades. However, some measurement systems do not include sufficient colorization of the point cloud. In order to detect, map and reference any damages to building facades, it is of great interest to transfer images from RGB and thermal infrared (TIR) cameras to the point cloud. This study aims to answer the research question if a flexible tool can be developed, which enable such measurements with high spatial resolution and flexibility. Therefore, an image-to-geometry registration approach for rendered point clouds is combined with a deep learning (DL)-based image feature matcher to estimate the camera pose of arbitrary images in relation to the geometry, i.e. the point cloud, to map color information. We developed a research design for multi-modal image matching to investigate the alignment of RGB and TIR camera images to a PLS point cloud with intensity information using calibrated and un-calibrated images. The accuracies of the estimated pose parameters reveal the best performance of the registration for pre-calibrated, i.e. undistorted, RGB camera images. The alignment of un-calibrated RGB and TIR camera images to a point cloud is possible if sufficient and well-distributed 2D-3D feature matches between image and point cloud are available. Our workflow enables the colorization of point clouds with high accuracy using images with very different radiometric characteristics and image resolutions. Only a rough approximation of the camera pose is required and hence the approach reliefs strict sensor synchronization requirements.

APPLICATION OF RGB-D SLAM IN 3D TUNNEL RECONSTRUCTION BASED ON SUPERPIXEL AIDED FEATURE TRACKING

Abstract. In large-scale projects such as hydropower and transportation, the real-time acquisition and generation of the 3D tunnel model can provide an important basis for the analysis and evaluation of the tunnel stability. The Simultaneous Localization And Mapping (SLAM) technology has the advantages of low cost and strong real-time, which can greatly improve the data acquisition efficiency during tunnel excavation. Feature tracking and matching are critical processes of traditional 3D reconstruction technologies such as Structure from Motion (SfM) and SLAM. However, the complicated rock mass structures on the tunnel surface and the limited lighting environment make feature tracking and matching difficult. Manhattan SLAM is a technology integrating superpixels and Manhattan world assumptions, in which both line features and planar features can be better extracted. Rock mass discontinuities including traces and structural planes are distributed on the inner surface of tunnels, which can be extracted for feature tracking and matching. Therefore, this paper proposes a 3D reconstruction pipeline for tunnels, in which, the Manhattan SLAM algorithm is applied for camera pose parameters estimation and the sparse point cloud generation, and the Patch-based Multi-View Stereo (PMVS) is adopted for dense reconstruction. In this paper, the Azure Kinect DK sensor is used for data acquisition. Experiments are proceeded and the results show that the proposed pipeline based on Manhattan SLAM and PMVS performs good robustness and feasibility for tunnels 3D reconstruction.

Genetic algorithms and bundle adjustment for the enhancement of 3D reconstruction

In this paper, we present a new technique of tridimensional reconstruction from a sequence of uncalibrated stereo images taken with cameras having varying parameters. At first, our system allows to recover initial coordinates of a set of 3D points. In this context, we have used our method of self-calibration based on the use of unknown 3D scene with its image projections and genetic algorithms to estimate all intrinsic parameters. After that extrinsic parameters are estimated based on classical pose estimation algorithms. Matching points and estimated value of intrinsic and extrinsic parameters are used to estimate initial 3D model that helps us in the initialization step. In order to have a reliable and relevant 3D reconstruction the proposed method is based on good and new exploitation of bundle adjustment (without camera poses initialization) technique based on Levenberg-Marquardt optimization with the aim to estimate our optimal 3D model that has special features compared to the classical case because it masks the pose parameters estimation in the optimization process. Finally, 3D mesh of the 3D scene is constructed with Delaunay algorithm and the 2D image is projected on the 3D model to generate the texture mapping. Experiments is conducted on real data to achieve demonstrate the validity and the performance of the proposed approach in terms of convergence, simplicity, stability and reconstruction quality.

Graph convolutional neural network for multi-scale feature learning

Automatic deformable 3D modeling is computationally expensive, especially when considering complex position, orientation and scale variations. We present a volume segmentation framework to utilize local and global regularizations in a data-driven approach. We introduce automated correspondence search to avoid manually labeling landmarks and improve scalability. We propose a novel marginal space learning technique, utilizing multi-resolution pooling to obtain local and contextual features without training numerous detectors or excessively dense patches. Unlike conventional convolutional neural network operators, graph-based operators allow spatially related features to be learned on the irregular domain of the multi-resolution space, and a graph-based convolutional neural network is proposed to learn representations for position and orientation classification. The graph-CNN classifiers are used within a marginal space learning framework to provide efficient and accurate shape pose parameter hypothesis prediction. During segmentation, a global constraint is initially non-iteratively applied, with local and geometric constraints applied iteratively for refinement. Comparison is provided against both classical deformable models and state-of-the-art techniques in the complex problem domain of segmenting aortic root structure from computerized tomography scans. The proposed method shows improvement in both pose parameter estimation and segmentation performance.

Annotated face model-based alignment: a robust landmark-free pose estimation approach for 3D model registration

Registering a 3D facial model onto a 2D image is important for constructing pixel-wise correspondences between different facial images. The registration is based on a 3 $$\times $$ 4 dimensional projection matrix, which is obtained from pose estimation. Conventional pose estimation approaches employ facial landmarks to determine the coefficients inside the projection matrix and are sensitive to missing or incorrect landmarks. In this paper, a landmark-free pose estimation method is presented. The method can be used to estimate the matrix when facial landmarks are not available. Experimental results show that the proposed method outperforms several landmark-free pose estimation methods and achieves competitive accuracy in terms of estimating pose parameters. The method is also demonstrated to be effective as part of a 3D-aided face recognition pipeline (UR2D), whose rank-1 identification rate is competitive to the methods that use landmarks to estimate head pose.

Image Pose Estimation Based on Procrustes Theory

This paper has established a high-precision hierarchical estimated pose parameters of image. Firstly, we select corresponding three image points of 3D points which constitute the largest area in image as a base, in order to estimate the depth and translate information; then based on the above method, we obtain the scale parameter of camera exterior information. And finally, the topic is transformed to a problem of estimating rotation relationship by vector, using Procrustes theory to obtain the best estimate of the angle elements of exterior parameters. The method can effectively solve problems which depth and coupling pose parameters cannot deal with. Experimental results show that this method of determining position and orientation parameter estimation model is of briefness, easy convergence and it can also achieve higher parameter estimation accuracy than the direct projection matrix factorization.

Realtime Reconstruction of an Animating Human Body from a Single Depth Camera.

We present a method for realtime reconstruction of an animating human body,which produces a sequence of deforming meshes representing a given performance captured by a single commodity depth camera. We achieve realtime single-view mesh completion by enhancing the parameterized SCAPE model.Our method, which we call Realtime SCAPE, performs full-body reconstruction without the use of markers.In Realtime SCAPE, estimations of body shape parameters and pose parameters, needed for reconstruction, are decoupled. Intrinsic body shape is first precomputed for a given subject, by determining shape parameters with the aid of a body shape database. Subsequently, per-frame pose parameter estimation is performed by means of linear blending skinning (LBS); the problem is decomposed into separately finding skinning weights and transformations. The skinning weights are also determined offline from the body shape database,reducing online reconstruction to simply finding the transformations in LBS. Doing so is formulated as a linear variational problem;carefully designed constraints are used to impose temporal coherence and alleviate artifacts. Experiments demonstrate that our method can produce full-body mesh sequences with high fidelity.

High-precision and real-time algorithms of multi-object detection, recognition and localization toward ARVD of cooperative spacecrafts

The spacecraft pose estimation based on vision measurement is widely used as an effective method for proximity operation in autonomous rendezvous and docking (ARVD). In order to estimate pose parameters more effectively, it is necessary to achieve high-precision object localization in video images. However, there are some difficulties to satisfy precision and real-time requirements simultaneously for the localization of multiple objects. In this paper, high-precision and real-time algorithms are proposed for multi-object detection, recognition and localization toward ARVD of cooperative spacecrafts. At first, through the denoise and characteristic analysis of video images, an object detection method is proposed based on adaptive threshold segmentation by threshold optimization with the variances of inter-classes and intra-class; then a recognition algorithm of multi-object is proposed by self-organization clustering; moreover, a high-precision localization algorithm of multi-object is proposed based on bilinear interpolation, which can achieve sub-pixel precision for object centroid estimation. A series of experimental results demonstrate that the proposed algorithms can achieve excellent object detection accuracy of 100% within 120m, high-precision object localization of 0.004–0.43pixel at 0.3–150m distance, and is also characterized with strong anti-disturbance and well real-time of 22ms average runtime so as to make a sound basis for further engineering tests.

Vision‐based isolated translations and rotations estimation for inverted robot approaching a target

PurposeVision‐based inverted robot control exhibits a complex and a multi‐parameter estimation task. Compromises over speed and accuracy must be made to reduce the cost of the system, if high profile techniques are not utilized. The purpose of this paper is to present such a technique where many parameter estimation problems admit partitioning process. The process eliminates the complexity by dividing such a high dimension task into several reduced dimension problems. The partition procedure is defined by human understanding of the task. Here a mechanical setup is designed that handles the pose parameters estimations as a set of sub‐problems. The estimated pose parameters are applied to an inverted robot approaching a target on the floor.Design/methodology/approachA novel technique that helps the inverted robot to approach its target is detailed here. The new methodology is based upon the modified versions of existing and proven techniques of pure translations and rotations estimations. Providing valid conditions, the parameters of the two vectors of the camera's pose are isolated. In the first pass, the roll angle is adjusted. A subsequent pass uses modified 2‐point algorithm to estimate partial translational vector. Introducing 4‐point algorithm; an extension of pure rotational vector estimation technique, to estimate partial rotational vector. Lastly, visual depth is estimated to complete the task. For simplicity, the robot dynamics are not detailed here. It is assumed that the robot can possibly achieve any position if the desired pose parameters are known.FindingsIt is found that the isolated vectors estimation process reduces the complexity of the system and so reduces the computational cost and processing time. The proposed technique is applied to a prototype inverted Cartesian robot having 3D rotary wrist. Through analysis, it is observed that the estimated parameters are very close to the actual pose parameters.Practical implicationsThe proposed technique can aid CNC vertical milling machines to countercheck the exact position and orientation of the tool w.r.t. the job. Similar systems in practice are DECKEL MAHO, DMU 60/80/100, 5‐axis package and DMC 35 V series.Originality/valueThe dynamic nature of the method proposed in the paper makes it more efficacious for mechanical/robotic systems with vertically downward tool under gravity effect.

Contour-based iterative pose estimation of 3D rigid object

Estimating pose parameters of a 3D rigid object based on a 2D monocular image is a fundamental problem in computer vision. State-of-the-art methods usually assume that certain feature correspondences are available a priori between the input image and object's 3D model. This presumption makes the problem more algebraically tractable. However, when there is no feature correspondence available a priori, how to estimate the pose of a truly 3D object using just one 2D monocular image is still not well solved. In this article, a new contour-based method which solves both the pose estimation problem and the feature correspondence problem simultaneously and iteratively is proposed. The outer contour of the object is firstly extracted from the input 2D grey-level image, then a tentative point correspondence relationship is established between the extracted contour and object's 3D model, based on which object's pose parameters will be estimated; the newly estimated pose parameters are then used to revise the tentative point correspondence relationship, and the process is iterated until convergence. Experiment results are promising, showing that the authors' method has fast convergence speed and good convergence radius.

Dynamic calibration of the relative pose and error analysis in a structured light system

We investigate the problem of dynamic calibration for our structured light system. First, a method is presented to estimate the rotation matrix and translation vector between the camera and the projector using plane-based homography. Then an approach is introduced to analyze theoretically the error sensitivity in the estimated pose parameters with respect to noise in the projection points. This algorithm is simple and easy to implement. Finally, some numerical simulations and real data experiments are carried out to validate our method.

Closed-form solution for circle pose estimation using binocular stereo vision

A simple method for pose estimation of a circle, with known circle radius, on the basis of binocular stereo vision is presented. The method is based on the projective equation of a circle, into which the stereo vision constraints are incorporated. The estimated pose parameters are compared with the ground truth and excellent performance is reported.

POSE ESTIMATION IN AUTOMATED VISUAL INSPECTION USING GENETIC ALGORITHM

In this paper, we propose a genetic algorithm based approach to determine the pose of an object in Automated Visual Inspection having three degrees of freedom. We have investigated the effect of noise at 20 dB SNR and also mismatch resulting from incorrect correspondences between the object space points and the image space points, on the estimation of pose parameters. The maximum error in translation parameters is less than 0.45 cm and rotational error is less than 0.2 degree at 20 dB SNR. The error in parameter estimation is insignificant upto 7 pairs of mismatched points out of 24 points in object space and the results skyrockets when 8 or more pairs of points are mismatched. We have compared our result with that obtained by least square technique and it shows that GA based method outperform the gradient based technique when the number of vertices of the object to be inspected is small. These results have clearly established the robustness of GA in estimating the pose of an object with small number of vertices in automated visual inspection.

Combining 3D flow fields with silhouette-based human motion capture for immersive video

In recent years, the convergence of computer vision and computer graphics has put forth a new field of research that focuses on the reconstruction of real-world scenes from video streams. To make immersive 3D video reality, the whole pipeline spanning from scene acquisition over 3D video reconstruction to real-time rendering needs to be researched. In this paper, we describe latest advancements of our system to record, reconstruct and render free-viewpoint videos of human actors. We apply a silhouette-based non-intrusive motion capture algorithm making use of a 3D human body model to estimate the actor’s parameters of motion from multi-view video streams. A renderer plays back the acquired motion sequence in real-time from any arbitrary perspective. Photo-realistic physical appearance of the moving actor is obtained by generating time-varying multi-view textures from video. This work shows how the motion capture sub-system can be enhanced by incorporating texture information from the input video streams into the tracking process. 3D motion fields are reconstructed from optical flow that are used in combination with silhouette matching to estimate pose parameters. We demonstrate that a high visual quality can be achieved with the proposed approach and validate the enhancements caused by the the motion field step.

Probabilistic Modeling and Recognition of 3-D Objects

This paper introduces a uniform statistical framework for both 3-D and 2-D object recognition using intensity images as input data. The theoretical part provides a mathematical tool for stochastic modeling. The algorithmic part introduces methods for automatic model generation, localization, and recognition of objects. 2-D images are used for learning the statistical appearance of 3-D objectss both the depth information and the matching between image and model features are missing for model generation. The implied incomplete data estimation problem is solved by the Expectation Maximization algorithm. This leads to a novel class of algorithms for automatic model generation from projections. The estimation of pose parameters corresponds to a non-linear maximum likelihood estimation problem which is solved by a global optimization procedure. Classification is done by the Bayesian decision rule. This work includes the experimental evaluation of the various facets of the presented approach. An empirical evaluation of learning algorithms and the comparison of different pose estimation algorithms show the feasibility of the proposed probabilistic framework.

Class-based recognition of 3D objects represented by volumetric primitives

This paper presents a novel approach to recognizing 3D complex objects that have similar geometric structure but belong to different subclasses. Test scenes are acquired by a laser striper as range images and the objects are modelled using a composite volumetric representation of superquadrics and geons. Matching is decomposed into two stages: first, an indexing scheme designed to make effective use of the symbolic keys of the representation is implemented in order to direct the search to the models containing the parts identified; second, a method is proposed where the hypotheses picked from the index are searched using an Interpretation Tree algorithm combined with a quality measure to evaluate the bindings and the final valid hypotheses based on Possibility Theory, or the Theory of Fuzzy Sets. The valid hypotheses ranked by the matching process are then passed to the pose estimation module. From the results shown here, we can conclude that shape and pose parameter estimation from range data is sufficiently accurate to allow the algorithms used here to discriminate between objects having the same general topology, but different specific shapes.

A quaternion solution to the pose determination problem for rendezvous and docking simulations

This paper presents the mathematical framework for the attitude and position estimation of an object from its image using three point correspondence information. A closed-form solution for the pose determination problem is derived by representing the relative attitude parameters using quaternions. The quaternion elements can be used to model the outputs of a vision sensor in attitude control simulations for the final approach phase of a rendezvous and docking mission. The algorithm has been validated with real images of a scaled model of the target features in a vision system test facility. Simulation results showing the comparison between the actual and estimated pose parameters are included.