Sensor Pose Research Articles

Distortion correction for the orthogonally-splitting-imaging pose sensor

Abstract The orthogonally-splitting-imaging pose sensor utilizes not only large field of view spherical lenses but also two sets of cylindrical ones to realize the high-speed, high-precision and wide-field pose measurement. Notable distortion, however, results from the wide-field lenses at the same time. Therefore, to obtain the best performance of the camera model, a distortion correction method is proposed in this paper, which combines the advantages of the high-stability of the Least Square fittings based on the orthogonal polynomials and the independence of the distortion correction based on the cross-ratio invariability. In this way, the ill-conditioned fitting matrix as well as the iteration and optimization procedures in solving extrinsic and intrinsic parameters can be avoided. Due to the wide fitness of the cross-ratio invariability and the orthogonal polynomials, this distortion correction technique is also suitable to other optical set up with different imaging structure. The experiment results that the corrected grids have superior precision and reliability with their original slopes demonstrate that the distortion model on the basis of orthogonal polynomial is validated and that the distortion correction is effective.

Three-dimensional integral imaging with flexible sensing.

We present to the best of our knowledge the first report on three-dimensional (3D) integral imaging capture and reconstruction method with unknown poses of sensors placed on a flexible surface. Compared to a conventional integral imaging system, where a lenslet or sensor array is commonly located on a planar surface, the flexible sensing integral imaging system allows sensors to be placed on a nonplanar surface that can increase the field of view of the 3D imaging system. To obtain the poses of the sensor array on a flexible surface, an estimation algorithm is developed based on two-view geometry theory and the camera projective model. In addition, a super-resolution image is generated from a sequence of low-resolution 2D images with sub-pixel shifts. Super-resolution 3D reconstruction results at different depths are presented to validate the proposed approach.

Structural Optimization of a Novel 6-DOF Pose Sensor System for Enhancing Noise Robustness at a Long Distance

In this paper, structural optimization of a sensor fusion system is newly presented to minimize noise sensitivity, thus maximizing the noise robustness in measuring 6-degree-of-freedom (6-DOF) poses of an object at a long distance with higher precision. The target sensor system, the so-called three-beam detector, is composed of three 1-D laser sensors and a vision camera. During the nonlinear computation, 6-DOF poses can be affected by sensor noises along with sensor configuration. Design parameters such as alignment angles, width, and height of the laser sensors are optimized in the sensor configuration that can minimize an objective function of the 6-DOF pose errors due to the sensor noise. To show sensing performance against the sensor noises, the sensing precision of the optimally configured system is compared with that of the parallel configuration system through numerical simulations and experiments. The proposed process of optimizing sensor configuration can be also applied to other sensor fusion systems to minimize the noise sensitivity.

Electromagnetic tracking in surgical and interventional environments: usability study.

Electromagnetic (EM) tracking of instruments within a clinical setting is notorious for fluctuating measurement performance. Position location measurement uncertainty of an EM system was characterized in various environments, including control, clinical, cone beam computed tomography (CBCT), and CT scanner environments. Static and dynamic effects of CBCT and CT scanning on EM tracking were evaluated. Two guidance devices were designed to solely translate or rotate the sensor in a non-interfering fit to decouple pose-dependent tracking uncertainties. These devices were mounted on a base to allow consistent and repeatable tests when changing environments. Using this method, position and orientation measurement accuracies, precision, and 95 % confidence intervals were assessed. The tracking performance varied significantly as a function of the environment-especially within the CBCT and CT scanners-and sensor pose. In fact, at a fixed sensor position in the clinical environment, the measurement error varied from 0.2 to 2.2 mm depending on sensor orientations. Improved accuracies were observed along the vertical axis of the field generator. Calibration of the measurements improved tracking performance in the CT environment by 50-85 %. EM tracking can provide effective assistance to surgeons or interventional radiologists during procedures performed in a clinical or CBCT environment. Applications in the CT scanner demand precalibration to provide acceptable performance.

Context-based automatic reconstruction and texturing of 3D urban terrain for quick-response tasks

Highly detailed 3D urban terrain models are the base for quick response tasks with indispensable human participation, e.g., disaster management. Thus, it is important to automate and accelerate the process of urban terrain modeling from sensor data such that the resulting 3D model is semantic, compact, recognizable, and easily usable for training and simulation purposes. To provide essential geometric attributes, buildings and trees must be identified among elevated objects in digital surface models. After building ground-plan estimation and roof details analysis, images from oblique airborne imagery are used to cover building faces with up-to-date texture thus achieving a better recognizability of the model. The three steps of the texturing procedure are sensor pose estimation, assessment of polygons projected into the images, and texture synthesis. Free geographic data, providing additional information about streets, forest areas, and other topographic object types, suppress false alarms and enrich the reconstruction results.

Indoor Autonomous Navigation Technology Research Based on Lidar for Quad-Copter Aerial Robot

An indoor autonomous navigation system without GPS has been developed, based on the platform of quad-copter, which may fulfill the task of searching, identifying and entering the target room in a building with multi-rooms corridor. A pose sensor was utilized to stabilize the aircraft. The SLAM (Simultaneous Localization and Mapping) and plan route in unknown environment have been created by a 2D lidar. A calibrated monocular camera has been used to recognize different marks to make sure the vehicle to enter the target room. The test result showed that the indoor autonomous navigation technology based on lidar for quad-copter aerial robot is feasible and successful.

Hand-held 3D scanner for surface-shape measurement without sensor pose tracking or surface markers

Current hand-held three-dimensional (3D) scanners for surface-shape measurement require sensor-tracking devices or surface markers to determine the position and orientation (pose) of the sensor at different viewpoints, thus limiting their functionality. This paper presents a hand-held 3D scanner without sensor tracking or surface markers. Multiple planes of light are simultaneously projected onto an object surface to provide sufficient surface information to enable surface fitting and fine interpolation between acquired data points. In surface scanning, the hand-held scanner acquires a sequence of dense overlapping range-images from different viewpoints with enough overlap between range views and abundant information within each range view to permit registration of all acquired data into a single 3D model. Calibration is performed by closed-form surface-fitting to map 2D image coordinates to 3D calibration-object coordinates. Conical diffraction adjustment corrects for projected-line curvature. The measurement accuracy was suitable for applications such as reverse engineering, and virtual and physical prototyping.

Fusion of Depth and Color Images for Dense Simultaneous Localization and Mapping

This paper presents a system for performing dense mapping of surface geometry and texture properties. Co-registered depth and grayscale images are incrementally fused into a global map in real-time as they are collected. The resulting dense map can be rendered into a virtual depth and grayscale image from any arbitrary pose. Comparison of the rendered and observed images provides a direct means of computing the sensor pose relative to the map allowing new data to be fused into the model. This frame-to-map tracking scheme, as opposed to frame-toframe tracking, improves system accuracy and robustness. Additionally, the use of both surface geometry and color texture better constrains the pose solution and reduces the risk of tracking failures. This paper describes an implementation of the proposed algorithm and provides experimental results.

Generation and weighting of 3D point correspondences for improved registration of RGB-D data

Abstract. Registration of RGB-D data using visual features is often influenced by errors in the transformation of visual features to 3D space as well as the random error of individual 3D points. In a long sequence, these errors accumulate and lead to inaccurate and deformed point clouds, particularly in situations where loop closing is not feasible. We present an epipolar search method for accurate transformation of the keypoints from 2D to 3D space, and define weights for the 3D points based on the theoretical random error of depth measurements. Our results show that the epipolar search method results in more accurate 3D correspondences. We also demonstrate that weighting the 3D points improves the accuracy of sensor pose estimates along the trajectory.

Sensitivity analysis of pose recovery from multi-center panoramas

A set of multi-view panoramas may consist of various types of panoramic images in cylindrical representation, such as single-center, multi-center, concentric, symmetric, or (after a transformation onto a cylinder) catadioptric panoramas. In comparison with single-center imaging models, there are fewer studies on multiple view geometry for the multi-center cases. A generalized epipolar curve equation has been derived in a book publication in 2008. This article extends such result and presents a cost function whose minimization solves the camera pose estimation problem. Due to the non-linearity of the multi-centered projection geometry, the modeling of sensor pose estimation typically results into non-linear and highly complicated forms which incur numerical instability. This article focuses on evaluating a method for solving the pose estimation problem under a minor geometrical constraint, namely leveled panoramas. Extensive synthetic and real experiments along with a formal sensitivity analysis are carried out to demonstrate the robustness of the proposed method.

UAV Pose Posterior Estimation with Vision Constraint

To estimate UAV(unmanned aerial vehicle) pose relative to airfield runway,a posterior parameter estimation method using the runway image and onboard pose sensors is presented.Firstly,pose parameters of UAV are measured by onboard sensors such as altimeter,GPS(global positioning system) and INS(inertial navigation system).Supposing the prior probability distribution of each parameter's measurement error is given and all errors are independent,the measured values from sensors can be considered as a sample of real ones.At the same time of sampling,an image of the runway is acquired from onboard calibrated camera,and the two edge lines of the runway are assumed to be extracted exactly from the image. Subject to the constraint of the perspective view of two parallel lines,the real parameters must locate on a constraint surface in the parameter space,and thereby the posterior mean estimation of the parameters can be obtained.Simulation results show that the method is effective.

Wide-angle vision for road views

AbstractThe field-of-view of a wide-angle image is greater than (say) 90 degrees, and so contains more information than available in a standard image. A wide field-of-view is more advantageous than standard input for understanding the geometry of 3D scenes, and for estimating the poses of panoramic sensors within such scenes. Thus, wide-angle imaging sensors and methodologies are commonly used in various road-safety, street surveillance, street virtual touring, or street 3D modelling applications. The paper reviews related wide-angle vision technologies by focusing on mathematical issues rather than on hardware.

Palmprint Verification by Scene Matching Approaches

This paper presents two novel approaches for palmprint verification by scene match- ing. In a sense, the process of palmprint verification can be regarded as matching one palmprint template with another palmprint sample taken under conditions of different illumination, sensor pose geometry, or types of sensor, and making a decision of whether they belong to the same person or not. Enlightened by the idea of scene matching algorithms, two efficient approaches by scene matching are presented for palmprint verification. A sub-image from the user’s training palmprint is intentionally extracted, and stored as a template. In the verification stage, the sample image is directly matched with the template without the step of ROI (Region of Interest) extraction. Whether the sample image and the template are from the same person or not is decided by their matching scores. Experimental evaluation results on two databases clearly demonstrate the effectiveness of the proposed approaches.

Multiple-Frame Subpixel Wildfire Tracking

We present a method to improve subpixel signal detection in airborne or orbital image sequences. The proposed technique recognizes stable interest point features in multiple overlapping frames. It estimates motion between consecutive frames and tracks candidate detections over time. The final detection decision combines signal strengths from multiple views to improve sensitivity. The algorithm is computationally tractable for real-time use on autonomous robotic platforms and spacecraft. Additionally, the interest points enable image-relative localization, obviating the need to transmit the entire image and reducing transmission bandwidth requirements by one or more orders of magnitude. This permits higher acquisition rates and potentially improved coverage for remote monitoring. Ground-based systems can reconstruct absolute positions from landmarks without measurements of sensor pose. We demonstrate the algorithm using airborne 4- μm imagery from multiple overpasses of a controlled wildfire.

A New Framework for Stereo Sensor Pose Through Road Segmentation and Registration

This paper proposes a new framework for real-time estimation of the onboard stereo head's position and orientation relative to the road surface, which is required for any advanced driver-assistance application. This framework can be used with all road types: highways, urban, etc. Unlike existing works that rely on feature extraction in either the image domain or 3-D space, we propose a framework that directly estimates the unknown parameters from the stream of stereo pairs' brightness. The proposed approach consists of two stages that are invoked for every stereo frame. The first stage segments the road region in one monocular view. The second stage estimates the camera pose using a featureless registration between the segmented monocular road region and the other view in the stereo pair. This paper has two main contributions. The first contribution combines a road segmentation algorithm with a registration technique to estimate the online stereo camera pose. The second contribution solves the registration using a featureless method, which is carried out using two different optimization techniques: 1) the differential evolution algorithm and 2) the Levenberg-Marquardt (LM) algorithm. We provide experiments and evaluations of performance. The results presented show the validity of our proposed framework.

A Fast Slam Approach to Freehand 3-D Ultrasound Reconstruction for Catheter Ablation Guidance in the Left Atrium

We present a method for real-time, freehand 3D ultrasound (3D-US) reconstruction of moving anatomy, with specific application towards guiding the catheter ablation procedure in the left atrium. Using an intracardiac echo (ICE) catheter with a pose (position/orientation) sensor mounted to its tip, we continually mosaic 2D-ICE images of a left atrium phantom model to form a 3D-US volume. Our mosaicing strategy employs a probabilistic framework based on simultaneous localization and mapping (SLAM), a technique commonly used in mobile robotics for creating maps of unexplored environments. The measured ICE catheter tip pose provides an initial estimate for compounding 2D-ICE image data into the 3D-US volume. However, we simultaneously consider the overlap-consistency shared between 2D-ICE images and the 3D-US volume, computing a “corrected” tip pose if need be to ensure spatially-consistent reconstruction. This allows us to compensate for anatomic movement and sensor drift that would otherwise cause motion artifacts in the 3D-US volume. Our approach incorporates 2D-ICE data immediately after acquisition, allowing us to continuously update the registration parameters linking sensor coordinates to 3D-US coordinates. This, in turn, enables real-time localization and display of sensorized therapeutic catheters within the 3D-US volume for facilitating procedural guidance.

Active vision in robotic systems: A survey of recent developments

In this paper we provide a broad survey of developments in active vision in robotic applications over the last 15 years. With increasing demand for robotic automation, research in this area has received much attention. Among the many factors that can be attributed to a high-performance robotic system, the planned sensing or acquisition of perceptions on the operating environment is a crucial component. The aim of sensor planning is to determine the pose and settings of vision sensors for undertaking a vision-based task that usually requires obtaining multiple views of the object to be manipulated. Planning for robot vision is a complex problem for an active system due to its sensing uncertainty and environmental uncertainty. This paper describes such problems arising from many applications, e.g. object recognition and modeling, site reconstruction and inspection, surveillance, tracking and search, as well as robotic manipulation and assembly, localization and mapping, navigation and exploration. A bundle of solutions and methods have been proposed to solve these problems in the past. They are summarized in this review while enabling readers to easily refer solution methods for practical applications. Representative contributions, their evaluations, analyses, and future research trends are also addressed in an abstract level.

Automatic Relocalization and Loop Closing for Real-Time Monocular SLAM

Monocular SLAM has the potential to turn inexpensive cameras into powerful pose sensors for applications such as robotics and augmented reality. We present a relocalization module for such systems which solves some of the problems encountered by previous monocular SLAM systems--tracking failure, map merging, and loop closure detection. This module extends recent advances in keypoint recognition to determine the camera pose relative to the landmarks within a single frame time of 33 ms. We first show how this module can be used to improve the robustness of these systems. Blur, sudden motion, and occlusion can all cause tracking to fail, leading to a corrupted map. Using the relocalization module, the system can automatically detect and recover from tracking failure while preserving map integrity. Extensive tests show that the system can then reliably generate maps for long sequences even in the presence of frequent tracking failure. We then show that the relocalization module can be used to recognize overlap in maps, i.e., when the camera has returned to a previously mapped area. Having established an overlap, we determine the relative pose of the maps using trajectory alignment so that independent maps can be merged and loop closure events can be recognized. The system combining all of these abilities is able to map larger environments and for significantly longer periods than previous systems.

The Study & Application on Shipborne Satellite Antenna Automatic Tracking System

For the shipborne satellite antenna system，owing to the continuous change of the carrier’s position and pose as the time passing, we must conduct real-time measurement on the pose of satellite antenna, so that the satellite can be aimed through servo control system. In this paper, the hardware and software about the shipborne satellite antenna system are introduced. Three-dimensional closed-loop servo control system which is made of three kinds of pose sensors is implemented to isolate the links between the antenna pose and the carrier’s position and pose. FSK serial communication is adopted to realize a reliable control commands and data transmission between the control unit above the deck and the control unit under the deck. Conical spiral scan based on the best AGC signal realizes automatic tracking satellite and locking it. The result of the examination shows that the pointing of antenna is of steadiness, precision(the point precision is less than 0.3 degrees), real time and speediness(the capture time is less than 25 seconds ).

Vehicle Localization in Outdoor Mountainous Forested Paths and Extension of Two-Dimensional Road Centerline Maps to Three-Dimensional Maps

This paper presents an approach for vehicle three-dimensional (3-D) localization in outdoor woodland environments where a previously available two-dimensional road centerline map is used in combination with a loosely coupled multi-sensor system to estimate the vehicle position in mountainous forested paths. The localization system is composed of a wheel encoder, an inertial measurement unit, a DGPS, a laser sensor and a barometer. An extended Kalman filter is used for sensor data fusion and pose estimation. When available, DGPS is used for 3-D dead reckoning accumulated error correction. During DGPS blackouts, the laser sensor is used for road extraction and measurement of the displacement of the vehicle to the road centerline, then the position is corrected towards the map. Moreover, the barometer that measures the height difference towards a reference is used to correct the estimated height in absence of DGPS 3-D data. The estimated height is added to the available road map to obtain a 3-D road centerline map that includes the road width measured with the laser sensor. Experimental results in large-scale real mountainous woodland environments show the robustness and simplicity of the proposed approach for vehicle localization and 3-D map extension.