Relative Orientation Parameters Research Articles

Geometric model and assessment of a dual‐fisheye imaging system

AbstractThis paper presents a technique for the calibration of a 360° dual imaging system (Ricoh Theta S) composed of two cameras with fisheye lenses specially mounted in a common structure. The experiments are based on calibration with a bundle adjustment using the equidistant fisheye lens model and relative orientation stability constraints, such as distance, base elements and rotation matrix between individual cameras’ coordinate systems. As a result, an equidistant mathematical model using the Conrady–Brown lens distortion model was shown to be suitable for estimating the interior and relative orientation parameters of the Ricoh Theta S. The main contribution presented in this work is a simultaneous bundle adjustment approach with stability constraints on base elements and rotation matrix angles, which achieved accuracies comparable with conventional photogrammetric calibration with perspective cameras.

Comprehensive Analysis of System Calibration between Optical Camera and Range Finder

This paper describes the comprehensive analysis of system calibration between an optical camera and a range finder. The results suggest guidelines for accurate and efficient system calibration enabling high-quality data fusion. First, self-calibration procedures were carried out using a testbed designed for both the optical camera and range finder. The interior orientation parameters of the utilized sensors were precisely computed. Afterwards, 92 system calibration experiments were carried out according to different approaches and data configurations. For comparison of the various experimental results, two measures, namely the matching rate of fusion data and the standard deviation of relative orientation parameters derived after system calibration procedures, were considered. Among the 92 experimental cases, the best result (the matching rate of 99.08%) was shown for the use of the one-step system calibration method and six datasets from multiple columns. Also, the root mean square values of the residuals after the self- and system calibrations were less than 0.8 and 0.6 pixels, respectively. In an overall evaluation, it was confirmed that the one-step system calibration method using four or more datasets provided more stable and accurate relative orientation parameters and data fusion results than the other cases.

Three-Dimensional Registration for Handheld Profiling Systems Based on Multiple Shot Structured Light.

In this article, a multi-view registration approach for the 3D handheld profiling system based on the multiple shot structured light technique is proposed. The multi-view registration approach is categorized into coarse registration and point cloud refinement using the iterative closest point (ICP) algorithm. Coarse registration of multiple point clouds was performed using relative orientation and translation parameters estimated via homography-based visual navigation. The proposed system was evaluated using an artificial human skull and a paper box object. For the quantitative evaluation of the accuracy of a single 3D scan, a paper box was reconstructed, and the mean errors in its height and breadth were found to be 9.4 μm and 23 μm, respectively. A comprehensive quantitative evaluation and comparison of proposed algorithm was performed with other variants of ICP. The root mean square error for the ICP algorithm to register a pair of point clouds of the skull object was also found to be less than 1 mm.

Modelling and automated calibration of a general multi‐projective camera

AbstractRecently, multi‐projective cameras (MPCs), often based on frame‐mounted multiple cameras with a small baseline and arbitrary overlap, have found a remarkable place in geomatics and vision‐based applications. This paper outlines the geometric calibration of a general MPC by presenting a mathematical model that describes its unknown generic geometry. A modified bundle block adjustment is employed to calibrate an industrial‐level 360° non‐metric camera. The structure of any MPC can be retrieved as a calibration set of relative and interior orientation parameters (as well as the pose of the MPC shots) using a calibration room which has been accurately determined by close range photogrammetry. To demonstrate the efficiency and precision of the model, a Panono camera (an MPC with 36 individual cameras) was calibrated. After the adjustment, sub‐pixel image residuals and acceptable object‐space errors were observed.

Practical In Situ Implementation of a Multicamera Multisystem Calibration

Consumer-grade cameras are generally low-cost and available off-the-shelf, so having multicamera photogrammetric systems for 3D reconstruction is both financially feasible and practical. Such systems can be deployed in many different types of applications: infrastructure health monitoring, cultural heritage documentation, bio-medicine, as-built surveys, and indoor or outdoor mobile mapping for example. A geometric system calibration is usually necessary before a data acquisition mission in order for the results to have optimal accuracy. A typical system calibration must address the estimation of both the interior and the exterior, or relative, orientation parameters for each camera in the system. This article reviews different ways of performing a calibration of a photogrammetric system consisting of multiple cameras. It then proposes a methodology for the simultaneous estimation of both the interior and the relative orientation parameters which can work in several different types of scenarios including a multicamera multisystem calibration. A rigorous in situ system calibration was successfully implemented and tested. The same algorithm is able to handle the equivalent to a traditional-style bundle adjustment, that is, a network solution without constraints, for a single or multicamera calibrations, and the proposed bundle adjustment with built-in relative orientation constraints for the calibration of a system or multiple systems of cameras.

PRECISION ANALYSIS OF VISUAL ODOMETRY BASED ON DISPARITY CHANGING

Abstract. This thesis aims to analyze the precision of Position and orientation of cameras on Mobile Mapping System (MMS) determined by disparity based VO (DBVO). Dual forwards taken cameras on MMS are applied to obtain a sequence of stereo pairs. The Interior Orientation Parameters (IOPs) and Relative Orientation Parameters (ROPs) are derived in advance. The pose estimation is achieved by DBVO without additional control data. The procedure of DBVO consists of four steps. First up, keypoint detection and matching is conducted to obtain tie points in consecutive images. Then, image rectification is implemented to transform tie points into epipolar image space. Next, parallax equation is applied to estimate the 3D coordinates of interest points in epipolar image 3D space. Since their image points have different disparity in neighboring stereo pairs, the 3D coordinates of interest points in neighboring pairs are different as well. Finally, 3D conformal transformation is employed to derive the transformation parameters between neighboring pairs according to changing of coordinates of interest points. The posteriori STDs are adopted to assess the quality of transformation. Besides, check data of ground trajectory derived by photo triangulation are applied to evaluate the result. The relative errors of horizontal and vertical translations derived by DBVO are 2 % and 3 % in non-viewing direction. However, the translation in viewing direction and three rotation angles derived by DBVO have significant systematic errors about 1 m, 3°, 3° and 10° respectively. The influence of error propagation is not significant according to the chart of error distance ratio. In open area, the trajectory of INS/GPS is similar to ground truth, while the trajectory derived by DBVO has 44 % relative error. In residential district, the trajectory derived by INS/GPS has drift error about 2 m, while the relative error of the trajectory derived by DBVO decreases to 38 %. It is presumed that the systematic error results from 3D coordinates estimated by parallax equation because of poor intersection geometry. It will be proved by adding sideward photographing cameras in the future.

GEOMETRIC STITCHING METHOD FOR DOUBLE CAMERAS WITH WEAK CONVERGENCE GEOMETRY

Abstract. In this paper, a new geometric stitching method is proposed which utilizes digital elevation model (DEM)-aided block adjustment to solve relative orientation parameters for dual-camera with weak convergence geometry. A rational function model (RFM) with affine transformation is chosen as the relative orientation model. To deal with the weak geometry, a reference DEM is used in this method as an additional constraint in the block adjustment, which only calculates the planimetry coordinates of tie points (TPs). After that we can use the obtained affine transform coefficients to generate virtual grid, and update rational polynomial coefficients (RPCs) to complete the geometric stitching. Our proposed method was tested on GaoFen-2(GF-2) dual-camera panchromatic (PAN) images. The test results show that the proposed method can achieve an accuracy of better than 0.5 pixel in planimetry and have a seamless visual effect. For regions with small relief, when global DEM with 1 km grid, SRTM with 90 m grid and ASTER GDEM V2 with 30 m grid replaced DEM with 1m grid as elevation constraint， it is almost no loss of accuracy. The test results proved the effectiveness and feasibility of the stitching method.

Geometric Calibration of an Aerial Multihead Camera System for Direct Georeferencing Applications

An aerial multihead camera consists of a photogrammetric system composed of multiple cameras, which are mounted together in a main structure. By combining the images acquired simultaneously from each camera, a single synthetic image with much larger coverage can be generated. Such systems are usually integrated with a position and orientation system (POS) to perform direct georeferencing (DG) or integrated sensor orientation (ISO). However, to obtain mapping products with high accuracy through a DG procedure, it is essential the implementation of the system geometric calibration. Usually, the aerial multihead camera manufacturers perform the geometric calibration using laboratory methods and only the camera interior orientation parameters (IOPs) and their relative orientation parameters (ROPs) are determined to generate the synthetic image (process known as “platform calibration”). The mounting parameters (lever arms and boresight misalignment angles) relating the synthetic image and GNSS/INS reference systems are usually defined using nominal installation values. The objective of this paper is to present an in-flight calibration methodology for multihead camera systems and its assessment for DG applications. The introduced methodology involves three steps: 1) determination of the cameras’ IOPs and their ROPs; 2) synthetic image generation; and 3) refinement of the IOPs of the synthetic image and the mounting parameters determination between the synthetic image and GNSS/INS reference systems using different methods. The results of the experiments shown the viability of the proposed methodology for DG applications involving photogrammetric procedures for large-scale mapping requirements.

Integrated Computational Approach to the Electron Paramagnetic Resonance Characterization of Rigid 310-Helical Peptides with TOAC Nitroxide Spin Labels.

We address the interpretation, via an integrated computational approach, of the experimental continuous-wave electron paramagnetic resonance (cw-EPR) spectra of a complete set of conformationally highly restricted, stable 310-helical peptides from hexa- to nonamers, each bis-labeled with nitroxide radical-containing TOAC (4-amino-1-oxyl-2,2,6,6-tetramethylpiperidine-4-carboxylic acid) residues. The usefulness of TOAC for this type of analysis has been shown already to be due to its cyclic piperidine side chain, which is rigidly connected to the peptide backbone α-carbon. The TOAC α-amino acids are separated by two, three, four, and five intervening residues. This set of compounds has allowed us to modulate both the radical···radical distance and the relative orientation parameters. To further validate our conclusion, a comparative analysis has been carried out on three singly TOAC-labeled peptides of similar main-chain length.

AUTOMATIC RECTIFICATION OF BUILDING FAÇADES

Abstract. Focusing mainly on the case of (near-)planar building façades, a methodology for their automatic projective rectification is described and evaluated. It relies on a suitably configured, calibrated stereo pair of an object expected to contain a minimum of vertical and/or horizontal lines for the purposes of levelling. The SURF operator has been used for extracting and matching interest points. The coplanar points have been separated with two alternative methods. First, the fundamental matrix of the stereo pair, computed using robust estimation, allows estimating the relative orientation of the calibrated pair; initial parameter values, if needed, may be estimated via the essential matrix. Intersection of valid points creates a 3D point set in model space, to which a plane is robustly fitted. Second, all initial point matches are directly used for robustly estimating the inter-image homography of the pair, thus directly selecting all image matches referring to coplanar points; initial values for the relative orientation parameters, if needed, may be estimated from a decomposition of the inter-image homography. Finally, all intersected coplanar model points yield the object-to-image homography to allow image rectification. The in-plane rotation required to finalize the transformation is found by assuming that rectified images contain sufficient straight linear segments to form a dominant pair of orthogonal directions which correspond to horizontality/verticality in 3D space. In our implementation, image edges from Canny detector are used in linear Hough Transform (HT) resulting in a 2D array (ρ, θ) with values equal to the sum of pixels belonging to the particular line. Quantization parameter values aim at absorbing possible slight deviations from collinearity due to thinning or uncorrected lens distortions. By first imposing a threshold expressing the minimum acceptable number of edge-characterized pixels, the resulting HT is accumulated along the ρ-dimension to give a single vector, whose values represent the number of lines of the particular direction. Since here the dominant pair of orthogonal directions has to be found, all vector values are added with their π/2-shifted counterpart. This function is then convolved with a 1D Gaussian function; the optimal angle of in-plane rotation is at the maximum value of the result. The described approach has been successfully evaluated with several building façades of varying morphology by assessing remaining line convergence (projectivity), skewness and deviations from horizontality/verticality. Mean estimated deviation from a metric result was 0°.2. Open questions are also discussed.

A stereo photographic physical model for ocean wave measurement

Unlike the traditional models, a new stereo photographic model requiring no sea control points that are difficult to lay on a sea surface is developed in this paper. It is realized according to the order of camera intrinsic parameters calibration, relative orientation and absolute orientation based on average sea surface, and we give the governing equations and its algorithms. First of all, in the paper we present an imaging model that adopts non-measurement camera with considering only the radial distortion coefficients, and then give the method that calculates the camera intrinsic parameters by combining the co-planar equations and the distance equations to form a closed system of equations. For verifying the convergence of the governing equations, we lay four parallel sand ridges on a level ground in laboratory, measure seven lines of different heights and take a pair of the images by two cameras. The intrinsic parameters are successfully calculated by using the method, and further analysis shows that the results are highly precise. When observing the ocean waves in offshore sea, we need to recalculate the relative orientation parameters. At this time the governing equations are the co-planar equations, but the accuracy of the image matching is seriously related to the accuracy of the relative orientation parameters. However, as the mirror reflection of the sea surface, the matching accuracy of the sea image is often low, so we select 407 optimal matching points from among 1600 conjugate points and calculate the parameters. It needs a sequence of sea image pairs to obtain the absolute orientation parameters, 256 pairs in the paper. For each pair, we select firstly 300300 feature points on the left image approximately evenly, then find the conjugate points on the right image by image matching, calculate the coordinates of the corresponding sea surface points in the left camera coordinate system, and then fit the points into a plane; thus a plane sequence can be obtained. We establish a special object coordinate system on the average sea surface that is obtained by summing the plane sequence, and calculate the absolute orientation parameters by the geometric relationship between the two coordinate systems. And it is proved that the methods of relative orientation and absolute orientation proposed in this paper are feasible by reconstructing sea surface height field. The model greatly reduces the difficulty in calibrating the stereoscopic photography for ocean wave measurement, which is beneficial to its popularization and application.

Performance Evaluation of Alternative Relative Orientation Procedures for UAV-based Imagery with Prior Flight Trajectory Information

Thanks to recent advances at the hardware (e.g., emergence of reliable platforms at low cost) and software (e.g., automated identification of conjugate points in overlapping images) levels, UAV-based 3D reconstruction has been widely used in various applications. However, mitigating the impact of outliers in automatically matched points in UAV imagery, especially when dealing with scenes that has poor and/or repetitive texture, remains to be a challenging task. In spite of the fact that existing literature has already demonstrated that incorporating prior motion information can play an important role in increasing the reliability of the matching process, there is a lack of methodologies that are mainly suited for UAV imagery. Assuming the availability of prior information regarding the trajectory of a UAV-platform, this paper presents a two-point approach for reliable estimation of Relative Orientation Parameters (ROPs) of UAV-based images. This approach is based on the assumption that the UAV platform is moving at a constant flying height while maintaining the camera in a nadir-looking orientation. For this flight scenario, a closed-form solution that can be derived using a minimum of two pairs of conjugate points is established. In order to evaluate the performance of the proposed approach, experimental tests using real stereo-pairs acquired from different UAV platforms have been conducted. The derived results from the comparative performance analysis against the Nistér five-point approach demonstrate that the proposed two-point approach is capable of providing reliable estimate of the ROPs from UAV-based imagery in the presence of poor and/or repetitive texture with high percentage of matching outliers.

Performance Evaluation of Alternative Relative Orientation Procedures for UAV-based Imagery with Prior Flight Trajectory Information

Abstract. Thanks to recent advances at the hardware (e.g., emergence of reliable platforms at low cost) and software (e.g., automated identification of conjugate points in overlapping images) levels, UAV-based 3D reconstruction has been widely used in various applications. However, mitigating the impact of outliers in automatically matched points in UAV imagery, especially when dealing with scenes that has poor and/or repetitive texture, remains to be a challenging task. In spite of the fact that existing literature has already demonstrated that incorporating prior motion information can play an important role in increasing the reliability of the matching process, there is a lack of methodologies that are mainly suited for UAV imagery. Assuming the availability of prior information regarding the trajectory of a UAV-platform, this paper presents a two-point approach for reliable estimation of Relative Orientation Parameters (ROPs) of UAV-based images. This approach is based on the assumption that the UAV platform is moving at a constant flying height while maintaining the camera in a nadir-looking orientation. For this flight scenario, a closed-form solution that can be derived using a minimum of two pairs of conjugate points is established. In order to evaluate the performance of the proposed approach, experimental tests using real stereo-pairs acquired from different UAV platforms have been conducted. The derived results from the comparative performance analysis against the Nistér five-point approach demonstrate that the proposed two-point approach is capable of providing reliable estimate of the ROPs from UAV-based imagery in the presence of poor and/or repetitive texture with high percentage of matching outliers.

Calibration of multi-camera photogrammetric systems

Abstract. Due to the low-cost and off-the-shelf availability of consumer grade cameras, multi-camera photogrammetric systems have become a popular means for 3D reconstruction. These systems can be used in a variety of applications such as infrastructure monitoring, cultural heritage documentation, biomedicine, mobile mapping, as-built architectural surveys, etc. In order to ensure that the required precision is met, a system calibration must be performed prior to the data collection campaign. This system calibration should be performed as efficiently as possible, because it may need to be completed many times. Multi-camera system calibration involves the estimation of the interior orientation parameters of each involved camera and the estimation of the relative orientation parameters among the cameras. This paper first reviews a method for multi-camera system calibration with built-in relative orientation constraints. A system stability analysis algorithm is then presented which can be used to assess different system calibration outcomes. The paper explores the required calibration configuration for a specific system in two situations: major calibration (when both the interior orientation parameters and relative orientation parameters are estimated), and minor calibration (when the interior orientation parameters are known a-priori and only the relative orientation parameters are estimated). In both situations, system calibration results are compared using the system stability analysis methodology.

POSE VERSUS STATE: ARE SENSOR POSITION AND ATTITUDE SUFFICIENT FOR MODERN PHOTOGRAMMETRY AND REMOTE SENSING?

Abstract. We investigate the advantages of using what we call sensor state parameters or sensor state to describe the geometrical relationship between a sensor and the 3D space or the 4D time-space that extend the traditional image pose or orientation (position and attitude) to the image state. In our concept, at some point in time t, the sensor state is a 12-dimensional vector composed of four 3-dimensional subvectors p(t), v(t), γ(t) and ω(t). Roughly speaking, p(t) is the sensor’s position, v(t) its linear velocity, γ(t) its attitude and ω(t) its angular velocity. It is clear that the state concept extends the pose or orientation ones and attempts to describe both a sensor’s statics (p(t), γ(t)) and dynamics (v(t), ω(t)). It is also clear that if p(t), γ(t) are known for all t within some time interval of interest, then v(t) and ω(t) can be derived from the former. We present three methods to compute the state parameters, two for the continuous case and one for the discrete case. The first two methods rely on the availability of inertial measurements and their derived time-Position-Velocity-Attitude (tPVA) trajectories. The first method extends the INS mechanization equations and the second method derives the IMU angular velocities from INS mechanization equations' output data. The third method derives lineal and angular velocities from relative orientation parameters. We illustrate how sensor states can be applied to solve practical problems. For this purpose we have selected three cases: multi-sensor synchronization calibration, correction of image motion blur (translational and rotational) and accurate orientation of images acquired with focal-plane shutters.

Epipolar image rectification through geometric algorithms with unknown parameters

Image processing in photogrammetry is commonly used for scene reconstruction. Although two-dimensional applications can be solved using isolated images, reconstruction of three-dimen- sional scenes usually requires the use of multiple images simultane- ously. Epipolar image rectification is a common technique for this purpose. It typically requires internal orientation parameters and, therefore, knowledge of camera calibration and relative orientation parameters between images. A reparameterization of the fundamen- tal matrix through a completely geometric algorithm of seven parameters that enables the epipolar image rectification of a photo- grammetric stereo pair without introducing any orientation parameters and without premarking ground control points is presented. The algo- rithm enables the generation of different stereoscopic models with a single photogrammetric pair from unknown cameras, scanned from a book, or frames from video sequences. Stereoscopic models with no parallaxes have been obtained with a standard deviation of <0.5 pixels. © 2013 SPIE and IS&T (DOI: 10.1117/1.JEI.22.4 .043021)

Photogrammetric modeling of the relative orientation in underwater environments

Underwater photogrammetry provides an efficient means for documentation of environments which are complex and have limited accessibility. Yet the establishment of reference control networks in such settings is oftentimes difficult. In this regard, use of the coplanarity condition, which requires neither knowledge of object space coordinates nor setting a reference control network, seems to be an attractive solution. However, the coplanarity relation does not hold in such environments because of the refraction effect, and methods that have been proposed thus far for geometrical modeling of its effect require knowledge of object-space quantities. Thus, this paper proposes a geometrically-driven approach which fulfills the coplanarity condition and thereby requires no knowledge of object space data. Such an approach may prove useful not only for object space reconstruction but also as a preparatory step for application of bundle block adjustment and for outlier detection. All are key features in photogrammetric practices. Results show that no unique setup is needed for estimating the relative orientation parameters using the model and that high levels of accuracy can be achieved.

Comparative Analysis of Different Approaches for Multi-camera System Calibration

Mobile mapping systems integrate a set of imaging sensors and a position and orientation system. To fully achieve the accuracy of the utilized sensors, a careful system calibration should be carried out. The system calibration involves individual sensor calibration and the mounting parameters calibration relating the system components. For multi-camera systems, the mounting parameters involve two sets of relative orientation parameters (ROP): the ROP among the cameras and the ROP between the cameras and the navigation sensors. This paper proposes a mathematical model for a single-step photogrammetric system calibration suitable for both single and multi-camera systems. As a special case of this model, indirect geo-referencing can be performed with relative orientation constraints (ROC). A general model which allows the estimation of ROP as wells as the incorporation of prior information on the ROP among the cameras during the integrated sensor orientation (ISO) is also proposed. This general model can be used for the indirect georeferencing with ROC as well as ISO without ROP among the cameras. To evaluate the performance of the single-step system calibration using the different models, real dataset captured by a hand-held multi-camera system is used. The system calibration is performed by using five different models and the performance is compared among these models.

MULTISPECTRAL IMAGE CAPTURING WITH FOVEON SENSORS

Abstract. This article describes a specific image quality problem using an UAV and the commercially available multispectral camera Tetracam ADC Lite. The tests were carried out with commercially available UAV Multirotor MR-X 8 performed under normal use and conditions. The ADC Lite shows a remarkable rolling shutter effect caused by the movement and vibrations of the UAV and a slow readout speed of the sensor. Based on these studies the current state of a sensor development is presented, which is composed of two compact cameras with Foveon sensors. These cameras allow to record high quality image data without motion blur or rolling shutter effect. One camera captures the normal colour range; the second camera is modified for the near infrared. The moving parts of both cameras are glued to ensure that a geometric camera calibration is valid over a longer period of time. The success of the gluing procedure has been proven by multiple calibrations. For the matching of the colour- and infrared image the usability of calibrated relative orientation parameters between both cameras were tested. Despite absolutely synchronous triggering of the cameras by an electrical signal, a time delay can be found up to 3/100 s between the images. This time delay in combination with the movement and rotation of the UAV while taking the photos results in a significant error in the previously calibrated relative orientation. These parameters should not be used in further processing. This article concludes with a first result of a 4-channel image and an outlook on the following investigations.

Photogrammetric Bundle Adjustment With Self-Calibration of the PrimeSense 3D Camera Technology: Microsoft Kinect

The Kinect system is arguably the most popular 3-D camera technology currently on the market. Its application domain is vast and has been deployed in scenarios where accurate geometric measurements are needed. Regarding the PrimeSense technology, a limited amount of work has been devoted to calibrating the Kinect, especially the depth data. The Kinect is, however, inevitably prone to distortions, as independently confirmed by numerous users. An effective method for improving the quality of the Kinect system is by modeling the sensor's systematic errors using bundle adjustment. In this paper, a method for modeling the intrinsic and extrinsic parameters of the infrared and colour cameras, and more importantly the distortions in the depth image, is presented. Through an integrated marker-and feature-based self-calibration, two Kinects were calibrated. A novel approach for modeling the depth systematic errors as a function of lens distortion and relative orientation parameters is shown to be effective. The results show improvements in geometric accuracy up to 53% compared with uncalibrated point clouds captured using the popular software RGBDemo. Systematic depth discontinuities were also reduced and in the check-plane analysis the noise of the Kinect point cloud was reduced by 17%.