BUNDLE ADJUSTMENT WITH SELF‐CALIBRATION USING STRAIGHT LINES

Increased use of digital imagery has facilitated the opportunity to use features, in addition to points, in photogrammetric applications. Straight lines are often present in object space, and prior research has focused on incorporating straight–line constraints into bundle adjustment for frame imagery. In the research reported in this paper, object–space straight lines are used in a bundle adjustment with self–calibration. The perspective projection of straight lines in the object space produces straight lines in the image space in the absence of distortions. Any deviations from straightness in the image space are attributed to various distortion sources, such as radial and decentric lens distortions. Before incorporating straight lines into a bundle adjustment with self–calibration, the representation and perspective transformation of straight lines between image space and object space should be addressed. In this investigation, images of straight lines are represented as a sequence of points along the image line. Also, two points along the object–space straight line are used to represent that line. The perspective relationship between image– and object–space lines is incorporated in a mathematical constraint. The underlying principle in this constraint is that the vector from the perspective centre to an image point on a straight–line feature lies on the plane defined by the perspective centre and the two object points defining the straight line. This constraint has been embedded in a software application for bundle adjustment with self–calibration that can incorporate point as well as straight–line features. Experiments with simulated and real data have proved the feasibility and the efficiency of the algorithm proposed.

The radial lens distortion correction technique based on least squares estimation corrects a distorted image by expanding it nonlinearly so that straight lines in the object space remain straight in the image space. An absolute pipelined architecture is designed to correct radial lens distortion in images by partitioning the distortion correction algorithm into four main modules. The architecture includes a COKDIC based rectangular to polar coordinate transformation module, a back mapping module for nonlinear transformation of corrected image space to distorted image space, a COKDIC based polar to rectangular coordinate transformation module, and a linear interpolation module to calculate the intensities of four pixels simultaneously in the corrected image space. The system parameters include the expanded/corrected image size, distorted image size, the back mapping coefficients, distortion center and the center of the corrected image. The hardware architecture can sustain a high throughput rate of 30 4-MegaPixel (Mpixels) frames per second (total of 120 Mpixels). The pipelined architecture design will facilitate the use of dedicated hardware that can be mounted along with the camera unit.

The aim of this paper is to present and assess techniques for orientation of pushbroom sensors that allow the estimation of the polynomial coefficients describing the platform trajectory, using linear features and linear features combined with points as ground control. The pushbroom image acquisition is not instantaneous and, as a consequence, six EOP (Exterior Orientation Parameters) for each scanned line must be estimated. The sensor position and attitude parameters are modeled with a time dependent polynomial. The proposed mathematic model is based on the equivalence property between planes, considering the pushbroom geometry. The relationship between object and image space is established through a mathematical model based on the equivalence between the vector normal to the projection plane in the image space and to the vector normal to the rotated projection plane in the object space. The model based on collinearity equations using points adapted to the pushbroom geometry was also implemented aiming at the comparison of the techniques, as well as to evaluate the use of control points and straight lines simultaneously. Some experiments using a CBERS scene were accomplished in order to test the developed techniques. The obtained results showed that is possible estimate the EOP of pushbroom images using straight lines and that the combination of straight lines with points is a more flexible alternative that allows good results.

A rational polynomial camera (RPC) model is a kind of generic sensor model that can be used in different remote sensing systems to model the relationship between object space and image space and transform image data to conform to a map projection. Unlike traditional physical camera models, an RPC model has many coefficients (a total of 80), and these coefficients do not have a physical interpretation. This represents a difficult challenge for the mapping community. For RPC refinement, many solutions, including direct and indirect methods, have been developed. One of them, the recent developed generic method has been shown to be a robust method. Because the generic method can simulate the camera's exterior parameters, it can be used in any geometric situation. Even so, the performance of bundle adjustment with the generic method is still unknown. In this paper, through experiments with a stereo pair and a stereo triplet, the capability of high-accuracy geopositioning based on the generic method is demonstrated. We first give a brief review of previous bundle adjustment methods based on RPC. Then, the bundle adjustment algorithm based on the generic method is introduced in detail. We finally present the experiments with both IKONOS and QuickBird imageries. The experiments show that the bundle adjustment based on the generic method can reach subpixel accuracy in image space and submeter accuracy in object space.

Automatic single photo resection (SPR) remains one of the challenging problems in digital photogrammetry. Visibility and uniqueness of distinct control points in the input imagery limit robust automation of the space resection procedure. Recent advances in photogrammetry mandate adopting higher‐level primitives, such as free‐form control linear features, for replacing traditional control points. Linear features can be automatically extracted from the image space. On the other hand, object space control linear features can be obtained from an existing GIS layer containing 3D vector data such as road networks or from newly developed terrestrial mobile mapping systems (MMS). In this paper, two different approaches are presented for simultaneously determining the position and attitude of the imagery as well as the correspondence between image and object space linear features. These approaches are based on two representation schemes of the linear features. The first one represents the linear feature by a sequence of 2D and 3D points along the linear feature in the image and object space, respectively. The second scheme assumes that the feature is modelled by polylines (a sequence of straight‐line segments). Neither approach requires one‐to‐one correspondence between image and object space primitives, which makes the suggested methodology robust against changes and/or discrepancies between the data‐sets involved. This characteristic will be helpful in detecting and dealing with changes between object and image space linear features (due to temporal effects for example). The parameter estimation and matching follow an optimal sequential procedure that is developed and described within this paper, which depends on the sensitivity of the mathematical model relating corresponding primitives at various image regions to incremental changes in the exterior orientation parameters (EOP). Experiments are conducted to compare the algorithms’ efficiency and the accuracy of the estimated EOP using both approaches. Experimental results using real data demonstrate the feasibility and robustness of both representation schemes as well as the methodologies developed. Moreover, different generalisation levels of the polylines representing the free‐form linear features are compared.

Wavefront aberrations in the image space are critical for visual perception, though the clinical available instruments usually give the wavefront aberrations in the object space. This study aims to compare the aberrations in the object and image spaces. With the measured wavefront aberrations over the horizontal and vertical ±15° visual fields, the in-going and out-going wide-field individual myopic eye models were constructed to obtain the wavefront aberrations in the object and image spaces of the same eye over ±45° horizontal and vertical visual fields. The average differences in the mean sphere and astigmatism were below 0.25 D between the object and image spaces over the horizontal and vertical ±45° visual fields under 3 mm and 6 mm pupil diameter. The wavefront aberrations in the object space are a proper representation of the aberrations in the image space at least for horizontal visual fields ranging from -35°to +35° and vertical visual fields ranging from -15°to +15°.

This paper proposes a method for calculating exterior orientation parameters (EOPs) of images based on the condition that the straight line in the image space and corresponding line in the object space are coplanar. Experimental results have proved the feasibility and validity of this method, and the accuracy of EOPs is within one pixel and similar to that of point-based method. The position and direction of line segments have influence on the precision of EOPs, and the influences are analyzed academically. The reliability of precision analytical results has proved by utilizing simulated image.

In this work a method is proposed to allow the indirect orientation of images using photogrammetric control extracted through integration of data derived from Photogrammetry and Light Detection and Ranging (LiDAR) system. The photogrammetric control is obtained by using an inverse photogrammetric model, which allows the projection of image space straight lines onto the object space. his mathematical model is developed based on the intersection between the collinearity-based straight line and a DSM of region, derived from LiDAR data. The mathematical model used in the indirect orientation of the image is known as the model of equivalent t planes. This mathematical model is based on the equivalence between the vector normal to the projection plane in the image space and to the vector normal to the rotated projection plane in the object space. The goal of this work is to verify the quality, efficiency and potential of photogrammetric control straight lines obtained with proposed method applied to the indirect orientation of images. The quality of generated photogrammetric control was statistically available and the results showed that proposed method is promising and it has potential for the indirect orientation of images.

The aim of this paper is to present an experimental assessment of two models that use “control lines” for the indirect orientation of pushbroom imagery. Since pushbroom image acquisition is not instantaneous, six exterior orientation parameters (EOPs) must be estimated for each scanned line. The sensor position and attitude parameters are modelled with a time‐dependent polynomial. The relationship between a straight line in the image space and its homologous form in the object space is established in the first model, based on the principle that the position vector containing an image point (projection ray) and the vector normal to the projection plane in the object space are orthogonal. The second model is based on the equivalence between the vector normal to the projection plane in the image space and the vector normal to the rotated projection plane in the object space. The equivalence property between planes was adapted to consider the pushbroom geometry. A model based on collinearity equations using points adapted to the pushbroom geometry was also implemented, aiming at a comparison of the methodologies. Six experiments using different sets of observations for indirect estimation of EOPs of images from the China–Brazil Earth Resources Satellite (CBERS) were carried out, by varying the geometric distribution and the number of straight lines. Also, experiments combining points and straight lines were accomplished. The results showed that an accuracy of around twice the ground sample distance (GSD) in the check points can be achieved with the models studied, which can then be used to estimate the EOPs of pushbroom images. Several other factors affecting the accuracy, such as the distribution and number of control features, were also assessed.

The aim of this paper is to present a model for orientation of pushbroom sensors that allows estimating the polynomial coefficients describing the trajectory of the platform, using linear features as ground control. Considering that pushbroom image acquisition is not instantaneous, six EOP (Exterior Orientation Parameters) for each scanned line must be estimated. The sensor position and attitude parameters are modeled with a time dependent polynomial. The relationship between object and image space is established through a mathematical model based on the equivalence between the vector normal to the projection plane in the image space and to the vector normal to the rotated projection plane in the object space. The equivalence property between planes was adapted to consider the pushbroom geometry. Some experiments with simulated data corresponding to CBERS scene (China-Brazil Earth Resource Satellite) were accomplished in order to test the developed model using straight lines. Moreover, experiments with points ground with the model based on collinearity equations adapted to the pushbroom geometry were also accomplished. The obtained results showed that the proposed model can be used to estimate the EOP of pushbroom images with suitable accuracy.

Motion parameter estimation by tracking stationary three-dimensional straight lines in image sequences

By implicit camera calibration, we mean the process of calibrating cameras without explicitly computing their physical parameters. We introduce a new implicit model based on a generalized mapping between an image plane and multiple, parallel calibration planes (usually between four to seven planes). This paper presents a method of computing a relationship between a point on a three-dimensional (3D) object and its corresponding two-dimensional (2D) coordinate in a camera image. This relationship is expanded to form a mapping of points in 3D space to points in image (camera) space and visa versa that requires only matrix multiplication operations. This paper presents the rationale behind the selection of the forms of four matrices and the algorithms to calculate the parameters for the matrices. Two of the matrices are used to map 3D points in object space to 2D points on the CCD camera image plane. The other two matrices are used to map 2D points on the image plane to points on user defined planes in 3D object space. The mappings include compensation for lens distortion and measurement errors. The number of parameters used can be increased, in a straight forward fashion, to calculate and use as many parameters as needed to obtain a user desired accuracy. Previous methods of camera calibration use a fixed number of parameters which can limit the obtainable accuracy and most require the solution of nonlinear equations. The procedure presented can be used to calibrate a single camera to make 2D measurements or calibrate stereo cameras to make 3D measurements. Positional accuracy of better than 3 parts in 10,000 have been achieved. The algorithms in this paper were developed and are implemented in MATLABR (registered trademark of The Math Works, Inc.). We have developed a system to analyze the path of optical fiber during high speed payout (unwinding) of optical fiber off a bobbin. This requires recording and analyzing high speed (5 microsecond exposure time), synchronous, stereo images of the optical fiber during payout. A 3D equation for the fiber at an instant in time is calculated from the corresponding pair of stereo images as follows. In each image, about 20 points along the 2D projection of the fiber are located. Each of these 'fiber points' in one image is mapped to its projection line in 3D space. Each projection line is mapped into another line in the second image. The intersection of each mapped projection line and a curve fitted to the fiber points of the second image (fiber projection in second image) is calculated. Each intersection point is mapped back to the 3D space. A 3D fiber coordinate is formed from the intersection, in 3D space, of a mapped intersection point with its corresponding projection line. The 3D equation for the fiber is computed from this ordered list of 3D coordinates. This process requires a method of accurately mapping 2D (image space) to 3D (object space) and visa versa.3173© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Object and image spaces are widely used in geometrical optics to describe the functions of optical systems. However, the mapping between these two spaces is non-linear, meaning that a function expressed in coordinate frames in the image space cannot be directly applied in the object space, and vice versa. For example, the relationship between the wavefront and ray aberrations given by [1] is valid for coordinate frames in the image space but fails for coordinate frames in the object space. To overcome this limitation, this study converts the wavefront-ray aberration relationship using the chain rule so that it can also be applied to coordinate frames in the object space. The numerical results obtained for the primary ray aberrations using the proposed converted relationship are shown to be in close agreement with the Zemax simulation results.

Precise georeferencing using the rigorous sensor model and rational function model for ZiYuan-3 strip scenes with minimum control

Space quantization between the object and image spaces of a microscopic stereovision system with a stereo light microscope