Conjugate Image Points Research Articles

A flexible targetless LiDAR–GNSS/INS–camera calibration method for UAV platforms

There is a growing need for 3D colored maps acquired from multi-sensor moving platforms. Accurate multi-sensor data alignment is an important prerequisite for the construction of 3D colored maps derived from simultaneously acquired camera and Light Detection and Ranging (LiDAR) data. However, current alignment methods are hampered by low automation, heavy computational costs or tedious system calibration set-ups. In this paper, we consider a LiDAR–global navigation satellite system (GNSS)/inertial navigation system (INS)–camera system mounted on an Unmanned Aerial Vehicle (UAV) platform. We present a detailed literature review of existing calibration methods for such systems. We propose a new versatile automatic and targetless calibration method of this system. This method involves estimating the calibration parameters by optimizing the correspondence between pairs of conjugate image points extracted from overlapping images and the projection of these points onto the georeferenced LiDAR point cloud. Experiments on actual data show the suitability of this method for the construction of 3D colored point clouds. Quantitative calibration results using checkpoints indicate that the obtained calibration accuracy is compatible with the accuracy of the georeferenced LiDAR point cloud, i.e. 5 cm. Further experiments on simulated data show the robustness of this approach to initial calibration parameters and low sensitivity to LiDAR point cloud density and noise. As this method is quite flexible, we believe it is more suitable for 3D color map generation than other methods proposed in the literature.

Determination of Optimum Tie Point Interval for SAR Image Coregistration by Decomposing Autocorrelation Coefficient

Interferometric synthetic aperture radar (InSAR) is a well-established technique for the generation of digital elevation models and/or the measurement of ground surface deformations. In InSAR data processing, the first step is the image coregistration achieved by using a set of tie points which are the conjugate image points on both master and slave images. To achieve reliable coregistration, a set of points is used instead of a single point. The interval of tie points, i.e., tie point density, will greatly affect the reliability of the coregistration. Hitherto, there have been no effective methods for the determination of optimum tie point interval for the purpose. In practice, this parameter is determined by experience. In this paper, a method for determination of optimum tie point interval is presented based on theoretical analysis of autocorrelation coefficient of SAR image, which reflects the similarity between image pixels. After computation of autocorrelation coefficient in range and azimuth, it is decomposed into low and high frequencies by a wavelet transform. Then, by a combined analysis of the variations of wave amplitudes with distance, the optimum interval for tie point matching can be determined. It is found that the optimum tie point interval in azimuth and range can be different. Two pairs of SAR images acquired by different platforms are used to validate the proposed method. An optimum tie point interval can be determined in this way for any pair of SAR images.

Geometric Self-Calibration of YaoGan-13 Images Using Multiple Overlapping Images.

Geometric calibration is an important means of improving the absolute positioning accuracy of space-borne synthetic aperture radar imagery. The conventional calibration method is based on a calibration field, which is simple and convenient, but requires a great deal of manpower and material resources to obtain ground control points. Although newer cross-calibration methods do not require ground control points, calibration accuracy still depends on a periodically updated reference image. Accordingly, this study proposes a geometric self-calibration method based on the positioning consistency constraint of conjugate image points to provide rapid and accurate calibration of the YaoGan-13 satellite. The proposed method can accurately calibrate geometric parameters without requiring ground control points or high-precision reference images. To verify the absolute positioning accuracy obtained using the proposed self-calibration method, YaoGan-13 Stripmap images of multiple regions were collected and evaluated. The results indicate that high-accuracy absolute positioning can be achieved with a plane accuracy of 3.83 m or better for Stripmap data, without regarding elevation error. Compared to the conventional calibration method using high-accuracy control data, the difference between the two methods is only about 2.53 m, less than the 3-m resolution of the image, verifying the effectiveness of the proposed self-calibration method.

A novel approach for epipolar resampling of cross-track linear pushbroom imagery using orbital parameters model

This paper presents a novel approach to epipolar resampling of cross-track linear pushbroom imagery using orbital parameters model (OPM). The backbone of the proposed method relies on modification of attitude parameters of linear array stereo imagery in such a way to parallelize the approximate conjugate epipolar lines (ACELs) with the instantaneous base line (IBL) of the conjugate image points (CIPs). Afterward, a complementary rotation is applied in order to parallelize all the ACELs throughout the stereo imagery. The new estimated attitude parameters are evaluated based on the direction of the IBL and the ACELs. Due to the spatial and temporal variability of the IBL (respectively changes in column and row numbers of the CIPs) and nonparallel nature of the epipolar lines in the stereo linear images, some polynomials in the both column and row numbers of the CIPs are used to model new attitude parameters. As the instantaneous position of sensors remains fix, the digital elevation model (DEM) of the area of interest is not required in the resampling process. According to the experimental results obtained from two pairs of SPOT and RapidEye stereo imagery with a high elevation relief, the average absolute values of remained vertical parallaxes of CIPs in the normalized images were obtained 0.19 and 0.28 pixels respectively, which confirm the high accuracy and applicability of the proposed method.

Epipolar Resampling of Cross-Track Pushbroom Satellite Imagery Using the Rigorous Sensor Model.

Epipolar resampling aims to eliminate the vertical parallax of stereo images. Due to the dynamic nature of the exterior orientation parameters of linear pushbroom satellite imagery and the complexity of reconstructing the epipolar geometry using rigorous sensor models, so far, no epipolar resampling approach has been proposed based on these models. In this paper for the first time it is shown that the orientation of the instantaneous baseline (IB) of conjugate image points (CIPs) in the linear pushbroom satellite imagery can be modeled with high precision in terms of the rows- and the columns-number of CIPs. Taking advantage of this feature, a novel approach is then presented for epipolar resampling of cross-track linear pushbroom satellite imagery. The proposed method is based on the rigorous sensor model. As the instantaneous position of sensors remains fixed, the digital elevation model of the area of interest is not required in the resampling process. Experimental results obtained from two pairs of SPOT and one pair of RapidEye stereo imagery with different terrain conditions shows that the proposed epipolar resampling approach benefits from a superior accuracy, as the remained vertical parallaxes of all CIPs in the normalized images are close to zero.

ORIENTATION MODELING FOR AMATEUR CAMERAS BY MATCHING IMAGE LINE FEATURES AND BUILDING VECTOR DATA

Abstract. With the popularity of geospatial applications, database updating is getting important due to the environmental changes over time. Imagery provides a lower cost and efficient way to update the database. Three dimensional objects can be measured by space intersection using conjugate image points and orientation parameters of cameras. However, precise orientation parameters of light amateur cameras are not always available due to their costliness and heaviness of precision GPS and IMU. To automatize data updating, the correspondence of object vector data and image may be built to improve the accuracy of direct georeferencing. This study contains four major parts, (1) back-projection of object vector data, (2) extraction of image feature lines, (3) object-image feature line matching, and (4) line-based orientation modeling. In order to construct the correspondence of features between an image and a building model, the building vector features were back-projected onto the image using the initial camera orientation from GPS and IMU. Image line features were extracted from the imagery. Afterwards, the matching procedure was done by assessing the similarity between the extracted image features and the back-projected ones. Then, the fourth part utilized line features in orientation modeling. The line-based orientation modeling was performed by the integration of line parametric equations into collinearity condition equations. The experiment data included images with 0.06 m resolution acquired by Canon EOS Mark 5D II camera on a Microdrones MD4-1000 UAV. Experimental results indicate that 2.1 pixel accuracy may be reached, which is equivalent to 0.12 m in the object space.

ORIENTATION MODELING FOR AMATEUR CAMERAS BY MATCHING IMAGE LINE FEATURES AND BUILDING VECTOR DATA

With the popularity of geospatial applications, database updating is getting important due to the environmental changes over time. Imagery provides a lower cost and efficient way to update the database. Three dimensional objects can be measured by space intersection using conjugate image points and orientation parameters of cameras. However, precise orientation parameters of light amateur cameras are not always available due to their costliness and heaviness of precision GPS and IMU. To automatize data updating, the correspondence of object vector data and image may be built to improve the accuracy of direct georeferencing. This study contains four major parts, (1) back-projection of object vector data, (2) extraction of image feature lines, (3) object-image feature line matching, and (4) line-based orientation modeling. In order to construct the correspondence of features between an image and a building model, the building vector features were back-projected onto the image using the initial camera orientation from GPS and IMU. Image line features were extracted from the imagery. Afterwards, the matching procedure was done by assessing the similarity between the extracted image features and the back-projected ones. Then, the fourth part utilized line features in orientation modeling. The line-based orientation modeling was performed by the integration of line parametric equations into collinearity condition equations. The experiment data included images with 0.06 m resolution acquired by Canon EOS Mark 5D II camera on a Microdrones MD4-1000 UAV. Experimental results indicate that 2.1 pixel accuracy may be reached, which is equivalent to 0.12 m in the object space.

LiDAR Point Cloud Registration by Image Detection Technique

In this letter, a novel approach that utilizes the spectrum information (i.e., images) provided in a modern light detection and ranging (LiDAR) sensor is proposed for the registration of multistation LiDAR data sets. First, the conjugate points in the images collected at varied LiDAR stations are extracted through the speedup robust feature technique. Then, by applying the image-object space mapping technique, the 3-D coordinates of the conjugate image points can be obtained. Those identified 3-D conjugate points are then fed into a registration model so that the transformation parameters can be immediately solved using the efficient noniterative solution to linear transformations technique. Based on numerical results from a case study, it has been demonstrated that, by implementing the proposed approach, a fully automatic and reliable registration of multistation LiDAR point clouds can be achieved without the need for any human intervention.

Optimum estimator for SPOT-pan and ALS-lidar intensity image registration via tie points

The characteristics of a Système Pour d’Observation de la Terre (SPOT) panchromatic image of a geographic area such as an airfield are different from those of an airborne laser scanner lidar intensity image of the same area. This study investigates the feasibility of locating tie points shared by the target (airborne lidar-intensity) and search (spaceborne SPOT-pan) images. At an approximate point position, image segmentation and enhancement are employed so that a pair of target and search window images tend to resemble each other. The similarity is then measured by utilizing a least-squares image matching algorithm which involves variance-component estimation. The proposed method has a 3–15% higher success rate than an equal-weight matching method. Coordinate differences of the conjugate image points serve as reference value to be used in the least-squares collocation method for pixel-by-pixel transformation. The image registration accuracy from a SPOT-pan to a lidar intensity image is estimated to be ±1.0 pixel.

Epipolar arrangement of satellite imagery by projection trajectory simplification

Abstract This paper presents an efficient algorithm for approximate epipolar image generation from stereoscopic satellite images by projection trajectory simplification. In this algorithm, the original non‐linear projection trajectory is simplified to a linear one by the least squares line fitting of sample points selected from the projection trajectory. The performance of the algorithm is evaluated using IRS‐P5, SPOT5‐HRG and IKONOS stereo‐images acquired in different stereoscopic imaging modes over different terrain types. The results obtained indicate that the vertical parallax of each pair of conjugate image points on the epipolar images thus generated achieves accuracy at the sub‐pixel level.

Conjugate image points by an adaptive estimator for the gray‐level variance

A least‐squares image matching algorithm was extended to adjust the different possible variances, auto‐ and cross‐covariances found in a gray‐level covariance matrix. Scaling variance and covariance components were estimated iteratively, based on some positive‐semidefinite accompanying matrices. The theory of a best linear, unbiased estimator for the scale factors has been presented in detail. Conjugate‐point accuracy results were obtained from 114 pairs of Radarsat‐1 crossroads and pond‐corner image windows. A 14% improvement in sub‐pixel matching accuracy was achieved by adhering to the proposed BLU‐Estimating algorithm, in particular for a category of the 11×11 image‐window size.

Determination of optimum window size for SAR image co‐registration with decomposition of auto‐correlation

AbstractInterferometric synthetic aperture radar (InSAR) is a promising recent technique for the generation of digital elevation models and/or the measurement of ground surface deformations. In InSAR data processing, the first step is the image co‐registration, achieved by using a set of tie points which are the conjugate image points on the master and slave images. Tie points are selected with the aim of finding the conjugate point automatically on the slave image for any given point on the master image, by the process known as image matching. To achieve reliable matching, a set of points within a window is used instead of a single point. The size of the window will affect the reliability of matching. Hitherto there have been no effective methods for the determination of optimum window size for this purpose. In practice, this parameter is determined by experience. In this paper, a pair of SAR images is used to test the effect of window size on the reliability of co‐registration. An optimum window size is selected on the basis of the experimental results. The determination of optimum window size for tie point matching is then examined theoretically, leading to a proposal for an automated method based on the auto‐correlation of the SAR images, which reflects the similarity between image pixels. The auto‐correlation function is decomposed into waves of various frequencies by a wavelet transform. By a combined analysis of the variations of wave amplitudes with frequency, the optimum window size for tie point matching can be determined. A procedure based on the use of Daubechies wavelet db1 is proposed in detail. An optimum window size could be determined in this way for any pair of SAR images.

Scanning Confocal Electron Microscopy

A scanning confocal microscope and methods are provided for configuring scanning confocal microscopes for imaging specimens, such as, high resolution imaging of thick non-optically transparent specimens including imaging of buried or subsurface features of thick non-optically transparent structures. The scanning confocal microscope, such as a scanning confocal electron microscope (SCEM), is configured to image structures buried in thick specimens, such as specimens greater than eight microns thick, utilizing confocal imaging principles. A scanning confocal microscope includes an illumination source, a specimen, and a detector. The illumination source provides a focused radiation beam that is applied to the specimen. The detector detects an interaction signal from the specimen. The scanning confocal microscope is configured to operate in the confocal imaging mode, where the imaging source, specimen and detector are arranged to be located at conjugate image points. The focused radiation beam provided by the illumination source includes an electron beam, a proton beam, an ion beam, or an x-ray beam. The focused radiation beam provided by the illumination source is capable of penetrating thick non-optically transparent specimens, unlike visible light or optical probes that cannot penetrate significant depths in optically dense specimens. The incident probe is sequentially scanned across a region of interest of the specimen and the net integrated confocal intensity at each point is detected and used to provide an image display. A scanning confocal electron microscope (SCEM) is provided that permits resolutions better than 100 nanometers for materials as thick as 8-10 microns. The image resolution provided is equal to or better than typical high flux x-ray sources, while operating at speeds up to one hundred times faster and the scanning confocal microscope can be located in a conventional laboratory space.

Noncontact ballistic motion measurement using a fiber-optic confocal sensor

A fiber-optic confocal sensor for noncontact ballistic measurements is described. Determination of motion at velocities of 1.7 km/s with an uncertainty as small as ±0.3% is demonstrated for both a projectile and a free-surface target. The fibers detect the passage of the object at their conjugate image points created by low F/# optics. This results in an output signal comprising a train of sharp pulses each precisely identifying when the ballistic object traverses an image point. Since the ballistic object does not contact the sensor at the time of imaging, the measurements do not perturb the motion, enabling multi-fragment measurement, as well as repetitive measurements of the same object point.

Three-Dimensional Positioning Using SPOT Stereostrips with Sparse Control

This paper describes an experiment for a proposed scheme using on­ board satellite data to perform three-dimensional positioning for a pair of SPOT stereostrips. Based on the sampling geometry of French SPOT satellite images, we first adjusted the on-board parameters by incorporating a small number of ground control points (GCPs). 1ben a procedure for three-dimensional modeling was per­ formed as follows. The central features of the proposed scheme are: (l) use of on­ board data to initialize orbit parameters and attitude data for the satellite; (2) cor­ rection of on-board parameters with low order polynomials; (3) use of reverse transformation to check the collinear relationship between object space and image space for the GCPs; (4) adjustment of attitude data for the satellite to minimize the error in terms of the GCP collinearity; and (5) space intersection. Experimental results indicate that, for a SPOT panchromatic stereostrip with a 0.6 base-to-height ratio and a 45 km by 280 km stereo model area, the root-mean-square-error for 3­ D positioning is less than 8 m in each direction when only 4 GCPs are used. Three-dimensional positioning for object points is an important task in the field of civil engineering when geometrical data processing is needed. Terrain data are often required in the planning, design, and construction stages of various transportation and hydraulic engineering projects (Epps and Corey 1990; Jones et al. 1990). Three-dimensional terrain information is also required for geometrical correction (Wiesel 1985) and terrain-dependent radiometric correction (Colby 1991) of remotely sensed images. Three-di­ mensional terrain data may be obtained from ground surveying with the ex­ pense of intensive labor, but a labor-saving approach is possible when pho­ tograrnmetric techniques are used. A higher efficiency may be anticipated by the use of a digital approach to automate the working procedure. Nowadays, because the spatial resolution of satellite images is significantly improved and stereoscopic sampling is available, the efficient extraction of 3-D terrain data becomes possible. Currently, French SPOT satellites have the highest tem­ poral and spatial resolutions among the available earth resource satellites. In addition, due to their side looking characteristics, SPOT satellites may pro­ vide cross-track stereo pairs. Thus, SPOT images have high potential for the extraction of 3-D terrain information. From the photograrnmetric point of view, collinearity condition equations are generally used in 3-D positioning for SPOT stereo pairs. Based on the equations, a bundle adjustment may be applied to model the satellite orien­ tation and the 3-D ground coordinates for a pair of conjugate image points (Konecny et al. 1987; Gugan and Dowman 1988). This approach may reach subpixel accuracy. However, a large number of ground control points (GCPs)

Conjugate imagery in the automated reproduction of three dimensional coordinates from two dimensional coordinate data

The three-dimensional video analysis of human motion commonly utilises automated image processing and digitisation processes to produce real-time unidentified two-dimensional coordinate data of segmental markers. In what can be a time-consuming process the two-dimensional data are then identified and tracked to produce three-dimensional coordinates. This paper presents an approach to the automated reproduction of three-dimensional coordinates from two-dimensional coordinate data. Conjugate imaging techniques were utilised in the development of four criterion measures for determining the validity of conjugate (corresponding) image points. An algorithm based on the criterion measures was then developed for the automated reproduction of three dimensional coordinates from camera image coordinate data. The algorithm was tested with a 55 point marker system viewed in four video cameras (digitisation error approx. 0.2%, lab point separation ≥ 6 cm). The success of the algorithm was dependent on the closeness of markers, the accuracy of the photogrammetric system, and the number of markers visible in two camera images. The present research has developed techniques based on conjugate imagery for the automated reproduction of three-dimensional coordinates from two-dimensional data, and provided a bases for further development of automated three-dimensional tracking.

Quantification of Pavement Rut Depth Using Stereovision Technology

An attempt to investigate the potential capabilities and accuracy of the PC-based stereovision system in quantification of pavement rut depth was performed. This process was demonstrated using three rutted sections of flexible pavement. Selected highway pavement sections were chosen, using representative criteria that assured randomness in the studied parameters, for semiautomatic rut depth quantification. Surface measurements were represented by two-dimensional and three-dimensional plots, as well as contour maps for the rutted areas. Results obtained by this technique were compared with those obtained by the manual conventional method using a straightedge to check accuracy. Comparison with the manual (conventional) method showed that stereovision using dual charged-coupled device cameras mounted on a stand on a moving vehicle has a consistent potential accuracy of about one pixel (i.e., about 1.5 cm) on average in measuring the rut depth for the three sections. This result was compatible with the human operator's ability to point to the conjugate image points, which has a potential accuracy of about one to two pixels. Advantages of using the stereovision technique to quantify rut depth include remoteness, safety, and real-time potential due to the use of digital images.