Exterior Orientation Of Images Research Articles

IMAGE ORIENTATION BY EMBEDDING IN A GAN LATENT SPACE

Abstract. Estimation of an image exterior orientation is required in multiple tasks including 3D reconstruction, autonomous driving and navigation, and single-photo 3D reconstruction. The problem can be easily solved if some reference points (keypoints) with known coordinates in the reference frame are detected in the image. While multiple robust keypoint detectors were developed, estimation of an image exterior orientation from a single image remains challenging in many cases. For example, repeating structures in the scene or absence of textures can reduce the performance of keypoint detectors. In this paper, we propose an algorithm for estimation of an image exterior orientation that leverages the latent space of Generative Adversarial Network (GAN). We propose a modification of the StyleGAN2 model that we term ExteriorGAN. Unlike the StyleGAN2 that generates random images from a random noise vector z, we aim training a mapping from a random vector z and a given image exterior orientation p G : (z,p) → A. Our model generates random images for a constant exterior orientation p and random z that have constant geometry but differs in the scene appearance (e.g. different light direction or intensity). We perform embedding of the image A into the latent space w and reconstruct the input noise vector z and exterior orientation parameters w using a stochastic gradient descent. We developed a dedicated dataset with 50k images and corresponding orientation parameters to train and validate our ExteriorGAN model. The results of evaluation demonstrate that our algorithm allows estimation of the exterior orientation of an image with respect to a known 3D scene. The accuracy of the exterior orientation is comparable with modern state-of-the-art methods. The camera pose can be recovered with a mean error of 50 mm for a working space of 5 by 5 meters.

ГЕОДЕЗІЯ, КАРТОГРАФІЯ І АЕРОФОТОЗНІМАННЯ

Aim. Determination of the elements of external spatial orientation of the surveying systems at the moment of image acquisition is the fundamental task in photogrammetry. Principally, this problem is solving in two ways. The first way is direct positioning and measuring of directions of camera optical axis in the geodetic space with the help of GNSS/INS equipment. The second way is the analytical solution of the problem using a set of reference information (often such information is a set of ground control points whose geodetic positions are known with sufficient accuracy and which are reliably recognised on aerial images of the photogrammetric block). The authors consider the task of providing reference and control information using the second approach, which has a number of advantages in terms of reliability and accuracy of determining the unknown image exterior orientation parameters. It is proposed to obtain additional images of ground control points by the method of their auxiliary aerial photography using an unmanned aerial vehicle (UAV) on a larger scale compared to the scale of the images of the photogrammetric block. The aim of the presented work is the implementation of the method of creating reference points and experimental confirmation of its effectiveness for photogrammetric processing. Methods and results. For the entire realization of the potential of the analytical way to determine the elements of external orientation of images, it is necessary to have a certain number of ground control points (GCP) and to keep the defined scheme of their location on the photogrammetric block. As the main source of input data authors use UAV aerial images of the terrain, which are obtained separately from the block of aerial survey, and have a better geometric resolution and which clearly depict the control reference points. Application of such auxiliary images gives the possibility of automated transferring of the position of ground control point into images of the main photogrammetric block. In our interpretation, these images of ground control points and their surroundings on the ground are called "control reference images". The basis of the work is to develop a method for obtaining the auxiliary control reference images and transferring of position of GCP depicted on them into aerial or space images of terrain by means of computer stereo matching. To achieve this goal, we have developed a processing method for the creation of control reference images of aerial image or a series of auxiliary multi-scale aerial images obtained by a drone from different heights above the reference point. The operator identifies and measures the GCP once on the auxiliary aerial image of the highest resolution. Then there is an automatic stereo matching of the control reference image in the whole series of auxiliary images in succession with a decrease in the resolution and, ultimately, directly with the aerial images of photogrammetric block. On this stage there are no recognition/cursor targeting by the human operator, and therefore there are no discrepancies, errors or mistakes related to it. In addition, if to apply fairly large size of control reference images, the proposed method can be used on a low-texture terrain, and therefore deal in many cases without the physical marking of points measured by GNSS method. And this is a way to simplify and reduce the cost of photogrammetric technology. The action of the developed method has been verified experimentally to provide the control reference information of the block of archival aerial images of the low-texture terrain. The results of the experimental approbation of the proposed method give grounds to assert that the method makes it possible to perform geodetic reference of photogrammetric projects more efficiently due to the refusal of the physical marking of the area before aerial survey. The proposed method can also be used to obtain the information for checking the quality of photogrammetric survey for provision of check points. The authors argue that the use of additional equipment - UAV of semi-professional class to obtain control reference images is economically feasible. Scientific novelty and practical relevance. The results of approbation of the "control reference image" method with obtaining stereo pairs of aerial images with vertical placement of the base are presented for the first time. There was implemented the study of the properties of such stereo pairs of aerial images to obtain images of reference points. The effectiveness of including reference images in the main block of the digital aerial triangulation network created on UAV’s images is shown.

TOPOGRAPHIC MAPPING WITH MANIPULATOR ARM CAMERA IN LUNAR SAMPLE RETURN MISSION

Abstract. The topographic mapping of sampling areas, providing basic sampling environment information, is crucial in sample return mission. The fixed monitoring cameras were designed for mapping of sampling areas in fixed effective resolution. In order to perform more detailed topographic analysis of sampling areas, this paper proposed a topographic mapping method based on the sequential sample images captured with the movements of manipulator arm. The tie point matching results and the image exterior orientation parameters obtained from measurements of manipulator arm joints were employed to the weighted bundle adjustment based optimization for the accurate topographic mapping. The simulated images were adopted to validate the effectiveness and accuracy of the proposed method.

ASSESSMENT OF RGB AND HYPERSPECTRAL UAV REMOTE SENSING FOR GRASS QUANTITY AND QUALITY ESTIMATION

Abstract. The information on the grass quantity and quality is needed for several times in a growing season for making optimal decisions about the harvesting time and the fertiliser rate, especially in northern countries, where grass swards quality declines and yield increases rapidly in the primary growth. We studied the potential of UAV-based photogrammetry and spectral imaging in grass quality and quantity estimation. To study this, a trial site with large variation in the quantity and quality parameters was established by using different nitrogen fertilizer application rates and harvesting dates. UAV-based remote sensing datasets were captured four times during the primary growth season in June 2017 and agricultural reference measurements including dry biomass and quality parameters, such as the digestibility (D-value) were collected simultaneously. The datasets were captured using a flying height of 50 m which provided a GSD of 0.7 cm for the photogrammetric imagery and 5 cm for the hyperspectral imagery. A rigorous photogrammetric workflow was carried out for all data sets aiming to determine the image exterior orientation parameters, camera interior orientation parameters, 3D point clouds and orthomosaics. The quantitative radiometric calibration included sensor corrections, atmospheric correction, and correction for the radiometric non-uniformities caused by illumination variations, BRDF correction and the absolute reflectance transformation. Random forest (RF) and multilinear regression (MLR) estimators were trained using spectral bands, vegetation indices and 3D features, extracted from the remote sensing datasets, and insitu reference measurements. From the FPI hyperspectral data, the 35 spectral bands and 11 spectral indices were used. The 3D features were extracted from the canopy height model (CHM) generated using RGB data. The most accurate results were obtained in the second measurement day (15th June) which was near to the optimal harvesting time and generally RF outperformed MLR slightly. When assessed with the leave-one-out-estimation, the best root mean squared error (RMSE%) were 8.9% for the dry biomass using 3D features. The best D-value estimation using RF algorithm (RMSE% = 0.87%) was obtained using spectral features. Using the estimators, we then calculated grass quality and quantity maps covering the entire test site to compare different techniques and to evaluate the variability in the field. The results showed that the low-cost drone remote sensing gave excellent precision both for biomass and quality parameter estimation if accurately calibrated, offering an excellent tool for efficient and accurate management of silage grass production.

Automated Aerial Triangulation for UAV-Based Mapping

Accurate 3D reconstruction/modelling from unmanned aerial vehicle (UAV)-based imagery has become the key prerequisite in various applications. Although current commercial software has automated the process of image-based reconstruction, a transparent system, which can be incorporated with different user-defined constraints, is still preferred by the photogrammetric research community. In this regard, this paper presents a transparent framework for the automated aerial triangulation of UAV images. The proposed framework is conducted in three steps. In the first step, two approaches, which take advantage of prior information regarding the flight trajectory, are implemented for reliable relative orientation recovery. Then, initial recovery of image exterior orientation parameters (EOPs) is achieved through either an incremental or global approach. Finally, a global bundle adjustment involving Ground Control Points (GCPs) and check points is carried out to refine all estimated parameters in the defined mapping coordinate system. Four real image datasets, which are acquired by two different UAV platforms, have been utilized to evaluate the feasibility of the proposed framework. In addition, a comparative analysis between the proposed framework and the existing commercial software is performed. The derived experimental results demonstrate the superior performance of the proposed framework in providing an accurate 3D model, especially when dealing with acquired UAV images containing repetitive pattern and significant image distortions.

Errors-in-Variables Anisotropic Extended Orthogonal Procrustes Analysis

This letter presents a novel total least squares (TLS) solution of the anisotropic row-scaling Procrustes problem. The ordinary LS Procrustes approach finds the transformation parameters between origin and destination sets of observations minimizing errors affecting only the destination one. In this letter, we introduce the errors-in-variables model in the anisotropic Procrustes analysis problem and present a solution that can deal with the uncertainty affecting both sets of observations. The algorithm is applied to solve the image exterior orientation problem. Experiments show that the proposed TLS method leads to an accuracy in the parameters estimation that is higher than the one reached with the ordinary LS anisotropic Procrustes solution when the number of points, whose coordinates are known in both the image and the external systems, is small.

A Study on In Situ Calibration of an Off-The-Shelf Digital Camera Integrated to a Lidar System

Photogrammetric procedures using the integration of imagery and light detection and ranging (Lidar) datasets are becoming common in mapping companies nowadays. When photogrammetric and Lidar surveys are performed simultaneously while having the camera connected to the Lidar system, the image's exterior orientation parameters computed from the Lidar system can be used to perform photogrammetric applications. To implement such approach, camera calibration procedures should be applied to obtain accurate interior orientation parameters. This paper shows the performed study and experimental results from an in situ self-calibration using simultaneously collected imagery and Lidar datasets. The experimental study is conducted to devise a methodology for the in situ self-calibration that uses a set of vertical control points, extracted from Lidar point cloud, as well as the three-dimensional coordinates of the camera exposure stations, computed from the Lidar trajectory. For the small photogrammetric block used in this study, the results from the performed experiments demonstrated that the in situ self-calibration is required to obtain horizontal accuracies of approximately one image pixel on the ground in bundle adjustment assisted by direct measured of the coordinates of the camera exposure stations.

The use of an unmanned aerial vehicle for fracture mapping within a marble quarry (Carrara, Italy): photogrammetry and discrete fracture network modelling

ABSTRACTThis paper describes the use of a drone in collecting data for mapping discontinuities within a marble quarry. A topographic survey was carried out in order to guarantee high spatial accuracy in the exterior orientation of images. Photos were taken close to the slopes and at different angles, depending on the orientation of the quarry walls. This approach was used to overcome the problem of shadow areas and to obtain detailed information on any feature desired. Dense three-dimensional (3D) point clouds obtained through image processing were used to rebuild the quarry geometry. Discontinuities were then mapped deterministically in detail. Joint attitude interpretation was not always possible due to the regular shape of the cut walls; for every discontinuity set we therefore also mapped the uncertainty. This, together with additional fracture characteristics, was used to build 3D discrete fracture network models. Preliminary results reveal the advantage of modern photogrammetric systems in producing detailed orthophotos; the latter allow accurate mapping in areas difficult to access (one of the main limitations of traditional techniques). The results highlight the benefits of integrating photogrammetric data with those collected through classical methods: the resulting knowledge of the site is crucially important in instability analyses involving numerical modelling.

Solving bundle block adjustment by generalized anisotropic Procrustes analysis

The paper presents a new analytical tool to solve the classical photogrammetric bundle block adjustment. The analytical model is based on the generalized extension of the anisotropic row-scaling Procrustes analysis, that has been recently proposed by the same authors to solve the image exterior orientation problem. The main advantage of the method is given by the fact that the problem solution does not require any approximate value of the unknown parameters, nor any linearization procedure. Moreover, the algorithm is exceedingly simple to describe and easy to implement. Empirical results indicate that a zero-information initialization of the iterative relaxation procedure leads almost always to the correct final least squares solution. Experiments confirm the accuracy of the proposed method, when compared to the results obtained by applying a classical photogrammetric bundle block adjustment.

Camera Pose Determination and 3-D Measurement From Monocular Oblique Images With Horizontal Right Angle Constraints

This letter introduces a novel method for camera pose determination from monocular urban oblique images. Horizontal right angles that widely exist in urban scenes are used as geometric constraints in the camera pose determination, and the proposed 3-D measurement method using a monocular image is presented and then used to check the accuracy of the recovered image's exterior orientation parameters. Compared to the available vertical-line-based camera pose determination method, our new method is more accurate.

Quality of 3D Models Generated by SFM Technology

Abstract Using various types of automation in digital photogrammetry is associated with questions such as the accuracy of a 3D model generated on various types of surfaces and textures, the financial costs of the equipment needed, and also the time costs of the processing. This topic deals with the actual technology of computer vision, which allows the automated exterior orientation of images, camera calibration, and the generation of 3D models directly from images of the object itself, based on the automatic detection of significant points. Detailed testing is done using the Agisoft PhotoScan system, and the camera configuration is solved with respect to the accuracy of the 3D model generated and the time consumption of the calculations for the different types of textures and the different settings for the processing.

AEROTRIANGULATION SUPPORTED BY CAMERA STATION POSITION DETERMINED VIA PHYSICAL INTEGRATION OF LIDAR AND SLR DIGITAL CAMERA

Abstract. Nowadays lidar and photogrammetric surveys have been used together in many mapping procedures due to their complementary characteristics. Lidar survey is capable to promptly acquire reliable elevation information that is sometimes difficult via photogrammetric procedure. On the other hand, photogrammetric survey is easily able to get semantic information of the objects. Accessibility, availability, the increasing sensor size and quick image acquisition and processing are properties that have raised the use of SLR digital cameras in photogrammetry. Orthoimage generation is a powerful photogrammetric mapping procedure, where the advantages of the integration of lidar and image datasets are very well characterized. However, to perform this application both datasets must be within a common reference frame. In this paper, a procedure to have digital images positioned and oriented in the same lidar frame via a combination of direct and indirect georeferencing is studied. The SLR digital camera was physically connected with the lidar system to calculate the camera station's position in lidar frame. After that, the aerotriangulation supported by camera station's position is performed to get image's exterior orientation parameters (EOP).

TOWARD AUTOMATED FAÇADE TEXTURE GENERATION FOR 3D PHOTOREALISTIC CITY MODELLING WITH SMARTPHONES OR TABLET PCS

Abstract. An automated model-image fitting algorithm is proposed in this paper for generating façade texture image from pictures taken by smartphones or tablet PCs. The façade texture generation requires tremendous labour work and thus, has been the bottleneck of 3D photo-realistic city modelling. With advanced developments of the micro electro mechanical system (MEMS), camera, global positioning system (GPS), and gyroscope (G-sensors) can all be integrated into a smartphone or a table PC. These sensors bring the possibility of direct-georeferencing for the pictures taken by smartphones or tablet PCs. Since the accuracy of these sensors cannot compared to the surveying instruments, the image position and orientation derived from these sensors are not capable of photogrammetric measurements. This paper adopted the least-squares model-image fitting (LSMIF) algorithm to iteratively improve the image's exterior orientation. The image position from GPS and the image orientation from gyroscope are treated as the initial values. By fitting the projection of the wireframe model to the extracted edge pixels on image, the image exterior orientation elements are solved when the optimal fitting achieved. With the exact exterior orientation elements, the wireframe model of the building can be correctly projected on the image and, therefore, the façade texture image can be extracted from the picture.

Integration of Chang'E-1 Imagery and Laser Altimeter Data for Precision Lunar Topographic Modeling

Lunar orbital imagery and laser altimeter data are two major data sources for lunar topographic modeling. Most of the previous work has processed imagery and laser altimeter data separately. Usually though, there are inconsistencies between the topographic models derived from them. This paper presents an endeavor to integrate the Chang'E-1 imagery and laser altimeter data for consistent and precision lunar topographic modeling. A combined adjustment model for the Chang'E-1 imagery and laser altimeter data is developed, in which the participants are the laser altimeter points, image exterior orientation (EO) parameters, and tie points collected from the stereo images. A weighting scheme is designed for the participants in the adjustment, and a local surface constraint is imposed to improve the adjustment performance. The output of the combined adjustment is the refined image EO parameters and laser ground points. Experimental results using the Chang'E-1 data in the Apollo 15 and 16 landing site areas show that the proposed combined adjustment approach can reduce the misregistrations between the imagery and the laser altimeter data by a maximum of 1-18 pixels in image space. The Japanese SELenological and ENgineering Explorer (SELENE) laser altimeter data at the Apollo 15 and 16 landing site areas are employed for comparison analysis. Small shifts between the SELENE and Chang'E-1 laser altimeter data were found. The topography derived from Chang'E-1 data after the combined adjustment shows a relatively consistent trend with the topography determined by the SELENE laser altimeter data.

Line Feature Correspondence between Object Space and Image Space

In recent years, linear features have been gradually integrated into photogrammetry-related applications to estimate orientation parameters and facilitate object reconstruction, to name just a few. Yet, the correspondence of the conjugate linear features between different spaces or coordinate systems remains crucial to the goal of photogrammetric automation. When establishing exterior orientation of images on the line feature basis, the identifications and measurements of control line features must be involved. In this study, the authors develop an approach for effectively matching line features between object space and image space. The proposed algorithms start from projecting 3D line features into 2D image space employing collinearity equations with approximate orientation parameters. Then, the candidate lines detected from the images are chosen and matched with the 3D line features by imposing, in a sequential manner, geometric constraints that include angle, distance, reference point, and imaging geometry checks. Furthermore, a two-stage matching strategy, where the outcome of the first matching stage by partial 3D line features provides confined matching candidates for the second matching stage, is found effective in lowering the computational load, especially when faced with a great amount of 3D line features. Preliminary tests demonstrate successful, as well as satisfactory, 3D to 2D line feature correspondences using the proposed approach and strategy.

Digital image georeferencing from a multiple camera system by GPS/INS

In this paper, the development and testing of an airborne fully digital multi-sensor system for digital mapping data acquisition is presented. The system acquires two streams of data, namely, navigation (georeferencing) data and imaging data. The navigation data are obtained by integrating an accurate strapdown inertial navigation system with a differential GPS system (DGPS). The imaging data are acquired by two low-cost digital cameras, configured in such a way so as to reduce their geometric limitations. The two cameras capture strips of overlapping nadir and oblique images. The GPS/INS-derived trajectory contains the full translational and rotational motion of the carrier aircraft. Thus, image exterior orientation information is extracted from the trajectory, during post-processing. This approach eliminates the need for ground control (GCP) when computing 3D positions of objects that appear in the field of view of the system imaging component. Two approaches for calibrating the system are presented, namely, terrestrial calibration and in-flight calibration. Test flights were conducted over the campus of The University of Calgary. Testing the system showed that best ground point positioning accuracy at 1:12,000 average image scale is 0.2 m (RMS) in easting and northing and 0.3 m (RMS) in height. Preliminary results indicate that major applications of such a system in the future are in the field of digital mapping, at scales of 1:5000 and smaller, and in the generation of digital elevation models for engineering applications.