Exterior Orientation Elements Research Articles

Применение сферической модели камеры для решения фотограмметрических задач

In this paper, we consider the Unified Camera Model and its properties, as well as investigate and apply the improved Double-Spherical one (DS-model) in the task of restoring exterior and interior orientation elements for a set of images with ultra-wide-angle fisheye lens. The use of panoramic cameras in photogrammetry issues is a new trend, while the spherical model fully inherits the properties of the classical central-projection sample. The mathematical concept of using it is presented, as well as the solution of the optimization problem of calculating the elements of the internal and external orientation of the camera through the Gauss-Newton method, which is relevant for the tasks of reconstructing a three-dimensional scene and developing an optical SLAM based on a multi-camera system with ultra-wide-angle lenses, is implemented. In this work, epipolar geometry methods, digital image processing methods, statistical, numerical, and optimization ones were used. The results obtained enable evaluating the applicability of the DS-model for three-dimensional reconstruction of scenes during external and internal monitoring of buildings and structures, restoration of the motion trajectory and position in space for mobile systems with an installed camera, solving matters such as Perspective-n-Point for restoring the relative orientation between different active and passive sensors

Automatic Point Cloud Colorization of Ground-Based LiDAR Data Using Video Imagery without Position and Orientation System

With the continuous development of three-dimensional city modeling, traditional close-range photogrammetry is limited by complex processing procedures and incomplete 3D depth information, making it unable to meet high-precision modeling requirements. In contrast, the integration of light detection and ranging and cameras in mobile measurement systems provides a new and highly effective solution. Currently, integrated mobile measurement systems commonly require cameras, lasers, position and orientation system and inertial measurement units; thus, the hardware cost is relatively expensive, and the system integration is complex. Therefore, in this paper, we propose a ground mobile measurement system only composed of a LiDAR and a GoPro camera, providing a more convenient and reliable way to automatically obtain 3D point cloud data with spectral information. The automatic point cloud coloring based on video images mainly includes four aspects: (1) Establishing models for radial distortion and tangential distortion to correct video images. (2) Establishing a registration method based on normalized Zernike moments to obtain the exterior orientation elements. The error of the result is only 0.5–1 pixel, which is far higher than registration based on a collinearity equation. (3) Establishing relative orientation based on essential matrix decomposition and nonlinear optimization. This involves uniformly using the speeded-up robust features algorithm with distance restriction and random sample consensus to select corresponding points. The vertical parallax of the stereo image pair model is less than one pixel, indicating that the accuracy is high. (4) A point cloud coloring method based on Gaussian distribution with central region restriction is adopted. Only pixels within the central region are considered valid for coloring. Then, the point cloud is colored based on the mean of the Gaussian distribution of the color set. In the colored point cloud, the textures of the buildings are clear, and targets such as windows, grass, trees, and vehicles can be clearly distinguished. Overall, the result meets the accuracy requirements of applications such as tunnel detection, street-view modeling and 3D urban modeling.

Исследование точности формирования снимка камерами со шторно-щелевым затвором при съемке с беспилотных воздушных судов

Quite often aerial survey made by unmanned aerial vehicle (UAV) is carried out using digital non-metric cameras that have rolling shutters. Variations in speed of a UAV and its angular elements of exterior orientation during aerial survey cause distortions in the images generated by the camera, which affects the accuracy of the processing results. The article shows the variations in the elements of exterior orientation during the operation of a rolling shutter and describes the way of its detection by flight recorder. The values of image distortion, caused by the variations in the angular elements of the exterior orientation of a snapshot, are calculated for the Sony Alpha A6000 digital non-metric camera.

Accurate Mapping Method for UAV Photogrammetry Without Ground Control Points in the Map Projection Frame

Unmanned aerial vehicle (UAV) photogrammetry without ground control points (GCPs) can effectively improve production efficiency and reduce production costs; this method is especially advantageous in areas that are difficult for people to reach. However, there are a series of problems in UAV photogrammetry without GCPs. One of the main problems is that the accurate camera parameters cannot be obtained through the on-the-job calibration method; furthermore, the inaccurate principal distance will have a serious impact on the elevation accuracy of object points. The other one is that the projection deformation and earth curvature also have impacts on the elevation accuracy, when the mapping task is carried out in the map projection frame. This article explains the specific reasons of elevation errors and proposes an effective solution. First, the camera self-calibration is performed in a geocentric frame with control strips. Then, the exterior orientation elements of the images are calculated in the map projection frame without control strips. Finally, the elevation errors that are caused by the map projection deformation and the earth’s curvature are corrected. The experimental results show that the proposed method can achieve accurate mapping, and the elevation accuracy has been significantly improved.

On-Orbit Calibration of Installation Matrix between Remote Sensing Camera and Star Camera Based on Vector Angle Invariance.

To achieve photogrammetry without ground control points (GCPs), the precise measurement of the exterior orientation elements for the remote sensing camera is particularly important. Currently, the satellites are equipped with a GPS receiver, so that the accuracy of the line elements of the exterior orientation elements could reach centimeter-level. Furthermore, the high-precision angle elements of the exterior orientation elements could be obtained through a star camera which provides the direction reference in the inertial coordinate system and star images. Due to the stress release during the launch and the changes of the thermal environment, the installation matrix is variable and needs to be recalibrated. Hence, we estimate the cosine angle vector invariance of a remote sensing camera and star camera which are independent of attitude, and then we deal with long-term on-orbit data by using batch processing to realize the accurate calibration of the installation matrix. This method not only removes the coupling of attitude and installation matrix, but also reduces the conversion error of multiple coordinate systems. Finally, the geo-positioning accuracy in planimetry is remarkably higher than the conventional method in the simulation results.

STUDY OF DIGITAL CAMERA CALIBRATION ON A FLAT TEST OBJECT

Abstract. The paper presents a method for calibration of digital cameras based on the use of a flat test object. The main distinctive feature of this method is that the camera is fixed and does not change its position in space. A series of images of a flat test object (for example, a chessboard) is taken at various distances and inclination of the test object. One should tilt the test object relative to the image plane in order to avoid ambiguity in solving the problem. All these images are measured separately, and processing is performed together, counting them as one image with common exterior orientation elements. Experimental studies have shown sufficient efficiency of such calibration, which is easily implemented and gives positive results in comparison with the classical calibration of the camera on a spatial test object. The results of experimental studies on real images have shown that proposed calibration method gives the comparable with conventional method accuracy. Thus, it is possible to use simple and cheap flat calibration test object instead of spatial one. To achieve the maximum result in camera calibration accuracy, use this method to tilt the chessboard at angles in the range of 30–40 degrees to the optical axis of the camera being calibrated.

OPTIMIZATION ALGORITHM OF TILTED IMAGE MATCHING BASED ON ADAPTIVE INITIAL OBJECT ASPECT

Abstract. When adopting the matching method of the least squares image based on object-patch to match tilted images, problems like the low degree of connection points for images with the discontinuity of depth or the discrepancy in elevation or low availability of aerotriangulation points would frequently appear. To address such problems, a tilted-image-matching algorithm based on an adaptive initial object-patch is proposed by this paper. By means of the existing initial values of the interior and exterior orientation elements of the tilted image and the information of object points generated in the matching process, the algorithm takes advantage of the method of multi-patch forward intersection and object variance partition so as to adaptively calculate the elevation of the object-patch and the initial value of the normal vector direction angle. Furthermore, this algorithm aims to solve the problem of difficulties in matching the tilted image with its corresponding points brought about by the low accuracy of the initial value of the tilted image when adopting the matching method of the least squares image based on object-patch to match the tilted image with high discrepancy in elevation. We adopt the algorithm as proposed in this paper and the least squares image matching method in which the initial state of the object-patch is horizontal to the object-patch respectively to conduct the verification process of comparing and matching two groups of tilted images. Finally, the effectiveness of the algorithm as proposed in this paper is verified by the testing results.

FAST AND ROBUST REGISTRATION OF AERIAL IMAGES AND LIDAR DATA BASED ON STRUCTURAL FEATURES AND 3D PHASE CORRELATION

Abstract. Co-Registration of aerial imagery and Light Detection and Ranging (LiDAR) data is quilt challenging because the different imaging mechanism causes significant geometric and radiometric distortions between such data. To tackle the problem, this paper proposes an automatic registration method based on structural features and three-dimension (3D) phase correlation. In the proposed method, the LiDAR point cloud data is first transformed into the intensity map, which is used as the reference image. Then, we employ the Fast operator to extract uniformly distributed interest points in the aerial image by a partition strategy and perform a local geometric correction by using the collinearity equation to eliminate scale and rotation difference between images. Subsequently, a robust structural feature descriptor is build based on dense gradient features, and the 3D phase correlation is used to detect control points (CPs) between aerial images and LiDAR data in the frequency domain, where the image matching is accelerated by the 3D Fast Fourier Transform (FFT). Finally, the obtained CPs are employed to correct the exterior orientation elements, which is used to achieve co-registration of aerial images and LiDAR data. Experiments with two datasets of aerial images and LiDAR data show that the proposed method is much faster and more robust than state of the art methods.

The Satellite’s On-Orbit Attitude System Error Compensation Technique Based on Stereo Models

According to the problem that the low measurement accuracy of TH-1 satellite star sensor, the low frequency and “slow drift” error which cannot be ignored in the attitude determination system, resulting in obvious random error in the horizontal position and elevation direction, and the change of the error with time and latitude, cannot be calibrated by the ground field of the real problem. In this paper, a low frequency detection model is established by using the principle of relative orientation, and the low frequency error is obtained by parallax elimination. Finally, the satellite attitude is compensated and the more accurate exterior orientation elements are obtained, thus improving the positioning accuracy and stability. The experimental results show that: the proposed methods are feasible, and by using the model to dynamically calibrate the exterior orientation angle elements on orbit, the plane and elevation errors of the ground points can be basically eliminated. The global uncontrollable positioning accuracy and stability of the photogrammetry satellite are improved.

Алгоритм определения элементов ориентирования тепловизионного снимка с беспилотного летательного аппарата по опорным точкам

The article deals with the algorithm for determining the exterior and interior orientation elements of an infrared image obtained from an unmanned aerial vehicle using four reference points. The idea of the proposed algorithm is to determine the true position of the image by the defined three-dimensional spatial coordinates of the reference points in the image at the time of shooting. The image plane is built up on the defined points. The coordinates of the principal point of the image are calculated by making a perpendicular from the perspective center to the plane of the image. The focal length is equal to the length of this perpendicular. Euler angles characterizing the position of the camera at the time of shooting are calculated after determining the axes’ directions of the inclined image coordinate system. The proposed algorithm is effective even in the case when all the elements of the image orientation are unknown. Calculations of the image elements on model examples with different initial data show high accuracy. The possibility of obtaining the necessary accuracy for the orthotransformation procedure was confirmed on real images.

Determining the Elements of Exterior Orientation in Aerial Triangulation Processing Using UAV Technology

Unmanned Aerial Vehicles (UAVs) are still an interesting and current research topic in photogrammetry. An important issue in this area is determining the elements of exterior orientation of image data acquired at low altitudes. The article presents selected mathematical methods (TGC, TIC, TAD) of estimating elements of exterior orientation for image data obtained at low altitudes. The measurement data for the experimental test were recorded by the Unmanned Aerial Vehicle platform Trimble UX-5. In the framework of the test photogrammetric flight, the authors obtained 506 images and navigation data specifying the position and orientation of the Unmanned Aerial Vehicle. As a result of the research, it is proven possible to show the usefulness of the mathematical models (TGC, TIC, TAD) in estimation of elements of exterior orientation.

DEVELOPMENT OF MULTI-VIEW STEREO CONSIDERING ACCURACY OF EXTERIOR ORIENTATION ELEMENTS

Abstract. Structure from motion (SfM) has been widely used to achieve automatic 3D reconstructions. However, as the 3D point clouds obtained via SfM are sparse, multi-view stereo (MVS) was developed to compensate for this sparseness. The accuracy of the 3D surface depends on the accuracy of the orientation elements based on the SfM. Additionally, in the case of an unmanned aerial vehicle (UAV), SfM exhibits a decrease in the accuracy of the orientation elements during complex camera movements. This paper proposes a patch-based MVS (PMVS) method considering the accuracy of the orientation elements. The proposed method involves applying the global SfM, estimating accuracy of exterior orientation (EO) elements, and introducing the accuracy of EO elements to PMVS. The PMVS approximates an object surface by using small rectangular patches, namely local tangent plane approximation. The patches are optimized by minimizing the sum of the photometric discrepancy scores. The accuracy of the EO elements is introduced to the patch optimization as weighting function. This accuracy is defined using the variances of the estimated parameters in the bundle adjustment. We also investigate the types of weighting functions. The results indicate that the proposed method is capable of considering geometric conditions during patch estimation. The proposed method was applied to the three types of image datasets, i.e., images captured using an SLR camera at ground level, images captured using a UAV equipped with a SLR camera, and images captured using an airplane equipped with an oblique camera. Through the experimental results, the improved accuracy and the effectiveness of the proposed method were confirmed.

Determining of correlation relationship between roll, pitch, and yaw for UAVs

Abstract Currently, UAVs are intensively being introduced into topographic-photogrammetric production for topographic digital aerial photography and laser scanning. These technologies have a number of advantages: they don’t require specially prepared platforms and launchers, they are relatively inexpensive unlike large aircrafts, and they are safe. However, there are still many unsolved problems for ultralight UAVs, especially when the aerial photography is made. As you know, the requirements for the implementation of the aerial survey process are quite stringent, first of all, for horizontal flight: the angles of inclination must be within 3–5 degrees, since exceeding these tolerances significantly affects the accuracy for determining the spatial coordinates of objects. Therefore, there was an idea to conduct researches of dependences between the pitch α, roll ω and yaw κ. For this purpose, 100 images obtained from aircraft-type UAV ‘Arrow’ developed and created by specialists from Lviv Polytechnic National University and ‘Abris’ were used. As a result of the study, the multiple correlation coefficient and the parameters of the linear regression equation for the angular elements of the exterior orientation of digital images were calculated. In addition, statistical quality evaluations for the obtained regression model were carried out. Analysis of the received data allows to assert that angular elements of exterior orientation are correlated with each other. Therefore, in the further imaging materials, processing it becomes possible to make compensation of this fact and to improve calculation accuracy of spatial coordinates of points.

Error analysis of exterior orientation elements on geolocation for a Moon-based Earth observation optical sensor

ABSTRACTThe Moon is a potential new platform for Earth observation. The advantages of its large-scale observational scope, long temporal duration, and multi-layer detecting of the Earth will undoubtedly advance our understanding of the Earth system. To carry out the observations from a Moon-based optical sensor, the geolocation error caused by exterior orientation elements need to be investigated. This paper analyses the error effects of exterior orientation elements on geolocation for an optical sensor. To estimate the error, we present a geometric image model and utilise some parameters to measure the image offsets. Through a large number of numerical simulations, the results demonstrate that the image offsets are not obvious influenced by the distance and observation angle at mid-high latitude of the Moon and have linear correlation with the increasing errors of the exterior orientation elements. Further, the relationship between the spatial resolution and errors of exterior orientation elements are revealed. Finally, the error characteristics for Moon-based Earth observation are discussed. It is expected that the conclusion drawn in this paper could support the study of a Moon-based Earth observation optical sensor.

BUILDING SHADOW DETECTION FROM GHOST IMAGERY

Abstract. Shadow is one of the basic features of remote sensing image, it expresses a lot of information of the object which is loss or interference, and the removal of shadow is always a difficult problem to remote sensing image processing. In this paper, it is mainly analyzes the characteristics and properties of shadows from the ghost image (traditional orthorectification). The DBM and the interior and exterior orientation elements of the image are used to calculate the zenith angle of sun. Then this paper combines the scope of the architectural shadows which has be determined by the zenith angle of sun with the region growing method to make the detection of architectural shadow areas. This method lays a solid foundation for the shadow of the repair from the ghost image later. It will greatly improve the accuracy of shadow detection from buildings and make it more conducive to solve the problem of urban large-scale aerial imagines.

EMERGENCY RESCUE MANAGEMENT SUPPORTED BY UAV REMOTE SENSING DATA

Abstract. In the proposed SAFEDAM system, aerial and satellite-based information is used for the monitoring of river bodies, flood monitoring during the event, and for post-disaster damage assessment. UAV constitute a valuable source of information about the current situation in the field during the operation of emergency services. Time is crucial, and the basic assumption to use UAV remote sensing data is to make them available immediately after landing. Therefore, the approach of automatic orthomosaics created based on the exterior orientation of the transmitted images using direct georeference was selected instead of sophisticated automatic on-the-fly image-matching and georeferencing. The article conveys the justification for selecting this option in order to process orthomosaics with lower localisation accuracy in a short time. The developed algorithm takes into account the elements of exterior and interior orientation of the camera as well as the digital terrain model. The evaluation of the orthomosaic was conducted based on theorthophoto created in post-processing. Thanks to the approach used, images from the platform are available in the near real time on the screen of the interventional mode of the SAFEDAM system and are an extremely valuable and informative source of data. The system also integrates tools, which support the management of the action and prepares site documentation.

Measurement and Precision Analysis of Exterior Orientation Element Based on Landmark Point Auxiliary Orientation

Obtaining the interior and exterior orientation elements and distortion parameters of camera is critical in the applications of photogrammetric survey and computer vision on recovering object position and angular orientation. While being used outside as supervision devices monitoring objects in the air, cameras are generally installed in non-calibration designated regions with high erection and elevation postures within their view distance over the sky. Therefore, it is really hard to obtain the exterior orientation element of camera. In order to solve this problem, this paper proposed a method to obtain the exterior orientation element of camera outdoors by applying landmark point auxiliary orientation and to obtain the interior and exterior orientation elements and distortion coefficient of camera by using self-check bundle adjustment in indoor three-dimensional control field. Thus, the transformational relations between the landmark points of different positions could be calculated according to the method of landmark point auxiliary orientation. The exterior orientation elements of camera could be recovered through transformational relations, and the position and posture of recovered camera in outdoor non-calibration designated field could be simulated accordingly. The experiential results show that the exterior orientation elements of camera can be effectively recovered through the method of landmark point auxiliary orientation while certain high intersection precision can be maintained.

Shipborne Over- and Under-Water Integrated Mobile Mapping System and Its Seamless Integration of Point Clouds

ABSTRACTThe rapid acquisition techniques of over- and under-water geoinformation is urgently needed. The Shipborne Over- and Under-water integrated Mobile Mapping System (SiOUMMS) applies plug-in and multi-process network technology to achieve integrated data acquisition and cooperative control of multi-sensors, such as Global Navigation Satellite System, Inertial Measurement Unit, laser scanner, and multi-beam echo sounder. The proposed system can meet the requirements of diverse application scenarios and sensors, allowing for flexibility and scalability. Meanwhile, coarse calibration based on a set of common points and fine calibration using the plane feature constraint are proposed for integratedly obtaining the exterior orientation elements of the laser scanner and the multi-beam echo sounder. Together, these methods provide theoretical support and technical help to the seamless integration of over- and under-water point clouds. Experiments are also described, proving that the root mean square error of over-water point clouds is 0.133 m, the average elevation clearance spacing between adjacent over- and under-water point clouds is less than 0.3 m, and the average horizontal clearance spacing is less than 0.2 m. At the same time, cases in this paper show that SiOUMMS has made important achievements in the integrated measurement of reefs, coastal zone, and inland waterways, and has broad prospects for future applications.

GEOPOSITIONING PRECISION ANALYSIS OF MULTIPLE IMAGE TRIANGULATION USING LRO NAC LUNAR IMAGES

Abstract. This paper presents an empirical analysis of the geopositioning precision of multiple image triangulation using Lunar Reconnaissance Orbiter Camera (LROC) Narrow Angle Camera (NAC) images at the Chang’e-3(CE-3) landing site. Nine LROC NAC images are selected for comparative analysis of geopositioning precision. Rigorous sensor models of the images are established based on collinearity equations with interior and exterior orientation elements retrieved from the corresponding SPICE kernels. Rational polynomial coefficients (RPCs) of each image are derived by least squares fitting using vast number of virtual control points generated according to rigorous sensor models. Experiments of different combinations of images are performed for comparisons. The results demonstrate that the plane coordinates can achieve a precision of 0.54 m to 2.54 m, with a height precision of 0.71 m to 8.16 m when only two images are used for three-dimensional triangulation. There is a general trend that the geopositioning precision, especially the height precision, is improved with the convergent angle of the two images increasing from several degrees to about 50°. However, the image matching precision should also be taken into consideration when choosing image pairs for triangulation. The precisions of using all the 9 images are 0.60 m, 0.50 m, 1.23 m in along-track, cross-track, and height directions, which are better than most combinations of two or more images. However, triangulation with selected fewer images could produce better precision than that using all the images.

GEOPOSITIONING PRECISION ANALYSIS OF MULTIPLE IMAGE TRIANGULATION USING LRO NAC LUNAR IMAGES

This paper presents an empirical analysis of the geopositioning precision of multiple image triangulation using Lunar Reconnaissance Orbiter Camera (LROC) Narrow Angle Camera (NAC) images at the Chang’e-3(CE-3) landing site. Nine LROC NAC images are selected for comparative analysis of geopositioning precision. Rigorous sensor models of the images are established based on collinearity equations with interior and exterior orientation elements retrieved from the corresponding SPICE kernels. Rational polynomial coefficients (RPCs) of each image are derived by least squares fitting using vast number of virtual control points generated according to rigorous sensor models. Experiments of different combinations of images are performed for comparisons. The results demonstrate that the plane coordinates can achieve a precision of 0.54 m to 2.54 m, with a height precision of 0.71 m to 8.16 m when only two images are used for three-dimensional triangulation. There is a general trend that the geopositioning precision, especially the height precision, is improved with the convergent angle of the two images increasing from several degrees to about 50°. However, the image matching precision should also be taken into consideration when choosing image pairs for triangulation. The precisions of using all the 9 images are 0.60 m, 0.50 m, 1.23 m in along-track, cross-track, and height directions, which are better than most combinations of two or more images. However, triangulation with selected fewer images could produce better precision than that using all the images.