Applications In Photogrammetry Research Articles

Mathematical Camera Array Optimization for Face 3D Modeling Application.

Camera network design is a challenging task for many applications in photogrammetry, biomedical engineering, robotics, and industrial metrology, among other fields. Many driving factors are found in the camera network design including the camera specifications, object of interest, and type of application. One of the interesting applications is 3D face modeling and recognition which involves recognizing an individual based on facial attributes derived from the constructed 3D model. Developers and researchers still face difficulty in reaching the required high level of accuracy and reliability needed for image-based 3D face models. This is caused among many factors by the hardware limitations and imperfection of the cameras and the lack of proficiency in designing the ideal camera-system configuration. Accordingly, for precise measurements, we still need engineering-based techniques to ascertain the specific level of deliverables quality. In this paper, an optimal geometric design methodology of the camera network is presented by investigating different multi-camera system configurations composed of four up to eight cameras. A mathematical nonlinear constrained optimization technique is applied to solve the problem and each camera system configuration is tested for a facial 3D model where a quality assessment is applied to conclude the best configuration. The optimal configuration is found to be a 7-camera array, comprising a pentagon shape enclosing two additional cameras, offering high accuracy. For those who prioritize point density, a 9-camera array with a pentagon and quadrilateral arrangement in the X-Z plane is a viable choice. However, a 5-camera array offers a balance between accuracy and the number of cameras.

Elevation Estimation-Driven Building 3-D Reconstruction From Single-View Remote Sensing Imagery

Building 3D reconstruction from remote sensing images has a wide range of applications in smart cities, photogrammetry and other fields. Methods for automatic 3D urban building modeling typically employ multi-view images as input to algorithms to recover point clouds and 3D models of buildings. However, such models rely heavily on multi-view images of buildings, which are time-intensive and limit the applicability and practicality of the models. To solve these issues, we focus on designing an efficient DSM estimation-driven reconstruction framework (Building3D), which aims to reconstruct 3D building models from the input single-view remote sensing image. Existing DSM estimation networks suffer from the imbalance between local features and global features, which leads to over-smooth DSM estimates at instance boundaries. To address this issue, we propose a Semantic Flow Field-guided DSM Estimation (SFFDE) network, which utilizes the proposed concept of elevation semantic flow to achieve the registration of local and global features. First, in order to make the network semantics globally aware, we propose an Elevation Semantic Globalization (ESG) module to realize the semantic globalization of instances. Further, in order to alleviate the semantic span of global features and original local features, we propose a Local-to-Global Elevation Semantic Registration (L2G-ESR) module based on elevation semantic flow. Our Building3D is rooted in the SFFDE network for building elevation prediction, synchronized with a building extraction network for building masks, and then sequentially performs point cloud reconstruction and surface reconstruction (or CityGML model reconstruction). On this basis, our Building3D can optionally generate CityGML models or surface mesh models of the buildings. Extensive experiments on ISPRS Vaihingen and DFC2019 datasets on the DSM estimation task show that our SFFDE significantly improves upon state-of-the-art and δ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , δ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> and δ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> metrics of our SFFDE are improved to 0.595, 0.897 and 0.970. Furthermore, our Building3D achieves impressive results in the 3D point cloud and 3D model reconstruction process.

A Precisely One-Step Registration Methodology for Optical Imagery and LiDAR Data Using Virtual Point Primitives

The registration of optical imagery and 3D Light Detection and Ranging (LiDAR) point data continues to be a challenge for various applications in photogrammetry and remote sensing. In this paper, the framework employs a new registration primitive called virtual point (VP) that can be generated from the linear features within a LiDAR dataset including straight lines (SL) and curved lines (CL). By using an auxiliary parameter (λ), it is easy to take advantage of the accurate and fast calculation of the one-step registration transformation model. The transformation model parameters and λs can be calculated simultaneously by applying the least square method recursively. In urban areas, there are many buildings with different shapes. Therefore, the boundaries of buildings provide a large number of SL and CL features and selecting properly linear features and transforming into VPs can reduce the errors caused by the semi-discrete random characteristics of the LiDAR points. According to the result shown in the paper, the registration precision can reach the 1~2 pixels level of the optical images.

Calibration and rectification of bi-telecentric lenses in Scheimpflug condition

In the lens-based imaging model, the Scheimpflug principle is expressed as the object plane, the image plane, and the lens plane intersect in a line. With this principle, the focused object plane in the lens's object side can be tilted by placing a tilted sensor at the image side of the lens; thereby, multi-cameras can be focused on the same object space with an overlapping field of view and depth of field. For Scheimpflug cameras, additional tilt angles between the camera sensor and the optical axis are introduced, which has been well studied in pinhole cameras' calibration methods. Telecentric lens, as a commonly used lens type, has constant magnification in the axial direction and has a wide range of applications in close-range photogrammetry. To calibrate and rectify the telecentric lenses in Scheimpflug conditions, we derived a concise imaging model by expressing the sensor tilt angles and the lens magnification into a simplified intrinsic matrix. Based on the derived imaging model, an integrated calibration algorithm without solving the tilt angles and a stereo-rectification method for stereo matching are developed. The effectiveness and accuracy of the proposed methods are verified by experiments, including the comparison with the traditional telecentric model and pinhole model. Combined with the experimental results, we analyzed the potential impact of the extrinsic rotation matrix's ambiguity, verified whether the lens distortion affects the re-projection error, and discussed how the calibration posture influences the calibration accuracy.

MOUNTING CALIBRATION OF A MULTI-VIEW CAMERA SYSTEM ON A UAV PLATFORM

Abstract. Multi-view camera systems are used more and more frequently for applications in close-range photogrammetry, engineering geodesy and autonomous navigation, since they can cover a large portion of the environment and are considerably cheaper than alternative sensors such as laser scanners. In many cases, the cameras do not have overlapping fields of view. In this paper, we report on the development of such a system mounted on a rigid aluminium platform, and focus on its geometric system calibration. We present an approach for estimating the exterior orientation of such a multi-camera system based on bundle adjustment. We use a static environment with ground control points, which are related to the platform via a laser tracker. In the experimental part, the precision and partly accuracy that can be achieved in different scenarios is investigated. While we show that the accuracy potential of the platform is very high, the mounting calibration parameters are not necessarily precise enough to be used as constant values after calibration. However, this disadvantage can be mitigated by using those parameters as observations and refining them on-the-job.

PREFACE: TECHNICAL COMMISSION II

Abstract. ISPRS Technical Commission II focuses, at various scales, on geometric, radiometric and multi-temporal aspects of image- and range-based 3D surveying and modeling. Specifically, Commission II deals with image orientation, point cloud generation and processing, 3D feature extraction, scene understanding, sensor and data fusion, sensor characterization, machine learning for geospatial data analysis and big data techniques for massive data processing. Applications in the fields of mapping, industry, heritage, space, underwater and environment are considered.The Volume related to Commission II contains 182 papers published in the ISPRS Archives and 118 published in the ISPRS Annals. The Archives were accepted on the base of an abstract review from originally submitted 262 abstracts. The Annals comprise reviewed articles accepted from the originally submitted 191 full papers.Considering the aforementioned research issues, challenges and needs, the papers published depict the latest developments in methodological aspects and the most interesting applications in photogrammetry.Great interest was dedicated to machine/deep learning methods used to solve various steps of the photogrammetric pipeline and in particular for the semantic enrichment of images and 3D data. Many papers reported new models and methods to extract features, geometrical primitives and objects from data acquired by airborne and/or terrestrial sensors, including object recognition and 3D object/scene reconstruction.Various contributions referred to the utilization, integration, modeling and performance analyses of imaging and ranging sensors in the industry sector, environment and heritage fields. Cultural Heritage (including Underwater) attracted many submissions with various worldwide applications which depicted how 3D documentation is increasingly used and crucial for conservation, preservation and valorisation needs.We believe these volumes nicely recap the state of the art, current trends and possible applications in photogrammetry, with a nice overlook at the forthcoming years of investigations.On behalf of Technical Commission II, we would like to thank the local organisers of the 2020 ISPRS Congress, the members of the international program committee, all Working Group officers and all reviewers for their hard organizational work and efforts in the paper reviewing process.

PREFACE: TECHNICAL COMMISSION II

Abstract. ISPRS Technical Commission II focuses, at various scales, on geometric, radiometric and multi-temporal aspects of image- and range-based 3D surveying and modeling. Specifically, Commission II deals with image orientation, point cloud generation and processing, 3D feature extraction, scene understanding, sensor and data fusion, sensor characterization, machine learning for geospatial data analysis and big data techniques for massive data processing. Applications in the fields of mapping, industry, heritage, space, underwater and environment are considered.The Volume related to Commission II contains 182 papers published in the ISPRS Archives and 118 published in the ISPRS Annals. The Archives were accepted on the base of an abstract review from originally submitted 262 abstracts. The Annals comprise reviewed articles accepted from the originally submitted 191 full papers.Considering the aforementioned research issues, challenges and needs, the papers published depict the latest developments in methodological aspects and the most interesting applications in photogrammetry.Great interest was dedicated to machine/deep learning methods used to solve various steps of the photogrammetric pipeline and in particular for the semantic enrichment of images and 3D data. Many papers reported new models and methods to extract features, geometrical primitives and objects from data acquired by airborne and/or terrestrial sensors, including object recognition and 3D object/scene reconstruction.Various contributions referred to the utilization, integration, modeling and performance analyses of imaging and ranging sensors in the industry sector, environment and heritage fields. Cultural Heritage (including Underwater) attracted many submissions with various worldwide applications which depicted how 3D documentation is increasingly used and crucial for conservation, preservation and valorisation needs.We believe these volumes nicely recap the state of the art, current trends and possible applications in photogrammetry, with a nice overlook at the forthcoming years of investigations.On behalf of Technical Commission II, we would like to thank the local organisers of the 2020 ISPRS Congress, the members of the international program committee, all Working Group officers and all reviewers for their hard organizational work and efforts in the paper reviewing process.

Modeling morphodynamic processes in a meandering river with unmanned aerial vehicle-based measurements

Change detection analysis for monitoring and modeling riverine systems requires detailed spatial and temporal surveying of river morphology dynamics. Accurate digital surface models are an essential part of the studies of morphodynamic modeling of rivers and river channel evolution. An accurate high-resolution surface model of the river channel and floodplain enables a more comprehensive view of the riverbed evolution and allows monitoring of morphological changes of the entire river channel more precisely compared with traditional methods. In current practice, unmanned aerial systems (UAS) are widely used for various applications in photogrammetry, especially for three-dimensional surface modeling and large-scale mapping of riverine environments. UASs offer many advantages in terms of cost and image resolution and approaches for mapping across varying spatial extents when compared with terrestrial photogrammetry and satellite remote sensing systems. UAS-based aerial mapping was implemented in order to produce very high resolution surface models of a flood-sensitive region based on structure from motion technique. Ultimately, multitemporal topographic data were produced, and morphodynamic processes in the lower course of the Buyuk Menderes River in Turkey were modeled by digital shoreline analysis system and digital elevation model of difference methods. These methods were employed to examine the changes in the shoreline and to analyze the size of geomorphological changes and spatial patterns. The results showed that the change in the shoreline of the meanders varied from 3 to 27 m, and the water levels varied between ∼0.3 and 3 m. Although there was both sediment erosion and deposition along the shoreline, the predominant process was identified as deposition in the shoreline. A significant correlation was found between the deposited sediments and the sinuosity index values ( r = 0.88) according to the changes in water level over the months. In addition, major changes on the deposition rate were found to occur mostly after the summer season. This research showed that UASs could provide a suitable measurement model for determining areal and volumetric eroded/deposited sediment quantities along the meandering fields.

A FEASIBILITY STUDY ON INCREMENTAL BUNDLE ADJUSTMENT WITH FISHEYE IMAGES AND LOW-COST SENSORS

Abstract. Low cost imaging and positioning sensors are opening new frontiers for applications in near real-time Photogrammetry. Omnidirectional cameras acquiring images with 360° coverage, when combined with information coming from GNSS (Global Navigation Satellite Systems) and IMU (Inertial Measurement Unit), can efficiently estimate orientation and object space structure. However, several challenges remain in the use of low-cost sensors and image observations acquired by sensors with non-perspective inner geometry. The accuracy of the measurement using low-cost sensors is affected by different sources of errors and sensor stability. Microelectromechanical systems (MEMS) present a large gap between predicted and actual accuracy. This work presents a study on the performance of an integrated sensor orientation approach to estimate sensor orientation and 3D sparse point cloud, using an incremental bundle adjustment strategy and data coming from a low-cost portable mobile terrestrial system composed by off-theshelf navigation systems and a poly-dioptric system (Ricoh Theta S). Experiments were performed in an outdoor area (sidewalk), achieving a trajectory positional accuracy of 0.33 m and a meter level 3D reconstruction.

AUTOMATIC CAMERA SYSTEM CALIBRATION WITH A CHESSBOARD ENABLING FULL IMAGE COVERAGE

Abstract. Geometric camera calibration is a mandatory prerequisite for many applications in computer vision and photogrammetry. Especially when requiring an accurate camera model the effort for calibration can increase dramatically. For the calibration of the stereo-camera used for optical navigation a new chessboard based approach is presented. It is derived from different parts of existing approaches which, taken separately, are not able to meet the requirements. Moreover, the approach adds one novel main feature: It is able to detect all visible chessboard fields with the help of one or more fiducial markers simply sticked on a chessboard (AprilTags). This allows a robust detection of one or more chessboards in a scene, even from extreme perspectives. Except for the acquisition of the calibration images the presented approach enables a fully automatic calibration. Together with the parameters of the interior and relative orientation the full covariance matrix of all model parameters is calculated and provided, allowing a consistent error propagation in the whole processing chain of the imaging system. Even though the main use case for the approach is a stereo camera system it can be used for a multi-camera system with any number of cameras mounted on a rigid frame.

Complete Initial Solutions for Iterative Pose Estimation From Planar Objects

Camera pose estimation from the image of a planar object has important applications in photogrammetry and computer vision. In this paper, an efficient approach to find the initial solutions for iterative camera pose estimation using coplanar points is proposed. Starting with homography, the proposed approach provides a least-squares solution for absolute orientation, which has a relatively high accuracy and can be easily refined into one optimal pose that locates local minima of the according error function by using Gauss-Newton scheme or Lu’s orthogonal iteration algorithm. In response to ambiguities that exist in pose estimation from planar objects, we propose a novel method to find initial approximation of the second pose, which is different from existing methods in its concise form and clear geometric interpretation. Thorough testing on synthetic data shows that combined with currently employed iterative optimization algorithm, the two initial solutions proposed in this paper can achieve the same accuracy and robustness as the best state-of-the-art pose estimation algorithms, while with a significant decrease in computational cost. Real experiment is also employed to demonstrate its performance.

From Photogrammetry to Computer Vision

We outline the history of photogrammetry from the aspects of, perspective geometry, camera, platform, measure methods and measure instruments, and summarize previous contributions to photogrammetry. A brief review of computer vision history is given. The tight connections between computer vision and photogrammetry are discussed in terms of geometric principles, and some differences in applications are also considered. From the aspect of semantics, we analyze the development of remote sensing and its relations to machine learning and computer vision, including their common approaches and different applications. The prevailing deep learning raised from connectionism is also reviewed and its successful applications in photogrammetry are analyzed. At last, we expect that the future development of photogrammetry will be more tightly cross-integrated with computer vision, machine learning and artificial intelligence.

A DATA DRIVEN METHOD FOR FLAT ROOF BUILDING RECONSTRUCTION FROM LiDAR POINT CLOUDS

Abstract. 3D building modeling is one of the most important applications in photogrammetry and remote sensing. Airborne LiDAR (Light Detection And Ranging) is one of the primary information sources for building modeling. In this paper, a new data-driven method is proposed for 3D building modeling of flat roofs. First, roof segmentation is implemented using region growing method. The distance between roof points and the height difference of the points are utilized in this step. Next, the building edge points are detected using a new method that employs grid data, and then roof lines are regularized using the straight line approximation. The centroid point and direction for each line are estimated in this step. Finally, 3D model is reconstructed by integrating the roof and wall models. In the end, a qualitative and quantitative assessment of the proposed method is implemented. The results show that the proposed method could successfully model the flat roof buildings using LiDAR point cloud automatically.

ASSESMENT OF THE INFLUENCE OF UAV IMAGE QUALITY ON THE ORTHOPHOTO PRODUCTION

Abstract. Over the past years a noticeable increase of interest in using Unmanned Aerial Vehicles (UAV) for acquiring low altitude images has been observed. This method creates new possibilities of using geodata captured from low altitudes to generate large scale orthophotos. Because of comparatively low costs, UAV aerial surveying systems find many applications in photogrammetry and remote sensing. One of the most significant problems with automation of processing of image data acquired with this method is its low accuracy. This paper presents the following stages of acquisition and processing of images collected in various weather and lighting conditions: aerotriangulation, generating of Digital Terrain Models (DTMs), orthorectification and mosaicking. In the research a compact, non-metric camera, mounted on a fuselage powered by an electric motor was used. The tested area covered flat, agricultural and woodland terrains. Aerotriangulation and point cloud accuracy as well as generated digital terrain model and mosaic exactness were examined. Dense multiple image matching was used as a benchmark. The processing and analysis were carried out with INPHO UASMaster programme. Based on performed accuracy analysis it was stated that images acquired in poor weather conditions (cloudy, precipitation) degrade the final quality and accuracy of a photogrammetric product by an average of 25%.

Automatic registration of optical imagery with 3D LiDAR data using statistical similarity

The development of robust and accurate methods for automatic registration of optical imagery and 3D LiDAR data continues to be a challenge for a variety of applications in photogrammetry, computer vision and remote sensing. This paper proposes a new approach for the registration of optical imagery with LiDAR data based on the theory of Mutual Information (MI), which exploits the statistical dependency between same- and multi-modal datasets to achieve accurate registration. The MI-based similarity measures quantify dependencies between aerial imagery, and both LiDAR intensity data and 3D point cloud data. The needs for specific physical feature correspondences, which are not always attainable in the registration of imagery with 3D point clouds, are avoided. Current methods for registering 2D imagery to 3D point clouds are first reviewed, after which the mutual MI approach is presented. Particular attention is given to adoption of the Normalised Combined Mutual Information (NCMI) approach as a means to produce a similarity measure that exploits the inherently registered LiDAR intensity and point cloud data so as to improve the robustness of registration between optical imagery and LiDAR data. The effectiveness of local versus global similarity measures is also investigated, as are the transformation models involved in the registration process. An experimental program conducted to evaluate MI-based methods for registering aerial imagery to LiDAR data is reported and the results obtained in two areas with differing terrain and land cover, and with aerial imagery of different resolution and LiDAR data with different point density are discussed. These results demonstrate the potential of the MI and especially the CMI methods for registration of imagery and 3D point clouds, and they highlight the feasibility and robustness of the presented MI-based approach to automated registration of multi-sensor, multi-temporal and multi-resolution remote sensing data for a wide range of applications.

E-LEARNING – BEST PRACTICE IN PHOTOGRAMMETRY, REMOTE SENSING AND GIS – STATUS AND CHALLENGES

Abstract. In addition to professional training, computer aided teaching has long tradition. The difficult economic situation, however, forced many customers to take drastic austerity measures in the field of learning. Cost pressure encouraged a new openness to innovative and tailored learning concepts. As a result e-learning gained more interest and importance promising great benefit to the user. Around the world a variety of well-designed e-learning products exist. The web pages of Commission VI/2 (http://www.igg.tuberlin. de/ISPRS/) provide a decision aid to locate relevant material. Links to websites known to the authors are listed; a search function allows selective access, taking account of quality criteria. This article describes best practice e-learning applications in photogrammetry, remote sensing and GIS. The rating is based on results of the Computer Assisted Teaching CONtest (CATCON) initiated by ISPRS, and on observation of recent developments.

Sampling Schemes for Multidimensional Signals With Finite Rate of Innovation

In this paper, we consider the problem of sampling signals that are nonband-limited but have finite number of degrees of freedom per unit of time and call this number the rate of innovation. Streams of Diracs and piecewise polynomials are the examples of such signals, and thus are known as signals with finite rate of innovation (FRI). We know that the classical (band-limited sine) sampling theory does not enable perfect reconstruction of such signals from their samples since they are not band-limited. However, the recent results on FRI sampling suggest that it is possible to sample and perfectly reconstruct such nonband-limited signals using a rich class of kernels. In this paper, we extend those results in higher dimensions using compactly supported kernels that reproduce polynomials (satisfy Strang-Fix conditions). In fact, the polynomial reproduction property of the kernel makes it possible to obtain the continuous moments of the signal from its samples. Using these moments and the annihilating filter method (Prony's method), the innovative part of the signal, and therefore, the signal itself is perfectly reconstructed. In particular, we present local (directional-derivatives-based) and global (complex-moments-based, Radon-transform-based) sampling schemes for classes of FRI signals such as sets of Diracs, bilevel, and planar polygons, quadrature domains (e.g., circles, ellipses, and cardioids), 2D polynomials with polygonal boundaries, and n-dimensional Diracs and convex polytopes. This work has been explored in a promising way in super-resolution algorithms and distributed compression, and might find its applications in photogrammetry, computer graphics, and machine vision.

Rejection of mismatched correspondences along the affine epipolar line

Identifying the correspondences of points between images is an important task. It finds applications in depth recovery, motion recovery, photogrammetry, and robotics. Commonly, one uses the epipolar geometry to constrain the problem from a two-dimensional to a one-dimensional search. However, the estimation of the epipolar parameters itself depends on finding a sufficiently accurate set of initial point correspondences. Mismatched correspondences present themselves as outliers in this parameter estimation. Even if the epipolar geometry is perfectly found, it is still not sufficient to disambiguate false matches whose points lie along the epipolar line. This paper attempts to identify (and thereby reject) such mismatched correspondences, within the framework of an affine projection model and the use of three images. The robust Least Median of Squares (LMedS) approach is utilized in two stages. First, it is used to estimate the epipolar parameters and reject those point-correspondences whose orthogonal distance to the epipolar lines are significant. Second, points are resolved along their epipolar lines and the LMedS is used to solve for a consistent rotation for all points using three images.

Building a Low Cost Image Space Plotter with Software and Hardware Components in Current Use

A photogrammetric plotting system based upon digitizing tablets, a personal computer and a CAD system is proposed. The system, which is capable of working stereoscopically with the aid of a mirror stereoscope, can perform inner, relative and absolute orientations, and functions like an image space plotter. Simultaneous plotting is achieved because the system runs from Autocad through a program written in Autolisp. In this way, all the CAD editing capabilities can be used and the final map can be transferred without problems into several well known geographic information systems. Applications in terrestrial photogrammetry are also possible. The accuracy depends directly on that of the digitizing tablets and it can be improved significantly with proper compensation of systematic deformations.

Applications of tree search methods in digital photogrammetry

Digital photogrammetry has emerged from the combination of analytical photogrammetry and digital image processing and interpretation techniques. Methods like resampling, contrast enhancement and image matching already found their way to many applications in photogrammetry. Other methods, like tree search, are less well known. This paper explores the possibilities for using tree search methods to solve photogrammetric problems. It is shown that tree search methods can be useful for line following and knowledge-based image analysis. Application of tree search methods to matching problems is also possible. This application, however, requires fairly good image segmentations, which usually cannot be obtained for aerial imagery.