Field Of Photogrammetry Research Articles

A local feature extraction method for UAV-based image registration based on virtual line descriptors

Image local feature extraction is extensively utilized in the field of photogrammetry where the spatial distribution of features is important in high-quality image matching, particularly in high-resolution unmanned aerial vehicle (UAV) image registration. Presently, the spatial distribution problems are considered in some local feature extraction methods, though these methods are designed for point descriptors. Line descriptors are more robust to repetitive patterns compared to point descriptors and have attracted extensive attention in recent years. Hence, a feature extraction method is designed in this paper for line descriptors based on the K-connected virtual line descriptors matching method. Using the four measures, the quality of local features is quantified, and a regular gridding strategy based on the quality of local features is applied in the feature selection procedure. The proposed feature extraction method was successfully applied to match various simulated and real UAV-based images. Based on the experimental results using real images, it is indicated that two evaluation criteria, namely the spatial distribution quality of features and the number of correct matches, are improved to at least 12% and 15%, respectively, for verifying the capability of the proposed method to enhance matching performance.

Urban Classification from Aerial and Satellite Images

Abstract When talking about land cover, we need to find a proper way to extract information from aerial or satellite images. In the field of photogrammetry, aerial images are generally acquired by optical sensors that deliver images in four bands (red, green, blue and near-infrared). Recent researches in this field demonstrated that for the image classification process is still place for improvement. From satellites are obtained multispectral images with more bands (e.g. Landsat 7/8 has 36 spectral bands). This paper will present the differences between these two types of images and the classification results using support-vector machine and maximum likelihood classifier. For the aerial and the satellite images we used different sets of classification classes and the two methods mentioned above to highlight the importance of choosing the classes and the classification method.

Building Extraction from Airborne Multi-Spectral LiDAR Point Clouds Based on Graph Geometric Moments Convolutional Neural Networks

Building extraction has attracted much attentions for decades as a prerequisite for many applications and is still a challenging topic in the field of photogrammetry and remote sensing. Due to the lack of spectral information, massive data processing, and approach universality, building extraction from point clouds is still a thorny and challenging problem. In this paper, a novel deep-learning-based framework is proposed for building extraction from point cloud data. Specifically, first, a sample generation method is proposed to split the raw preprocessed multi-spectral light detection and ranging (LiDAR) data into numerous samples, which are directly fed into convolutional neural networks and completely cover the original inputs. Then, a graph geometric moments (GGM) convolution is proposed to encode the local geometric structure of point sets. In addition, a hierarchical architecture equipped with GGM convolution, called GGM convolutional neural networks, is proposed to train and recognize building points. Finally, the test scenes with varying sizes can be fed into the framework and obtain a point-wise extraction result. We evaluate the proposed framework and methods on the airborne multi-spectral LiDAR point clouds collected by an Optech Titan system. Compared with previous state-of-the-art networks, which are designed for point cloud segmentation, our method achieves the best performance with a correctness of 95.1%, a completeness of 93.7%, an F-measure of 94.4%, and an intersection over union (IoU) of 89.5% on two test areas. The experimental results confirm the effectiveness and efficiency of the proposed framework and methods.

HIERARCHICAL APPROACH FOR DETECTING CHANGES WITH THE USE OF DIFFERENT PYRAMID LEVELS IN DENSE IMAGE MATCHING

Abstract. Many cities order spatial data systematically, in particular aerial nadir images and orthophotomaps. However, only the orthoimages and orthophotomaps are usually used by the city administration, particularly in spatial planning. Some of the users are not aware of the possibilities as to how the aerial images can be used. Spatial data users, who may not be specialists in photogrammetry, are sometimes not aware that it is possible to obtain 3D information from 2D images as a point cloud. The idea of dense image matching (DIM) is well-known and described in the field of photogrammetry. Although dense image matching is a time- and memory-consuming process, this does not present a major drawback with modern computing. Images for the test area – Warsaw – are characterised by Ground Sampling Distance (GSD) equal to 8 cm. These images can be successfully used in change detection processes, comparing the dense image matching point cloud from two different dates. What is important while considering land cover change detection, is that it is not necessary to generate a detailed and high-density point cloud, e.g. in order to detect changes in buildings. The main idea of the article is to present the possibility of using higher levels of images pyramid in dense image matching within the change detection process as a way to optimize the processing time and point cloud accuracy. Which level of pyramid is needed to detect different changes in urban land cover will also be discussed.

Application and analysis of image modeling based on UAV

As one of the important geographic information products, orthophotos play an extremely important role in the field of geographic information. With the rapid development of China’s economic construction and the continuous improvement of photogrammetry technology, traditional orthophoto has been replaced by real orthophoto to meet the requirements of large-scale accurate mapping. This article discusses the method of generating real shot images, uses drone images, and uses Zhanwei New Village in Wuhan as the survey area to lay out the ground control points. Based on Pix4D modeling, the air and three encryptions are completed and based on high-precision DSM Quickly generate real radiography products. The research in this paper shows that the corrected real image can eliminate the phenomenon of blocking the wall and the problem of occlusion. It has a good effect and can be used in the field of line drawing maps. It provides a simple and quick solution for the rapid acquisition of orthophotos in the field of photogrammetry.

Using Total Probability in Image Template Matching

Image template matching is a main task in photogrammetry and computer vision. The matching can be used to automatically determine the 3D coordinates of a point. A firstborn image matching method in fields of photogrammetry and computer vision is area-based matching, which is based on correlation measuring that uses normalised cross-correlation. However, this method fails at a discontinuous edge and at the area of low illumination or at geometric distortion because of changes in imaging location. Thus, these points are considered outliers. The proposed method measures correlations, which is based on normalised cross-correlation, at each point by using various sizes of window and then considering the probability of correlations for each window. Thereafter, the determined probability values are integrated. On the basis of a specific threshold value, the point of maximum total probability correlation is recognised as a corresponding point. The algorithm is applied to aerial images for Digital Surface Model (DSM) generation. Results show that the corresponding points are identified successfully at different locations, especially at a discontinuous point, and that a Digital Surface Model of high resolution is generated.

MEASUREMENT OF FLUID-STRUCTURE INTERACTION OF WIND TURBINES IN WIND TUNNEL EXPERIMENTS: CONCEPT AND FIRST RESULTS

Abstract. Fluid-structure interactions are crucial for the design of rotor blades of wind power systems. Up to now, the mutual interactions between rotor blades and turbulent wind flows have been treated by complex simulations or were observed at individual discrete points. In this paper, a measurement concept is presented where spatial information of the motion/deformation of a rotating wind turbine as well as the wind flow are recorded in wind tunnel experiments. Wind flow and motion behaviour are recorded simultaneously and contactless. Techniques from the field of photogrammetry and flow measurement techniques are combined, resulting in high demands on the measurement concept. Furthermore, solutions for the realisation of a common coordinate system as well as for the synchronisation of both measuring systems are presented. In addition, the validation of the entire measurement concept is carried out based on of some wind tunnel tests in which a single rotor blade is used for the moment. This showed that the measurement concept and the proposed solutions for the simultaneous recording of wind flows and rotor blade movements are suitable in principle and that movements can be recorded and reconstructed with high accuracy.

Image Registration With Fourier-Based Image Correlation: A Comprehensive Review of Developments and Applications

Fourier-based image correlation is a powerful area-based image registration technique, which involves aligning images based on a translation model or similarity model by means of the image information and operation in the frequency domain. In recent years, Fourier-based image correlation has made significant progress and attracted extensive research interest in a variety of applications, especially in the field of photogrammetry and remote sensing, leading to the development of a number of subpixel methods that have improved the accuracy and robustness. However, to date, a detailed review of the literature related to Fourier-based image correlation is still lacking. In this review, we aim at providing a comprehensive overview of the fundamentals, developments, and applications of image registration with Fourier-based image correlation methods. Specifically, this review introduces the principal laws underlying these methods, presents a survey of the existing subpixel methods calculated both in the spatial domain and in the frequency domain, summarizes the major applications from three aspects, and discusses the challenges and possible directions of future research. This review is expected to be beneficial for researchers working in the relevant fields to obtain an insight into the current state of the art, to develop new variants, to explore potential applications, and to suggest promising future trends of image registration with Fourier-based image correlation.

Multi-Scale Local Context Embedding for LiDAR Point Cloud Classification

The semantic interpretation using point clouds, especially regarding light detection and ranging (LiDAR) point cloud classification, has attracted a growing interest in the fields of photogrammetry, remote sensing, and computer vision. In this letter, we aim at tackling a general and typical feature learning problem in 3-D point cloud classification— how to represent geometric features by structurally considering a point and its surroundings in a more effective and discriminative fashion? Recently, enormous efforts have been made to design the geometric features, yet it is less investigated to fully explore the potentials of the features. For that, there have been many filter-based studies proposed by selecting a subset of the whole feature space for better representing the local geometry structure. However, such a hard-threshold selection strategy inevitably suffers from information loss. In addition, the construction of the geometric features is relatively sensitive to the size of the neighborhood. To this end, we propose to extract multi-scaled feature representations and locally embed them into a low-dimensional and robust subspace where a more compact representation with the intrinsic structure preservation of the data is expected to be obtained, thereby further yielding a better classification performance. In our case, we apply a popular manifold learning approach, that is, locality-preserving projections, for the task of learning low-dimensional embedding. Experimental results conducted on one LiDAR point cloud data set provided by the 2018 IEEE Data Fusion Contest demonstrate the effectiveness of the proposed method in comparison with several commonly used state-of-the-art baselines.

A hybrid deep learning–based model for automatic car extraction from high-resolution airborne imagery

Automatic car extraction (ACE) from high-resolution airborne imagery (i.e., true-orthophoto) has been a hot research topic in the field of photogrammetry and machine learning. ACE from high-resolution airborne imagery is the most suitable method for control and monitoring practices in large cities such as traffic management. The use of deep learning–based feature extraction methods, such as convolutional neural networks, have been providing state-of-the-art performance in the last few years, particularly, these techniques have been successfully applied to automatic object extraction from images. In this paper, we proposed a novel hybrid method to take advantage of the semantic segmentation of high-resolution airborne imagery to ACE that is realized based on the combination of deep convolutional neural networks and restricted Boltzmann machine (RBM). This hybrid method is called RBMDeepNet. We trained and tested our model on the ISPRS Potsdam and Vaihingen benchmark datasets (non-big data) which is more challenging for ACE. Here, Potsdam data is a true-color dataset, and Vaihingen data is a false-color dataset. The results obtained in the present study showed that the proposed method for ACE from high-resolution airborne imagery achieves a 7% improvement in accuracy with about 10% improvement in processing time compared to similar methods.

A new hierarchical method for automatic road centerline extraction in urban areas using LIDAR data

Road detection and road extraction are important and challenging issues in the fields of photogrammetry and remote sensing. Researchers have conducted wide research in this regard based on multispectral images and achieved relatively useful results. Image-data driven methods have some shortcomings such as shadows, eliminating small and long vehicles, geometric distortions, and occlusions. In recent years, in order to overcome the above limitations and complexities, several attempts have been done based on LIDAR data. The present paper proposes an automatic hierarchical road detection and extraction method. The main goal of this research is to increase the level of continuity of the road detection and extraction processes. This method includes the preprocessing of intensity data using a local minima filter, applying the Mean Shift segmentation to the refined intensity data, and finally integrating it with various nDSM-based Products. The proposed method involves not only considering both small and long vehicles as road features but also neglecting some parts of large parking lots based on the nearby neighborhood of parked vehicles as much as possible as non-road features. The next step was the process of road centerline extraction by adopting a Voronoi-diagram based approach and then removing dangle lines in several iterations. The proposed method was applied to the Vaihingen and Toronto datasets (ISPRS). The completeness of the two datasets is 95.85% and 88%, and the correctness of these datasets are 83.68% and 72.2%, respectively. The results were indicative of the great potential of the proposed method for effective road centerline extraction in urban areas.

360° IMAGES FOR UAV MULTISENSOR DATA FUSION: FIRST TESTS AND RESULTS

Abstract. A relevant research topic in the photogrammetry field is related with experimenting, at different levels, data fusion and sensors integration, aiming at the development of rapid mapping systems, capable of quickly delivering accurate data, for a wide range of applications. The presented contribute aims at exploiting the potentialities of spherical images and videos acquired using a 360° camera mounted on board of a medium-sized Unmanned Aerial Vehicle (UAV). More specifically, the focus has been the development and testing of a rapid-mapping hybrid system, capable of fast acquisition of data from both the on-board sensors. The reduction of the data acquisition time has been, and still is, a hot topic faced by the researchers in the Geomatics community, especially linked to some hazardous operative scenarios, where the reduction of the time on the field is crucial for operators’ safety reasons (like in post-earthquake early damage assessment surveys).After a deep analysis of the available scientific literature, it turned out that researchers have been more focused on terrestrial applications of these emerging sensors (360° cameras), and no significant studies for aerial application have been conducted yet. First of all, some laboratory tests have been carried out, in order to evaluate the metric accuracy of the 3D models generated using the employed 360° sensor; thereafter, a solution for acquiring spherical images from 360° camera mounted on a light UAV has been designed. Problems and issues have been addressed and discussed, and results and improvements are, at the end of the paper, evaluated and proposed.

Reliable image matching via photometric and geometric constraints structured by Delaunay triangulation

Image matching is a basic task in the field of photogrammetry and remote sensing. By using the advantages of the Delaunay triangulation, this paper proposes a novel image matching method. First, neighboring structures of randomly distributed feature points are formed with the assistance of the Delaunay triangulation and its corresponding graph, and the image planes are simultaneously divided into patches of near-regular triangles. Second, two constraints, a photometric constraint and a geometric constraint, are implemented based on the constructed neighboring structures, which incorporate the hierarchical elimination and left-right checking strategies to deliver the influences of outliers on the decision of inliers and ensure the high precision of the final matches. The former utilizes a line descriptor as a second-order photometric constraint, and the latter adopts the spatial angular order (SAO) to achieve a geometric constraint for the calculation of dissimilarity scores between correspondences. In addition, with the constraints between triangles of the refined Delaunay triangulation and its corresponding graph, a match expansion is designed to exploit as many inliers as possible. Finally, a reliable image matching algorithm is proposed by sequentially executing the three constraints for outlier elimination and match expansion. Under comprehensive analysis and comparison with five state-of-the-art algorithms, the performance of the proposed method is verified by using both rigid and non-rigid datasets. The experimental results demonstrate that the Delaunay triangulation is sufficient to construct neighboring structures for the implementation of local photometric and geometric constraints, and the proposed method can achieve good performance in terms of the precision, recall and number of inliers, and provide reliable matches for stereo image pairs with both rigid and non-rigid transformations.

Co-Segmentation and Superpixel-Based Graph Cuts for Building Change Detection from Bi-Temporal Digital Surface Models and Aerial Images

Thanks to the recent development of laser scanner hardware and the technology of dense image matching (DIM), the acquisition of three-dimensional (3D) point cloud data has become increasingly convenient. However, how to effectively combine 3D point cloud data and images to realize accurate building change detection is still a hotspot in the field of photogrammetry and remote sensing. Therefore, with the bi-temporal aerial images and point cloud data obtained by airborne laser scanner (ALS) or DIM as the data source, a novel building change detection method combining co-segmentation and superpixel-based graph cuts is proposed in this paper. In this method, the bi-temporal point cloud data are firstly combined to achieve a co-segmentation to obtain bi-temporal superpixels with the simple linear iterative clustering (SLIC) algorithm. Secondly, for each period of aerial images, semantic segmentation based on a deep convolutional neural network is used to extract building areas, and this is the basis for subsequent superpixel feature extraction. Again, with the bi-temporal superpixel as the processing unit, a graph-cuts-based building change detection algorithm is proposed to extract the changed buildings. In this step, the building change detection problem is modeled as two binary classifications, and acquisition of each period’s changed buildings is a binary classification, in which the changed building is regarded as foreground and the other area as background. Then, the graph cuts algorithm is used to obtain the optimal solution. Next, by combining the bi-temporal changed buildings and digital surface models (DSMs), these changed buildings are further classified as “newly built,” “taller,” “demolished”, and “lower”. Finally, two typical datasets composed of bi-temporal aerial images and point cloud data obtained by ALS or DIM are used to validate the proposed method, and the experiments demonstrate the effectiveness and generality of the proposed algorithm.

Эволюция применения геометрических преобразований в фотограмметрии

Эволюция применения геометрических преобразований в фотограмметрии

3D MODELLING AND DIGITAL ARCHIVING OF CULTURAL HERITAGE: THE EXAMPLE OF ISMOIL SOMONIY TOMB IN UZBEKISTAN

The protection of cultural heritage consists of research, examination, diagnosis, conservation and documentation. The most important factor for the success of the study is the accuracy of the identification and documentation study. In recent years, in addition to traditional documentation studies, technological developments have started to be used in the documentation studies of cultural heritage. These studies have shown themselves in the field of photogrammetry. Photogrammetry is a science that helps to create a precise 3D model of the object by making use of at least two pictures taken from different angles of the object. Photogrammetry increases the possibilities for use in cultural heritage studies, research projects and engineering applications that require highly accurate spatial data.In this study, 3D model of Ismoil Somoniy Tomb/Samanid Mausoleum in Bukhara was made in photomodeler software. All architectural details of the tomb were documented with the help of the model. The digital data obtained as a result of the study can be used by all professional disciplines in the studies related to the subject.

AUTOMATIC 3D RECONSTRUCTION OF BUILDINGS ROOF TOPS IN DENSELY URBANIZED AREAS

Abstract. 3D reconstruction of the urban environment constitutes a well-studied problem in the field of photogrammetry and computer vision, attracting the growing interest of the scientific community, for many years. Although the current state of the art present very impressive results, there is still room for improvements. The production of reliable and accurate 3D reconstructions is useful for a wide range of applications, such as urban planning, GIS, tax assessment, cadastre, insurance, 3D city modelling, etc. In this paper, a methodology for the automatic 3D reconstruction of buildings roof tops in densely urbanized areas, utilizing dense point clouds data, is proposed. It consists of three (3) main phases, each of which comprises a set of processing steps. In the first phase, the point cloud is simplified and smoothed. Outliers and non-roof elements are detected and removed utilizing shape, position and area criteria. In the second phase, the geometry buildings roof tops is optimized, by detecting and normalizing the edges. In the last phase, the reconstruction of the buildings roof tops is conducted. A progressive process, utilizing a plane fitting algorithm in combination with Screened Poisson Surface Reconstruction is performed. Buildings roof tops surfaces are produced and optimized. A software tool is developed and utilized for the implementation of the proposed methodology. The produced results are assessed and a comparison with another open-source software is conducted. The proposed methodology seems to be effective providing satisfactory results, as it can manage properly the really noisy point clouds of densely urbanized environments.

Registration of Laser Scanning Point Clouds: A Review.

The integration of multi-platform, multi-angle, and multi-temporal LiDAR data has become important for geospatial data applications. This paper presents a comprehensive review of LiDAR data registration in the fields of photogrammetry and remote sensing. At present, a coarse-to-fine registration strategy is commonly used for LiDAR point clouds registration. The coarse registration method is first used to achieve a good initial position, based on which registration is then refined utilizing the fine registration method. According to the coarse-to-fine framework, this paper reviews current registration methods and their methodologies, and identifies important differences between them. The lack of standard data and unified evaluation systems is identified as a factor limiting objective comparison of different methods. The paper also describes the most commonly-used point cloud registration error analysis methods. Finally, avenues for future work on LiDAR data registration in terms of applications, data, and technology are discussed. In particular, there is a need to address registration of multi-angle and multi-scale data from various newly available types of LiDAR hardware, which will play an important role in diverse applications such as forest resource surveys, urban energy use, cultural heritage protection, and unmanned vehicles.

An object-based approach for tree species extraction from digital orthophoto maps

Abstract. Tree segmentation is an active and ongoing research area in the field of photogrammetry and remote sensing. It is more challenging due to both intra-class and inter-class similarities among various tree species. In this study, we exploited various statistical features for extraction of hazelnut trees from 1 : 5000 scaled digital orthophoto maps. Initially, the non-vegetation areas were eliminated using traditional normalized difference vegetation index (NDVI) followed by application of mean shift segmentation for transforming the pixels into meaningful homogeneous objects. In order to eliminate false positives, morphological opening and closing was employed on candidate objects. A number of heuristics were also derived to eliminate unwanted effects such as shadow and bounding box aspect ratios, before passing them into the classification stage. Finally, a knowledge based decision tree was constructed to distinguish the hazelnut trees from rest of objects which include manmade objects and other type of vegetation. We evaluated the proposed methodology on 10 sample orthophoto maps obtained from Giresun province in Turkey. The manually digitized hazelnut tree boundaries were taken as reference data for accuracy assessment. Both manually digitized and segmented tree borders were converted into binary images and the differences were calculated. According to the obtained results, the proposed methodology obtained an overall accuracy of more than 85 % for all sample images.

SOLVING THE RATIONAL POLYNOMIAL COEFFICIENTS BASED ON L CURVE

Abstract. The rational polynomial coefficients (RPC) model is a generalized sensor model, which can achieve high approximation accuracy. And it is widely used in the field of photogrammetry and remote sensing. Least square method is usually used to determine the optimal parameter solution of the rational function model. However the distribution of control points is not uniform or the model is over-parameterized, which leads to the singularity of the coefficient matrix of the normal equation. So the normal equation becomes ill conditioned equation. The obtained solutions are extremely unstable and even wrong. The Tikhonov regularization can effectively improve and solve the ill conditioned equation. In this paper, we calculate pathological equations by regularization method, and determine the regularization parameters by L curve. The results of the experiments on aerial format photos show that the accuracy of the first-order RPC with the equal denominators has the highest accuracy. The high order RPC model is not necessary in the processing of dealing with frame images, as the RPC model and the projective model are almost the same. The result shows that the first-order RPC model is basically consistent with the strict sensor model of photogrammetry. Orthorectification results both the firstorder RPC model and Camera Model (ERDAS9.2 platform) are similar to each other, and the maximum residuals of X and Y are 0.8174 feet and 0.9272 feet respectively. This result shows that RPC model can be used in the aerial photographic compensation replacement sensor model.