Automatic 3D Building Reconstruction Research Articles

Automatic 3D Building Reconstruction from OpenStreetMap and LiDAR Using Convolutional Neural Networks

This paper presents the implementation of an automatic method for the reconstruction of 3D building maps. The core innovation of the proposed method is the supplementation of OpenStreetMap data with LiDAR data to reconstruct 3D urban environments automatically. The only input of the method is the area that needs to be reconstructed, defined by the enclosing points in terms of the latitude and longitude. First, area data are requested in OpenStreetMap format. However, there are certain buildings and geometries that are not fully received in OpenStreetMap files, such as information on roof types or the heights of buildings. To complete the information that is missing in the OpenStreetMap data, LiDAR data are read directly and analyzed using a convolutional neural network. The proposed approach shows that a model can be obtained with only a few samples of roof images from an urban area in Spain, and is capable of inferring roofs in other urban areas of Spain as well as other countries that were not used to train the model. The results allow us to identify a mean of 75.57% for height data and a mean of 38.81% for roof data. The finally inferred data are added to the 3D urban model, resulting in detailed and accurate 3D building maps. This work shows that the neural network is able to detect buildings that are not present in OpenStreetMap for which in LiDAR data are available. In future work, it would be interesting to compare the results of the proposed method with other approaches for generating 3D models from OSM and LiDAR data, such as point cloud segmentation or voxel-based approaches. Another area for future research could be the use of data augmentation techniques to increase the size and robustness of the training dataset.

DIGITAL TWIN CREATION FOR SLUMS IN BRAZIL BASED ON UAV DATA

Abstract. Favelas are the most common type of informal settlements found in Brazil. The Housing Secretariat, City Hall, Sao Paulo, has conducted surveys using Unmanned Aerial Vehicles (UAVs) for the favelas to facilitate the slum upgrading projects and has taken the initiative to create a digital twin of the slum areas. This study illustrates the feasibility of developing a methodological workflow to create a digital twin by automatic 3D building reconstruction in slums from the UAV point cloud and the 2D building footprints with a continuous link for updating the building semantic information. This study focuses on facilitating data integration and updating the semantic information into the 3D model to provide additional information about the individual buildings in the slums. The assessments concluded that the proposed workflow is suitable for creating digital twins for slums based on the UAV and 2D cadastral data. However, the 3D slum model had a few limitations, which are discussed in this paper.

Automatic 3D building reconstruction from multi-view aerial images with deep learning

The study presented in this paper introduced a new fully automatic three-dimensional building reconstruction method that can generate first level of detail (LoD 1) building models from multi-view aerial images without any assistance from other data. The accuracy and completeness of our reconstructed models have approached that of manually delineated models to a large extent. The presented method consists of three parts: (1) efficient dense matching and Earth surface reconstruction, (2) reliable building footprint extraction and polygon regularization, and (3) highly accurate height inference of building roofs and bases. First, our novel deep learning-based multi-view matching method, composed of a convolutional neural network, gated recurrent convolutions, and a multi-scale pyramid matching structure, is used to reconstruct the digital surface model (DSM) and digital orthophoto map (DOM) efficiently without generating epipolarly rectified images. Second, our three-stage 2D building extraction method is introduced to deliver reliable and accurate building contours. Deep-learning based segmentation, assisted with DSM, is used to segment buildings from backgrounds; and the generated building maps are fused with a terrain classification algorithm to reach better segmentation results. A polygon regularization algorithm and a level set algorithm are thereafter employed to transfer the binary segmentation maps to structured vector-form building polygons. Third, a novel method is introduced to infer the height of building roofs and bases using adaptive local terrain filtering and neighborhood buffer analysis. We tested our method on a large experimental area that covered 2284 aerial images and 782 various types of buildings. Our results as far as correctness and completeness exceeded the results of other similar methods in a between-method comparison by at least 15% for individual 3D building models with many of them comparable to manual delineation results.

ADVANCED APPROACH FOR AUTOMATIC RECONSTRUCTION OF 3D BUILDINGS FROM AERIAL IMAGES

Abstract. In this work, a novel automatic 3D building reconstruction approach is proposed to extract accurate LoD1 building models from multi-view aerial images. The proposed approach consists of three main parts. The first step is to generate digital surface models (DSMs) from aerial images, which is implemented with the Smart3D software and can be replaced by other open-source multi-view stereo (MVS) algorithms as well. The second step is to produce structured 2D building footprints using combined deep learning and regularization. The initial building segmentation maps are obtained by the multi-scale aggregation fully convolutional network (MA-FCN), which takes both the images and DSM as input, through supervised learning. The initial segmentation maps are further refined with another segmentation maps that are derived from the DSM. After that, the contour extraction and regularization technology are applied to produce structured building footprints. In the last step, the elevations of the top and base of each individual building are reliably estimated by adopting an adaptive terrain generation approach and the neighbourhood buffer analysis. The georeferenced building footprint polygons and the elevations of building top and base form the watertight LoD1 building models. The qualitative and quantitative evaluations in Jinghai District, Tianjin, China demonstrate the robustness and effectiveness of the proposed approach.

AUTOMATIC 3D RECONSTRUCTION OF COMPLEX BUILDINGS FROM INCOMPLETE POINT CLOUDS WITH TOPOLOGICAL-RELATION CONSTRAINTS

Abstract. Automatic 3D building reconstruction from laser scanning or photogrammetric point clouds has gained increasing attention in the past two decades. Although many efforts have been made, the complexity of buildings and incompletion of point clouds, i.e., data missing, still make it a challenging task for automatic 3D reconstruction of buildings in large-scale urban scenes with various architectural styles. This paper presents an innovative approach for automatic generation of 3D models of complex buildings from even incomplete point clouds. The approach first decomposes the 3D space into multiple space units, including 3D polyhedral cells, facets and edges, where the facets and edges are also encoded with topological-relation constraints. Then, the units and constraints are used together to approximate the buildings. On one hand, by extracting facets from 3D cells and further extracting edges from facets, this approach simplifies complicated topological computations. On the other hand, because this approach models buildings on the basis of polyhedral cells, it can guarantee that the models are manifold and watertight and avoid correcting topological errors. A challenging dataset containing 105 buildings acquired in Central, Hong Kong, was used to evaluate the performance of the proposed approach. The results were compared with two previous methods and the comparisons suggested that the proposed approach outperforms other methods in terms of robustness, regularity, and accuracy of the models, with an average root-mean-square error of less than 0.9 m. The proposed approach is of significance for automatic 3D modelling of buildings for urban applications.

VOLUME BASED DTM GENERATION FROM VERY HIGH RESOLUTION PHOTOGRAMMETRIC DSMS

Abstract. In this paper we propose a new algorithm for digital terrain (DTM) model reconstruction from very high spatial resolution digital surface models (DSMs). It represents a combination of multi-directional filtering with a new metric which we call normalized volume above ground to create an above-ground mask containing buildings and elevated vegetation. This mask can be used to interpolate a ground-only DTM. The presented algorithm works fully automatically, requiring only the processing parameters minimum height and maximum width in metric units. Since slope and breaklines are not decisive criteria, low and smooth and even very extensive flat objects are recognized and masked. The algorithm was developed with the goal to generate the normalized DSM for automatic 3D building reconstruction and works reliably also in environments with distinct hillsides or terrace-shaped terrain where conventional methods would fail. A quantitative comparison with the ISPRS data sets Potsdam and Vaihingen show that 98-99% of all building data points are identified and can be removed, while enough ground data points (~66%) are kept to be able to reconstruct the ground surface. Additionally, we discuss the concept of size dependent height thresholds and present an efficient scheme for pyramidal processing of data sets reducing time complexity to linear to the number of pixels, O(WH).

VOLUME BASED DTM GENERATION FROM VERY HIGH RESOLUTION PHOTOGRAMMETRIC DSMS

In this paper we propose a new algorithm for digital terrain (DTM) model reconstruction from very high spatial resolution digital surface models (DSMs). It represents a combination of multi-directional filtering with a new metric which we call <i>normalized volume above ground</i> to create an above-ground mask containing buildings and elevated vegetation. This mask can be used to interpolate a ground-only DTM. The presented algorithm works fully automatically, requiring only the processing parameters <i>minimum height</i> and <i>maximum width</i> in metric units. Since slope and breaklines are not decisive criteria, low and smooth and even very extensive flat objects are recognized and masked. The algorithm was developed with the goal to generate the normalized DSM for automatic 3D building reconstruction and works reliably also in environments with distinct hillsides or terrace-shaped terrain where conventional methods would fail. A quantitative comparison with the ISPRS data sets <i>Potsdam</i> and <i>Vaihingen</i> show that 98-99% of all building data points are identified and can be removed, while enough ground data points (~66%) are kept to be able to reconstruct the ground surface. Additionally, we discuss the concept of <i>size dependent height thresholds</i> and present an efficient scheme for pyramidal processing of data sets reducing time complexity to linear to the number of pixels, <i>O(WH)</i>.

INCREMENTAL REFINEMENT OF FAÇADE MODELS WITH ATTRIBUTE GRAMMAR FROM 3D POINT CLOUDS

Data acquisition using unmanned aerial vehicles (UAVs) has gotten more and more attention over the last years. Especially in the field of building reconstruction the incremental interpretation of such data is a demanding task. In this context formal grammars play an important role for the top-down identification and reconstruction of building objects. Up to now, the available approaches expect offline data in order to parse an a-priori known grammar. For mapping on demand an on the fly reconstruction based on UAV data is required. An incremental interpretation of the data stream is inevitable. This paper presents an incremental parser of grammar rules for an automatic 3D building reconstruction. The parser enables a model refinement based on new observations with respect to a weighted attribute context-free grammar (WACFG). The falsification or rejection of hypotheses is supported as well. The parser can deal with and adapt available parse trees acquired from previous interpretations or predictions. Parse trees derived so far are updated in an iterative way using transformation rules. A diagnostic step searches for mismatches between current and new nodes. Prior knowledge on fac¸ades is incorporated. It is given by probability densities as well as architectural patterns. Since we cannot always assume normal distributions, the derivation of location and shape parameters of building objects is based on a kernel density estimation (KDE). While the level of detail is continuously improved, the geometrical, semantic and topological consistency is ensured.

INCREMENTAL REFINEMENT OF FAÇADE MODELS WITH ATTRIBUTE GRAMMAR FROM 3D POINT CLOUDS

Abstract. Data acquisition using unmanned aerial vehicles (UAVs) has gotten more and more attention over the last years. Especially in the field of building reconstruction the incremental interpretation of such data is a demanding task. In this context formal grammars play an important role for the top-down identification and reconstruction of building objects. Up to now, the available approaches expect offline data in order to parse an a-priori known grammar. For mapping on demand an on the fly reconstruction based on UAV data is required. An incremental interpretation of the data stream is inevitable. This paper presents an incremental parser of grammar rules for an automatic 3D building reconstruction. The parser enables a model refinement based on new observations with respect to a weighted attribute context-free grammar (WACFG). The falsification or rejection of hypotheses is supported as well. The parser can deal with and adapt available parse trees acquired from previous interpretations or predictions. Parse trees derived so far are updated in an iterative way using transformation rules. A diagnostic step searches for mismatches between current and new nodes. Prior knowledge on fac¸ades is incorporated. It is given by probability densities as well as architectural patterns. Since we cannot always assume normal distributions, the derivation of location and shape parameters of building objects is based on a kernel density estimation (KDE). While the level of detail is continuously improved, the geometrical, semantic and topological consistency is ensured.

Automatic urban 3D building reconstruction from multi-ray photogrammetry

Abstract. Over the last 20 years the use of, and demand for, three dimensional (3D) building models has meant there has been a vast amount of research conducted in automating the extraction and reconstruction of these models from airborne sensors. Whilst many different approaches have been suggested, full automation is yet to be achieved and research has suggested that the combination of data from multiple sources is required in order to achieve this. Developments in digital photogrammetry have delivered improvements in spatial resolution whilst higher image overlap to increase the number of pixel correspondents between images, giving the name multi-ray photogrammetry, has improved the resolution and quality of its by-products. In this paper the extraction of roof geometry from multiray photogrammetry will be covered, which underpins 3D building reconstruction. Using orthophotos, roof vertices are extracted using the Canny edge detector. Roof planes are detected from digital surface models (DSM) by extracting information from 2D cross sections and measuring height differences. To eliminate overhanging vegetation, the segmentation of trees is investigated by calculating the characteristics of a point within a local neighbourhood of the photogrammetric point cloud. The results highlight the complementary nature of these information sources, and a methodology for integration and reconstruction of roof geometry is proposed.

3D Building Adjustment Using Planar Half-Space Regularities

Abstract. The automatic reconstruction of 3D building models with complex roof shapes is still an active area of research. In this paper we present a novel approach for local and global regularization rules that integrate building knowledge to improve both the shape of the reconstructed building models and their accuracy. These rules are defined for the planar half-space representation of our models and emphasize the presence of symmetries, co-planarity, parallelism, and orthogonality. By not adjusting building features separately (e.g. ridges, eaves, etc.) we are able to handle more than one feature at a time without considering dependencies between different features. Additionally, we present a new method for reconstructing buildings with concave outlines using half-spaces that avoids the need to partition the models into smaller convex parts. We present both extensions in the context of a fully automatic feature-driven 3D building reconstruction approach where the whole process is suited for processing large urban areas with complex building roofs.

An update on automatic 3D building reconstruction

The development of tools for the generation of 3D city models started almost two decades ago. From the beginning, fully automatic reconstruction systems were envisioned to fulfil the need for efficient data collection. However, research on automatic city modelling is still a very active area. The paper will review a number of current approaches in order to comprehensively elaborate the state of the art of reconstruction methods and their respective principles. Originally, automatic city modelling only aimed at polyhedral building objects, which mainly reflects the respective roof shapes and building footprints. For this purpose, airborne images or laser scans are used. In addition to these developments, the paper will also review current approaches for the generation of more detailed facade geometries from terrestrial data collection.

Automatic 3D building reconstruction by integration of digital map and stereo imagery for urban area

Three dimensional model of urban areas is useful in several applications. In urban administration, perspective views derived from three dimensional model are used by city planners for landscape analysis. Three dimensional models are also used in modeling telecommunication environment. Building heights are important in searching for the optimal locations of transmission antennas especially in mobile telecommunication for cell planning. Furthermore, reconstructed 3D buildings are also essential in generating ‘true’ digital orthophotos where occlusion areas caused by tall buildings are eliminated. This paper introduces an automated method of building height and eventually 3D building reconstruction which involves the integration of existing 2D vector geodata (i.e., digital maps) and stereo aerial imagery. Accurate 2D vector geodata of major cities in Korea are available at the scale of 1/1,000. The vertical line locus method has been deployed to recover building heights. Building footprint in the 2D vector geodata is used in locating the vertical guideline along the building edges formed by building sides. Digital elevation model (DEM) was also generated from the contour layer which is included in the vector geodata. DEM values at the foot of the buildings were used as initial searching locations. At a preset incremental value of height along the vertical guideline, evaluation based on the cross correlation image matching technique was carried out to test if the top of the building has been reached. Buildings are reconstructed with flat roof surfaces that are typical shape of the buildings in the study site.

Reconstruction of 3D building models from aerial images and maps

Automatic 3D building reconstruction has becoming increasingly important for a number of applications. The reconstruction of buildings using only aerial images as data source has been proven to be a very difficult problem. The complexity of the reconstruction can be greatly reduced by combining the aerial images with other data sources. In this paper, we describe a 3D building reconstruction method that integrates the aerial image analysis with information from large-scale 2D Geographic Information System (GIS) databases and domain knowledge. By combining the images with GIS data, the specific strengths of both the images (high resolution, accuracy, and large-information content) and the GIS data (relatively simple interpretation) are exploited.