Dense Image Matching Research Articles

Photogrammetric Processing of Tianwen-1 HiRIC Imagery for Precision Topographic Mapping on Mars

The High-Resolution Imaging Camera (HiRIC) onboard China’s Tianwen-1 Mars probe aims to acquire detailed imagery of the Martian surface to comprehensively investigate its topography and geomorphology. The HiRIC is a pushbroom camera comprising three CCDs to simultaneously achieve sub-meter resolution and a large swath. However, processing HiRIC images using the conventional photogrammetric workflow is difficult due to the large shifts and narrow overlapping among the CCD lines. This paper presents a novel approach for photogrammetric processing of HiRIC images for precision topographic mapping that incorporates 1) the fitting of the initial Rational Polynomial Coefficients (RPCs) of images from the HiRIC position and pointing data, 2) a deep-learning-based method for tie-point matching between adjacent CCD images and cross-orbit images, 3) the bundle adjustment of multiple CCD images for tripled-epipolar image generation to ensure inner-orbit consistency, 4) the block adjustment of multiple orbit images to ensure cross-orbit consistency, and 5) dense image matching and space intersection based on the refined RPCs to generate Digital Elevation Models (DEMs). Experimental analyses were conducted using HiRIC images covering the landing region of the Zhurong rover. The results revealed that subpixel accuracy was achieved for image residuals among multiple-CCD or multiple-orbit images. Comparison with the reference data (HiRISE and MOLA DEMs) revealed a mean deviation of less than 7 m in terms of the geometric accuracy and the subtle topographic details of the HiRIC DEM. The presented approach offers a reliable solution for using the new dataset of HiRIC imagery for Mars topographic mapping.

Methodology of Detection and Classification of Selected Aviation Obstacles Based on UAV Dense Image Matching

Currently, more and more accurate data provided by UAVs make it possible to analyze land cover, which requires the detection of objects and their individual elements. Object detection and determination of their geometric features is possible thanks to dense point clouds generated based on imagery obtained from low altitudes. 3D data from UAVs turn out to be extremely useful for ensuring safety in the airspace in the close vicinity of the airport. This article presents the methodology of automatic aviation obstacle detection based on low altitude data (UAV). The research was carried out on a dense 3D point cloud. The developed methodology for detecting aviation obstacles consists of three main stages. The first is point cloud filtration based on height–preliminary identification of aviation obstacles, followed by 3D point cloud segmentation using a modified RANSAC algorithm, supplemented with two-dimensional vector data of aviation obstacles to improve the accuracy of the segmentation process. The last stage is the classification of aviation obstacles according to the adopted height and cross-section criterion. The proposed method of detecting aviation obstacles is characterized by high accuracy. The mean error of fitting the point cloud to the obstacle database ranged from ± 0.04 m to ± 0.07 m.

Re-engineering the Past: Countrywide Geo-referencing of Archival Aerial Imagery

Archival aerial imagery (AAI) represents a unique and relatively unexploited resource for assessing long-term environmental changes at a very high spatial resolution. A major constraint for the wider use of AAI often lies in the difficulties of establishing precise geo-referencing, namely in the difficult and time-consuming task of assigning ground reference through manual digitization of Ground control points (GCPs). We present a highly automated photogrammetric workflow for orientation of AAI. The workflow substitutes manual GCP measurements by generating image matches to a digital reference. The resulting abundant observations are algorithmically filtered and used in a bundle block adjustment (BBA) to obtain final image orientations. The proposed workflow has successfully been employed to process a complete coverage of AAI over the territory of Switzerland based on images acquired between 1985 and 1991. The accuracies obtained from the orientation process are very satisfying and allow for generating meaningful 2D and 3D products. The absolute accuracy for derived orthophotos and their mosaics is about 1 m. The relative accuracies are in the subpixel range and allow for generation of country-wide Digital surface models (DSMs) through dense-image matching. The obtained accuracies are comparable to those obtained at the authors’ affiliation using classical workflows that involve manual GCP identification from digital reference data. With regard to human working time, the workflow has, in our case, proven to be at least five times more efficient than classical workflows whilst the required computational resources are very moderate.

GRAPH NEURAL NETWORK BASED MULTI-FEATURE FUSION FOR BUILDING CHANGE DETECTION

Abstract. Building Change Detection (BCD) via multi-temporal remote sensing images is essential for various applications such as urban monitoring, urban planning, and disaster assessment. However, most building change detection approaches only extract features from different kinds of remote sensing images for change index determination, which can not determine the insignificant changes of small buildings. Given co-registered multi-temporal remote sensing images, the illumination variations and misregistration errors always lead to inaccurate change detection results. This study investigates the applicability of multi-feature fusion from both directly extract 2D features from remote sensing images and 3D features extracted by the dense image matching (DIM) generated 3D point cloud for accurate building change index generation. This paper introduces a graph neural network (GNN) based end-to-end learning framework for building change detection. The proposed framework includes feature extraction, feature fusion, and change index prediction. It starts with a pre-trained VGG-16 network as a backend and uses U-net architecture with five layers for feature map extraction. The extracted 2D features and 3D features are utilized as input into GNN based feature fusion parts. In the GNN parts, we introduce a flexible context aggregation mechanism based on attention to address the illumination variations and misregistration errors, enabling the framework to reason about the image-based texture information and depth information introduced by DIM generated 3D point cloud jointly. After that, the GNN generated affinity matrix is utilized for change index determination through a Hungarian algorithm. The experiment conducted on a dataset that covered Setagaya-Ku, Tokyo area, shows that the proposed method generated change map achieved the precision of 0.762 and the F1-score of 0.68 at pixel-level. Compared to traditional image-based change detection methods, our approach learns prior over geometrical structure information from the real 3D world, which robust to the misregistration errors. Compared to CNN based methods, the proposed method learns to fuse 2D and 3D features together to represent more comprehensive information for building change index determination. The experimental comparison results demonstrated that the proposed approach outperforms the traditional methods and CNN based methods.

OPEN URBAN AND FOREST DATASETS FROM A HIGH-PERFORMANCE MOBILE MAPPING BACKPACK – A CONTRIBUTION FOR ADVANCING THE CREATION OF DIGITAL CITY TWINS

Abstract. With this contribution, we describe and publish two high-quality street-level datasets, captured with a portable high-performance Mobile Mapping System (MMS). The datasets will be freely available for scientific use. Both datasets, from a city centre and a forest represent area-wide street-level reality captures which can be used e.g. for establishing cloud-based frameworks for infrastructure management as well as for smart city and forestry applications. The quality of these data sets has been thoroughly evaluated and demonstrated. For example, georeferencing accuracies in the centimetre range using these datasets in combination with image-based georeferencing have been achieved. Both high-quality multi sensor system street-level datasets are suitable for evaluating and improving methods for multiple tasks related to high-precision 3D reality capture and the creation of digital twins. Potential applications range from localization and georeferencing, dense image matching and 3D reconstruction to combined methods such as simultaneous localization and mapping and structure-from-motion as well as classification and scene interpretation. Our dataset is available online at: https://www.fhnw.ch/habg/bimage-datasets

INVESTIGATING 3D RECONSTRUCTION OF NON-COLLABORATIVE SURFACES THROUGH PHOTOGRAMMETRY AND PHOTOMETRIC STEREO

Abstract. 3D digital reconstruction techniques are extensively used for quality control purposes. Among them, photogrammetry and photometric stereo methods have been for a long time used with success in several application fields. However, generating highly-detailed and reliable micro-measurements of non-collaborative surfaces is still an open issue. In these cases, photogrammetry can provide accurate low-frequency 3D information, whereas it struggles to extract reliable high-frequency details. Conversely, photometric stereo can recover a very detailed surface topography, although global surface deformation is often present. In this paper, we present the preliminary results of an ongoing project aiming to combine photogrammetry and photometric stereo in a synergetic fusion of the two techniques. Particularly, hereafter, we introduce the main concept design behind an image acquisition system we developed to capture images from different positions and under different lighting conditions as required by photogrammetry and photometric stereo techniques. We show the benefit of such a combination through some experimental tests. The experiments showed that the proposed method recovers the surface topography at the same high-resolution achievable with photometric stereo while preserving the photogrammetric accuracy. Furthermore, we exploit light directionality and multiple light sources to improve the quality of dense image matching in poorly textured surfaces.

STATE OF THE ART IN DENSE IMAGE MATCHING COST COMPUTATION FOR HIGH-RESOLUTION SATELLITE STEREO

Abstract. Large-scale Digital Surface Model (DSM) generated with high-resolution satellite images (HRSI) are comparable, cheaper, and more accessible when comparing to Light Detection and Ranging (LiDAR) data and aerial remotely sensed images. Several photogrammetric commercial/open-source software packages are being developed for satellite image-based 3D reconstruction, in which, most of them adopt a modified version of Semi-Global Matching (SGM) algorithm for dense image matching. With the continuous development of matching cost computation methods, the existing methods can be divided into classical (low-level) and learning-based algorithms (non-end-to-end learning and end-to-end learning methods). On Middlebury and KITTI datasets, learning-based algorithms has shown their superiority compared to SGM derived methods. In this context, we assume that matching cost is the key factor of DIM. This paper reviews and evaluates Census Transform, and MC-CNN on a WorldView-3 typical city scene satellite stereo images on the premise that the overall SGM framework remains unchanged, providing a preliminary comparison for academic and industrial. We first compute the cost valume of these two methods, obtains the final DSM after semi-global optimization, and compares their gemetric accuracy with the corresponding LiDAR derived ground truth. We presented our comparison and findings in the experimental section.

Building Change Detection of Airborne Laser Scanning and Dense Image Matching Point Clouds using Height and Class Information

Abstract. Detecting changes is an important task to update databases and find irregularities in spatial data. Every couple of years, national mapping agencies (NMAs) acquire nation-wide point cloud data from Airborne Laser Scanning (ALS) as well as from Dense Image Matching (DIM) using aerial images. Besides deriving several other products such as Digital Elevation Models (DEMs) from them, those point clouds also offer the chance to detect changes between two points in time on a large scale. Buildings are an important object class in the context of change detection to update cadastre data. As detecting changes manually is very time consuming, the aim of this study is to provide reliable change detections for different building sizes in order to support NMAs in their task to update their databases. As datasets of different times may have varying point densities due to technological advancements or different sensors, we propose a raster-based approach, which is independent of the point density altogether. Within a raster cell, our approach considers the height distribution of all points for two points in time by exploiting the Jensen-Shannon distance to measure their similarity. Our proposed method outperforms simple threshold methods on detecting building changes with respect to the same or different point cloud types. In combination with our proposed class change detection approach, we achieve a change detection performance measured by the mean F1-Score of about 71% between two ALS and about 60% between ALS and DIM point clouds acquired at different times.

Automatic Workflow for Roof Extraction and Generation of 3D CityGML Models from Low-Cost UAV Image-Derived Point Clouds

Developments in UAV sensors and platforms in recent decades have stimulated an upsurge in its application for 3D mapping. The relatively low-cost nature of UAVs combined with the use of revolutionary photogrammetric algorithms, such as dense image matching, has made it a strong competitor to aerial lidar mapping. However, in the context of 3D city mapping, further 3D modeling is required to generate 3D city models which is often performed manually using, e.g., photogrammetric stereoplotting. The aim of the paper was to try to implement an algorithmic approach to building point cloud segmentation, from which an automated workflow for the generation of roof planes will also be presented. 3D models of buildings are then created using the roofs’ planes as a base, therefore satisfying the requirements for a Level of Detail (LoD) 2 in the CityGML paradigm. Consequently, the paper attempts to create an automated workflow starting from UAV-derived point clouds to LoD 2-compatible 3D model. Results show that the rule-based segmentation approach presented in this paper works well with the additional advantage of instance segmentation and automatic semantic attribute annotation, while the 3D modeling algorithm performs well for low to medium complexity roofs. The proposed workflow can therefore be implemented for simple roofs with a relatively low number of planar surfaces. Furthermore, the automated approach to the 3D modeling process also helps to maintain the geometric requirements of CityGML such as 3D polygon coplanarity vis-à-vis manual stereoplotting.

Automatic Extraction of Buildings from UAV-Based Imagery Using Artificial Neural Networks

The rapid growth of major cities across the counties demands accurate building extraction techniques. Unmanned Aerial Vehicles (UAV) help in obtaining terrain information that can be used to extract urban features. Recent advancement has led to the capture of aerial images of the earth surface in micro-detail using UAV. These aerial images enable us to perform classification, feature extraction, and height estimation at a finer scale. In this work, aerial images of the university campus were captured using a quadcopter drone equipped with high-resolution camera and satellite navigation system. Approximately 500 images were captured in the study area with necessary overlap and side lap. Captured images were subjected to aerial triangulation, dense image matching, and point cloud generation to produce Digital Surface Models (DSM) and orthophoto. Various machine learning algorithms—random forest (RF), support vector machine (SVM), naive Bayes (NB) and artificial neural networks (ANN)—have been used to extract building rooftops from derivatives of UAV-captured imageries, and accuracies were compared. Algorithms were trained using both spectral and elevation information to extract building rooftops, and improvements shown due to the addition of elevation data in training the model are observed. The proposed method is aimed at improving building-level information extraction and providing accurate building information to aid authorities for better planning and management.

Stereo Dense Image Matching by Adaptive Fusion of Multiple-Window Matching Results

Traditional stereo dense image matching (DIM) methods normally predefine a fixed window to compute matching cost, while their performances are limited by the matching window sizes. A large matching window usually achieves robust matching results in weak-textured regions, while it may cause over-smoothness problems in disparity jumps and fine structures. A small window can recover sharp boundaries and fine structures, while it contains high matching uncertainties in weak-textured regions. To address the issue above, we respectively compute matching results with different matching window sizes and then proposes an adaptive fusion method of these matching results so that a better matching result can be generated. The core algorithm designs a Convolutional Neural Network (CNN) to predict the probabilities of large and small windows for each pixel and then refines these probabilities by imposing a global energy function. A compromised solution of the global energy function is utilized by breaking the optimization into sub-optimizations of each pixel in one-dimensional (1D) paths. Finally, the matching results of large and small windows are fused by taking the refined probabilities as weights for more accurate matching. We test our method on aerial image datasets, satellite image datasets, and Middlebury benchmark with different matching cost metrics. Experiments show that our proposed adaptive fusion of multiple-window matching results method has a good transferability across different datasets and outperforms the small windows, the median windows, the large windows, and some state-of-the-art matching window selection methods.

Multiscale Integration of High-Resolution Spaceborne and Drone-Based Imagery for a High-Accuracy Digital Elevation Model Over Tristan da Cunha

Very high-resolution (VHR) optical Earth observation (EO) satellites as well as low-altitude and easy-to-use unmanned aerial systems (UAS/drones) provide ever-improving data sources for the generation of detailed 3-dimensional (3D) data using digital photogrammetric methods with dense image matching. Today both data sources represent cost-effective alternatives to dedicated airborne sensors, especially for remote regions. The latest generation of EO satellites can collect VHR imagery up to 0.30 m ground sample distance (GSD) of even the most remote location from different viewing angles many times per year. Consequently, well-chosen scenes from growing image archives enable the generation of high-resolution digital elevation models (DEMs). Furthermore, low-cost and easy to use drones can be quickly deployed in remote regions to capture blocks of images of local areas. Dense point clouds derived from these methods provide an invaluable data source to fill the gap between globally available low-resolution DEMs and highly accurate terrestrial surveys. Here we investigate the use of archived VHR satellite imagery with approx. 0.5 m GSD as well as low-altitude drone-based imagery with average GSD of better than 0.03 m to generate high-quality DEMs using photogrammetric tools over Tristan da Cunha, a remote island in the South Atlantic Ocean which lies beyond the reach of current commercial manned airborne mapping platforms. This study explores the potentials and limitations to combine this heterogeneous data sources to generate improved DEMs in terms of accuracy and resolution. A cross-validation between low-altitude airborne and spaceborne data sets describes the fit between both optical data sets. No co-registration error, scale difference or distortions were detected, and a quantitative cloud-to-cloud comparison showed an average distance of 0.26 m between both data sets. Both point clouds were merged applying a conventional georeferenced approach. The merged DEM preserves the rich detail from the drone-based survey and provides an accurate 3D representation of the entire study area. It provides the most detailed model of the island so far, suitable to support practical and scientific applications. This study demonstrates that combination archived VHR satellite and low-altitude drone-based imagery provide inexpensive alternatives to generate high-quality DEMs.

An improved semi-global matching method with optimized matching aggregation constraint

Stereo dense image matching (DIM) is a key technique in generating dense 3D point clouds at low cost, among which semi-global matching (SGM) is one of the best compromise between the matching accuracy and the time cost. Most commercial or open-source DIM software packages therefore adopt SGM as the core algorithm for the 3D point generation, which computes matching results in 2D image space by simply aggregating the matching results of multi-directional 1D paths. However, such aggregations of SGM did not consider the disparity consistency between adjacent pixels in 2D image space, which will finally decrease the matching accuracy. To achieve higher-accuracy while keep the high time efficiency of SGM, this paper proposes an improved SGM with a novel matching aggregation optimization constraint. The core algorithm formulates the matching aggregation as the optimization of a global energy function, and a local solution of the energy function is utilized to impose the disparity consistency between adjacent pixels, which is capable of removing noises in the matching aggregation results and increasing the final matching accuracy at low time cost. Experiments on aerial image dataset show that the proposed method outperformed the traditional SGM method and another improved SGM method. Compared with the traditional SGM, our proposed method can increase the average matching accuracy by at most 11%. Therefore, our proposed method can applied in some smart 3D applications, e.g. 3D change detection, city-scale reconstruction, and global survey mapping.

Post‐filtering with surface orientation constraints for stereo dense image matching

AbstractDense image matching (DIM) is a critical technique when computing accurate 3D geometric information for many photogrammetric applications. Most DIM methods adopt first‐order regularisation priors for efficient matching, which often introduce stepped biases (also called fronto‐parallel biases) into the matching results. To remove these biases and compute more accurate matching results, this paper proposes a novel post‐filtering method by adjusting the surface orientation of each pixel in the matching process. The core algorithm formulates the post‐filtering as the optimisation of a global energy function with second‐order regularisation priors. A compromise solution of the energy function is computed by breaking the optimisation into a collection of sub‐optimisations of each pixel in a local adaptive window. The proposed method was compared with several state‐of‐the‐art post‐filtering methods on indoor, aerial and satellite datasets. The comparisons demonstrate that the proposed method obtains the highest post‐filtering accuracies on all datasets.

INVESTIGATING STANDARDIZED 3D INPUT DATA FOR SOLAR PHOTOVOLTAIC POTENTIALS IN THE NETHERLANDS

Abstract. This paper presents our contribution to the development of a standardized 3D input data model for solar photovoltaic potential estimation. Presently, different input data and processing steps influence the calculation for estimating the potential of solar energy in the Netherlands. The variety in characteristics of input data and issues with temporal accuracy extracted from the national registers and databases makes it challenging to obtain a consistent and reliable result. To address this issue, we created a point cloud dataset that integrated from LiDAR point cloud and dense image matching which is complete, recent and positionally accurate. Furthermore, we made a 3D building model from the integrated point cloud and identified the effect of finer resolution in the photovoltaic potential analysis.

IMPROVING QUALITY AND INCLUSIVE EDUCATION ON PHOTOGRAMMETRY: NEW TEACHING APPROACHES AND MULTIMEDIA SUPPORTING MATERIALS

Abstract. The democratization and accessibility of low-cost devices for image acquisition and the development of highly automated procedures for orientation and dense image matching allow almost every person to be a potential producer of photogrammetric models. The diffusion of image-based technologies to produce 3D models amongst wider audiences entails however some risks, as the lack of critical awareness of the final quality of the outputs. Information and education about potentialities and limitations of reality-based digitization by photogrammetry may help spreading procedures and methods for the correct use of this technology. This paper presents the results of one of the funded projects within the 2018 ISPRS Capacity Building Initiatives “Education and training resources on digital photogrammetry”. The production of multimedia material for supporting smart educational teaching and learning approaches will be reported, as well as experiences on their application on case studies. Blended innovative teaching and learning pedagogical approaches have been tested, as Flipped Classroom (FC), Learning-by-doing (LBD), Collaborative Learning (CL), and Challenge-Based Learning (CBL), supported by multimedia tools for capacity-building and knowledge transfer. The implementation of multimedia materials for supporting teaching strategies resulted in the production of updated and engaging resources, as videos, tutorials, and datasets to be used during courses, workshops, and seminars targeted to different user groups. The combination of teaching strategies and multimedia supporting materials were tested within national and international projects, from academic courses to complete non-experts, from activities on the field to online and distance learning.

QUALITY ASPECTS OF TRUE ORTHOPHOTO IN URBAN AREAS

Abstract. Orthophotos are one of the most popular photogrammetric products and have been a leading source of up-to-date 2D data of urban areas for years. In the last few years, together with innovations in the area of Dense Image Matching, Digital Surface Models created with dense image matching start to be utilized as the height source during orthorectification. Recently this production workflow of true orthophotos were adopted to production standard in many countries. The aim of the presented research was to evaluate recent developments in the area of automatic true orthophoto generation for urban areas and to define factors which have the main influence on the quality of the final product. Obtained results showed that besides of the image overlap, the main factors which have direct influence on the resulted true orthophoto are the occurrence of shadows and vegetation (trees). One of the outcomes of the presented research was that the quantitative methods develop for quality evaluation of Digital Surface Models and Point Clouds are not directly transferable on the quality evaluation of true orthophotos.

HIERARCHICAL APPROACH FOR DETECTING CHANGES WITH THE USE OF DIFFERENT PYRAMID LEVELS IN DENSE IMAGE MATCHING

Abstract. Many cities order spatial data systematically, in particular aerial nadir images and orthophotomaps. However, only the orthoimages and orthophotomaps are usually used by the city administration, particularly in spatial planning. Some of the users are not aware of the possibilities as to how the aerial images can be used. Spatial data users, who may not be specialists in photogrammetry, are sometimes not aware that it is possible to obtain 3D information from 2D images as a point cloud. The idea of dense image matching (DIM) is well-known and described in the field of photogrammetry. Although dense image matching is a time- and memory-consuming process, this does not present a major drawback with modern computing. Images for the test area – Warsaw – are characterised by Ground Sampling Distance (GSD) equal to 8 cm. These images can be successfully used in change detection processes, comparing the dense image matching point cloud from two different dates. What is important while considering land cover change detection, is that it is not necessary to generate a detailed and high-density point cloud, e.g. in order to detect changes in buildings. The main idea of the article is to present the possibility of using higher levels of images pyramid in dense image matching within the change detection process as a way to optimize the processing time and point cloud accuracy. Which level of pyramid is needed to detect different changes in urban land cover will also be discussed.

USE OF HISTORICAL AERIAL IMAGES FOR 3D MODELLING OF GLACIERS IN THE PROVINCE OF TRENTO

Abstract. Historical aerial images represent a source of information of great value for glacier monitoring, as they cover the area of interest at a well-defined epoch and allow for visual interpretation and metric analysis. Typically, the aerial images are used to produce orthophotos and manually digitize the perimeters of the glaciers for analysis of the surface extent of the glaciers, while the extraction of height information is more challenging due to data quality and characteristics. This article discusses the potential of historical aerial images for glacier modelling. More specifically, it analyses the impact of their coverage, radiometric- and geometric accuracy, state of preservation and completeness on the photogrammetric workflow. The data set used consists of scans of 300 (analog) aerial images acquired between August and October 1954 by the U.S. Air Force with a Fairchild KF7660 camera over the entire Province of Trento. For the modelling of the glaciers, different techniques such as manual stereoscopic measurement and dense image matching were tested on sample glaciers and the results were analysed in detail. Due to local radiometric saturation in a large part of the glacial surfaces and other disturbances affecting the historical images (e.g. scratches, scanning errors, dark shadows), dense image matching did not produce any valuable results, and stereo plotting could be used only on images (or image parts) with acceptable quality. The derived Digital Terrain Models (DTMs) were compared with a reference DTM obtained with dense image matching from digital aerial images acquired in September 2015 with an UltraCam Eagle sensor, and, for some glaciers, to a DTM obtained with dense image matching from scanned aerial images acquired in September 1983 with a RC30 analog camera. The differences between 1954 and 2015 DTMs showed values up to 70–80 m in height and a behaviour that is confirmed by the models employed by the glaciology experts in Trento.

END-TO-END BUILDING CHANGE DETECTION MODEL IN AERIAL IMAGERY AND DIGITAL SURFACE MODEL BASED ON NEURAL NETWORKS

Abstract. Multi-temporal building change detection is one of the most essential major issues of photogrammetry and remote sensing at current stage, which is of great significance for wide applications as offering real estate indicators as well as monitoring urban environment. Although current photogrammetry methodologies could be applicated to 2-D remote sensing imagery for rectification with sensor parameters, multi-temporal aerial or satellite imagery is not adequate to offer spectral and textual features for building change detection. Alongside recent development of Dense Image Matching (DIM) technology, the acquisition of 3-D point cloud and Digital Surface Model (DSM) has been generally realized, which could be combined with imagery, making building change detection more effective with greater spatial structure and texture information. Over the past years, scholars have put forward vast change detection techniques including traditional and model-based solutions. Nevertheless, existing appropriate methodology combined with Neural Networks (NN) for accurate building change detection with multi-temporal imagery and DSM remains to be of great research focus currently due to the inevitable limitations and omissions of existing NN-based methods, which is of great research prospect. This study proposed a novel end-to-end model framework based on deep learning for pixel-level building change detection from high-spatial resolution aerial ortho imagery and corresponding DSM sharing same resolution, which is from the dataset of Tokyo whole area.