Abstract
Abstract. This research is the first to apply MeshCNN – a deep learning model that is specifically designed for 3D triangular meshes – in the photogrammetry domain. We highlight the challenges that arise when applying a mesh-based deep learning model to a photogrammetric mesh, especially w.r.t. data set properties. We provide solutions on how to prepare a remotely sensed mesh for a machine learning task. The most notable pre-processing step proposed is a novel application of the Breadth-First Search algorithm for chunking a large mesh into computable pieces. Furthermore, this work extends MeshCNN such that photometric features based on the mesh texture are considered in addition to the geometric information. Experiments show that including color information improves the predictive performance of the model by a large margin. Besides, experimental results indicate that segmentation performance could be advanced substantially with the introduction of a high-quality benchmark for semantic segmentation on meshes.
Highlights
Semantic segmentation is one of the fundamental problems in computer vision; it is extensively researched both in two-dimensional images and three-dimensional representations such as voxel grids, point clouds, or mesh grids
We address the compatibility of meshes generated by photogrammetry with MeshCNN and, thereby, highlight the impact of the mesh properties on the model’s predictive performance
The Center of Gravity (COG) cloud baseline model and MeshCNN’s predictive performances are significantly better when photometric features are added
Summary
Semantic segmentation is one of the fundamental problems in computer vision; it is extensively researched both in two-dimensional images and three-dimensional representations such as voxel grids, point clouds, or mesh grids. In the context of aerial imagery, most publications focused on the segmentation of point clouds or their voxelized representations, respectively. These are obvious choices since those data representations – especially voxel grids – enable the adaption of approaches proven to be effective on 2D pixel data. In the field of remote sensing and photogrammetry, triangular textured meshes gradually replace point clouds as a final user product (Laupheimer et al, 2020a,b)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
