Vectorizing planar roof structure from very high resolution remote sensing images using transformers

Abstract

ABSTRACT Accurately predicting the geometric structure of a building's roof as a vectorized representation from a raster image is a challenging task in building reconstruction. In this paper, we propose an efficient and precise parsing method called Roof-Former, based on a vision Transformer. Our method involves three steps: (1) Image encoder and edge node initialization, (2) Image feature fusion with an enhanced segmentation refinement branch, and (3) Edge filtering and structural reasoning. Our method outperforms previous works on the vectorizing world building dataset and the Enschede dataset, with vertex and edge heat map F1-scores increasing from 87.1 % , 76.2 % to 89.1 % , 78.1 % , and from 69.7 % , 68.8 % to 71.2 % , 69.5 % , respectively. Furthermore, our method demonstrates superior performance compared to the current state-of-the-art based on qualitative evaluations, indicating its effectiveness in extracting global image information while maintaining the consistency and topological validity of the roof structure.