PLANES4LOD2: Reconstruction of LoD-2 building models using a depth attention-based fully convolutional neural network

Abstract

Level of detail (LoD)-2 reconstruction is an inevitable task in digital twin-related applications such as disaster management, flood simulation, landslide simulation and solar panel recommendation. However, there is a lack of capable methods that can exploit fine details in RGB imagery and mitigate noise in photogrammetric digital surface models (DSMs). Our investigation is focused on the use of roof planes to achieve a geometrically complete and correct, and topologically consistent LoD-2 building reconstruction. Using UNet with the EfficientNet-B3 backbone, the developed approach starts with jointly predicting building sections and roof planes from the orthorectified RGB imagery and a photogrammetric DSM. The detected sections and planes are then vectorized by employing tree search and simplified with the Douglas Peucker algorithm. Subsequently, height values from the noisy input DSM and the vectorized image-based (and simplified) roof planes are used to derive 3D-planes. Finally, the building model is formed by computing plane intersections as the ridge lines. This study demonstrates that a well-designed depth attention module (DAM), which is the bottleneck of the UNet, can achieve a very good use of both spectral and depth features. The resultant 1-to-n correspondence between building section and roof plane benefits accurate and consistent building model reconstruction. Furthermore, it leads to a superior generalization capability of the proposed method. Experiments with 1437 buildings from the cities Cologne and Braunschweig, Germany, demonstrate the success of the proposed workflow in reconstructing compound buildings with complex roof structures. The achieved geometric mean absolute error (MAE) is 1.06m and 0.24m respectively. Comprehensive comparative evaluations showcase the superiority of the approach in terms of geometric completeness and accuracy, and topological consistence with. The improvement over SAT2LOD2 (Gui and Qin, 2021) is 1.12m in Cologne (data accessible at https://github.com/dlrPHS/GPUB) and 0.47m in Braunschweig in geometrical MAE.