Deep Learning-Based Building Detection Using High-Resolution RGBI Orthophotos and DSMs

Mohamed Fawzy, Attila Juhász, Árpád Barsi

Abstract. Deep learning techniques have demonstrated a promising efficacy for building feature extraction, presenting practical strategies to lessen the labour-intensive work of map updating, change detection, and urban growth monitoring. To address the labour-consuming challenges, a U-Net-based framework is developed to generate building maps automatically using high-resolution RGBI orthophoto and DSM data. The approach shows the effectiveness of the U-Net-based semantic segmentation for urban scene analysis. The presented procedures collect, preprocess, and combine orthophoto with DSM to train, apply, and assess the U-Net model for building extraction in urban environments using two input scenarios: (1) solely RGBI orthophoto and (2) RGBI orthophoto integrated with DSM. Four standard metrics: completeness, correctness, quality, and overall accuracy are applied to evaluate the model outputs, comparing the single orthophoto input to the combined orthophoto with DSM for building detection. The significant impact of the DSM and RGBI pairing is demonstrated by the heightened reliability of the data integration strategy when estimating buildings within nearby similar objects like roads and impervious surfaces. However, a few challenges related to the model's generalisation are noticed across complex urban contexts, including tree occlusions, unreferenced building extensions, and height irregularities surrounding structures. The findings highlight the potential of multimodal data fusion in urban investigations and reveal how it can improve the built-up asset mapping. Final results argue that DSM incorporation significantly enhances building classification performance using deep learning frameworks for geospatial applications, particularly in complex urban environments where single data and traditional image-based segmentation methods are inadequate.