Modelling sunlight and shading distribution on 3D trees and buildings: Deep learning augmented geospatial data construction from street view images

In complex urban environments, accurately estimating the shading effects of trees on three-dimensional (3D) building surfaces is crucial to facilitate building design and urban greenery implementation. However, there is a long-unsolved challenge in efficiently and elaborately modelling trees and simulating spatiotemporally heterogeneous shading effects of trees on 3D urban envelopes. To overcome the challenge, this study proposes a research framework that: (i) employs transfer learning to build a deep learning model for accurately segmenting geo-objects in Street View Images (SVIs), (ii) utilizes semantic segmentation results to fit regressions between the pixels of specific geo-objects in the SVIs and the corresponding real-world lengths of standard geo-objects, develops a 3D space geometric projection model for calculating tree coordinates and 3D geometries, and identifies the real spatial relationships between buildings and trees to calibrate errors caused by segmentation inaccuracies for subsequent simulations, and (iii) integrates the calibrated 3D tree models with 3D building models to construct a unified 3D urban model for estimating the spatiotemporal distribution of sunlight and shading. Using Singapore as the study area, we adopted DeepLabV3+, a widely used pre-trained semantic segmentation model, to achieve IoU of 91.51 % for buildings and 76.29 % for trees, with F1-scores of 97.93 % and 88.19 % respectively. Additionally, data calibration optimized initial tree polygons in 39.03 % of the SVIs, reducing outliers and improving modeling accuracy and robustness. The results demonstrate that the proposed framework efficiently and accurately models high-density urban environments, providing a practical solution to complex shading problems and reducing data acquisition and processing costs.