Machine learning potentials for global multi-timescale diffuse irradiance estimation: Synthesizing ground observations, time-series, and environmental features

Abstract

Separating diffuse horizontal irradiance (DHI) from ground-based global horizontal irradiance observations is critical owing to lack of direct DHI observations. Existing models are often site-specific and time-bound, thereby limiting their universal applicability. To address this, this study thoroughly explores machine learning (ML) for constructing global separation models. We develop 36 models using three ML algorithms—light gradient boosting machine (LightGBM), generalized additive model (GAM), and geographically weighted artificial neural network (GWANN)—alongside three data splitting strategies for minutely, 10-minutely, hourly, and daily timescales. LightGBM and GAM outperform GWANN in accuracy, with LightGBM excelling in interpretability, efficiency, and handling missing values, while GWANN exhibits superior stability. Compared with completely random splitting, model accuracy decreases by 2.26 % and 2.16 % for station- and date-based splitting, respectively. Prediction accuracy varies across timescales, with minutely models typically considered the best. Key predictors are Kt, solar zenith angle, and altitude, complemented by the significant influence of time-series Kt, aerosol, and cloud features. The influence of various factors on model accuracy differs and is scale-dependent. Among them, time-series Kt variability factors notably enhance ML-based predictions, especially at minutely scales. The LightGBM model outperforms classic models in overall and site-specific accuracy and adaptability, but its computational efficiency declines, especially with time-series variability factors. The findings highlight that ML-based separation models necessitate careful selection of algorithm, data splitting strategy, and input variables, fully considering study region, data situation, and temporal and spatial scales. This research offers a potential solution for constructing global separation models and insights for model improvement.