Physics-Informed Parallel Hybrid Ensemble of LightGBM and Transformer for Ultra-Accurate 15-Minute Solar PV Power Forecasting

Amira S. Mohamed Amira S. Mohamed, Frederic Andres Frederic Andres

The increasing penetration of photovoltaic (PV) systems demands highly accurate short-term power forecasting to ensure grid stability, facilitate energy trading, and optimize renewable energy operations. This study presents a comprehensive evaluation of seven machine learning and deep learning models for 15-minute ahead solar PV power forecasting using high-resolution data from the Desert Knowledge Australia Solar Centre (DKASC) Alice Springs Site 13. A robust physics-informed preprocessing pipeline was implemented, incorporating outlier clipping based on physical constraints, physics-aware imputation, and extensive feature engineering—including cyclical temporal encodings, multi-scale lag features, rolling statistics, interaction terms, and a causally constructed clear-sky index—to preserve temporal causality and enhance model performance. The study introduces a novel Parallel Hybrid Ensemble framework that strategically combines the strengths of LightGBM’s gradient boosting with the Transformer’s self-attention mechanism. Through Optuna-based hyperparameter optimization and rigorous statistical validation—including Friedman tests, bootstrap confidence intervals, seasonal robustness analysis, and multiple leakage-prevention protocols—the proposed ensemble achieved state-of-the-art performance on the independent test set: R² = 0.9993, RMSE = 0.0375 kW, and MAE = 0.0271 kW, representing a 90.5% reduction in RMSE compared to the 15-Minute persistence baseline. The ensemble maintained exceptional consistency across all seasons, attaining a peak seasonal R² of 0.9996 in Spring. Among standalone models, LightGBM significantly outperformed deep learning architectures (R² = 0.9963), followed by LSTM (0.9478), MLP (0.9461), Transformer (0.9429), and CNN (0.9226). The superior performance of the hybrid ensemble is attributed to the complementary capabilities of tree-based models in capturing nonlinear feature interactions and attention mechanisms in modeling temporal dependencies. These results establish the proposed Parallel Hybrid Ensemble as a highly accurate and robust solution for ultra-short-term solar PV power forecasting, offering substantial value for grid integration, energy management, and renewable energy systems.