Accurate prediction of solar irradiance is essential for the successful integration of solar power plants into electrical systems. Despite recent advancements in deep learning technology yielding impressive results in solar forecasting, their lack of interpretability has hindered their widespread adoption. In this paper, we propose a novel approach that integrates a Temporal Fusion Transformer (TFT) with a McClear model to achieve accurate and interpretable forecasting performance. The TFT is a deep learning model that provides transparency in its predictions through the use of interpretable self-attention layers for long-term dependencies, recurrent layers for local processing, specialized components for feature selection, and gating layers to suppress extraneous components. The model is capable of learning temporal associations between continuous time-series variables, namely, historical global horizontal irradiance (GHI) and clear sky GHI, accounting for cloud cover variability and clear sky conditions that are often ignored by most machine learning solar forecasters. Additionally, it minimizes a quantile loss during training to produce accurate probabilistic forecasts. In this study, we evaluate the performance of hourly GHI forecasts on eight diverse datasets with varying climates: temperate, cold, arid, and equatorial, for multiple temporal horizons of 2, 3, 6, 12, and 24 h. The model is benchmarked against both climatological persistence for deterministic forecasting and Complete History Persistence Ensemble for probabilistic forecasting. To prove that our model is not location locked, it has been blind tested on four completely different datasets. The results demonstrate that the proposed model outperforms its counterparts across all forecast horizons.