Solar irradiance forecasting is essential in renewable energy grids amongst others for back-up programming, operational planning, and short-term power purchases. This study focuses on forecasting hourly solar irradiance using data obtained from the Southern African Universities Radiometric Network at the University of Pretoria radiometric station. The study compares the predictive performance of long short-term memory (LSTM) networks, support vector regression and feed forward neural networks (FFNN) models for forecasting short-term solar irradiance. While all the models outperform principal component regression model, a benchmark model in this study, the FFNN yields the lowest mean absolute error and root mean square error on the testing set. Empirical results show that the FFNN model produces the most accurate forecasts based on mean absolute error and root mean square error. Forecast combination of machine learning models' forecasts is done using convex combination and quantile regression averaging (QRA). The predictive performance we found is statistically significant on the Diebold Mariano and Giacomini-White tests. Based on all the forecast accuracy measures used in this study including the statistical tests, QRA is found to be the best forecast combination method. QRA was also the best forecasting model compared with the stand-alone machine learning models. The median method for combining interval limits gives the best results on prediction interval widths analysis. This is the first application of LSTM on South African and African solar irradiance data to the best of our knowledge. This study has shown that providing adequate and detailed evaluation metrics, including statistical tests in forecasting gives more insight into the developed forecasting models.