In this study, we aimed to combine support vector regression (SVR) with an autoregressive (AR) model and a generalized autoregressive conditional heteroscedastic (GARCH) or asymmetric GARCH model. We compared the performances of these combined models with the maximum likelihood estimation (MLE)-based models under a normal, Student’s t, or skewed Student’s t distributional assumption. We employed such models to protect the heteroscedastic risks of four major cryptocurrencies, namely, Bitcoin, Ethereum, Tether, and Binance Coin, before and during COVID-19. We accomplished this task by forecasting their volatility, Value-at-Risk (VaR), and Expected Shortfall (ES). The empirical results revealed that in terms of the root-mean-square error (RMSE) and mean-absolute error (MAE) metrics, the SVR-based models with a polynomial or radial basis kernel function, which is nonlinear, resulted in a more accurate volatility forecast compared to the MLE-based models under any distributional assumption. The superiority of the former models was more apparent when applied to Tether exhibiting the most stable volatility. This supports evidence of nonlinear characteristics in the cryptocurrency return and volatility series. A combination between SVR and an asymmetric GARCH model also confirms the presence of an asymmetrically nonlinear relationship between the returns and volatility. In addition, the correct-VaR and Kupiec’s backtesting results showed that the SVR-based models tended to produce the VaR forecasts for the four cryptocurrencies with better accuracy at the 1%, 5%, and 10% significance levels compared to the MLE-based ones. The use of the correct-ES metric also leads to the superior ES forecasting performance of the former models at any level. However, McNeil and Frey’s backtesting results demonstrated their worse ES forecasting performance at the 1% level. As the COVID-19 pandemic progressed, we also found an increase in the Bitcoin and Binance Coin market risks as well as a decrease in the Ethereum and Tether market risks.