The existing literature on interest rate volatility using standard affine, arbitrage-free mod- els of the term structure is still inconclusive as to which factors are causing interest rate volatility. This is partly due to the latent nature of the factors in such models, and partly due to the difficulty in estimating their parameters. In this paper, to address both issues simulta- neously, we choose to focus on the widely used Nelson-Siegel yield curve model with its three factors representing the level, slope, and curvature of the yield curve. Christensen, Diebold, and Rudebusch (CDR, 2007) introduced the affine arbitrage-free class of Nelson-Siegel models and showed that it is easy to estimate and delivers robust estimates of the model parameters in addition to fitting the yield curve well in sample and forecasting yields well out of sample. However, their model class is characterized by constant volatility and, hence, has little to bear on the important issue of interest rate volatility. We introduce three new classes (with a total of five subclasses) of affine arbitrage-free models that incorporate stochastic volatility fac- tors, while preserving as much of the Nelson-Siegel factor loading structure as possible. We examine the in-sample and out-of-sample properties of various specifications, ranging from the most flexible to the most parsimonious. The models' yields are compared to those of the benchmark CDR model with constant volatility. We further compare the models' volatility dynamics to realized volatility measures across the maturity spectrum. Our goal is to identify a class of models that preserves the ability to forecast the yield term structure at least on par with the CDR model, while also being able to match yield volatility.