Forecasting market volatility, especially high-volatility incidents, is a critical issue in financial market research and practice. Business news as an important source of market information is often exploited by artificial intelligence–based volatility forecasting models. Computationally, deep learning architectures, such as recurrent neural networks, on extremely long input sequences remain infeasible because of time complexity and memory limitations. Meanwhile, understanding the inner workings of deep neural networks is challenging because of the largely black box nature of large neural networks. In this work, we address the first challenge by proposing a long- and short-term memory retrieval (LASER) architecture with flexible memory and horizon configurations to forecast market volatility. Then, we tackle the interpretability issue by devising a BEAM algorithm that leverages a large pretrained language model (GPT-2). It generates human-readable narratives verbalizing the evidence leading to the model prediction. Experiments on a Wall Street Journal news data set demonstrate the superior performance of our proposed LASER-BEAM pipeline in predicting high-volatility market scenarios and generating high-quality narratives compared with existing methods in the literature. History: Accepted by Ram Ramesh, Area Editor for Date Science & Machine Learning. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0055 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0055 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .