Real-Time Probabilistic Tropical Cyclone Forecasting in the Cloud

Abstract

Abstract Despite improvements in predicting the track and intensity of tropical cyclones (TCs), these storms with major societal and economic impacts continue to pose challenges for statically provisioned computational resources. The number of active storms varies from day to day, leading to regular bursts of irregular computational loads atop an already busy production schedule for weather prediction centers. The emergence of high-resolution ensemble TC prediction to quantify the uncertainty in track and intensity exacerbates this problem by requiring multiple forecasts run for each storm, each representing a possible outcome. With more than a decade of progress in the literature describing research and real-time numerical weather prediction in the cloud, we set out to evaluate if the commercial cloud environment could cope with the unique demands of TC ensemble forecasts. We describe a demonstration using a high-performance computing environment within the Microsoft Azure cloud to test dynamic resource provisioning to address time-varying resource challenges. We deployed existing operational models, implemented a combination of vendor-provided and open-source tools to orchestrate the cycling production workflows, and developed techniques for automatic error handling to keep production on schedule with minimal operator intervention. Despite challenges, our production pipeline from data ingest, forecast integration, graphics generation, and dissemination via social media was able to produce real-time forecasts of storm track and intensity with product latencies commensurate with existing operational forecasting systems.