Seasonal Ensemble Forecasting Application On Dependable Sumegha Scientific Cloud Infrastructure

Abstract

Despite several advances in understanding the behaviour of monsoon variability, innovations in the numerical modeling and the availability of higher computational capabilities, accurate prediction of Indian summer monsoon still remains a serious challenge. Seasonal Forecast Model (SFM), developed for seasonal forecast and climate research, is used for forecasting the Indian summer monsoon in advance of a season. Ensemble forecasting method helps us in finding and minimizing the uncertainty inherent in seasonal forecast. The inherent parallel nature and the bursty computational demands of the ensemble forecasting method allows it to effectively utilize the Infrastructure-as-a-Service (IaaS) model on the cloud platform. However, realizing huge scientific experiments is still a challenge to the cloud service providers as well as to the climate modeling community. To start with prototype experiments using SFM model were conducted at T-62 resolution (~ 200 km x 200 km grid). The experience gained from the prototype runs were used by the SuMegha operational team to fine tune the configuration of SuMegha Cloud resources to improve the quality of service. High resolution SFM at T-320 (~ 37 km x 37 km grid) was also configured and experiments were conducted to understand the scalability, computational performance of the application and the reliability of SuMegha Cloud. In this paper, we use SFM as a case study to present the key problems found by climate applications, and propose a framework to run the same on SuMegha Cloud infrastructure to allow a climate model to take advantage of these cloud resources in a seamless and reliable way. The framework uses classification and outlier detection techniques to classify the resources and also to identify the faulty resources. It addresses the challenges such as unexpected hardware failures, power outages, failed porting and software bugs. We share our experience in conducting the ensemble forecasting experiments on SuMegha Cloud using the proposed framework. We also attempt to provide a perspective on the desirable features of a scientific cloud infrastructure, for easier adaptation by the climate modeling community to conduct large scientific experiments.