Accelerating Community-Wide Evaluation of AI Models for Global Weather Prediction by Facilitating Access to Model Output

Abstract

Listen

Translate article

Abstract Numerous Artificial Intelligence-based Weather Prediction (AIWP) models have emerged over the past two years, mostly in the private sector. There is an urgent need to evaluate these models from a meteorological perspective, but access to the output of these models is limited. We detail two new resources to facilitate access to AIWP model output data in the hope of accelerating the investigation of AIWP models by the meteorological community. First, a three year (and growing) reforecast archive beginning in September of 2020 containing twice daily 10-day forecasts for FourCastNet v2-small, Pangu-Weather, and GraphCast Operational is now available via an Amazon S3 bucket through NOAA’s Open Data Dissemination (NODD) program (https://noaa-oar-mlwp-data.s3.amazonaws.com/index.html). This reforecast archive was initialized with both NOAA”s Global Forecast System (GFS) and ECMWF’s Integrated Forecast System (IFS) initial conditions in the hopes that users can begin to perform feature-based verification of impactful meteorological phenomena. Second, real-time output for these three models is visualized on our web page (https://aiweather.cira.colostate.edu) along with output from the GFS and the IFS. This allows users to easily compare output between each AIWP model and traditional, physics-based models with the goal of familiarizing users with the characteristics of AIWP models and determine whether the output aligns with expectations, is physically consistent and reasonable, and/or is trustworthy. We view these two efforts as a first step towards evaluating whether these new AIWP tools have a place in forecast operations.