“Big Data Assimilation” Toward Post-Petascale Severe Weather Prediction: An Overview and Progress

Takemasa Miyoshi,Masaru Kunii,Michiko Otsuka,Seiya Nishizawa,Shigenori Otsuka,Ryuji Yoshida,Juan Ruiz,Hiromu Seko,Tomoo Ushio,Sachiho A Adachi,Kotaro Bessho,Hirofumi Tomita,Kozo Okamoto,Yasumitsu Maejima,Jianwei Liao,Guo-Yuan Lien,Yutaka Ishikawa,Balazs Gerofi,Shinsuke Satoh

doi:10.1109/jproc.2016.2602560

Abstract

Following the invention of the telegraph, electronic computer, and remote sensing, “big data” is bringing another revolution to weather prediction. As sensor and computer technologies advance, orders of magnitude bigger data are produced by new sensors and high-precision computer simulation or “big simulation.” Data assimilation (DA) is a key to numerical weather prediction (NWP) by integrating the real-world sensor data into simulation. However, the current DA and NWP systems are not designed to handle the “big data” from next-generation sensors and big simulation. Therefore, we propose “big data assimilation” (BDA) innovation to fully utilize the big data. Since October 2013, the Japan’s BDA project has been exploring revolutionary NWP at 100-m mesh refreshed every 30 s, orders of magnitude finer and faster than the current typical NWP systems, by taking advantage of the fortunate combination of next-generation technologies: the 10-petaflops K computer, phased array weather radar, and geostationary satellite Himawari-8. So far, a BDA prototype system was developed and tested with real-world retrospective local rainstorm cases. This paper summarizes the activities and progress of the BDA project, and concludes with perspectives toward the post-petascale supercomputing era.

Highlights

Weather prediction has been revolutionized due to new technological developments such as the invention of the telegraph, electronic computer, and remote sensing
As for the input data used in our experiments, we use the Scalable Computing for Advanced Library and Environment (SCALE)-LETKF to perform the data assimilation cycle for regional weather analysis employing realworld observations to test the efficiency when equipped with the proposed I/O middleware
Output data after data simulation, which data will be read by the corresponding LETKF process as the input data for assimilation

Summary

Introduction

Weather prediction has been revolutionized due to new technological developments such as the invention of the telegraph, electronic computer, and remote sensing. The telegraph enabled plotting weather charts in the mid19th century once we could exchange weather observation data for a long distance quickly. In 1955, the U.S Joint Numerical Weather Prediction Unit started operational numerical weather prediction (NWP) using one of the earliest mainframe machines IBM 701. The computer has been growing rapidly, and NWP with high-performance computers (HPCs) became a key technology for weather prediction. In the 1970s, space exploration technologies enabled satellite remote sensing. A number of weather satellites have been launched, and it was essential to incorporate as soon as possible these new data into NWP. The present weather prediction largely relies on NWP with HPCs and satellite data. The method of integrating computer simulations and real-world data is known as data assimilation (DA), which plays an important role in NWP as the time-evolving simulation model

Objectives

Methods

Results

Conclusion