Increasing resolution and forecast length with a parallel ocean model

Abstract

A 3-dimensional baroclinic model of the North Sea and the Baltic Sea has delivered operational forecasts since 1995. The model is mainly forced by the operational atmospheric model (HIRLAM) at the Swedish Meteorological and Hydrological Institute (SMHI), but also by monthly means of river runoff and wave radiation stress from a wind wave model. The model is running with a nested grid, where a 12 nautical mile grid covers the whole area while Skagerrak, Kattegat, the Belt Sea and the Baltic Sea is covered with a 3 nautical mile grid. A crucial parameter in ocean modelling is how well the model resolves the meso-scale dynamics with a scale of the same order as the internal Rossby radius. For the Baltic Sea, this scale is roughly from 3 to 6 km, i.e. of the same size as the present resolution of 3 nautical miles. When moving to a massive parallel computer with distributed memory, it is now possible to increase the resolution to 1 nautical mile and still run a 48-hours forecast within 1 hour CPU-time. The parallelisation has been performed in two steps. At first the whole computational domain has been partitioned into a large number of rectangular blocks to avoid inactive land-points. The blocks have then been projected into the number of available parallel processors. Secondly, a set of F90 modules have been developed to manage the different grids and blocks while hiding parallelisation details. Only one grid block at a time is presented to the original F77 code. Finally, a new solver has been introduced for the linear equation systems for water level and sea ice dynamics, which now are solved with a distributed multi-frontal solver. We have found that the production of HIROMB, forecasts can successfully be moved from the present vector computer C90 to the massive parallel computer T3E while increasing the resolution from 3 to 1 nautical mile. However, speedup and load balance could still be further improved.