A Barotropic Solver for High-Resolution Ocean General Circulation Models

Xiaodan Yang,Weiguo Liu,Shengchang Zhou,Shan Zhou,Zhenya Song

doi:10.3390/jmse9040421

Abstract

High-resolution global ocean general circulation models (OGCMs) play a key role in accurate ocean forecasting. However, the models of the operational forecasting systems are still not in high resolution due to the subsequent high demand for large computation, as well as the low parallel efficiency barrier. Good scalability is an important index of parallel efficiency and is still a challenge for OGCMs. We found that the communication cost in a barotropic solver, namely, the preconditioned conjugate gradient (PCG) method, is the key bottleneck for scalability due to the high frequency of the global reductions. In this work, we developed a new algorithm—a communication-avoiding Krylov subspace method with a PCG (CA-PCG)—to improve scalability and then applied it to the Nucleus for European Modelling of the Ocean (NEMO) as an example. For PCG, inner product operations with global communication were needed in every iteration, while for CA-PCG, inner product operations were only needed every eight iterations. Therefore, the global communication cost decreased from more than 94.5% of the total execution time with PCG to less than 63.4% with CA-PCG. As a result, the execution time of the barotropic modes decreased from more than 17,000 s with PCG to less than 6000 s with CA-PCG, and the total execution time decreased from more than 18,000 s with PCG to less than 6200 s with CA-PCG. Besides, the ratio of the speedup can also be increased from 3.7 to 4.6. In summary, the high process count scalability when using CA-PCG was effectively improved from that using the PCG method, providing a highly effective solution for accurate ocean simulation.

Highlights

Ocean general circulation models (OGCMs) have become increasingly important for understanding oceanic dynamic processes and ocean environment forecasting
We found that the poor scaling performance of the preconditioned conjugate gradient (PCG) method was caused by the operation of global reductions, which could account for more than 96% of the total execution time when using more than 8000 processes (Figure 2)
The model results simulated using the communication-avoiding Krylov subspace method with a PCG (CA-PCG) solver were accurate, and the CA-PCG solver can be used in the Nucleus for European Modelling of the Ocean (NEMO) model in future research

Summary

Introduction

Ocean general circulation models (OGCMs) have become increasingly important for understanding oceanic dynamic processes and ocean environment forecasting. OGCMs have been developed with finer resolution (10–100 km for eddy-resolving ocean models) and more physical procedures due to increasing scientific requirements. OGCMs with finer horizontal resolution can provide more accurate results [1]. The computational cost will be almost three orders of magnitude if the horizontal resolution is fined by one order of magnitude [2,3]. The parallel efficiency becomes a great challenge for OGCMs. Due to the low parallel efficiency, current global operational forecasting systems are still in 0.1◦ (∼10 km) to 1◦ (∼100 km) resolution, which falls far short of expectations. With the resolution finer, the demand for improving computational performance is more and more urgent

Methods

Results

Conclusion