Abstract
AbstractCloud computing platforms can facilitate the use of Earth science models by providing immediate access to fully configured software, massive computing power, and large input data sets. However, slow internode communication performance has previously discouraged the use of cloud platforms for massively parallel simulations. Here we show that recent advances in the network performance on the Amazon Web Services cloud enable efficient model simulations with over a thousand cores. The choices of Message Passing Interface library configuration and internode communication protocol are critical to this success. Application to the Goddard Earth Observing System (GEOS)‐Chem global 3‐D chemical transport model at 50‐km horizontal resolution shows efficient scaling up to at least 1,152 cores, with performance and cost comparable to the National Aeronautics and Space Administration Pleiades supercomputing cluster.
Highlights
Cloud computing platforms can provide scientists immediate access to complex Earth science models and large datasets, greatly facilitating scientific research and collaboration (Zhuang et al, 2019)
We apply this new capability to the Goddard Earth Observing System (GEOS)-Chem model of atmospheric chemistry (Eastham et al, 2018), and present an easy-to-follow research workflow for GEOS-Chem users on the Amazon Web Services (AWS) cloud that can serve as template for other Earth science models
We found that only the Intel-Message Passing Interface (MPI) library can efficiently utilize Elastic Fabric Adapter (EFA); the OpenMPI library still had to use Transmission Control Protocol (TCP) for inter-node communication. 4.2 Network benchmark results
Summary
Cloud computing platforms can provide scientists immediate access to complex Earth science models and large datasets, greatly facilitating scientific research and collaboration (Zhuang et al, 2019). We show that new technology makes it possible to conduct computationally- and cost-efficient simulations with over a thousand cores on cloud platforms We apply this new capability to the GEOS-Chem model of atmospheric chemistry (Eastham et al, 2018), and present an easy-to-follow research workflow for GEOS-Chem users on the Amazon Web Services (AWS) cloud that can serve as template for other Earth science models. The AWS cloud has recently introduced (1) the “Nitro Hypervisor” for virtual machine management with extremely low overhead and “near-bare-metal” performance (Gregg, 2017); (2) a new instance type “C5n” with 100 Gb/s bandwidth, 4× higher than the previous “C5” type (Amazon, 2018e); and (3) a low-latency network interface called Elastic Fabric Adapter (EFA) to improve the inter-node communication performance for Message Passing Interface (MPI) libraries (Amazon, 2019e) Due to these and other improvements, it is time to revisit the suitability of cloud computing platforms for HPC applications. Detailed online instructions are available for reproducing this work (See “Acknowledgments, Samples, and Data”)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
