Abstract
Compute grids are used widely in many areas of environmental science, but there has been limited uptake of grid computing by the climate modelling community, partly because the characteristics of many climate models make them difficult to use with popular grid middleware systems. In particular, climate models usually produce large volumes of output data, and running them also involves complicated workflows implemented as shell scripts. A new grid middleware system that is well suited to climate modelling applications is presented in this paper. Grid Remote Execution (G-Rex) allows climate models to be deployed as Web services on remote computer systems and then launched and controlled as if they were running on the user's own computer. Output from the model is transferred back to the user while the run is in progress, to prevent it from accumulating on the remote system and to allow the user to monitor the model. G-Rex has a representational state transfer (REST) architectural style, featuring a Java client program that can easily be incorporated into existing scientific workflow scripts. Some technical details of G-Rex are presented, with examples of its use by climate modellers.
Highlights
Computer models of the oceans and atmosphere are important tools in the study of the Earth’s system
GRID REMOTE EXECUTION (G-REX) exposes remotely installed climate models as Web services using the representational state transfer (REST)2 architectural style (Fielding 2000), in which the services and their resources are all identified by URLs, and the client communicates with the server using standard HTTP operations such as GET and POST
If the run appears to have a problem, the scientist can terminate it to prevent unnecessary resource usage. (vi) When an output file has been downloaded, it will be automatically deleted from the server; this happens during the model run, preventing accumulation of output files on the remote resource. (vii) When the run finishes, the G-REX system automatically deletes any remaining files associated with the run that are still left on the server
Summary
Computer models of the oceans and atmosphere are important tools in the study of the Earth’s system. To carry out scientific workflows involving meteorological forcing and data preand post-processing prior to job resubmission Models such as NEMO are typically run on expensive national high-performance computing (HPC) resources. To be effective for climate modelling applications, grid middleware must address two issues in particular: the large volume of output data, which must not be allowed to accumulate on the remote system; and the complicated scientific workflow scripts, with which the middleware must be able to integrate . These are the two challenges that the middleware described in this paper was designed to tackle
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
