Abstract

Cloud platforms are becoming more prevalent in every computational domain, particularly in e-Science. A typical scientific workload will have a long execution time or be data intensive. Providing an execution environment for these applications, which belong to different tenants, has to deal with the horizontal scaling of execution flows (i.e. threads) and an effective allocation of resources that takes into account the effective progress made by each tenant. While this is trivial for Bag-of-Tasks and embarrassingly parallel jobs, it is hard for HPC single-process multi-threaded applications because they cannot be scaled up automatically just by adding more virtual machines to execute the workload. In this paper we present MengTian, a distributed execution environment or platform capable of addressing the issues above. It encompasses several extensions to the Java execution environment, ranging from middleware to the virtual machine code and libraries. Our Java-based platform provides a Single System Image abstraction supported by a Partially Global Address Space to transparently spawn threads across a cluster of machines. It monitors progress with different levels-of-detail and accounts and restricts resource consumption. The overall goal is to redistribute resources among different JVM instances, increasing the unitary outcome of the progress vs. resource usage ratio over time.

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.