Transient Server Research Articles

Robust and effective ab initio molecular dynamics simulations on the GPU cloud infrastructure using the Schrödinger Materials Science Suite

Ab initio Born-Oppenheimer molecular dynamics (AIMD) is a valuable method for simulating physico-chemical processes of complex systems, including reactive systems, and for training machine learning models and force fields. Speed and stability issues on traditional hardware preclude routine AIMD simulations for larger systems and longer timescales. We postulate that any practically useful AIMD simulation must generate a trajectory of a minimum 1000 MD steps a day on a moderate cloud resource. In this work, we implement a computing workflow that enables routine calculations at this throughput and demonstrate results for several non-trivial atomistic dynamical systems. In particular, we have employed the GPU implementation of the Quantum ESPRESSO code which we will show increases AIMD productivity compared to the CPU version. In order to take advantage of transient servers (which are more cost and energy effective compared to the stable servers), we have implemented automatic restart/continuation of the AIMD runs within the Schrödinger Materials Science Suite. Finally, to reduce simulation size and thus reduce compute time when modeling surfaces, we have implemented a wall potential constraint. Our benchmarks using several reactive systems (lithium anode surface/solvent interface, hydrogen diffusion in an iron grain boundary) show a significant speed up when running on a GPU-enabled transient server using our updated implementation.

SciSpot: Scientific Computing On Temporally Constrained Cloud Preemptible VMs

Scientific computing applications are being increasingly deployed on cloud computing platforms. Transient servers such as EC2 spot instances and Google Preemptible VMs, can be used to lower the costs of running applications on the cloud by up to <inline-formula><tex-math notation="LaTeX">$10\times$</tex-math></inline-formula> . However, the frequent preemptions and resource heterogeneity of these transient servers introduces many challenges in their effective and efficient use. In this paper, we develop techniques for modeling and mitigating preemptions of transient servers, and present SciSpot, a software framework that enables low-cost scientific computing on the cloud. SciSpot deploys applications on Google Cloud Preemptible Virtual Machines that exhibit temporally constrained preemptions: VMs are always preempted in a 24 hour interval. Our empirical analysis shows that the preemption rate is generally bathtub shaped, which raises multiple fundamental challenges in performance modeling and policy design. We develop a new reliability model for temporally constrained preemptions, and use statistical mechanics to show why the bathtub shape is generally exhibited. SciSpot’s design is guided by our observation that many emerging scientific computing applications that integrate machine learning with simulations, can be deployed as “bags” of jobs, which represent multiple instantiations of the same computation with different physical model parameters. For a bag of jobs, SciSpot finds the optimal transient server on-the-fly, by taking into account the price, performance, and preemption rates of different servers. SciSpot reduces costs by <inline-formula><tex-math notation="LaTeX">$5\times$</tex-math></inline-formula> compared to conventional cloud deployments, and reduces makespans by up to <inline-formula><tex-math notation="LaTeX">$10\times$</tex-math></inline-formula> compared to conventional high performance computing clusters.

The Spectral Reclassification of Nearby (z &lt; 0.02) Type IIn Supernovae

Abstract Type IIn supernovae (SNe IIn) are an enigmatic subclass of SNe that show heterogeneity in their photometric and spectroscopic properties. Complex, multi-component Balmer line profiles are a defining property of SNe IIn. Perhaps owing to the heterogeneity in the SN IIn class, online databases often conflict and hold outdated or erroneous classifications. This is compounded by SN “impostors” and H ii region contamination. We outline a classification scheme based on multi-component Hα profiles and apply it to a sample of 115 nearby (redshift z &lt; 0.02) SNe that have held a SN IIn classification on the Open Supernova Catalogue (OSC). We find that 25 of our transients were misclassified as SNe IIn. We also note that there is disagreement between OSC and the Transient Name Server for 51 of the transients.

Cost-Effective Cloud Server Provisioning for Predictable Performance of Big Data Analytics

Cloud datacenters are underutilized due to server over-provisioning. To increase datacenter utilization, cloud providers offer users an option to run workloads such as big data analytics on the underutilized resources, in the form of cheap yet revocable transient servers (e.g., EC2 spot instances, GCE preemptible instances). Though at highly reduced prices, deploying big data analytics on the unstable cloud transient servers can severely degrade the job performance due to instance revocations. To tackle this issue, this paper proposes iSpot, a cost-effective transient server provisioning framework for achieving predictable performance in the cloud, by focusing on Spark as a representative Directed Acyclic Graph (DAG)-style big data analytics workload. It first identifies the stable cloud transient servers during the job execution by devising an accurate Long Short-Term Memory (LSTM)-based price prediction method. Leveraging automatic job profiling and the acquired DAG information of stages, we further build an analytical performance model and present a lightweight critical data checkpointing mechanism for Spark, to enable our design of iSpot provisioning strategy for guaranteeing the job performance on stable transient servers. Extensive prototype experiments on both EC2 spot instances and GCE preemptible instances demonstrate that, iSpot is able to guarantee the performance of big data analytics running on cloud transient servers while reducing the job budget by up to 83.8 percent in comparison to the state-of-the-art server provisioning strategies, yet with acceptable runtime overhead.

Portfolio-driven Resource Management for Transient Cloud Servers

Cloud providers have begun to offer their surplus capacity in the form of low-cost transient servers, which can be revoked unilaterally at any time. While the low cost of transient servers makes them attractive for a wide range of applications, such as data processing and scientific computing, failures due to server revocation can severely degrade application performance. Since different transient server types offer different cost and availability tradeoffs, we present the notion of server portfolios that is based on financial portfolio modeling. Server portfolios enable construction of an "optimal" mix of severs to meet an application's sensitivity to cost and revocation risk. We implement model-driven portfolios in a system called ExoSphere, and show how diverse applications can use portfolios and application-specific policies to gracefully handle transient servers. We show that ExoSphere enables widely-used parallel applications such as Spark, MPI, and BOINC to be made transiency-aware with modest effort. Our experiments show that allowing the applications to use suitable transiency-aware policies, ExoSphere is able to achieve 80% cost savings when compared to on-demand servers and greatly reduces revocation risk compared to existing approaches.

Portfolio-driven Resource Management for Transient Cloud Servers

Cloud providers have begun to offer their surplus capacity in the form of low-cost transient servers, which can be revoked unilaterally at any time. While the low cost of transient servers makes them attractive for a wide range of applications, such as data processing and scientific computing, failures due to server revocation can severely degrade application performance. Since different transient server types offer different cost and availability tradeoffs, we present the notion of server portfolios that is based on financial portfolio modeling. Server portfolios enable construction of an "optimal" mix of severs to meet an application's sensitivity to cost and revocation risk. We implement model-driven portfolios in a system called ExoSphere, and show how diverse applications can use portfolios and application-specific policies to gracefully handle transient servers. We show that ExoSphere enables widely-used parallel applications such as Spark, MPI, and BOINC to be made transiency-aware with modest effort. Our experiments show that allowing the applications to use suitable transiency-aware policies, ExoSphere is able to achieve 80% cost savings when compared to on-demand servers and greatly reduces revocation risk compared to existing approaches.

Web-Scale Datacenters

This issue of Internet Computing surveys issues surrounding Web-scale datacenters, particularly in the areas of cloud provisioning as well as networking optimization and configuration. They include workload isolation, recovery from transient server availability, network configuration, virtual networking, and content distribution.

Determination of the Lumped-Capacitance Parameters of Air-Cooled Servers Through Air Temperature Measurements

Computer servers can be represented by lumped thermal capacitances for the purpose of simulating server and data center transient thermal response to changes in operating conditions or equipment failures. Two parameters are needed to characterize the transient behavior of a lumped-capacitance server: its thermal capacitance and its thermal conductance, heat transfer effectiveness, or time constant. To avoid the laborious task of obtaining these parameters from measurements or estimations of the thermal characteristics of internal components of the server, a method is proposed to derive these parameters from external measurements that can be easily obtained without performing an “autopsy” on the server. In this paper, we present the mathematical formulation underlying the proposed method and describe how the parameters are to be obtained from external air-temperature measurements using the mathematical model. We then present validation test cases using experimental data from server shut-down and inlet-temperature ramp tests. The experimentally obtained parameters are implemented into a computational fluid dynamics (CFD) case study of server shutdown in which the transient server exit air temperature is computed from the lumped-capacitance parameters via a user-defined function. The results thus obtained are in excellent agreement with the experimental data.