Network capabilities of cloud services for a real time scientific application

Abstract

Dedicating high-end servers for executing scientific applications that run intermittently, such as severe weather detection or generalized weather forecasting, wastes resources. While the Infrastructure-as-a-Service (IaaS) model used by today's cloud platforms is well-suited for the bursty computational demands of these applications, it is unclear if the network capabilities of today's cloud platforms are sufficient. In this paper, we analyze the networking capabilities of multiple commercial (Amazon's EC2 and Rackspace) and research (GENICloud and ExoGENI cloud) platforms in the context of a Nowcasting application, a forecasting algorithm for highly accurate, near-term, e.g., 5-20 minutes, weather predictions. The application has both computational and network requirements. While it executes rarely, whenever severe weather approaches, it benefits from an IaaS model; However, since its results are time-critical, enough bandwidth must be available to transmit radar data to cloud platforms before it becomes stale. We conduct network capacity measurements between radar sites and cloud platforms throughout the country. Our results indicate that ExoGENI cloud performs the best for both serial and parallel data transfer with an average throughput of 110.22 Mbps and 17.2 Mbps, respectively. We also found that the cloud services perform better in the distributed data transfer case, where a subset of nodes transmit data in parallel to a cloud instance. Ultimately, we conclude that commercial and research clouds are capable of providing sufficient bandwidth for our real-time Nowcasting application.