Applying SDN based data network on HPC Big Data Computing – Design, Implementation, and Evaluation

Abstract

Large scale storage data networks are difficult to conFigure and tricky to maintain [1][2][3]. As storage data volumes grow and the pace of change accelerates, it can be a struggle to keep up the integrity of a large scale storage network. Software Defined Networking (SDN) [5][6][7][8][10][11] provides a method to centrally conFigure and manage physical and virtual network devices such as routers, switches, and gateways in HPC datacenter. Current HPC computer cluster and data centers are using homogeneous data network technologies such as the Infiniband network (QDR, FDR, EDR, and HDR). Due to the connector, cable, and bandwidth backward compatibility issues, once an aged HPC computing system was retired, we have to demolish all established computer clusters, data network, and data storage. Eventually we lost all costly investment on the data network and storage. Those current deployment approaches do not provide a feasible and compatible growing path and cannot meet the need for future extreme scale HPC computing systems [4][12][13].