High-performance Data Transfer Research Articles

A method for efficient radio astronomical data gridding on multi-core vector processor

Gridding is the performance-critical step in the data reduction pipeline for radio astronomy research, allowing astronomers to create the correct sky images for further analysis. Like the 2D stencil computation, gridding iteratively updates the output cells by convolution, where the value at each output cell in the space is computed as a weighted sum of neighboring point values. Existing state-of-the-art works have achieved performance improvement of gridding by using multi-core CPUs and GPUs in real-world applications, and their study proved that gridding is a type of scientific computation with high-density computing characteristics. However, low computational performance or high power consumption becomes the main limitation for their processing of large-scale astronomical data. The high-density computing feature of gridding provides opportunities to accelerate it on the multi-core vector processor with vector-SIMD architectures. However, existing works’ (such as those implemented on CPUs or GPUs) task parallelization and data transfer strategies are inefficient to perform gridding directly on the vector processor without any dedicated mapping algorithm.M-DSP is a multi-core vector processor with vector-SIMD architectures designed for the next-generation exascale supercomputer, delivering high performance with ultra-low power consumption. In this paper, we present, for the first time, a novel method to achieve efficient gridding on the M-DSP. Specifically, we propose a gridding workflow designed for the vector-SIMD architectures and present a vectorized version of the gridding convolution algorithm to fully exploit the computational power of the M-DSP. In addition, centering on the processor architectures, we propose task-based parallelization strategies for block and line computing as well as different data loading strategies to achieve high parallel performance and high data transfer efficiency. Experimental results show that our work on M-DSP exhibits very competitive performance compared to other methods running on CPUs or GPUs. This demonstrates the efficiency of our method and the fact that the vector-SIMD architecture is beneficial for scientific computing with ”high density” characteristics, which can exploit its wide vector core and achieve higher performance than its competitors.

High-performance data transfer for full data replication between iter and the remote experimentation centre

A high-performance data transfer method has been developed for the Remote Experimentation Centre (REC) of ITER in Japan for the first time. The developed technology shows the technical feasibility to establish the REC with full data replication between ITER and REC for remote experiments. Test results showed that it achieved a data transfer rate of approximately 7.9 Giga-bits per second (Gbps) on a 10-Gbps network. The new double-layer storage structure can accelerate the storage read/write speed up to 2 GByte/s. Moreover, the Internet and a layer-2 virtual private network (L2VPN) comparison tests demonstrated that the latter is superior in both security and speed. This technology shows great potential for near real-time full data replication between ITER and REC, which may provide a new style of world-wide remote experimentation.

Cloud-Centric and Logically Isolated Virtual Network Environment Based on Software-Defined Wide Area Network

Recent development of distributed cloud environments requires advanced network infrastructure in order to facilitate network automation, virtualization, high performance data transfer, and secured access of end-to-end resources across regional boundaries. In order to meet these innovative cloud networking requirements, software-defined wide area network (SD-WAN) is primarily demanded to converge distributed cloud resources (e.g., virtual machines (VMs)) in a programmable and intelligent manner over distant networks. Therefore, this paper proposes a logically isolated networking scheme designed to integrate distributed cloud resources to dynamic and on-demand virtual networking over SD-WAN. The performance evaluation and experimental results of the proposed scheme indicate that virtual network convergence time is minimized in two different network models such as: (1) an operating OpenFlow-oriented SD-WAN infrastructure (KREONET-S) which is deployed on the advanced national research network in Korea, and (2) Mininet-based experimental and emulated networks.

High performance data transfer

The exponentially increasing need for high speed data transfer is driven by big data, and cloud computing together with the needs of data intensive science, High Performance Computing (HPC), defense, the oil and gas industry etc. We report on the Zettar ZX software. This has been developed since 2013 to meet these growing needs by providing high performance data transfer and encryption in a scalable, balanced, easy to deploy and use way while minimizing power and space utilization. In collaboration with several commercial vendors, Proofs of Concept (PoC) consisting of clusters have been put together using off-the- shelf components to test the ZX scalability and ability to balance services using multiple cores, and links. The PoCs are based on SSD flash storage that is managed by a parallel file system. Each cluster occupies 4 rack units. Using the PoCs, between clusters we have achieved almost 200Gbps memory to memory over two 100Gbps links, and 70Gbps parallel file to parallel file with encryption over a 5000 mile 100Gbps link.

MdtmFTP and its evaluation on ESNET SDN testbed

To address the high-performance challenges of data transfer in the big data era, we are developing and implementing mdtmFTP: a high-performance data transfer tool for big data. mdtmFTP has four salient features. First, it adopts an I/O centric architecture to execute data transfer tasks. Second, it more efficiently utilizes the underlying multicore platform through optimized thread scheduling. Third, it implements a large virtual file mechanism to address the lots-of-small-files (LOSF) problem. Finally, mdtmFTP integrates multiple optimization mechanisms, including–zero copy, asynchronous I/O, pipelining, batch processing, and pre-allocated buffer pools–to enhance performance. mdtmFTP has been extensively tested and evaluated within the ESNET 100G testbed. Evaluations show that mdtmFTP can achieve higher performance than existing data transfer tools, such as GridFTP, FDT, and BBCP.

Analysis of NUMA effects in modern multicore systems for the design of high-performance data transfer applications

To capitalize on multicore power, modern high-speed data transfer applications usually adopt multi-threaded design and aggregate multiple network interfaces. However, NUMA introduces another dimension of complexity to these applications. In this paper, we undertook comprehensive experiment on real systems to illustrate the importance of NUMA-awareness to applications with intensive memory accesses and network I/Os. Instead of simply attributing the NUMA effect to the physical layout, we provide an in-depth analysis of underlying interactions inside hardware devices. We profile the system performance by monitoring relevant hardware counters, and reveal how the NUMA penalty occurs during prefetch and cache synchronization processes. Consequently, we implement a thread mapping module in a bulk data transfer software, BBCP, as a practical example of enabling NUMA-awareness. The enhanced application is then evaluated on our high-performance testbed with storage area networks (SAN). Our experimental results show that the proposed NUMA optimizations can significantly improve BBCP’s performance in memory-based tests with various contention levels and realistic data transfers involving SAN-based storage.

Evaluating the transport layer of the ALFA framework for the Intel® Xeon Phi™ Coprocessor

The ALFA framework supports the software development of major High Energy Physics experiments. As part of our research effort to optimize the transport layer of ALFA, we focus on profiling its data transfer performance for inter-node communication on the Intel Xeon Phi Coprocessor. In this article we present the collected performance measurements with the related analysis of the results. The optimization opportunities that are discovered, help us to formulate the future plans of enabling high performance data transfer for ALFA on the Intel Xeon Phi architecture.

Design and testbed evaluation of RDMA-based middleware for high-performance data transfer applications

Providing high-speed data transfer is vital to various data-intensive applications supported by data center networks. We design a middleware layer of high-speed communication based on Remote Direct Memory Access (RDMA) that serves as the common substrate to accelerate various data transfer tools, such as FTP, HTTP, file copy, sync and remote file I/O. This middleware offers better end-to-end bandwidth performance than the traditional TCP-based alternatives, while it hides the heterogeneity of the underlying high-speed architecture. This paper describes this middleware's function modules, including resource abstraction and task synchronization and scheduling, that maximize the parallelism and performance of RDMA operations. For networks without RDMA hardware acceleration, we integrate Linux kernel optimization techniques to reduce data copy and processing in the middleware. We provide a reference implementation of the popular file-transfer protocol over this RDMA-based middleware layer, called RFTP. Our experimental results show that our RFTP outperforms several TCP-based FTP tools, such as GridFTP, while it maintains very low CPU consumption on a variety of data center platforms. Furthermore, those results confirm that our RFTP tool achieves near line-speed performance in both LAN and WAN, and scales consistently from 10Gbps Ethernet to 40Gbps Ethernet and InfiniBand environments.

StorNet: Integrated Dynamic Storage and Network Resource Provisioning and Management for Automated Data Transfers

StorNet is a joint project of Brookhaven National Laboratory (BNL) and Lawrence Berkeley National Laboratory (LBNL) to research, design, and develop an integrated end-to-end resource provisioning and management framework for high-performance data transfers. The StorNet framework leverages heterogeneous network protocols and storage types in a federated computing environment to provide the capability of predictable, efficient delivery of high-bandwidth data transfers for data intensive applications. The framework incorporates functional modules to perform such data transfers through storage and network bandwidth co-scheduling, storage and network resource provisioning, and performance monitoring, and is based on LBNL's BeStMan/SRM, BNL's TeraPaths, and ESNet's OSCARS systems.

Service-Oriented Collaborative Framework for High-Performance Data Transfer in Grids

As the number of data-intensive applications increases in various domains, scientists need to save, retrieve, and analyze increasingly large datasets. The huge volume of data and the long latency of data transfer on the Internet make it very difficult to ensure high-performance access to Data Grids. Thus, data replication techniques have been widely adopted to solve the latency problem. In this paper, we propose an efficient data replication algorithm for multi-source data transfer, whereby a data replica can be assembled in parallel from multiple distributed data sources and adapted to the fluctuation of network bandwidths. The experimental results show that the proposed algorithm can obtain more aggregated bandwidth, reduce connection overheads, and achieve superior load balance. We also develop a Service-oriented Collaborative Framework that not only implements the proposed data replication algorithm in a real Grid system, but also provides a virtual data management environment that integrates distributed and heterogeneous storage systems and enables users to share their data with group members for collaborative work.

Moving huge scientific datasets over the Internet

Modern scientific experiments can generate hundreds of gigabytes to terabytes or even petabytes of data that may be maintained in large numbers of relatively small files. Frequently, these data must be disseminated to remote collaborators or computational centers for data analysis. Moving this dataset with high performance and strong robustness and providing a simple interface for users are challenging tasks. We present a data transfer framework comprising a high-performance data transfer library based on GridFTP, an extensible data scheduler with four data scheduling policies, and a GUI that allows users to transfer their dataset easily, reliably, and securely. This system incorporates automatic tuning mechanisms to select at runtime the number of concurrent threads to be used for transfers. Also included are restart mechanisms for handling client, network, and server failures. Experimental results indicate that our data transfer system can significantly improve data transfer performance and can recover well from failures. Copyright © 2011 John Wiley & Sons, Ltd.

High performance data transfer and monitoring for RHIC and USATLAS

Modern nuclear and high energy physics experiments yield large amounts of data and thus require efficient and high capacity storage and transfer. BNL, the hosting site for RHIC experiments and the US ATLAS Tier 1 Center, plays a pivotal role in transferring between other sites in the US and around the world for data distribution and processing. Each component in the infrastructure from data acquisition system to local analysis facility must be tuned, tested, monitored, and sometimes diagnosed to transfer such a massive volume of data, often over long distances. Maximal performance can be reached by performing a combination of hardware optimization, TCP tuning, transfer application tuning and network architecture optimization.

A Dynamic Performance-Based Flow Control Method for High-Speed Data Transfer

New types of specialized network applications are being created that need to be able to transmit large amounts of data across dedicated network links. TCP fails to be a suitable method of bulk data transfer in many of these applications, giving rise to new classes of protocols designed to circumvent TCP's shortcomings. It is typical in these high-performance applications, however, that the system hardware is simply incapable of saturating the bandwidths supported by the network infrastructure. When the bottleneck for data transfer occurs in the system itself and not in the network, it is critical that the protocol scales gracefully to prevent buffer overflow and packet loss. It is therefore necessary to build a high-speed protocol adaptive to the performance of each system by including a dynamic performance-based flow control. This paper develops such a protocol, performance adaptive UDP (henceforth PA-UDP), which aims to dynamically and autonomously maximize performance under different systems. A mathematical model and related algorithms are proposed to describe the theoretical basis behind effective buffer and CPU management. A novel delay-based rate-throttling model is also demonstrated to be very accurate under diverse system latencies. Based on these models, we implemented a prototype under Linux, and the experimental results demonstrate that PA-UDP outperforms other existing high-speed protocols on commodity hardware in terms of throughput, packet loss, and CPU utilization. PA-UDP is efficient not only for high-speed research networks, but also for reliable high-performance bulk data transfer over dedicated local area networks where congestion and fairness are typically not a concern.

A new revolution in enterprise storage architecture

Network attached storage (NAS) and storage area network (SAN) are the enterprise storage architectures that are commonly deployed in data centers. However, as storage demands escalate over time, none of these storage networks support aggressive concurrency and per server throughput requirements for the applications on highly scalable computing clusters. In recent times, efforts have been set to incorporate the strengths of existing storage architectures into a unique framework that accomplished the performance, scalability, manageability, and a high level of data security. The approach has led to the design of novel storage architecture that support next generation object based storage technology. One of the main design features in object based storage architecture is the separation of metadata from the data management. This is to avoid the metadata server (MDS) from becoming the performance bottleneck, as seen in NAS architecture. MDS does not contain any user data. It is essentially used as a global resource for locating objects, mitigating secure access to objects, and assisting objects management. Data are transferred directly between the application servers (also referred as clients) and the object based storage devices (OSD) during its read/write operations without any intervention from MDS. An MDS cluster allows enterprise storages to scale to a very large finite number of objects (possibly 1030) that can be managed simultaneously, and at the same time sustaining their access performance. Object based storage architecture allows application servers to access its storage devices in parallel to cultivate high performance data transfers.

Architecture Design of Computing Intensive SoCs

Most existing system-on-chip (SoC) architectures are for microprocessor-centric designs. They are not suitable for computing intensive SoCs, which have their own configurability, extendibility, performance, and data exchange characteristics. This paper analyzes these characteristics and gives design principles for computing intensive SoCs. Three architectures suitable for different situations are compared with selection criteria given. The architectural design of a high performance network security accelerator (HPNSA) is used to elaborate on the design techniques to fully exploit the performance potential of the architectures. A behavior-level simulation system is implemented with the C++ programming language to evaluate the HPNSA performance and to obtain the optimum system design parameters. Simulations show that this architecture provides high performance data transfer.

Performance enhancement of TCP in high-speed networks

GridFTP is a secure and reliable high-performance parallel data transfer protocol used for transferring massive amounts of widely distributed data. Currently it allows users to configure the number of parallel streams and socket buffer size. However, its tuning procedure for optimal combination is a time consuming task. The socket handlers and buffers are important system resources and must therefore be carefully managed. In this paper, we propose a scheme to achieve high throughput even with a smaller buffer size, and also derive a regression equation to predict the optimal combination of resources for a connection. To improve the performance, the TCP based on our scheme obtains higher throughput and spends less memory for the same throughput than the original TCP scheme. In addition, the regression equation is verified by comparing measured and predicted values, and we apply the equation to an actual experiment on the KOrea advanced REsearch Network (KOREN). The result demonstrates that the equation provides excellent predictions with only an 8% error boundary.

Grid-based Data Access to Nucleotide Sequence Database

The International Nucleotide Sequence Database Collaboration (INSDC) exchanges sequence data on a daily basis across its three member organizations in the USA, UK and Japan. This paper studies how this sequence database in MySQL can best take advantage of the increased transfer bandwidth of a Grid-optimized data communication protocol. Within the context of the UK Government Project Grid-oriented Storage (GOS) and the EC Project EuroAsiaGrid, GOS File System (GOS-FS) has been developed in our lab, which melds distributed file system technology with high performance data transfer techniques to meet the needs of WAN/Grid-based virtual organizations. A real-world test shows that the INSDC sequence database backing up operation, mysqldump, over the GOS-FS protocol beats those over the classic NFS protocol by 6 times over the link between Cambridge and Tokyo. Best of all, the multi-streamed GOS-FS protocol remains fully compatible with existing IP infrastructures.

Data management services and transfer scheme in ChinaGrid

ChinaGrid is a nationwide grid project supported by the Chinese government that aims at integrating massive heterogeneous resources distributed on the China Education and Research Network (CERNET). The ChinaGrid Supporting Platform (CGSP) is a grid middleware developed for ChinaGrid. It provides a series of system-level services for ChinaGrid, helps to build application portals and integrate grid resources, and supports the secondary development of grid services. Data Management Services (DMS) is a group of system-level services in CGSP for managing storage and data resources, supporting transparent data access, and guaranteeing high-performance data transfer on the grid. It consists of metadata management service, storage resource management service, replication management service, storage agent and transfer client. It offers the fundamental support for data access on ChinaGrid. In this paper, we introduce the design principle and implementation of DMS and its transfer scheme.

The File Mover: high-performance data transfer for the grid

The exploration in many scientific disciplines (e.g. High-Energy Physics, Climate Modeling, and Life Sciences) involves the production and the analysis of massive data collections, whose archival, retrieval, and analysis require the coordinated usage of high-capacity computing, network, and storage resources. To obtain satisfactory performance, these applications require the availability of a high-performance, reliable data transfer mechanisms, able to minimize the data transfer time that often dominates their execution time. In this paper we present the File Mover, an efficient data transfer system specifically tailored to the needs of data-intensive applications, that exploits the overlay networks paradigm to provide superior performance with respect to conventional file transfer systems. An extensive experimental evaluation, carried out by means of a proof-of-concept implementation of the File Mover for a variety of network scenarios, shows the ability of the File Mover to outperform alternative data transfer systems. Copyright © 2007 John Wiley & Sons, Ltd.

A basic R&D for an analysis framework distributed on wide area network

Present status is reported on several R&D issues related to world-wide analysis for forthcoming high-energy collider experiments. This study includes (1) high-density computer farm with effective configuration and management capability, (2) high throughput data storage mechanisms covering disk arrays and hierarchical storage architecture, and (3) high-performance massive data transfer techniques on large latency WAN. Understanding these basic technologies is necessary for the deployment of developing data grid computing.