Data Transfer Nodes Research Articles

AIDTN: Towards a Real-Time AI Optimized DTN System With NVMeoF

Large-scale data transport for data-intensive sciences is a complex multidimensional challenge. The challenge includes optimizing the end-to-end Big Data movement performance in real-time, supporting direct remote data access using NVMe over Fabrics (NVMeoF) and deploying to existing research platforms. AIDTN is the first effort to provide a unique AI system designed to incorporate NVMe over Fabrics (NVMeoF) and optimize coordination among multiple components supporting large-scale, multi-domain Wide Area Network (WAN) data-intensive science. AIDTN's research objective is to integrate next-generation storage architecture using NVMeoF, specialized network design using high-performance network appliances, Data Transfer Nodes (DTNs), catalysts in driving data transport, and a unique AI system explicitly designed for high-performance data movement challenges. AIDTN is the first system that uses network and system features to predict the end-to-end performance of high-performance data movement and further extends the model with NVMe-specific features for NVMeoF remote data access. As a result, AIDTN improves data movement performance by up to 284% while minimizing packet loss compared to other heuristics approaches. It also has a prediction error rate as low as 0.16 compared to AI models with the only network (error rate = 0.29) or network and system features (error rate = 0.19).

Integrated Waste Management System with IOT-Based Centralized Control towards a Smart Eco Campus-Telkom University

Telkom University is a private university in Indonesia that has implemented an eco-campus or green campus program. The Greenmetric Program is one part of Telkom University's strategic planning in realizing a campus that cares and has an environmental culture by managing the environment in a systematic and sustainable manner. One of the programs is to reduce waste production. Therefore, it is necessary to conduct research related to optimizing waste management by implementing Onsite Waste Treatment Technology-Integrated-Independent-Environmentally Friendly (TOSS-TMRL). TOSS-TMRL Management System with Centralized Control Based on the Internet of Things (IoT) Towards a Smart ECO Campus to be completed on the spot. This will reduce pollution, operational costs, and waste management investment and can even function as Living Labs for environmental-based courses. In addition, optimization is also one of the efforts to realize a smart eco campus towards the best-ranking Green Metric Telkom University. Wireless communication technology is very efficient, effective, and easy to use. One of the applications can be applied to determine the condition of environmental cleanliness in a campus area by integrating microcontroller technology and detection devices (sensors, QR codes, AR/imaging, and CCTV) with Wireless Sensor Networks (WSN) and IoT applications as tools to increase awareness of the importance of sorting waste so that the cleanliness of the campus area can be maintained. In this study, a system and network architecture design were carried out based on the number of sensor nodes, the number of recycle bins, the number of garbage transportation equipment, the number of buildings, the number of areas, and the ability of Data Transfer Nodes (DTN) to provide information to the server taken from the recycle bin of each room/building, business areas, SME facilities, canteens, and dormitories or local areas. With this system, it is hoped that there will be no accumulation of garbage and allow users/garbage managers to interact with the system through a web browser and android application.

Integration of Multi-Class Service Paradigm With Generic Trust Mechanism for Innovation, Customization and Adaptability in MANETs

Trust-based mechanisms are widely used in wireless networks of different kinds for providing security against attack. Trust mechanism provides security from various attacks using both detective and preventive manner. This paper presents a quality service paradigm that can be integrated with any underlying trust mechanism. The paradigm includes different flags corresponding to different services incorporated in various routing packets. The paradigm provides flexible customization and adaptability as per the demand of communicating nodes for effective data transfer. Various quality service classes are designed to formulate route as per the requirement to minimize the routing overhead and balancing of load among nodes. This paradigm power is the trust mechanism with proactive action for detection of malicious nodes in the network. The proposed paradigm is incorporated in an established trust mechanism and compared with standard version of that trust mechanism for packet delivery ratio and routing overhead.

Performance Evaluation of NVMe-over-TCP Using Journaling File Systems in International WAN

Distributing Big Data for science is pushing the capabilities of networks and computing systems. However, the fundamental concept of copying data from one machine to another has not been challenged in collaborative science. As recent storage system development uses modern fabrics to provide faster remote data access with lower overhead, traditional data movement using Data Transfer Nodes must cope with the paradigm shift from a store-and-forward model to streaming data with direct storage access over the networks. This study evaluates NVMe-over-TCP (NVMe-TCP) in a long-distance network using different file systems and configurations to characterize remote NVMe file system access performance in MAN and WAN data moving scenarios. We found that NVMe-TCP is more suitable for remote data read than remote data write over the networks, and using RAID0 can significantly improve performance in a long-distance network. Additionally, a fine-tuning file system can improve remote write performance in DTNs with a long-distance network.

Energy Optimization Algorithm for Path Selection in Wireless Body Sensor Networks

Recent advancement in technologies for wireless sensor networks has led to the emergence of wireless body sensor networks (WBSN). These networks are composed of various sensor nodes that are placed on the human body and have the ability to constantly detect, process and transmit any sensed vital signs of patients to physicians without confining the patients to their hospital beds or restraining their movement. In common with a good number of other sensor network applications, the sensors are constrained by not having sufficient energy for them to function, a situation which often leads to unexpected failures in the network. However, recent research has shown that the use of mobile nodes for data transfer can significantly reduce energy consumption of the network. Hence, in this paper, an energy efficient hybrid algorithm using particle swarm optimization (PSO) algorithm and teaching-learning-based optimization (TLBO) algorithm is presented. The proposed algorithm takes into consideration the residual energy and the distance of each node from the base station to select routes from the sensing nodes to the base station. The hybridization is performed by the incorporation of the teaching and learning factors of TLBO algorithm into the velocity equation of PSO algorithm in order to improve the convergence of the algorithm into global optimum. The performance of the new hybrid algorithm is compared to similar optimization algorithms. Extensive simulation results show the potential of the proposed algorithm to optimize the energy of wireless body sensor networks.

Particle swarm optimization-based energy efficient clustering protocol in wireless sensor network

The nodes in the wireless sensor network are furnished with restricted and irreplaceable battery power. The continuous sensing, computation, and communication drain out the energy of sensors very quickly. The optimal utilization of the sensor energy has always been a key issue for all the applications in the wireless sensor network. To manage the energy issue of nodes, various approaches were proposed in the past which focused on designing the proper energy management methods. The clustering of sensors is one of the most popular techniques used to manage the energy-related concerns of networks. In this paper, a particle swarm optimization-based energy efficient clustering protocol (PSO-EEC) is proposed to enhance the network lifetime and performance. The proposed protocol uses the particle swarm optimization technique to select the cluster head and relay nodes for the network. The cluster head is selected by employing the particle swarm optimization based fitness function which considers the energy ratio (initial energy and residual energy) of nodes, distance between nodes and cluster head, and node degree to appoint the most optimal node for the cluster head job. For the data transfer to base station, the proposed scheme uses the fitness value based on residual energy of cluster head and distance to base station parameters to nominate the relay nodes for the multi-hop data transfer to the base station. The performance of the proposed protocol is compared with the various existing approaches in terms of different performance parameters such as energy expenditure, network lifetime, and throughput to evaluate its effectiveness. The proposed scheme has improved the lifetime of the network by 238%, 136%, 106%, and 71% as compared to the existing MDCH-PSO, MCHEOR, MOPSO, and HSA-PSO techniques used in the simulation results for the comparison purpose. The stability period of the network in proposed scheme is approximately 396%, 321%, 246%, and 126% more than the existing MDCH-PSO, MCHEOR, MOPSO, and HSA-PSO protocol .

A Trust-Based Access Control Mechanism for Intra-Sensor Network Communication in Internet of Things

The Internet of Things (IoT) can integrate both the physical and the information world to provide applications and services. IoT networks permit users, applications and devices in various physical locations for easy and seamless communication with each other. Data are sometimes quite sensitive in nature based on whether they are from one single IoT subsystem or an aggregation of multiple systems. Securing data becomes crucial and data access will have to be controlled. The IoT cloud can pose several challenges to the scale of IoT systems that help in identifying all real challenges in the cloud services. The main components of the access control mechanisms are authentication and authorization. Trust can permit the device with a natural judging method for other devices and the way in which security can be handled along with access control. A trust relationship between two different devices will help in influencing the future behaviour of interactions. Trust-based systems have been proposed in the literature, but not much work has been done to decrease the computational time in encryption system which is critical for low powered sensor devices. This work proposed a novel authentication mechanism ”Trust-Level-Public Key Authentication” which is used to connect between the nodes for data transfer which is efficient. This work was supported by a system of remote key management in the cloud and used for aggregating encrypted data. The levels of authorization in the nodes were controlled remotely. The experimental results showed that average consumption of energy (J) with the trust system to have performed better by about 11.5%, 12.6%, 17.6%, 14.2%, 13.5% and by about 15.9% when compared to the average consumption of energy (J) without a trust system that is used for the number of nodes 250, 500, 750, 1000, 1250, and 1500, respectively.

Scalable design and algorithm for science DMZ by considering the nature of research traffic

This paper explores the nature of scientific research traffic on the Korea research environment open network. Based on these investigations, we propose a scalable design and algorithm for the science demilitarization zone (DMZ). The proposed design allows users to share a data-transfer node (DTN), which is essential but costly equipment in the Science DMZ. The proposed iterative greedy algorithm attempts to minimize the peak traffic of the shared DTNs. By considering state-of-the-art DTN and practical research traffic, the proposed design and algorithm achieve up to 79% capital expenditure (CAPEX) reduction from that of a reference design of the Science DMZ where a DTN is allocated per user. The proposed algorithm achieves a 5-order-of-magnitude reduction in computation time at the cost of acceptable CAPEX overheads compared to those of the minimum-CAPEX solutions.

EPLAODV: Energy Priority and Link Aware Ad-hoc On-demand Distance Vector Routing Protocol for Post-disaster Communication Networks

In Mobile Ad-hoc Networks (MANETs) the major issue between nodes for data transfer is link quality. The majority of the MANET routing protocols try to maintain the link quality between nodes but still need improvement. This paper focuses on link quality between nodes in an ad hoc network in emergency scenarios through the design of the Energy Priority and Link Aware Ad-hoc On Demand Distance Vector (EPLAODV) routing protocol. EPLAODV improves the link quality between nodes through SNR. For performance analysis, the NS2.35 network simulator was used. Simulation time and traffic load were the major simulation parameters. From the simulation results, it is observed that EPLAODV performs better than AODV.

Intelligent Agent-Based Assessment of a Resilient Multi-hop Routing Protocol for Dynamic WSN

Wireless sensor networks (WSN) constitute a current field of interest in which the major concerns are related to mobility, spatial distribution, connectivity and dynamic creation of networks between autonomous nodes for cooperative detection and data transfer in diverse areas (e.g., healthcare, environmental or industrial monitoring). To this end, the present work describes the theoretical and practical development of a communication protocol for WSNs based on Bluetooth. The interaction between mobile nodes is performed with a multi-hop scheme in response to traffic needs without requiring a scatternet formation procedure. The interest of this algorithm—based on the concept of routing vector—is that it was designed to withstand changes in the node distribution for high-mobility scenarios, thus allowing the implementation of a robust data routing in low-resource microcontrolled devices with no operability loss. As the main contribution, we present the hardware and software implementation of the communication protocol in real devices along several case studies. With this aim, a long-term experimentation in a dense scenario has been carried out through an intelligent agent-based approach to formally validate the protocol considering three different performance metrics: packet delivery ratio, feedback overhead and round-trip time.

Network in Belle II

Belle II has started the Phase 3 data taking with a fully equipped detector. The data flow at the maximum luminosity is expected to be 12PB of data/year and will be analysed by a cutting-edge computing infrastructure spread over 26 Countries. Several of the major computing centres for HEP in Europe, USA and Canada will store the second copy of RAW data. In this scenario, the international network infrastructure for research plays a key role in supporting and orchestrating all the activities of data analysis and replication. The large-scale network data challenge will also take advantage from LHCONE VRF service and the support of network experts of KEKCC, Belle II sites and NREN. The program of major upgrade in 2019 empowered the connection among Japan, Europe and USA over a 100Gb geographic ring. In this work, we summarize the network requirements needed to accomplish all the tasks provided by the Belle II computing model. We also highlight the status of the major network links that support and advance Belle II. Lastly, we present the results of the last Network Data Challenge campaign performed between KEK and the main RAW data centres with the additional usage of the Data Transfer Node service provided by GÉANT.

Game theory-based Routing for Wireless Sensor Networks: A Comparative Survey

Wireless sensor networks (WSNs) have become an important and promising technology owing to their wide range of applications in disaster response, battle field surveillance, wildfire monitoring, radioactivity monitoring, etc. In WSNs, routing plays a significant role in delivery latency, energy consumption, and packet delivery ratio. Furthermore, as these applications are used in critical operations with limited irreplaceable batteries, routing protocols are required to be flawless as well as energy efficient. The dynamic environment also requires intelligent and adaptive routing. Game theory is widely used for designing routing protocols in WSNs to achieve not only reduced energy consumption but also increased packet delivery ratio. The core features of efficiently designed game theory-based routing protocols include optimal cluster head selection in hierarchical routing, energy-efficient and delay-aware route discovery, fault-tolerant data delivery, and coalition forming and grouping among nodes for stringent data transfer. In this paper, different routing protocols based on various types of games are extensively reviewed, which have been reported so far for improving energy consumption, delay, route establishment time, packet delivery ratio, and network lifetime. The different game theory-based routing protocols are qualitatively compared with each other in terms of major features, advantages, limitations, and key characteristics. For each protocol, possible applications and future improvements are summarized. Certain important open concerns and challenges are also discussed, along with future research directions.

Energy-Effective Cooperative and Reliable Delivery Routing Protocols for Underwater Wireless Sensor Networks

Underwater deployed sensors nodes are energy-constrained. Therefore, energy efficiency becomes crucial in underwater wireless sensor networks (U-WSNs). The adverse channel corrupts the packets and challenges their reliability. To handle these challenges, two routing schemes are introduced in this paper. They are effective energy and reliable delivery (EERD) and cooperative effective energy and reliable delivery (CoEERD). In EERD, the packets follow single-path routing and the best forwarder node is selected using a weight function such that packets are transferred via the reliable paths with low energy usage. Packet transfer via a single route in EERD has, however, compromised reliability as the undersea links bear harshness and unpredictability. Therefore, the CoEERD scheme adds cooperative routing to EERD, in which a relay node is introduced between a source–destination pair. The destination requests the relay when the packets it gets from the source are corrupted beyond a threshold value. Selection of weight function is unique and considers many factors to ensure low energy usage with reliability while considering nodes for data transfer. This also helps in selecting a single relay node rather than many relays in the conventional cooperative routing model. Based on simulation results, the EERD and CoEERD protocols have improved performance in energy usage, reliable packet transfer and delay.

A Comprehensive Tutorial on Science DMZ

Science and engineering applications are now generating data at an unprecedented rate. From large facilities such as the Large Hadron Collider to portable DNA sequencing devices, these instruments can produce hundreds of terabytes in short periods of time. Researchers and other professionals rely on networks to transfer data between sensing locations, instruments, data storage devices, and computing systems. While general-purpose networks, also referred to as enterprise networks, are capable of transporting basic data, such as e-mails and Web content, they face numerous challenges when transferring terabyte- and petabyte-scale data. At best, transfers of science data on these networks may last days or even weeks. In response to this challenge, the Science Demilitarized Zone (Science DMZ) has been proposed. The Science DMZ is a network or a portion of a network designed to facilitate the transfer of big science data. The main elements of the Science DMZ include: 1) specialized end devices, referred to as data transfer nodes (DTNs), built for sending/receiving data at a high speed over wide area networks; 2) high-throughput, friction-free paths connecting DTNs, instruments, storage devices, and computing systems; 3) performance measurement devices to monitor end-to-end paths over multiple domains; and 4) security policies and enforcement mechanisms tailored for high-performance environments. Despite the increasingly important role of Science DMZs, the literature is still missing a guideline to provide researchers and other professionals with the knowledge to broaden the understanding and development of Science DMZs. This paper addresses this gap by presenting a comprehensive tutorial on Science DMZs. The tutorial reviews fundamental network concepts that have a large impact on Science DMZs, such as router architecture, TCP attributes, and operational security. Then, the tutorial delves into protocols and devices at different layers, from the physical cyberinfrastructure to application-layer tools and security appliances, that must be carefully considered for the optimal operation of Science DMZs. This paper also contrasts Science DMZs with general-purpose networks, and presents empirical results and use cases applicable to current and future Science DMZs.

AmoebaNet: An SDN-enabled network service for big data science

Data transfer is now an essential function for science discoveries, particularly within big data environments. To support data transfer for big data science, there is a need for high performance, scalable, end-to-end, and programmable networks that enable science applications to use the network most efficiently. The existing network paradigm that support big data science consists of three major components: terabit networks that provide high network bandwidths, Data Transfer Nodes (DTNs) and Science DMZ architecture that bypasses the performance hotspots in typical campus networks, and on-demand secure circuits/paths reservation systems, such as ESNet OSCARS and Internet2 AL2S, which provides automated, guaranteed bandwidth service in WAN. This network paradigm has proven to be very successful. However, to reach its full potentials, we claim that existing network paradigm for big data science must address three major problems: the last mile problem, the scalability problem, and the programmability problem. To address these problems, we proposed a solution called AmoebaNet. AmoebaNet applies Software Defined Networking (SDN) technology to provide “QoS-guaranteed” network services in campus or local area networks. AmoebaNet complements existing network paradigm for big data science: it allows application to program networks at run-time for optimum performance; and, in conjunction with WAN circuits/paths reservation system such as ESNet OSCARS and Internet2 AL2S; it solves the last mile problem and the scalability problem.

Toward a smart data transfer node

Scientific computing systems are becoming significantly more complex, with distributed teams and complex workflows spanning resources from telescopes and light sources to fast networks and Internet of Things sensor systems. In such settings, no single, centralized administrative team and software stack can coordinate and manage all resources used by a single application. Indeed, we have reached a critical limit in manageability using current human-in-the-loop techniques. We therefore argue that resources must begin to respond automatically, adapting and tuning their behavior in response to observed properties of scientific workflows. Over time, machine learning methods can be used to identify effective strategies for autonomic, goal-driven management behaviors that can be applied end-to-end across the scientific computing landscape. Using the data transfer nodes that are widely deployed in modern research networks as an example, we explore the architecture, methods, and algorithms needed for a smart data transfer node to support future scientific computing systems that self-tune and self-manage.

Enhancing Data Transfer Performance Utilizing a DTN between Cloud Service Providers

The rapid transfer of massive data in the cloud environment is required to prepare for unexpected situations like disaster recovery. With regard to this requirement, we propose a new approach to transferring cloud virtual machine images rapidly in the cloud environment utilizing dedicated Data Transfer Nodes (DTNs). The overall procedure is composed of local/remote copy processes and a DTN-to-DTN transfer process. These processes are coordinated and executed based on a fork system call in the proposed algorithm. In addition, we especially focus on the local copy process between a cloud controller and DTNs and improve data transfer performance through the well-tuned mount techniques in Network File System (NFS)-based connections. Several experiments have been performed considering the combination of synchronous/asynchronous modes and the network buffer size. We show the results of throughput in all the experiment cases and compare them. Consequently, the best throughput in write operations has been obtained in the case of an NFS server in a DTN and an NFS client in a cloud controller running entirely in the asynchronous mode.

Automatic Between-Pulse Analysis of DIII-D Experimental Data Performed Remotely on a Supercomputer at Argonne Leadership Computing Facility

For the first time, an automatically triggered, between-pulse fusion science analysis code was run on-demand at a remotely located supercomputer at Argonne Leadership Computing Facility (ALCF, Lemont, Illinois) in support of in-process experiments being performed at DIII-D (San Diego, California). This represents a new paradigm for combining geographically distant experimental and high-performance computing facilities to provide enhanced data analysis that is quickly available to researchers. Enhanced analysis improves the understanding of the current pulse, translating into a more efficient use of experimental resources and quality of the resultant science. The analysis code used here, called SURFMN, calculates the magnetic structure of the plasma using Fourier transform. Increasing the number of Fourier components provides a more accurate determination of the stochastic boundary layer near the plasma edge by better resolving magnetic islands, but requires 26 min to complete using local DIII-D resources, putting it well outside the useful time range for between-pulse analysis. These islands relate to confinement and edge-localized mode suppression, and may be controlled by adjusting coil currents for the next pulse. ALCF has ensured on-demand execution of SURFMN by providing a reserved queue, a specialized service that launches the code after receiving an automatic trigger, and network access from the worker nodes for data transfer. Runs are executed on 252 cores of ALCF’s Cooley cluster and the data are available locally at DIII-D within 3 min of triggering. The original SURFMN design limits additional improvements with more cores; however, our work shows a path forward where codes that benefit from thousands of processors can run between pulses.

Data transfer nodes and demonstration of 100-400 Gbps wide area throughput using the Caltech SDN testbed

We review and demonstrate the design of efficient data transfer nodes (DTNs), from the perspective of the highest throughput over both local and wide area networks, as well as the highest performance per unit cost. A careful system-level design is required for the hardware, firmware, OS and software components. Furthermore, additional tuning of these components, and the identification and elimination of any remaining bottlenecks is needed once the system is assembled and commissioned, in order to obtain optimal performance. For high throughput data transfers, specialized software is used to overcome the traditional limits in performance caused by the OS, file system, file structures used, etc. Concretely, we will discuss and present the latest results using Fast Data Transfer (FDT), developed by Caltech. We present and discuss the design choices for three generations of Caltech DTNs. Their transfer capabilities range from 40 Gbps to 400 Gbps. Disk throughput is still the biggest challenge in the current generation of available hardware. However, new NVME drives combined with RDMA and a new NVME network fabric are expected to improve the overall data-transfer throughput and simultaneously reduce the CPU load on the end nodes.

Site in a box: Improving the Tier 3 experience

The Pacific Research Platform is an initiative to interconnect Science DMZs between campuses across the West Coast of the United States over a 100 gbps network. The LHC @ UC is a proof of concept pilot project that focuses on interconnecting 6 University of California campuses. It is spearheaded by computing specialists from the UCSD Tier 2 Center in collaboration with the San Diego Supercomputer Center. A machine has been shipped to each campus extending the concept of the Data Transfer Node to a cluster in a box that is fully integrated into the local compute, storage, and networking infrastructure. The node contains a full HTCondor batch system, and also an XRootD proxy cache. User jobs routed to the DTN can run on 40 additional slots provided by the machine, and can also flock to a common GlideinWMS pilot pool, which sends jobs out to any of the participating UCs, as well as to Comet, the new supercomputer at SDSC. In addition, a common XRootD federation has been created to interconnect the UCs and give the ability to arbitrarily export data from the home university, to make it available wherever the jobs run. The UC level federation also statically redirects to either the ATLAS FAX or CMS AAA federation respectively to make globally published datasets available, depending on end user VO membership credentials. XRootD read operations from the federation transfer through the nearest DTN proxy cache located at the site where the jobs run. This reduces wide area network overhead for subsequent accesses, and improves overall read performance. Details on the technical implementation, challenges faced and overcome in setting up the infrastructure, and an analysis of usage patterns and system scalability will be presented.