Data Transfer Performance Research Articles

The Implication of Different Transmission Protocols for Vehicular Networks using NS-2

A vehicular network, categorized as an ad-hoc network, operates without fixed base station infrastructure or centralized management. In this setup, each network entity can serve as a router or a host, facilitating communication via multiple links. Unlike a mobile ad-hoc network, a vehicular network allows nodes to freely join or leave the network. For data transmission, various network layers are involved in delivering packets, with TCP and UDP protocols commonly employed. However, prior research has highlighted the limitations of Vehicular Ad-Hoc Networks (VANETs), particularly due to the surge in vehicular numbers on roads. This situation can trigger issues such as network congestion, leading to increased packet delivery delays and potential packet drops, ultimately compromising network performance. Moreover, when vehicles travel at high speeds on highways, quick message transmission is crucial, often without waiting for acknowledgments from both the transmitter and receiver. To address these challenges, some of the benefits of the User Datagram Protocol (UDP) can be integrated into vehicular network technologies. Consequently, this study aims to assess the data transfer performance between Transmission Control Protocol (TCP) and UDP, focusing on metrics like packet delivery ratio, packet loss, and packet drop, while varying vehicle density. The research location was Persiaran Permai, Seksyen 7, Shah Alam, Malaysia, and simulation was conducted using tools such as JOSM, MOVE, and NS-2. The findings reveal noteworthy differences: TCP exhibits a superior average packet delivery ratio (99.44% vs. 83.56%) and lower packet loss (0.56% vs. 16.45%) compared to UDP. Conversely, UDP demonstrates better performance in terms of average packet drop (0 compared to 6.75) when compared to TCP.

Improved Accurate Localization using PSO and the Weighted Dijkstra Algorithm in Software Defined Wireless Sensor Networks

Wireless Sensor Networks (WSNs) are crucial in various fields, including monitoring the environment, surveillance, and healthcare. They rely on localization services for accurate data transfer and optimal network performance. Traditional WSN techniques struggle to adapt to dynamic environmental changes beyond the intended task scope. A synergy between Software-Defined Networking (SDN) and WSN has been suggested to address this issue. This research paper presents proposed approach for machine learning-based localization in Software Defined Wireless Sensor Networks (SDWSNs) using Particle Swarm Optimization (PSO) technique and the Weighted Dijkstra algorithm. PSO technique is used for clustering, the weighted Dijkstra algorithm (WDA) for finding the shortest path and sending data packets, and machine learning algorithms like AdaBoost and Naïve Bayes for data classification. The effectiveness of the proposed approach is measured using energy consumption, throughput, network lifespan, and packet delivery ratio, outperforming existing models like OEERP, LEACH, DRINA, and BCDCA. The machine learning algorithms' performance is also evaluated, with Naïve Bayes achieving the highest accuracy of 78.24% and AdaBoost 98.90%.

Computer Hardware and Network Data Transmission based on Internet of Things Communication Technology

In order to meet the requirements of computer hardware and network data transmission security, a research based on Internet of Things communication technology is proposed. The main content of this research is the research based on the communication technology of the Internet of Things, through the description of the communication protocol of the Internet of Things, the system hardware design and implementation methods are used, and finally the research method based on the communication technology of the Internet of Things is constructed through experiments and analysis. The core technologies of 5G connectivity are being used to construct the IOT. As a result, the IOT might gain momentum. The experimental findings demonstrate that the delays are all within 200 Ms. When the message size is short (within 1KB), the transmission of diverse hardware is average, and the transmission quality standards of QoS1 are fulfilled. The transmission quality standards of QoS1 can match the communication reliability and security needs of the Internet of Things. This article evaluates the performance of data transfers with lengths of 20 byte, 30 byte, 50 byte, and 70 byte, respectively. This paper evaluates the efficiency of Wi-Fi access configuration by sending data packets of varying sizes i.e., 10 bytes, 30 bytes, 50 bytes, and 70 bytes over a distribution network. The results show that, on average, the network takes 0.6692s, 1.3546s, 2.8600s, and 4.7319s to deliver each packet, with success rates of 100% in each case. The system's increased network distribution efficiency is observed from the experimentation. The research based on the Internet of Things communication technology can meet the needs of computer hardware and network data transmission security.

Opportunities of IoT in Fog Computing for High Fault Tolerance and Sustainable Energy Optimization

Today, the importance of enhanced quality of service and energy optimization has promoted research into sensor applications such as pervasive health monitoring, distributed computing, etc. In general, the resulting sensor data are stored on the cloud server for future processing. For this purpose, recently, the use of fog computing from a real-world perspective has emerged, utilizing end-user nodes and neighboring edge devices to perform computation and communication. This paper aims to develop a quality-of-service-based energy optimization (QoS-EO) scheme for the wireless sensor environments deployed in fog computing. The fog nodes deployed in specific geographical areas cover the sensor activity performed in those areas. The logical situation of the entire system is informed by the fog nodes, as portrayed. The implemented techniques enable services in a fog-collaborated WSN environment. Thus, the proposed scheme performs quality-of-service placement and optimizes the network energy. The results show a maximum turnaround time of 8 ms, a minimum turnaround time of 1 ms, and an average turnaround time of 3 ms. The costs that were calculated indicate that as the number of iterations increases, the path cost value decreases, demonstrating the efficacy of the proposed technique. The CPU execution delay was reduced to a minimum of 0.06 s. In comparison, the proposed QoS-EO scheme has a lower network usage of 611,643.3 and a lower execution cost of 83,142.2. Thus, the results show the best cost estimation, reliability, and performance of data transfer in a short time, showing a high level of network availability, throughput, and performance guarantee.

Smart Data Placement Using Storage-as-a-Service Model for Big Data Pipelines.

Big data pipelines are developed to process data characterized by one or more of the three big data features, commonly known as the three Vs (volume, velocity, and variety), through a series of steps (e.g., extract, transform, and move), making the ground work for the use of advanced analytics and ML/AI techniques. Computing continuum (i.e., cloud/fog/edge) allows access to virtually infinite amount of resources, where data pipelines could be executed at scale; however, the implementation of data pipelines on the continuum is a complex task that needs to take computing resources, data transmission channels, triggers, data transfer methods, integration of message queues, etc., into account. The task becomes even more challenging when data storage is considered as part of the data pipelines. Local storage is expensive, hard to maintain, and comes with several challenges (e.g., data availability, data security, and backup). The use of cloud storage, i.e., storage-as-a-service (StaaS), instead of local storage has the potential of providing more flexibility in terms of scalability, fault tolerance, and availability. In this article, we propose a generic approach to integrate StaaS with data pipelines, i.e., computation on an on-premise server or on a specific cloud, but integration with StaaS, and develop a ranking method for available storage options based on five key parameters: cost, proximity, network performance, server-side encryption, and user weights/preferences. The evaluation carried out demonstrates the effectiveness of the proposed approach in terms of data transfer performance, utility of the individual parameters, and feasibility of dynamic selection of a storage option based on four primary user scenarios.

A Novel Stochastic Blockchain-Based Energy Management in Smart Cities Using V2S and V2G

This paper proposes a secure management framework for optimal scheduling and operation of energy systems in smart cities, which will bring balance in modern power systems. The expanded infiltration of energy systems along with stochastic performance and high data transfer rate can bring new challenges that may severely affect precise energy management. This research work investigates the optimal and secure operation of energy systems in the presence of smart transportation systems in a correlated energy system. To achieve the maximum efficiency from the transportation system, the ideas of vehicle-to-subway (V2S) and vehicle-to-grid (V2G) are utilized to develop a bilateral power flow strategy (charging/discharging or idle mode) to provide the chief charging requirements of energy systems. In this regard, the proposed model suggests a novel stochastic architecture based on unscented transformation (UT) to handle the operational uncertainties of the transportation systems when considering the correlated power generation of wind turbines (WTs). Moreover, blockchain technology is deployed to guarantee the security of data transfer within the smart city. Hence, there is a secure collaboration within all sub-systems in the smart cities, which has triggered the integrated management in this structure. The simulation results are discussed on a typical smart city test system to show the quality of the proposed model.

Reliable Wide-Area Data Transfers for Streaming Workflows

Many large science projects rely on remote clusters for (near) real-time data processing, thus they demand reliable wide-area data transfer performance for smooth end-to-end workflow executions. However, data transfers are often exposed to performance variations due to the changing network (e.g., background traffic) and dataset (e.g., average file size) conditions, necessitating adaptive solutions to meet stringent performance requirements of delay-sensitive streaming workflows. In this article, we propose <i>FStream++</i> to provide reliable transfer performance for large streaming science applications by dynamically adjusting transfer settings to adapt to changing transfer conditions. <i>FStream++</i> combines three optimization methods as <i>dynamic tuning</i> , <i>online profiling</i> , and <i>historical analysis</i> to swiftly and accurately discover optimal transfer settings that can meet workflow requirements. Dynamic tuning uses a heuristic model to predict the values of transfer parameters based on dataset characteristics and network settings. Since heuristic models fall short to incorporate many important factors such as I/O throughput and resource interference, we complement it with online profiling to execute a real-time search for a subset of transfer settings. Finally, historical analysis takes advantage of the long-running nature of streaming workflows by storing and analyzing previous performance observations to shorten the execution time of online profiling. We evaluate the performance of <i>FStream++</i> by transferring several synthetic and real-world workloads in high-performance production networks and show that it offers up to <inline-formula><tex-math notation="LaTeX">$3.6x$</tex-math></inline-formula> performance improvement over legacy transfer applications and up to 24% over our previous work <i>FStream</i> .

Moving small files in a networked environment

Globally distributed computing infrastructures, such as clouds and supercomputers, are currently used to manage data that is generated with an unprecedented speed from a variety of resources. Coping with this trend, the volume of data exchanged across distant sites increases substantially. To accelerate data transfer, high-speed networks are provided to connect remote sites. Most existing data movement solutions are optimized for moving large files. However, it is still challenging to transfer a large number of small files across networks. This disadvantage not only lowers data transfer performance, but also decreases overall system utilization. We identify that moving small files is mainly constrained by degraded file system throughput, not just network performance as might be suspected. We have built a data transfer pipeline model to analyze the impact of small network I/O and storage I/O on data movement. Extending one of the widely used open source data movement solutions, GridFTP, we demonstrate several appropriate engineering approaches that mitigate the bottleneck and increase data transfer efficiency. We show optimizations that improve data transfer performance more than 5 times. In comparison to existing solutions, our approaches can save a significant amount of system resources for moving lots of small files.

A method for efficient radio astronomical data gridding on multi-core vector processor

Gridding is the performance-critical step in the data reduction pipeline for radio astronomy research, allowing astronomers to create the correct sky images for further analysis. Like the 2D stencil computation, gridding iteratively updates the output cells by convolution, where the value at each output cell in the space is computed as a weighted sum of neighboring point values. Existing state-of-the-art works have achieved performance improvement of gridding by using multi-core CPUs and GPUs in real-world applications, and their study proved that gridding is a type of scientific computation with high-density computing characteristics. However, low computational performance or high power consumption becomes the main limitation for their processing of large-scale astronomical data. The high-density computing feature of gridding provides opportunities to accelerate it on the multi-core vector processor with vector-SIMD architectures. However, existing works’ (such as those implemented on CPUs or GPUs) task parallelization and data transfer strategies are inefficient to perform gridding directly on the vector processor without any dedicated mapping algorithm.M-DSP is a multi-core vector processor with vector-SIMD architectures designed for the next-generation exascale supercomputer, delivering high performance with ultra-low power consumption. In this paper, we present, for the first time, a novel method to achieve efficient gridding on the M-DSP. Specifically, we propose a gridding workflow designed for the vector-SIMD architectures and present a vectorized version of the gridding convolution algorithm to fully exploit the computational power of the M-DSP. In addition, centering on the processor architectures, we propose task-based parallelization strategies for block and line computing as well as different data loading strategies to achieve high parallel performance and high data transfer efficiency. Experimental results show that our work on M-DSP exhibits very competitive performance compared to other methods running on CPUs or GPUs. This demonstrates the efficiency of our method and the fact that the vector-SIMD architecture is beneficial for scientific computing with ”high density” characteristics, which can exploit its wide vector core and achieve higher performance than its competitors.

How to Optimize Connection Between PACS and Clinical Data Warehouse: A Web Service Approach Based on Full Metadata Integration.

Clinical image data analysis is an active area of research. Integrating such data in a Clinical Data Warehouse (CDW) implies to unlock the PACS and RIS and to address interoperability and semantics issues. Based on specific functional and technical requirements, our goal was to propose a web service (I4DW) that allows users to query and access pixel data from a CDW by fully integrating and indexing imaging metadata. Here, we present the technical implementation of this workflow as well as the evaluation we carried out using a prostate cancer cohort use case. The query mechanism relies on a Dicom metadata hierarchy dynamically generated during the ETL Process. We evaluated the Dicom data transfer performance of I4DW, and found mean retrieval times of 5.94 seconds and 0.9 seconds to retrieve a complete DICOM series from the PACS and all metadata of a series. We could retrieve all patients and imaging tests of the prostate cancer cohort with a precision of 0.95 and a recall of 1. By leveraging the CMOVE method, our approach based on the Dicom protocol is scalable and domain-neutral. Future improvement will focus on performance optimization and de identification.

TPBF: Two-Phase Bloom-Filter-Based End-to-End Data Integrity Verification Framework for Object-Based Big Data Transfer Systems

Computational science simulations produce huge volumes of data for scientific research organizations. Often, this data is shared by data centers distributed geographically for storage and analysis. Data corruption in the end-to-end route of data transmission is one of the major challenges in distributing the data geographically. End-to-end integrity verification is therefore critical for transmitting such data across data centers effectively. Although several data integrity techniques currently exist, most have a significant negative influence on the data transmission rate as well as the storage overhead. Therefore, existing data integrity techniques are not viable solutions in high performance computing environments where it is very common to transfer huge volumes of data across data centers. In this study, we propose a two-phase Bloom-filter-based end-to-end data integrity verification framework for object-based big data transfer systems. The proposed solution effectively handles data integrity errors by reducing the memory and storage overhead and minimizing the impact on the overall data transmission rate. We investigated the memory, storage, and data transfer rate overheads of the proposed data integrity verification framework on the overall data transfer performance. The experimental findings showed that the suggested framework had 5% and 10% overhead on the total data transmission rate and on the total memory usage, respectively. However, we observed significant savings in terms of storage requirements, when compared with state-of-the-art solutions.

Awrapper algorithm improve data transfer and synchornization performance of RSYNC

Data transfer involving large files between two geographically distributed distant points and the problem of synchronization in a distributed environment still remain as challenging tasks. The synchronization problem exists in several data transfer applications such as web caching network, data mirroring on the web, network storage, searching engine on the web, and some online data mining systems. In this paper, a wrapper algorithm is proposed to improve the performance of the well-known RSYNC algorithm for data transfer and to improve the synchronization of deep hierarchical folders on a wide area network. The proposed algorithm can be used not only for RSYNC but also for other applications involving data transfer and synchronization. The experimental results show that the proposed framework and techniques achieve significant improvements over RSYNC with regard to data transfer speed and synchronization time.

Enterprise Data Sharing with Privacy-Preserved Based on Hyperledger Fabric Blockchain in IIOT's Application.

Internet of Things (IoT) technology is now widely used in energy, healthcare, services, transportation, and other fields. With the increase in industrial equipment (e.g., smart mobile terminals, sensors, and other embedded devices) in the Internet of Things and the advent of Industry 4.0, there has been an explosion of data generated that is characterized by a high volume but small size. How to manage and protect sensitive private data in data sharing has become an urgent issue for enterprises. Traditional data sharing and storage relies on trusted third-party platforms or distributed cloud storage, but these approaches run the risk of single-node failure, and third parties and cloud storage providers can be vulnerable to attacks that can lead to data theft. To solve these problems, this paper proposes a Hyperledger Fabric blockchain-based secure data transfer scheme for enterprises in the Industrial Internet of Things (IIOT). We store raw data in the IIoT in the InterPlanetary File System (IPFS) network after encryption and store the Keyword-index table we designed in Hyperledger Fabric blockchain, and enterprises share the data by querying the Keyword-index table. We use Fabric’s channel mechanism combined with our designed Chaincode to achieve privacy protection and efficient data transmission while using the Elliptic Curve Digital Signature Algorithm (ECDSA) to ensure data integrity. Finally, we performed security analysis and experiments on the proposed scheme, and the results show that overall the data transfer performance in the IPFS network is generally better than the traditional network, In the case of transferring 5 MB file size data, the transmission speed and latency of IPFS are 19.23 mb/s and 0.26 s, respectively, and the IPFS network is almost 4 times faster than the TCP/IP network while taking only a quarter of the time, which is more advantageous when transferring small files, such as data in the IIOT. In addition, our scheme outperforms the blockchain systems mainly used today in terms of both throughput, latency, and system overhead. The average throughput of our solution can reach 110 tps (transactions are executed per second), and the minimum throughput in experimental tests can reach 101 tps.

Investigating data transfer performance bottlenecks and optimizations in modern programmable OpenGL

Investigating data transfer performance bottlenecks and optimizations in modern programmable OpenGL

The NetSage measurement and analysis framework in practice

Data sharing is required for research collaborations, but effective data transfer performance continues to be difficult to achieve. The NetSage Measurement and Analysis Framework can assist in understanding research data movement. It collects a broad set of monitoring data and builds performance Dashboards to visualize the data. Each Dashboard is specifically designed to address a well-defined analysis need of the stakeholders. This paper describes the design methodology, the resulting architecture, the development approach and lessons learned, and a set of discoveries that NetSage Dashboards made possible.

Cache management for large data transfers and multipath forwarding strategies in Named Data Networking

Named Data Networking (NDN) is a promising approach to provide fast in-network access to compact muon solenoid (CMS) datasets. It proposes a content-centric rather than a host-centric approach to data retrieval. Data packets with unique and immutable names are retrieved from a content store (CS) using Interest packets. The current NDN architecture relies on forwarding strategies that are only dependent upon on-path caching. Such a design does not take advantage of the cached content available on the adjacent off-path routers in the network, thus reducing data transfer efficiency. In this work, we propose a software-defined, storage-aware routing mechanism that leverages NDN router cache-states, software defined networking (SDN) and multipath forwarding strategies to improve the efficiency of very large data transfers. First, we propose a novel distributed multipath (D-MP) forwarding strategy and enhancements to the NDN Interest forwarding pipeline. In addition, we develop a centralized SDN-enabled control for the multipath forwarding strategy (S-MP), which leverages the global knowledge of NDN network states that distributes Interests efficiently. We perform extensive evaluations of our proposed methods on an at-scale wide area network (WAN) testbed spanning six geographically separated sites. Our proposed solutions easily outperform the existing NDN forwarding strategies. The D-MP strategy results in performance gains ranging between 10.4x to 12.5x over the default NDN implementation without in-network caching, and 12.2x to 18.4x with in-network caching enabled. For S-MP strategy, we demonstrate a performance improvement of 10.6x to 12.6x, and 12.9x to 18.5x, with in-network caching disabled and enabled, respectively. Further, we also present a comprehensive analysis of NDN cache management for large data transfers and propose a novel prefetching mechanism to improve data transfer performance. Due to the inherent capacity limitations of the NDN router caches, we use SDN to provide an intelligent and efficient solution for data distribution and routing across multiple NDN router caches. We demonstrate how software-defined control can be used to partition and distribute large CMS files based on NDN router cache-state knowledge. Further, SDN control will also configure the router forwarding strategy to retrieve CMS data from the network. Our proposed solution demonstrates that the CMS datasets can be retrieved 28%–38% faster from the NDN routers’ caches than existing NDN approaches. Lastly, we develop a prefetching mechanism to improve the transfer performance of files that are not available in the router’s cache.

Skyrmion ratchet in funnel geometries

Using a particle-based model, we simulate the behavior of a skyrmion under the influence of asymmetric funnel geometries and ac driving at zero temperature. We specifically investigate possibilities for controlling the skyrmion motion by harnessing a ratchet effect. Our results show that as the amplitude of a unidirectional ac drive is increased, the skyrmion can be set into motion along either the easy or hard direction of the funnel depending on the ac driving direction. When the ac drive is parallel to the funnel axis, the skyrmion flows in the easy direction and its average velocity is quantized. In contrast, when the ac drive is perpendicular to the funnel axis, a Magnus-induced ratchet effect occurs, and the skyrmion moves along the hard direction with a constant average velocity. For biharmonic ac driving of equal amplitude along both the parallel and perpendicular directions, we observe a reentrant pinning phase where the skyrmion ratchet vanishes. For asymmetric biharmonic ac drives, the skyrmion exhibits a combination of effects and can move in either the easy or hard direction depending on the configuration of the ac drives. These results indicate that it is possible to achieve controlled skyrmion motion using funnel geometries, and we discuss ways in which this could be harnessed to perform data transfer operations.

Design and Experimental Validation of a Compact Low-Cost Weather Station for Solar Photovoltaic Applications

This paper presents a compact low-cost weather station specially dedicated to renewable energy applications based on solar photovoltaic (PV) technologies. The main objective of the weather station is to verify which technology of solar PV modules would be more suitable for the specific location where the weather station is installed. Therefore, the developed weather station includes three technologies of PV modules (polycrystalline, monocrystalline, and amorphous silicon), each one connected to a dedicated DC-DC power converter with a maximum power point control (MPPC) functionality, as well as a set of sensors (solar irradiance, temperature, humidity, wind speed, and wind direction) used to measure the local weather. The acquired data is processed and stored locally in the weather station and, when necessary, the user can download the data to an Android mobile device through a Bluetooth Low Energy (BLE) wireless network connection using the developed mobile app, where the transferred data is stored in a SQLite database and can be visualized in graphs. Throughout the paper, the design of the developed weather station and the associated technologies are described, as well as the details of the mobile app. The developed system comprising the weather station and the mobile app was validated through a set of experimental tests ranging from the data acquisition to its visualization, as well as the achieved wireless data transfer performance.

Compiler-directed scratchpad memory data transfer optimization for multithreaded applications on a heterogeneous many-core architecture

The heterogeneous many-core architecture plays an important role in the fields of high-performance computing and scientific computing. It uses accelerator cores with on-chip memories to improve performance and reduce energy consumption. Scratchpad memory (SPM) is a kind of fast on-chip memory with lower energy consumption compared with a hardware cache. However, data transfer between SPM and off-chip memory can be managed only by a programmer or compiler. In this paper, we propose a compiler-directed multithreaded SPM data transfer model (MSDTM) to optimize the process of data transfer in a heterogeneous many-core architecture. We use compile-time analysis to classify data accesses, check dependences and determine the allocation of data transfer operations. We further present the data transfer performance model to derive the optimal granularity of data transfer and select the most profitable data transfer strategy. We implement the proposed MSDTM on the GCC complier and evaluate it on Sunway TaihuLight with selected test cases from benchmarks and scientific computing applications. The experimental result shows that the proposed MSDTM improves the application execution time by 5.49times and achieves an energy saving of 5.16times on average.

Exploratory analysis and performance prediction of big data transfer in High-performance Networks

Big data transfer in large-scale scientific and business applications is increasingly carried out over connections with guaranteed bandwidth provisioned in High-performance Networks (HPNs) via advance bandwidth reservation. Provisioning agents need to carefully schedule data transfer requests, compute network paths, and allocate appropriate bandwidths. Such reserved bandwidths, if not fully utilized, could be simply wasted due to the exclusive access during the approved time window, and cause extra overhead and complexity for resource management. This calls for accurate performance prediction to reserve bandwidths that match actual needs and avoid over-provisioning. We employ machine learning algorithms to predict big data transfer performance based on extensive performance measurements collected in the past several years from data transfer tests using different protocols and toolkits between various end sites on several real-life physical or emulated testbeds. We first analyze the performance patterns in response to a comprehensive list of parameters in end-host systems, network connections, and data transfer applications, which motivate the use of machine learning and also help us identify the effects of latent factors. We then propose threshold- and clustering-based methods to eliminate negative effects of latent factors in data preprocessing and build a robust performance predictor based on customized domain-oriented loss functions. The performance of the proposed methods is verified by extensive experiments using SVR and RFR as well as theoretical analysis of the general performance bound.