Data Distribution Algorithm Research Articles

Research on sensor data optimization technology for thermal hydraulic experiment of nuclear reactor

The sensor signals collected by the nuclear reactor thermal hydraulic experimental system are mixed with complex noise information. The uncertainty of sensor data directly affects the analysis and evaluation effect of machine learning algorithms on the operating status of the experimental system. Traditional denoising methods have poor adaptability to different types of noisy data, are highly dependent on researchers and have high design cost. This paper proposes a data optimization technology based on a hybrid adaptive real-time denoising (HART) model, which can realize data distribution self-adaptation and algorithm hyperparameter self-optimization according to the characteristics of noisy source data. Through the joint denoising algorithm with Variational Mode Decomposition (VMD) algorithm as the core, the partition denoising of source data is realized. In addition, the practical application of the source data of thermal hydraulic experiments in the nuclear field and the verification of simulation and noise addition experiments have been completed. The results show that the denoising optimization model proposed in this paper has the characteristics of good adaptability, self-optimization and real-time processing for different sensor signals. It can effectively remove noise information, restrain data uncertainty, and provide high-quality data source for experimental data analysis based on machine learning. At the same time, the optimization technology can be further applied to the optimization of sensor data of nuclear power plants (NPP).

Controller node driven hop count based data distribution algorithm in ring connected binary tree network-on-chip for parallel processing

Controller node driven hop count based data distribution algorithm in ring connected binary tree network-on-chip for parallel processing

An Optimized AMS Based Cloud Downloading Service with Advanced Caching and Intelligent Data Distribution Mechanism

The popularity of peer-to-peer video content downloading has surged due to diverse content availability and convenient sharing among users. However, scaling systems to accommodate the growing number of users and content items poses a challenge. This research aims to optimize video content downloading in peer-to-peer systems. The objective is to improve performance by developing advanced caching mechanisms, an intelligent data distribution algorithm, and efficient bandwidth resource management. The proposed approach involves implementing innovative caching mechanisms that store frequently accessed content closer to users, reducing download time. An intelligent data distribution algorithm minimizes bottlenecks and maximizes download speeds. Efficient bandwidth resource management ensures fair allocation. Results demonstrate significant enhancements in download time and overall system performance, leading to improved user experience. This research addresses the need for an optimized video content downloading system to handle increasing user and content volumes. The findings hold the potential to enhance user experiences, facilitate seamless video sharing, and advance peer-to-peer video content downloading.

Data Distribution for Heterogeneous Storage Systems

The exponential growth of data in many science and engineering domains poses significant challenges to storage systems. Data distribution is a critical component in large-scale distributed storage systems and plays a vital role in placing petabytes of data and beyond, among tens to hundreds of thousands of storage devices. Meantime, heterogeneous storage systems, such as those having devices with hard disk drives (HDDs) and storage class memories (SCMs), have become increasingly popular for massive data storage due to their distinct and complement characteristics. This paper presents a new data distribution algorithm called SUORA (Scalable and Uniform storage via Optimally-adaptive and Random number Addressing) specifically for heterogeneous devices to maximize the benefits of them. SUORA provides a fully symmetric, highly efficient methodology to distribute data across a hybrid and tiered storage cluster. It divides heterogeneous devices into different buckets and segments, and adopts pseudo-random functions to map data onto them with the balanced consideration of capacity, performance and life-time. By analyzing hotness and access patterns, SUORA gradually moves hot data from HDDs to SCMs to optimize the throughput, and moves cold data reversely for load balance. It combines data replication with migration to significantly reduce movement overhead while making data placement more adaptive to different workloads. Extensive evaluations on simulation and Sheepdog storage system show that, with considering distinct characteristics of various devices thoroughly, SUORA improves the overall performance efficiency of heterogeneous storage systems.

An Encryption Approach Based on Formal Method for Securing Distributed Big Data Storage in Cloud Environment

Cloud computing is the technology used to store massive amounts of data (Big data) on Servers which locations are unknown to the user of the service. One of the major source of concern in security and privacy is the fact that cloud operators have chances to reach the sensitive data. This makes Cloud computing adoption by organizations with users’ sensitive details including the banking sector and government agencies marred by resentment. Therefore, a cryptography approach is proposed, named Secured Efficient Distributed Storage SecDcloud) model that is designed to obtain an efficient mass distributed service, and is supported by the Modified Alternative Data Distribution (MAD2) Algorithm Modified Secured Data distribution (MSED2) Algorithm as well as Improved Data Conflation (IDCon) Algorithm based on formal method. Our proposed mechanism aims to split, encrypt sensitive data and store the data to the different cloud servers without causing big overheads using formal method which prevents cloud service providers from directly accessing the user’s data. Our experiments, evaluated on security and efficiency of our technique, and is compared with state of the art Advanced Encryption Standard(AES) Algorithm and the results show that it is capable of effectively defending against the most common cloud-based threats while still maintaining a reasonable amount of processing time.

Data Cooperative Distribution Mechanism of Internet of Vehicles Using D2D Technology

In recent years, with the advancement of the automotive industry and the innovation of electronic technology, issues related to the field of Internet of Vehicles (IoV for short) have received increasing attention. Existing communication technologies can no longer meet the demands of high mobility, low latency, high reliability, and massive streaming data distribution faced by IoV in the future. As one of the key technologies of 5G, D2D communication technology can effectively alleviate the above challenges faced by IoV. This technology not only allows terminals to perform point-to-point direct communication within a certain range but also allows data forwarding between receiving terminals, thereby improving the data distribution efficiency of the IoV, which injects new impetus into the development of the IoV business in the future. However, existing cooperative distribution algorithms between terminals do not fully consider the differences in D2D links, making it difficult to achieve efficient use of spectrum resources. To address this problem, this paper proposes a cooperative data distribution algorithm based on multihop relay in the IoV environment where D2D is used to realize flexible terminal communication between vehicles, including both multicast and unicast modes. The algorithm adaptively selects the optimal relay, route, and number of transmission hops based on the D2D link quality to find the optimal cooperative distribution scheme to achieve the goal of optimizing the energy efficiency of the requesting vehicle. Simulation results show that, compared with the existing one-hop unicast and one-hop multicast forwarding algorithms, the multihop relay data cooperative distribution algorithm proposed in this paper exhibits strong global optimization ability and can significantly improve D2D forwarding resource utilization and data transmission rate and improve the throughput of data distribution services, thereby effectively improving the energy efficiency of the requesting vehicle.

Efficient Text Classification with Deep Learning on Imbalanced Data Improved with Better Distribution

Technological developments and the widespread use of the internet cause the data produced on a daily basis to increase exponentially. An important part of this deluge of data is text data from applications such as social media, communication tools, customer service. The processing of this large amount of text data needs automation. Significant successes have been achieved in text processing recently. Especially with deep learning applications, text classification performance has become quite satisfactory. In this study, we proposed an innovative data distribution algorithm that reduces the data imbalance problem to further increase the text classification success. Experiment results show that there is an improvement of approximately 3.5% in classification accuracy and over 3 in F1 score with the algorithm that optimizes the data distribution.

Load Balancing Cloud Storage Data Distribution Strategy of Internet of Things Terminal Nodes considering Access Cost.

With the rapid development of Internet of Things (IoT) technology, IoT terminal nodes are facing many challenges in data storage, distribution, and data management. In particular, in the IoT terminal nodes considering access cost, the corresponding data distribution and storage are professional, complex, and miscellaneous. Based on the abovementioned current situation, this article innovatively proposes a complex sensor data placement algorithm based on the cloud storage distribution of IoT terminal nodes. Under this algorithm, the accurate division of IoT data I/O methods is realized through reasonable configuration. Through the adaptive sensing algorithm, while fully considering the access cost of the algorithm, the performance of the IoT data storage system is further optimized. In the corresponding terminal node load balancing problem, this article innovatively proposes the terminal node data sorting and distribution algorithm through the node data. The sorting and distribution algorithm realizes the precise segmentation of the IoT data to be processed, thereby realizing the improvement of data reading and processing speed. Based on the proposed algorithm, this article designs a load balancing cloud storage data distribution optimization system of IoT terminal nodes considering access cost and carries out experimental verification in a real environment. The experimental results show that the data pattern division accuracy corresponding to the proposed distribution strategy is improved to 97.13% and the corresponding data access efficiency is improved to 98.3%, compared with the traditional distribution strategy. Therefore, the data distribution strategy proposed in this article has obvious performance advantages and further promotion value.

Hardware-based security approach for secure distributed cloud data storage and retrieval

The East African Community (EAC) recognizes the fundamental role of science and technology in economic development. This led the Partner States to promote cooperation in the development and application of science and technology within the Community. The adoption of cloud computing in East Africa Community member states may induce a number of paths for Internet-based service provisioning to satisfy various demands in the community. However, data privacy and security has turned into crucial issues that resist cloud adoption for the past decades in the area of computerized systems. The dominating issue regarding security and privacy is pointed out on the data accessibility whereby cloud service providers and consumers simply gain access to secret information (data). The issue resulted in the cloud service consumer’s fear and slowed down the adoption speed of cloud computing from diverse areas of applications, i.e., governmental, medical and financial institutions; therefore, in this article we address this issue and propose a robust Hardware-Based Security (HBS) approach, whereby the cloud service provider and consumers are limited to simply access any stored data in the cloud data centers. This approach halves data files into data segments and distributes encrypted file segments to different cloud data centers. Our approach is built on Secure Data Distribution (SDD) and Data Retrieval (DR) Algorithms described in a Secure Efficient Data Distribution Model (SEDD). The analysis shows robust data security and less computation power which can resist unauthorized data accessibility with less latency. Therefore, once the East African Community member states adopt a secure cloud services, the development and application of science and technology within the community will boost the sustainable economic development.

K-means clustering algorithm for data distribution in cloud computing environment

K-means clustering algorithm for data distribution in cloud computing environment

K-means clustering algorithm for data distribution in cloud computing environment

This study analyses the data structure in cluster analysis. It is a clustering method that randomly selects a known number of points and then continues to expand. Through the comparative experiment...

GRAPH-BASED FOG COMPUTING NETWORK MODEL

IoT networks generate numerous amounts of data that is then transferred to the cloud for processing. Transferring data cleansing and parts of calculations towards these edge-level networks improves system’s, latency, energy consumption, network bandwidth and computational resources utilization, fault tolerance and thus operational costs. On the other hand, these fog nodes are resource-constrained, have extremely distributed and heterogeneous nature, lack horizontal scalability, and, thus, the vanilla SOA approach is not applicable to them. Utilization of Software Defined Network (SDN) with task distribution capabilities advocated in this paper addresses these issues. Suggested framework may utilize various routing and data distribution algorithms allowing to build flexible system most relevant for particular use-case. Advocated architecture was evaluated in agent-based simulation environment and proved its’ feasibility and performance gains compared to conventional event-stream approach.

Modeling of Bitcoin's Blockchain Delivery Network

In this paper, we provide a comprehensive analytical model for Bitcoin's blockchain distribution network. Components of the model are derived from recent measurements and business analysis reports. We model the data distribution algorithm using branching processes in the network with random distribution of node connectivity. Then, we apply Jackson network model to the entire network in which individual nodes operate as priority M/G/1 queuing systems. Data arrival to the nodes is modeled as a non-homogeneous Poisson process where the distribution of arrival rate to the nodes is derived from the analytical model of data delivery protocol. Within performance results, we present probability distributions of block and transaction distribution time, node response time, forking probabilities, network partition sizes, and duration of ledger's inconsistency period.

Serverless Management of Sensing Systems for Fog Computing Framework

In this paper, we propose a framework for the management of the Internet of Things (IoT) devices in a smart building to model services based on the serverless computing paradigm. The deployment of an IoT compatible serverless paradigm consists of a hierarchical structural design across the edge, fog, and cloud computing layers. The fog/edge nodes collect the data generated from various sensors, process the data in the intermediate nodes, and then forward certain data to a cloud for future analysis. The framework consists of a heterogeneous IoT network. We proposed a data distribution algorithm in the framework to make sure management, maintenance and availability of heterogeneous IoT network in the serverless computing paradigm are effective and efficient. The experiments conducted are validated at the developed fog and edge gateways using API mechanism. The response times for an application doing the computation at fog level and at the cloud level are compared. The experimentation shows that latency is less for the fog model as compared to the data sent to the cloud model.

A Data Distribution Algorithm Based on Neighbors’ Locations in VANET

Abstract Effective data distribution algorithm plays an important role in the Vehicular Ad Hoc Networks (VANETs). However, the broadcast storm often appears when VANET uses broadcast to communicate which means some nodes receive too many packages to deal with. Moreover, the algorithm often becomes inefficient due to the frequency changes in the different network densities. To address these issues, the beacons are used to judge the network state and then the nodes can select the corresponding data distribution mode under different vehicle densities.Simulation results show that the proposed algorithm has better performance in the coverage rate, delivery rate, and packet loss rate than the Flooding and the AID(An adaptive approach for information dissemination).

A resilient and distributed near real-time traffic forecasting application for Fog computing environments

In this paper we propose an architecture for a city-wide traffic modeling and prediction service based on the Fog Computing paradigm. The work assumes an scenario in which a number of distributed antennas receive data generated by vehicles across the city. In the Fog nodes data is collected, processed in local and intermediate nodes, and finally forwarded to a central Cloud location for further analysis. We propose a combination of a data distribution algorithm, resilient to back-haul connectivity issues, and a traffic modeling approach based on deep learning techniques to provide distributed traffic forecasting capabilities. In our experiments, we leverage real traffic logs from one week of Floating Car Data (FCD) generated in the city of Barcelona by a road-assistance service fleet comprising thousands of vehicles. FCD was processed across several simulated conditions, ranging from scenarios in which no connectivity failures occurred in the Fog nodes, to situations with long and frequent connectivity outage periods. For each scenario, the resilience and accuracy of both the data distribution algorithm, and the learning methods were analyzed. Results show that the data distribution process running in the Fog nodes is resilient to back-haul connectivity issues and is able to deliver data to the Cloud location even in presence of severe connectivity problems. Additionally, the proposed traffic modeling and forecasting method exhibits better behavior when run distributed in the Fog instead of centralized in the Cloud, especially when connectivity issues occur that force data to be delivered out of order to the Cloud.

Распределенный алгоритм отображения распределенных многомерных данных на многомерный мультикомпьютер в системе фрагментированного программирования LuNA

Distributed algorithm with local interactions Patch is presented in the paper. Patch is intended for data distribution and dynamic load balancing in the LuNA fragmented programming system. LuNA system is used for creation of parallel implementations of large-scale numerical models for distributed memory systems. Execution of a fragmented program is controlled by LuNA run-time system, which uses different data and computation distribution algorithms to enable efficient use of resources and minimize total execution time of the program. Patch algorithm, developed to be used in the LuNA system, enables distribution of multidimensional data meshes on a multidimensional lattice of computational nodes of a supercomputer. The algorithm uses mapping of data to multidimensional lattice of cells (coordinates), which in their turn are mapped to computational nodes. That mapping makes it possible to account for data dependencies and preserve data locality during dynamic load balancing. Patch algorithm was compared with another LuNA data distribution algorithm Rope, fragmented realisation of a real numerical problem was used for experiments. Experiments showed that Patch algorithm provides a general reduction in total computational volume and distances, as compared to Rope algorithm.

SLA based healthcare big data analysis and computing in cloud network

Large volume of multi-structured and low-latency patient data are generated in healthcare services, which is achallenging task to process and analyze within the Service Level Agreement (SLA). In this paper, a Parallel Semi-Naive Bayes (PSNB) based probabilistic method is used to process the healthcare big data in cloud for future health condition prediction. In order to improve the accuracy of PSNB method, a Modified Conjunctive Attribute (MCA) algorithm is proposed for reducing the dimension. Emergency condition of the patient is considered by setting a global priority among the patients and an Optimal Data Distribution (ODD) algorithm is proposed to position both batch and streaming patient data into the Spark nodes. Further, a Dynamic Job Scheduling (DJS) algorithm is designed to schedule the jobs efficiently to the most suitable nodes for processing the data taking SLA into account. Our proposed PSNB algorithm provides better accuracy of 87.8% for both batch and streaming data, which is 12.8% higher than the original Naive–Bayes (NB) algorithm and can conveniently be employed in various patient monitoring applications.

Efficient Community-Based Data Distribution Over Multicast Trees

Communities are important attributes of today’s networking, since people who join networks tend to join communities. Thus, community detection is an important issue in designing algorithms for delay-tolerant networks (DTNs). Multicasting is an appropriate method to share information within a community or between communities, because it delivers messages from a source to a group of targets using limited resources. This paper addresses the problem of detecting communities in weighted networks with irregular topologies. Based on communities, we propose an efficient data distribution algorithm for DTNs. The distribution strategy is based on the construction of a number of multicast trees, where each tree can be used to select the best relay node for each target to improve multicast efficiency. The proposed strategy provides better delivery ratio compared with other strategies and it reduces latencies when multiple nodes from a community need to multicast to other community members.

Predictive complex event processing based on evolving Bayesian networks

In the Big Data era, large volumes of data are continuously and rapidly generated from sensor networks, social network, the Internet, etc. Predicting from online event stream is an important task since users usually need to predict some future states and take some actions in advance. Many applications need online prediction models which can evolve automatically with data distribution drift and algorithms which can support single-pass processing of data, which are still faced with many challenges. In this paper, the authors propose a predictive complex event processing method based on evolving Bayesian networks. The Bayesian model is designed based on event type and time with inference method based on Gaussian mixture model and EM algorithm. When learning the structure of Bayesian network from event streams, this method supports calculating score metric incrementally when new data is arrived or edges in the network are changed. Evolving Bayesian network structure is supported based on hill-climbing method. The system can continuously monitor the Bayesian network model and modify it if it is found to be not appropriate for the new incoming data. The method of this paper is evaluated in road traffic domain with both real application data and data produced by a simulated transportation system. The total percentage error is 8.12% for real data and 7.78% for simulated data, while the best result for other methods is 11.79% for real data and 14.59% for simulated data. The experimental evaluations show that this method is effective for predictive complex event processing and it outperforms other popular methods when processing traffic prediction in intelligent transportation systems.