Dynamic Partitioning Algorithm Research Articles

Dynamic stream partitioning for time-triggered traffic in Time-Sensitive Networking

Time-Sensitive Networking (TSN) is a promising network technology that can ensure bounded latency and jitter for industrial real-time scenarios. It establishes the IEEE 802.1 Qbv standard to precisely control the periodic transmission time of the time-triggered (TT) traffic. However, it is still a challenge to efficiently schedule the TT traffic in large-scale networks with high transmission performance. In this paper, we propose a novel method called dynamic stream partitioning (DSP) to solve this problem. It satisfies the IEEE 802.1 Qbv standard and can significantly reduce the scheduling time by reducing the number of constraints. Based on the DSP method, we analyze the impact of stream partitioning on the transmission performance of the TT traffic, and propose an indicator called Normalized Degradation of Performance (NDOP) to quantify it dynamically. Furthermore, we design a partitioning-aware dynamic routing (PADR) to expand the scheduling space. Integrated with NDOP and PADR, we propose a joint routing and dynamic stream partitioning (JR/DSP) algorithm. Extensive simulation experiments verify that the proposed JR/DSP algorithm has higher scalability, transmission performance, and schedulability compared to the static stream partitioning algorithms.

A distributed PageRank-based dynamic partition algorithm to improve distributed energy storages participation in frequency regulation

In high renewable energy sources penetrated power systems, a large number of distributed small-capacity energy storages (ESs) are expected to form virtual power plants (VPPs) and provide comparable frequency regulation capacity to conventional power plants. In this paper, a dynamic and distributed partition method is proposed to cluster ESs into VPPs by selecting the electrical distance and local power demands as the index for partitioning. Utilizing the partition indices, the Personalized PageRank (PPR) scores are used to measure the similarity of all ESs with respect to other ESs in parallel. Then Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm is utilized to partition the ESs into different VPPs. Within the VPPs, the secondary frequency control strategy is adopted to achieve the frequency regulation and state of charge (SoC) balancing. Finally, the performance comparison of the proposed partition method with several classic clustering algorithms is conducted to validate the efficiency and the quality of the partition. And numerical studies are performed in an IEEE 39-bus system to verify that the proposed control method can eliminate the frequency deviation and reduce power losses.

Time-Aware Data Partition Optimization and Heterogeneous Task Scheduling Strategies in Spark Clusters

Abstract The Spark computing framework provides an efficient solution to address the major requirements of big data processing, but data partitioning and job scheduling in the Spark framework are the two major bottlenecks that limit Spark’s performance. In the Spark Shuffle phase, the data skewing problem caused by unbalanced data partitioning leads to the problem of increased job completion time. In response to the above problems, a balanced partitioning strategy for intermediate data is proposed in this article, which considers the characteristics of intermediate data, establishes a data skewing model and proposes a dynamic partitioning algorithm. In Spark heterogeneous clusters, because of the differences in node performance and task requirements, the default task scheduling algorithm cannot complete scheduling efficiently, which leads to low system task processing efficiency. In order to deal with the above problems, an efficient job scheduling strategy is proposed in this article, which integrates node performance and task requirements, and proposes a task scheduling algorithm using greedy strategy. The experimental results prove that the dynamic partitioning algorithm for intermediate data proposed in this article effectively alleviates the problem that data skew leads to the decrease of system task processing efficiency and shortens the overall task completion time. The efficient job scheduling strategy proposed in this article can efficiently complete the job scheduling tasks under heterogeneous clusters, allocate jobs to nodes in a balanced manner, decrease the overall job completion time and increase the system resource utilization.

Wildfire detection in large-scale environments using force-based control for swarms of UAVs

Wildfires affect countries worldwide as global warming increases the probability of their appearance. Monitoring vast areas of forests can be challenging due to the lack of resources and information. Additionally, early detection of wildfires can be beneficial for their mitigation. To this end, we explore in simulation the use of swarms of uncrewed aerial vehicles (UAVs) with long autonomy that can cover large areas the size of California to detect early stage wildfires. Four decentralised control algorithms are tested: (1) random walking, (2) dispersion, (3) pheromone avoidance and (4) dynamic space partition. The first three adaptations are known from literature, whereas the last one is newly developed. The algorithms are tested with swarms of different sizes to test the spatial coverage of the system in 24 h of simulation time. Best results are achieved using a version of the dynamic space partition algorithm (DSP) which can detect 82% of the fires using only 20 UAVs. When the swarm consists of 40 or more aircraft 100% coverage can also be achieved. Further tests of DSP show robustness when agents fail and when new fires are generated in the area.

Privacy-Preserving Scheme With Account-Mapping and Noise-Adding for Energy Trading Based on Consortium Blockchain

The maturity in information technology and new energy technologies enables participants to generate, buy, and sell energy in energy trading systems. Although applying blockchain technology to energy trading has solved some drawbacks in traditional centralized energy systems, the openness and transparency characteristics make the trading records stored on the blockchain vulnerable to data-mining attacks that may cause indispensable privacy leakage. Due to high efficiency and low overhead, noise-addition is an appropriate solution for privacy preservation. Nonetheless, recent research on noise-addition needs to generate massive accounts, which brings a certain amount of waste and inconvenience for later regulation and management. To avoid the aforementioned issues, this paper proposes a consortium blockchain-enabled scheme to ensure the privacy of data stored on the blockchain and resist linking attacks initiated by data mining algorithms. Our scheme utilizes a dynamic partition algorithm to leverage an account mapping algorithm and a virtual token algorithm. Specifically, the account mapping algorithm utilizes a dynamic account allocation method to hide the trading distribution of active users. Furthermore, the virtual token algorithm applies Laplace noise to hide the actual energy consumption of inactive users and curb excessive accounts generation. Finally, we formally demonstrate the privacy and effectiveness of our proposed scheme in security analysis and experiment evaluations.

Group Reassignment for Dynamic Edge Partitioning

Graph partitioning is a mandatory step in large-scale distributed graph processing. When partitioning real-world power-law graphs, the edge partitioning algorithm performs better than the traditional vertex partitioning algorithm, because it can cut a single vertex into multiple replicas to apportion the computation. Many advanced edge partitioning methods are designed for partitioning a static graph from scratch. However, the real-world graph structure changes continuously, which leads to a decrease in partition quality and affects the performance of the graph applications. Some studies are devoted to offline repartitioning or batch incremental partitioning, but how to deal with dynamics in real-time is still worthy of in-depth study. In this article, we discuss the impact of dynamic change on partition and discover that both insertion and deletion will lead to local suboptimal partitioning, which is the reason for the degradation of partition quality. As a solution, a dynamic edge partitioning algorithm is proposed to partition dynamics in real-time. Specifically, we deal with dynamics by a distributed stream and improve partition quality by reassigning some closely connected edges. Experiments show that it is robust to initial partition quality, dynamic scale and type, and distributed scale. Compared with the state-of-the-art dynamic partitioner, it can reduce vertex-cuts by 29.5 percent. Compared with the repartitioning algorithms, it can save the partitioning time by 91.0 percent. Applied on the graph task, it can reduce the increase of communication cost and the increase of the total time of task by 41.5 and 71.4 percent.

Metaheuristics-based routing optimisation, balanced workload distribution and security strategy in IoT environment

The advancement of wireless sensor networks (WSNs) made the society ever so slightly gain new degrees of freedom, intrinsically with respect to connectivity. The emergence of internet of things (IoT) together with the onset and development of evolutionary computing, address the various needs of the growing urbanisation with a significant allusion altogether. Applications built on cloud infrastructure are deployed for job segregation, rapid processing and utilisation but with limitation of proper computation in high density IoT environment. This paper strives on resolving some of the issues so accosted by the IoT paradigm, with the help of a new metaheuristics-based data routing hybrid algorithm based on directed artificial bat algorithm (DABA) and particle swarm optimisation (PSO) to optimise connection issues such as real time delay and network congestion. This approach also introduces the clustering concept together with fog computing to distribute the network stress and to optimise the usage of bandwidth using the dynamic graph partitioning algorithm. The paper further proposes advance metaheuristics constricted PSO (C-PSO) in the hybrid algorithm and effective proposal for security strategy to enhance the efficiency of the IoT environment.

Metaheuristics-based routing optimisation, balanced workload distribution and security strategy in IoT environment

The advancement of wireless sensor networks (WSNs) made the society ever so slightly gain new degrees of freedom, intrinsically with respect to connectivity. The emergence of internet of things (IoT) together with the onset and development of evolutionary computing, address the various needs of the growing urbanisation with a significant allusion altogether. Applications built on cloud infrastructure are deployed for job segregation, rapid processing and utilisation but with limitation of proper computation in high density IoT environment. This paper strives on resolving some of the issues so accosted by the IoT paradigm, with the help of a new metaheuristics-based data routing hybrid algorithm based on directed artificial bat algorithm (DABA) and particle swarm optimisation (PSO) to optimise connection issues such as real time delay and network congestion. This approach also introduces the clustering concept together with fog computing to distribute the network stress and to optimise the usage of bandwidth using the dynamic graph partitioning algorithm. The paper further proposes advance metaheuristics constricted PSO (C-PSO) in the hybrid algorithm and effective proposal for security strategy to enhance the efficiency of the IoT environment.

Patient clustering using dynamic partitioning on correlated and uncertain biomedical data

Background and objectivesHealth professionals look for specific patterns by correlating multiple physiological data in the process of deciding treatments to remedy clinical abnormalities. Biomedical data exhibit some common patterns in the event of identical clinical illnesses. The primary interest of this work is automatic discovery of such patterns in vital sign data (e.g. heart rate, blood pressure) using unsupervised learning and utilising them to identify patients with similar clinical conditions.MethodsA patient clustering method is developed that efficiently isolates patients into multiple groups by discovering dynamic patterns in multi-dimensional vital sign data. A dynamic partitioning algorithm and a patient clustering approach is proposed by introducing a measure namely aggregated instance-wise uncertainty (AIU) computed from multi-dimensional physiological time-series data.ResultsThe developed model is evaluated qualitatively using principal component analysis and silhouette value; and quantitatively in terms of its ability of clustering patients associated with different clinical situations. Experiments are conducted using real-world biomedical data of patients having various clinical conditions. Thee observed accuracy was 82.85% and 91.17% on two experimental datasets comprised of 35 and 34 patients data respectively.The comparisons show that the proposed approached outperformed than other methods in state-of-the-art approach.ConclusionsThe experimental outcomes demonstrate the effectiveness of the proposed approach in discovering distinct patterns with predictive significance.

A Dynamic Plane Prediction Method Using the Extended Frame in Smart Dust IoT Environments.

Internet of Things (IoT) technologies are undeniably already all around us, as we stand at the cusp of the next generation of IoT technologies. Indeed, the next-generation of IoT technologies are evolving before IoT technologies have been fully adopted, and smart dust IoT technology is one such example. The concept of smart dust IoT technology, which features very small devices with low computing power, is a revolutionary and innovative concept that enables many things that were previously unimaginable, but at the same time creates unresolved problems. One of the biggest problems is the bottlenecks in data transmission that can be caused by this large number of devices. The bottleneck problem was solved with the Dual Plane Development Kit (DPDK) architecture. However, the DPDK solution created an unexpected new problem, which is called the mixed packet problem. The mixed packet problem, which occurs when a large number of data packets and control packets mix and change at a rapid rate, can slow a system significantly. In this paper, we propose a dynamic partitioning algorithm that solves the mixed packet problem by physically separating the planes and using a learning algorithm to determine the ratio of separated planes. In addition, we propose a training data model eXtended Permuted Frame (XPF) that innovatively increases the number of training data to reflect the packet characteristics of the system. By solving the mixed packet problem in this way, it was found that the proposed dynamic partitioning algorithm performed about 72% better than the general DPDK environment, and 88% closer to the ideal environment.

An Energy-Efficient Coverage Enhancement Strategy for Wireless Sensor Networks Based on a Dynamic Partition Algorithm for Cellular Grids and an Improved Vampire Bat Optimizer.

Sensor nodes perform missions based on the effectual invariable coverage of events, and it is commonly guaranteed by the determinate deployment for sensor nodes who deviate from the optimum site frequently. To reach the optimal coverage effect with the lowest costs is a primary goal of wireless sensor networks. In this paper, by splicing the sensing area optimally with cellular grids, the best deployment location for sensors and the required minimum number of them are revealed. The optimization problem of coverage rate and energy consumption is converted into a task assignment problem, and a dynamic partition algorithm for cellular grids is also proposed to improve the coverage effect when the number of sensors is variable. Furthermore, on the basis of solving the multi-objective problem of reducing and balancing the energy cost of sensors, the vampire bat optimizer is improved by introducing virtual bats and virtual preys, and finally solves the asymmetric assignment problem once the number of cellular grids is not equal to that of sensors. Simulation results indicate that the residual energy of sensors during redeployment is balanced notably by our strategy when compared to three other popular coverage-enhancement algorithms. Additionally, the total energy cost of sensor nodes and coverage rate can be optimized, and it also has a superior robustness when the number of nodes changes.

Exploring GPU-Accelerated Routing for FPGAs

Field Programmable Gate Arrays (FPGAs) are reconfigurable architectures able to provide a good balance between energy efficiency and flexibility with respect to CPUs and ASICs. The main drawback in using FPGAs, however, is their timing-consuming routing process, significantly hindering the designer productivity. An emerging solution to this problem is to accelerate the routing by parallelization. Existing attempts of parallelizing the FPGA routing either do not fully exploit the parallelism or suffer from an excessive quality loss. Massive parallelism using GPUs has the potential to solve this issue but faces non-trivial challenges. To cope with these challenges, this paper explores GPU-accelerated routing approach for FPGAs. We leverage the idea of problem size reduction by limiting the single-net routing in a small subgraph rather than in an entire graph, further enabling the GPU-friendly shortest path algorithm to be used in FPGA routing. We maintain the convergence after problem size reduction by using the dynamic expansion of the routing resource subgraph, where the routing region of subgraph will be progressively expanded to find a feasible solution to each net. In addition, we are based on a GPU platform to explore the fine-grained single-net parallel routing in three ways and propose a hybrid approach to combine the static and dynamic parallelization for better speedup in FPGA routing. To explore the coarse-grained multi-net parallelization, We propose a dynamic programming-based partitioning algorithm to parallelize the routing of multiple nets while generating the equivalent routing results as the original single-net routing. Experimental results show that our proposed approach can provide an average of about 21.53× speedup on a single GPU with a tolerable loss in the routing quality and maintain a scalable speedup on large-scale routing resource graphs. To our knowledge, this is the first work to demonstrate the effectiveness of GPU-accelerated routing for FPGAs.

A Dynamic Partitioning Algorithm for Sip Detection using a Bottle-Attachable IMU Sensor

Hydration tracking technologies are a promising tool for improving health outcomes across a variety of populations. As a non-wearable solution that is reconfigurable across containers, bottle-attachable inertial measurement unit (IMU) sensors offer numerous advantages versus alternative tracking approaches. This paper proposes a novel dynamic temporal partitioning and classification algorithm for spotting drinks within the streaming data generated by such sensors. By exploiting the distinguishing characteristics of the container’s estimated inclination during drinking, the algorithm identifies candidate drink intervals for subsequent classification using a Threshold-Merge-Discard framework. The proposed approach is benchmarked against a slight variation of a previously introduced sliding window classifier for a series of experiments replicating the intended use case of the device. The new algorithm is shown to increase the true-positive detection rate by 23.7%, while reducing the number of required classification operations by more than an order of magnitude.

Application of surrogate parameters in characteristic UV–vis absorption bands for rapid analysis of water contaminants

In recent years, absorption spectroscopy is being widely used for water quality monitoring. However, much of the spectral analysis results do not provide any physical meaning and the use of traditional parameters for monitoring water quality, such as chemical oxygen demand (COD), is not a radical research method because of the large diversity of chemical components in water. Here in, we studied the absorption spectra of eight typical contaminants and proposed four integral parameters in different spectral ranges, which were relative to different functional groups. The proposed surrogate parameters can be used to rapidly analyze and predict the categories of contaminants based on their changes without using any modeling. We used a dynamic partitioning algorithm based on the fourth-order derivative spectrum to adapt the movement of absorption bands. The impact of turbidity was eliminated by deducting the turbidity component from surrogate parameters based on the proportion of four parameters in formazine turbid solution. Using this method, we conducted long-term monitoring of the wastewater released by a dyeing and printing plant. The rapid analysis and prediction potential of different categories of water contaminants was demonstrated. The prediction results can provide evidence for more targeted and detailed analysis and water quality monitoring.

Adaptive Path Isolation for Elephant and Mice Flows by Exploiting Path Diversity in Datacenters

Resource competition and conflicts in datacenter networks (DCNs) are frequent and intense. They become inevitable when mixing elephant and mice flows on shared transmission paths, resulting in arbitration between throughput and latency and performance degradation. We propose a novel flow scheduling scheme, Freeway, that leverages on path diversity in the DCN topology to guarantee, simultaneously, mice flow completion within deadline and high network utilization. Freeway adaptively partitions the available paths into low latency and high throughput paths and provides different transmission services for each category. A M/G/1-based model is developed to theoretically obtain the highest value of average delay over the path that will guarantee for 99% of mice flows their completion time before the deadline. Based on this bound, Freeway proposes a dynamic path partitioning algorithm to adjust dynamically with varying traffic load the number of low latency and high throughput paths. While mice flows are transmitted over low latency paths using a simple equal cost multiple path (ECMP) scheduling, Freeway load balances elephant flows on different high-throughput paths. We evaluate Freeway in a series of simulation on a large scale topology and use real traces. Our evaluation results show that Freeway significantly reduces the mice flows completion time within deadlines, while achieving remarkable throughput compared with current schemes. It is remarkable that Freeway does not need any change of DCN switch fabrics or scheduling algorithms and can be deployed easily on any generic datacenter network with switches implementing VLANs and trunking.

Leopard

This paper introduces a dynamic graph partitioning algorithm, designed for large, constantly changing graphs. We propose a partitioning framework that adjusts on the fly as the graph structure changes. We also introduce a replication algorithm that is tightly integrated with the partitioning algorithm, which further reduces the number of edges cut by the partitioning algorithm. Even though the proposed approach is handicapped by only taking into consideration local parts of the graph when reassigning vertices, extensive evaluation shows that the proposed approach maintains a quality partitioning over time, which is comparable at any point in time to performing a full partitioning from scratch using a state-the-art static graph partitioning algorithm such as METIS. Furthermore, when vertex replication is turned on, edge-cut can improve by an order of magnitude.

Jump-Diffusion Approximation of Stochastic Reaction Dynamics: Error Bounds and Algorithms

Biochemical processes in living cells are comprised of reactions with vastly varying speeds and molecular counts of the reactant species. Classical deterministic or stochastic approaches to modeling often fail to exploit this multiscale nature of the reaction systems. In this paper, we propose a jump-diffusion approximation to these types of multiscale systems that couples the two traditional modeling approaches. An error bound of the proposed approximation is derived and used to partition the reactions into two sets, where one set is modeled by continuous stochastic differential equations and the other by discrete state processes. The methodology leads to a very efficient dynamic partitioning algorithm which has been implemented for several multiscale reaction systems. The gain in computational efficiency is evident in all these examples, which include a realistically sized model of a signal transduction cascade coupled to a gene expression dynamics.

Research on Dynamic Bandwidth Partition Algorithm for Control Channel of Vehicular Ad-Hoc Networks

In IEEE 802.11p protocol of Vehicular Ad-Hoc network (VANET), the multiple channels are divided into one control channel (CCH) and six service channels (SCHs). CCH is used for broadcasting safety messages related to road conditions, so that CCH access algorithm has great influence on efficiency of VANET. This paper proposes a contention-free bandwidth partition algorithm, which can dynamically partition the CCH interval and bandwidth assigned to on board units (OBUs) located in vehicles according to real-time VANET environments. Meanwhile, a frequency hopping communication scheme based on chaos scrambling is applied to our algorithm to improve the reliability of VANET. The performance of proposed algorithm are verified by theoretical analysis, the results show that it is able to enhance the transmission efficiency of safety packets in CCH with high reliability.

A Dynamic Algorithm of Partitioning Urban Road Network into Traffic Control Subareas

The objective of this study is to develop a dynamic algorithm for traffic control subarea partition. Influencing factors are classified into static factors and dynamic factors. We mainly focus on the three dynamic factors which are cycle length, platoon length and traffic flow continuation. An integrated correlation model is established to quantify the correlation degree of adjacent intersections and then a dynamic partition algorithm is developed based on the model. Both this algorithm and Changs algorithm, which is a representative achievement in subarea partition cited by Traffic control systems Handbook, are applied to a network including 20 signalized intersections in VISSIM to compare their benefits. Statistical results show that: (a) adjacent intersections with high correlation degree can be partitioned into one subarea to execute signal coordination according to our algorithm; (b) implementing our algorithm can save average vehicle delay by 8.9% and average vehicle stops by 7.8% than implementing Changs algorithm.

Energy Management in Wireless Mobile Systems Using Dynamic Task Assignment

Wireless mobile sensing systems are hierarchical and heterogeneous in nature with components that have different energy and performance capabilities. In such systems allocation of tasks to different devices affects both performance and the entire system battery lifetime. In this paper we formulate the problem of optimal task assignment with objectives related to minimizing the total system energy consumption and maximizing the system lifetime as an Integer Linear Program (ILP). We describe a heuristic algorithm and two dynamic graph-based partitioning algorithms that are computationally efficient and that are able to adapt in real-time to changing system conditions. ILP based solutions are able to achieve optimal task assignment, but cannot be used in dynamically changing conditions due to their computationally expensive nature. We evaluate the performance of our three dynamic algorithms using Qualnet and show that they have up to 88% longer system lifetime than the ILP based solutions.