Partitioning Load Balancing Research Articles

Task Migration with Partitioning for Load Balancing in Collaborative Edge Computing

Multi-access edge computing (MEC) has emerged as a promising technology to facilitate efficient vehicular applications, such as autonomous driving, path planning and navigation. By offloading tasks from vehicles to MEC servers (MECSs), the MEC system can facilitate computation-intensive applications with hard latency constraints in vehicles with limited computing resources. However, owing to the mobility of vehicles, the vehicles are not evenly distributed across the MEC system. Therefore, some MECSs are heavily congested, whereas others are lightly loaded. If a task is offloaded to a congested MECS, it can be blocked or have high latency. Moreover, service interruption would occur because of the high mobility and limited coverage of the MECS. In this paper, we assume that the task can be divided into a set of subtasks and computed by multiple MECSs in parallel. Therefore, we propose a method of task migration with partitioning. To balance loads, the MEC system migrates the set of subtasks of tasks in an overloaded MECS to one or more underloaded MECSs according to the load difference. Simulations have indicated that, compared with conventional methods, the proposed method can increase the satisfaction of quality-of-service requirements, such as low latency, service reliability, and MEC system throughput by optimizing load balancing and task partitioning.

Frequent Pattern Mining over Unstructured Data using Semi-Structured Doc-Model and Pattern Ranking

Frequent pattern mining is an essential data-mining task, with a goal of discovering knowledge in the form of repeated patterns. Many efficient pattern-mining algorithms have been discovered in the last two decades, yet most do not scale to the type of data we are presented with today, the so-called “Big Data”. Scalable parallel algorithms hold the key to solving the problem in this context. This paper reviews recent advances in parallel frequent pattern mining, analysing them through the Big Data lens. Load balancing and work partitioning are the major challenges to be conquered. These challenges always invoke innovative methods to do, as Big Data evolves with no limits. The biggest challenge than before is conquering unstructured data for finding frequent patterns. To accomplish this Semi Structured Doc-Model and ranking of patterns are used.

Load balancing in reducers for skewed data in MapReduce systems by using scalable simple random sampling

MapReduce has demonstrated itself to be as a highly efficient programming model for processing massive dataset on the distributed system. One of the most important obstacles hindering the performance of MapReduce is data skewness. The presence of data skewness leads to considerable load imbalance on the reducers and performance degradation. In this paper, the problem of how to efficiently accommodate intermediate data to even up the load of all reducers is studied when encountering skewed data. A scalable sampling algorithm is used which it can observe a more precise approximate distribution of the keys by sampling only a small fraction of the intermediate data. Afterwards, it is applied to evaluate the overall distribution of the keys. In addition, we propose a sorted-balance algorithm based on sampling results: sorted-balance algorithm using scalable simple random sampling (SBaSC). This work not only puts forward a load-balanced partitioning strategy, but also proves a significant approximation ratio of SBaSC. The experiments confirm that our solution attains a better execution time and load balancing results.

Highly scalable SFC-based dynamic load balancing and its application to atmospheric modeling

Load balance is one of the major challenges for efficient supercomputing, especially for applications that exhibit workload variations. Various dynamic load balancing and workload partitioning methods have been developed to handle this issue by migrating workload between nodes periodically during the runtime. However, on today’s top HPC systems–and even more so on future exascale systems–runtime performance and scalability of these methods becomes a concern, due to the costs exceeding the benefits of dynamic load balancing. In this work, we focus on methods based on space-filling curves (SFC), a well-established and comparably fast approach for workload partitioning. SFCs reduce the partitioning problem from n dimensions to one dimension. The remaining task, the so-called 1D partitioning problem or chains-on-chains partitioning problem, is to decompose a 1D workload array into consecutive, balanced partitions. While published parallel heuristics for this problem cannot reliably deliver the required workload balance, especially at large scale, exact algorithms are infeasible due to their sequential nature. We therefore propose a hierarchical method that combines a heuristic and an exact algorithm and allows to trade-off between these two approaches. We compare load balance, execution time, application communication, and task migration of the algorithms using real-life workload data from two different applications on two different HPC systems. The hierarchical method provides a significant speed-up compared to exact algorithms and yet achieves nearly the optimal load balance. On a Blue Gene/Q system, it is able to partition 2.6 million tasks for 524288 processes with over 99% of the optimal balance in 23.4 ms only, while a published fast exact algorithm requires 6.4 s. We also provide a comparison to parallel load balancing methods implemented in the Zoltan library and present results from applying our methods to COSMO-SPECS+FD4, a detailed atmospheric simulation model that requires frequent dynamic load balancing to run efficiently at large scale.

Distributed User Association with Resource Partitioning in Heterogeneous Cellular Networks

To fully utilize the resources of heterogeneous cellular networks (HCNs), an effective approach that offloads users to the underloaded small cells from overloaded macrocells is widely advocated. However, this operation often leads to a bad result that the offloaded users achieve lower signal-to-interference-plus-noise-ratio (SINR) than these users in macrocells. Thus, some appropriate interference avoidance techniques should be adopted to partially alleviate the SINR degradation. For this, we consider the resource (frequency) partitioning that turns off some fraction of such resources in a macrocell. Naturally, an optimal offloading strategy should be closely coupled with resource partitioning, and in turn the optimal partition decides the offloading performance. In this paper, we propose a distributed association strategy with joint offloading and resource partitioning for HCNs. We reveal that load balancing, by itself, is insufficient, and additional resource partitioning is required to improve system performance. Meanwhile, we also show that, compared with the best power association and range extension association, the proposed scheme provides better association performance.

Distributed In-Memory Processing of All k Nearest Neighbor Queries

A wide spectrum of Internet-scale mobile applications, ranging from social networking, gaming and entertainment to emergency response and crisis management, all require efficient and scalable All k Nearest Neighbor (AkNN) computations over millions of moving objects every few seconds to be operational. Most traditional techniques for computing AkNN queries are centralized, lacking both scalability and efficiency. Only recently, distributed techniques for shared-nothing cloud infrastructures have been proposed to achieve scalability for large datasets. These batch-oriented algorithms are sub-optimal due to inefficient data space partitioning and data replication among processing units. In this paper, we present Spitfire , a distributed algorithm that provides a scalable and high-performance AkNN processing framework. Our proposed algorithm deploys a fast load-balanced partitioning scheme along with an efficient replication-set selection algorithm, to provide fast main-memory computations of the exact AkNN results in a batch-oriented manner. We evaluate, both analytically and experimentally, how the pruning efficiency of the Spitfire algorithm plays a pivotal role in reducing communication and response time up to an order of magnitude, compared to three other state-of-the-art distributed AkNN algorithms executed in distributed main-memory.

Joint User Association and Resource Partitioning with QoS Support for Heterogeneous Cellular Networks

In heterogeneous cellular networks (HCNs), it is desirable that users in macrocells should be offloaded to small cells with less congested traffic. However, these offloaded users often have lower signal-to-interference-plus-noise-ratio (SINR) than them in macrocells. The degradation of SINR can be partially alleviated by adopting interference avoidance techniques, such as resource partitioning, whereby the macrocell lets some fraction of such resources be unavailable. Naturally, an optimal offloading scheme should be tightly coupled with resource partitioning; the optimal partition in turn decides the number of offloaded users. Moreover, in order to reduce the call blocking probability of users, we need to consider their quality of service (QoS) requirements. Thus, in this paper, we propose a distributed QoS-aware strategy with joint offloading and resource partitioning for HCNs. We reveal that load balancing, by itself, is insufficient, and additional resource partitioning should be considered to reduce the call blocking probability. Meanwhile, we also show that compared with the best power association and range expansion association, the proposed scheme has more obvious superiorities over call blocking probability.

Joint Resource Partitioning and Offloading in Heterogeneous Cellular Networks

In heterogeneous cellular networks (HCNs), it is desirable to offload mobile users to small cells, which are typically significantly less congested than the macrocells. To achieve sufficient load balancing, the offloaded users often have much lower SINR than they would on the macrocell. This SINR degradation can be partially alleviated through interference avoidance, for example time or frequency resource partitioning, whereby the macrocell turns off in some fraction of such resources. Naturally, the optimal offloading strategy is tightly coupled with resource partitioning; the optimal amount of which in turn depends on how many users have been offloaded. In this paper, we propose a general and tractable framework for modeling and analyzing joint resource partitioning and offloading in a two-tier cellular network. With it, we are able to derive the downlink rate distribution over the entire network, and an optimal strategy for joint resource partitioning and offloading. We show that load balancing, by itself, is insufficient, and resource partitioning is required in conjunction with offloading to improve the rate of cell edge users in co-channel heterogeneous networks.

Balancing reducer workload for skewed data using sampling-based partitioning

MapReduce has emerged as a popular tool for distributed processing of massive data. However, it is not efficient when handling skewed data and it often leads to reducer load imbalance. In this paper, we address the problem of how to efficiently partition intermediate keys to balance the workload of all reducers when processing skewed data. We present a sampling scheme to compute the approximate distribution of key frequency, estimate the overall distribution and then make a partition scheme in advance. Then, we apply it to map phase of the executing MapReduce job. This work not only provides a load-balanced partition strategy, but also keeps a high performance of synchronous mode of MapReduce. We also propose two partition methods based on sampling results: cluster combination and cluster split combination. The experimental results show that our methods achieve a better time and load balancing results.

Automatic Load-Balanced Partitioning Strategy in Parallel Multilevel Fast Multipole Algorithm

In this paper, we propose an automatic load-balanced partitioning strategy for parallel multilevel fast multipole algorithm(MLFMA) based on distributed-memory architectures to solve the large scale electromagnetic scattering problems. We focus on the automatic load-balancing partitioning strategy because that our original scheme requires that users input the transition level, which is usually determined on the users’ experience and sometimes lead to a bad load-balancing partition. By introducing the automatic load-balancing algorithm to our pervious partitioning technique, our implementation can automatically achieve the best load-balancing and consequently attain better parallel efficiency. To present the effectiveness of the new strategy, we analyze results of the previous implementation according to different inputs of transition level and compare them with the result of implementation using the new algorithm.

Task partitioning, scheduling and load balancing strategy for mixed nature of tasks

Load balancing and task partitioning are important components of distributed computing. The optimum performance from the distributed computing system is achieved by using effective scheduling and load balancing strategy. Researchers have well explored CPU, memory, and I/O-intensive tasks scheduling, and load balancing techniques. But one of the main obstacles of the load balancing technique leads to the ignorance of applications having a mixed nature of tasks. This is because load balancing strategies developed for one kind of job nature are not effective for the other kind of job nature. We have proposed a load balancing scheme in this paper, which is known as Mixed Task Load Balancing (MTLB) for Cluster of Workstation (CW) systems. In our proposed MTLB strategy, pre-tasks are assigned to each worker by the master to eliminate the worker's idle time. A main feature of MTLB strategy is to eradicate the inevitable selection of workers. Furthermore, the proposed MTLB strategy employs Three Resources Consideration (TRC) for load balancing (CPU, Memory, and I/O). The proposed MTLB strategy has removed the overheads of previously proposed strategies. The measured results show that MTLB strategy has a significant improvement in performance.

Avatar mobility in user-created networked virtual worlds: measurements, analysis, and implications

We collected mobility traces of avatars spanning multiple regions in Second Life, a popular user-created virtual world. We analyzed the traces to characterize the dynamics of the avatars' mobility and behavior, both temporally and spatially. We discuss the implications of our findings on the design of peer-to-peer architecture, interest management, mobility modeling of avatars, server load balancing and zone partitioning, caching, and prefetching for user-created virtual worlds.

A repartitioning hypergraph model for dynamic load balancing

In parallel adaptive applications, the computational structure of the applications changes over time, leading to load imbalances even though the initial load distributions were balanced. To restore balance and to keep communication volume low in further iterations of the applications, dynamic load balancing (repartitioning) of the changed computational structure is required. Repartitioning differs from static load balancing (partitioning) due to the additional requirement of minimizing migration cost to move data from an existing partition to a new partition. In this paper, we present a novel repartitioning hypergraph model for dynamic load balancing that accounts for both communication volume in the application and migration cost to move data, in order to minimize the overall cost. The use of a hypergraph-based model allows us to accurately model communication costs rather than approximate them with graph-based models. We show that the new model can be realized using hypergraph partitioning with fixed vertices and describe our parallel multilevel implementation within the Zoltan load balancing toolkit. To the best of our knowledge, this is the first implementation for dynamic load balancing based on hypergraph partitioning. To demonstrate the effectiveness of our approach, we conducted experiments on a Linux cluster with 1024 processors. The results show that, in terms of reducing total cost, our new model compares favorably to the graph-based dynamic load balancing approaches, and multilevel approaches improve the repartitioning quality significantly.

Structure of Supercritical Fluid Krypton at Small Scattering Angle Using Parallel Molecular Dynamics Simulation

We present a parallel algorithm for molecular dynamics involving short-range two- and three-body potentials and the pair-correlation function, g(r). The method is based on a spatial decomposition of the simulation box that takes advantage of a linked-cell list, and allows a load balanced partition of the computations of both the forces and g(r) over the processors. The tests of the program is conducted by evaluating the efficiency for both the thermalization phase and the production phase of the simulation. This method is successfully applied to the calculation of the direct correlation function of fluid krypton at small scattering angle along the T = 297 K supercritical isotherm.

A Computation+Communication Load Balanced Loop Partitioning Method for Distributed Memory Systems

Due to a significant communication overhead of sending and receiving data, the loop partitioning approaches on distributed memory systems must guarantee not just the computation load balance but computation+communication load balance. The previous approaches in loop partitioning have achieved a communication-free, computation load balanced iteration space partitioning solution for a limited subset of DOALL loops. But a large category of DOALL loops inevitably result in communication and the trade-offs between computation and communication must be carefully analyzed for these loops in order to balance out the combined computation time and communication overheads. In this work, we describe a partitioning approach based on the above motivation for the general cases of DOALL loops. Our goal is to achieve a computation+communication load balanced partitioning through static data and iteration space distribution. Our approach first performs partitioning of iteration and data spaces of a loop nest by analyzing communication and parallelism; it then performs architecture-dependent analysis to adjust the granularity of partitions, load balance each partition with respect to total computation+communication, and then performs mapping of partitions onto the available number of processors. This multiphase partitioning method works as follows. First, the code partitioning phase analyzes the references in the body of the DOALL loop nest and determines a set of directions for reducing a larger degree of communication by trading a lesser degree of parallelism. The partitioning is carried out in the iteration space of the loop by cyclically following a set of direction vectors such that the data references are maximally localized and reused, eliminating a larger communication volume than parallelism. We then perform data space partitioning based on a new larger partition owns rule to minimize the communication overhead for a compute intensive partition by localizing its references relatively more than a smaller noncompute intensive partition. A partition interaction graph is then constructed which is used by the architecture-dependent analysis phase to merge the partitions to achieve granularity adjustment, computation+communication load balance, and mapping on the actual number of available processors. Relevant theory and algorithms are developed along with a performance evaluation on the Cray T3D.

Matrix partitioning on a virtual shared memory parallel machine

The general problem considered in the paper is partitioning of a matrix operation between processors of a parallel system in an optimum load-balanced way without potential memory contention. The considered parallel system is defined by several features the main of which is availability of a virtual shared memory divided into segments. If partitioning of a matrix operation causes parallel access to the same memory segment with writing data to the segment by at least one processor, then contention between processors arises which implies performance degradation. To eliminate such situation, a restriction is imposed on a class of possible partitionings, so that no two processors would write data to the same segment. On the resulting class of contention-free partitionings, a load-balanced optimum partitioning is defined as satisfying independent minimax criteria. The main result of the paper is an algorithm for finding the optimum partitioning by means of analytical solution of respective minimax problems. The paper also discusses implementation and performance issues related to the algorithm, on the basis of experience at Kendall Square Research Corporation, where the partitioning algorithm was used for creating high-performance parallel matrix libraries.