Synchronization Cost Research Articles

Methodology of Effective Task Planning and Algorithm for Multivariate Computation of the Characteristics of a Quantum Rotation Sensor on a Hybrid Supercomputer Cluster

The problem of planning supercomputer computations is considered using the example of computing the characteristics of a quantum rotation sensor model. A methodology for applying the task scheduling algorithm is proposed. For this purpose, time estimates were obtained for computations at cluster nodes. The possibility of dividing settlements by cluster nodes without additional synchronization costs is shown. A start-up algorithm for improving the efficiency of multivariate computations is presented, and the efficiency of the planning algorithm in its use is estimated.

Modeling and intelligent optimization of social collective behavior with online public opinion synchronization

Today, the public opinion synchronization on the network platform is becoming one of important issues worthy of careful study. In this paper, we take synchronization evolution phenomenon as an objective and adopt artificial bee colony (ABC) to evaluate network synchronization effects with optimization theory to find out an appropriate network structure. Firstly, we use the Kuramoto oscillators as a metaphor of the social system collective behavior. Secondly, combined with the social network characteristics obtained from the data of Sina Micro-Blog, a synchronization evolution model of Internet public opinion based on Kuramoto one is established. Subsequently, evolutionary multi-objective optimization model is set up and the ABC method is used to optimize the level of network synchronization, the synchronization starting time and the cost of public opinion synchronization. Finally, case analysis on “Double Eleven” Internet Marketing as well as Cadmium Poisoned Rice Event demonstrates that the synchronization performance of weak coupled system can be enhanced by offering reasonable configuration of the connection cost and the synchronization duration cost. In addition, a certain degree of increase in input cost can promote the synchronization performance and extend the synchronization duration significantly.

On the Semantics of Distributed Reactive Programming: The Cost of Consistency

The reactive programming paradigm aims to simplify the development of reactive systems. It provides abstractions to define time-changing values that are automatically updated by the runtime according to their dependencies. The benefits of reactive programming in distributed settings have been recognized for long. Yet, existing solutions for distributed reactive programming enforce the same semantics as in single processes, introducing communication and synchronization costs that hamper scalability. Establishing suitable abstractions for distributed reactive programming demands for a deeper investigation of the semantics of change propagation. This paper takes a foundational approach and defines precise propagation semantics in terms of consistency guarantees that constrain the order and isolation of value updates. We study the benefits and costs of these consistency guarantees both theoretically and empirically, using case studies and synthetic benchmarks. We show that different applications require different levels of consistency and that manually implementing the required level on a middleware that provides a lower one annuls the abstraction improvements of reactive programming. This motivates a framework that enables the developers to select the best trade-off between consistency and overhead for the problem at hand. To this end, we present DREAM, a distributed reactive programming middleware with flexible consistency guarantees.

Time Synchronization Algorithm for Wireless Sensor Networks Based on Frequency Offset Estimation

Aiming at solving the poor the classical synchronous algorithm stability in wireless sensor network and high overhead of clock phase offset and frequency offset, a synchronization algorithm (CSMS algorithm) was designed for wireless sensor networks based on frequency offset estimation. The CSMS algorithm used the low overhead phase bias and frequency offset estimation method to improve the synchronization accuracy and stability of the pair nodes. At the same time, a synchronization strategy was built based on layering and broadcast monitoring, which ensured the stability and synchronization accuracy of the algorithm, realized the synchronization with neighbor nodes and root nodes, and optimized the total synchronization cost. Among them, the CSMS algorithm was mainly divided into two stages: level discovery phase, which was used for generating a layered structure of network; synchronization phase, used to estimate clock offset and frequency offset between pairs of nodes. The experimental results showed that the CSMS algorithm can effectively balance the synchronization energy consumption, synchronization accuracy and synchronization stability. As a result, it is summed up that dynamic adjustment of the nodes clock deviation is realized, the long-term stability of synchronization is ensured, and the precision of synchronization is improved.

FBSGraph: Accelerating Asynchronous Graph Processing via Forward and Backward Sweeping

Graph algorithm is pervasive in many applications ranging from targeted advertising to natural language processing. Recently, Asynchronous Graph Processing (AGP) is becoming a promising model to support graph algorithm on large-scale distributed computing platforms because it enables faster convergence speed and lower synchronization cost than the synchronous model for no barrier between iterations. However, existing AGP methods still suffer from poor performance for inefficient vertex state propagation. In this paper, we propose an effective and low-cost forward and backward sweeping execution method to accelerate state propagation for AGP, based on a key observation that states in AGP can be propagated between vertices much faster when the vertices are processed sequentially along the graph path within each round. Through dividing graph into paths and asynchronously processing vertices on each path in an alternative forward and backward way according to their order on this path, vertex states in our approach can be quickly propagated to other vertices and converge in a faster way with only little additional overhead. In order to efficiently support it over distributed platforms, we also propose a scheme to reduce the communication overhead along with a static priority ordering scheme to further improve the convergence speed. Experimental results on a cluster with 1,024 cores show that our approach achieves excellent scalability for large-scale graph algorithms and the overall execution time is reduced by at least 39.8 percent, in comparison with the most cutting-edge methods.

Distributed Newton Methods for Deep Neural Networks.

Deep learning involves a difficult nonconvex optimization problem with a large number of weights between any two adjacent layers of a deep structure. To handle large data sets or complicated networks, distributed training is needed, but the calculation of function, gradient, and Hessian is expensive. In particular, the communication and the synchronization cost may become a bottleneck. In this letter, we focus on situations where the model is distributedly stored and propose a novel distributed Newton method for training deep neural networks. By variable and feature-wise data partitions and some careful designs, we are able to explicitly use the Jacobian matrix for matrix-vector products in the Newton method. Some techniques are incorporated to reduce the running time as well as memory consumption. First, to reduce the communication cost, we propose a diagonalization method such that an approximate Newton direction can be obtained without communication between machines. Second, we consider subsampled Gauss-Newton matrices for reducing the running time as well as the communication cost. Third, to reduce the synchronization cost, we terminate the process of finding an approximate Newton direction even though some nodes have not finished their tasks. Details of some implementation issues in distributed environments are thoroughly investigated. Experiments demonstrate that the proposed method is effective for the distributed training of deep neural networks. Compared with stochastic gradient methods, it is more robust and may give better test accuracy.

GPU-accelerated Gibbs sampling: a case study of the Horseshoe Probit model

Gibbs sampling is a widely used Markov chain Monte Carlo (MCMC) method for numerically approximating integrals of interest in Bayesian statistics and other mathematical sciences. Many implementations of MCMC methods do not extend easily to parallel computing environments, as their inherently sequential nature incurs a large synchronization cost. In the case study illustrated by this paper, we show how to do Gibbs sampling in a fully data-parallel manner on a graphics processing unit, for a large class of exchangeable models that admit latent variable representations. Our approach takes a systems perspective, with emphasis placed on efficient use of compute hardware. We demonstrate our method on a Horseshoe Probit regression model and find that our implementation scales effectively to thousands of predictors and millions of data points simultaneously.

Argobots: A Lightweight Low-Level Threading and Tasking Framework

In the past few decades, a number of user-level threading and tasking models have been proposed in the literature to address the shortcomings of OS-level threads, primarily with respect to cost and flexibility. Current state-of-the-art user-level threading and tasking models, however, either are too specific to applications or architectures or are not as powerful or flexible. In this paper, we present Argobots, a lightweight, low-level threading and tasking framework that is designed as a portable and performant substrate for high-level programming models or runtime systems. Argobots offers a carefully designed execution model that balances generality of functionality with providing a rich set of controls to allow specialization by end users or high-level programming models. We describe the design, implementation, and performance characterization of Argobots and present integrations with three high-level models: OpenMP, MPI, and colocated I/O services. Evaluations show that (1) Argobots, while providing richer capabilities, is competitive with existing simpler generic threading runtimes; (2) our OpenMP runtime offers more efficient interoperability capabilities than production OpenMP runtimes do; (3) when MPI interoperates with Argobots instead of Pthreads, it enjoys reduced synchronization costs and better latency-hiding capabilities; and (4) I/O services with Argobots reduce interference with colocated applications while achieving performance competitive with that of a Pthreads approach.

A resource efficient framework to run automotive embedded software on multi-core ECUs

The increasing functionality and complexity of automotive applications requires not only the use of more powerful hardware, e.g., multi-core processors, but also efficient methods and tools to support design decisions. Component-based software engineering proved to be a promising solution for managing software complexity and allowing for reuse. However, there are several challenges inherent in the intersection of resource efficiency and predictability of multi-core processors when it comes to running component-based embedded software. In this paper, we present a software design framework addressing these challenges. The framework includes both mapping of software components onto executable tasks, and the partitioning of the generated task set onto the cores of a multi-core processor. This paper aims at enhancing resource efficiency by optimizing the software design with respect to: 1) the inter-software-components communication cost, 2) the cost of synchronization among dependent transactions of software components, and 3) the interaction of software components with the basic software services. An engine management system, one of the most complex automotive sub-systems, is considered as a use case, and the experimental results show a reduction of up to 11.2% total CPU usage on a quad-core processor, in comparison with the common framework in the literature.

Easy PRAM-Based High-Performance Parallel Programming with ICE

Parallel machines have become more widely used. Unfortunately parallel programming technologies have advanced at a much slower pace except for regular programs. For irregular programs, this advancement is inhibited by high synchronization costs, non-loop parallelism, non-array data structures, recursively expressed parallelism and parallelism that is too fine-grained to be exploitable. We present ICE, a new parallel programming language that is easy-to-program, since: (i) ICE is a synchronous, lock-step language so there is no need for programmer-specified synchronization; (ii) for a PRAM algorithm its ICE program amounts to directly transcribing it; and (iii) the PRAM algorithmic theory offers unique wealth of parallel algorithms and techniques. We propose ICE to be a part of an ecosystem consisting of the XMT architecture, the PRAM algorithmic model, and ICE itself, that together deliver on the twin goal of easy programming and efficient parallelization of irregular programs. The XMT architecture, developed at UMD, can exploit fine-grained parallelism in irregular programs. We have built the ICE compiler which translates the ICE language into the multithreaded XMTC language; the significance of this is that multi-threading is a feature shared by practically all current scalable parallel programming languages thus providing a method to compile ICE code. As one indication of ease of programming, we observed a reduction in code size in 11 out of 16 benchmarks as compared to hand-optimized XMTC. For these programs, the average reduction in number of lines of code was 35.5 percent. The remaining 5 benchmarks had almost the same code size for both ICE and hand-optimized XMTC. Our main result is perhaps surprising: The run-time was comparable to XMTC with a 0.53 percent average gain for ICE across all benchmarks.

Non-clairvoyant online scheduling of synchronized jobs on virtual clusters

Although virtualization technology is recently applied to next-generation distributed high-performance computing systems, theoretical aspects of scheduling jobs on these virtualized environments are not sufficiently studied, especially in online and non-clairvoyant cases. Virtualization of computing resources results in interference and virtualization overheads that negatively impact the load balancing objectives on commonly used cluster of multi-core physical machines. We present a technique for non-clairvoyant online scheduling of globally synchronized jobs, each of which spawns tasks to execute compute-intensive works. Our technique considers both load balancing of physical cores and per job synchronization cost minimization. We show that in the presence of arbitrary virtualization overheads, interference effects and synchronization cost, the problem can be reduced to an online unrelated parallel machine scheduling, which is solved using routing of virtual circuits. We present a new opportunity cost model to reduce the problem to the routing of virtual circuits and prove the effectiveness of our scheduling technique using mathematical analysis and simulative experiments.

Learning-based synchronous approach from forwarding nodes to reduce the delay for Industrial Internet of Things

The Industrial Internet of Things (IIoTs) is creating a new world which incorporates machine learning, sensor data, and machine-to-machine (M2M) communications. In IIoTs, the length of the transmission delay is one of the pivotal performance because dilatory communication will cause heavy losses to industrial applications. In this paper, a learning-based synchronous (LS) approach from forwarding nodes is proposed to reduce the delay for IIoTs. In an asynchronous Media Access Control protocol, when senders need to send data, they always require to wait for their corresponding receiver to wake up. Thus, the delay here is greater than in the synchronous network. However, the synchronization cost of the whole network is enormous, and it is difficult to maintain. Therefore, LS mechanism uses a partial synchronization approach to reduce synchronization costs while effectively reducing delay. In LS approach, instead of synchronizing the nodes in the entire network, only sender nodes and part of the nodes in their forwarding node set are synchronized by self-learning methods, and accurate synchronization is not required here. Thus, the delay can be effectively reduced under the low cost. Secondly, the nodes near sink maintain the original duty cycle, while the nodes in the regions away from the sink use their remaining energy and perform synchronization operations, so as not to damage the network lifetime. Finally, because the synchronization in this paper is based on different synchronization periods among different nodes, it can improve the network performance by reducing the conflict between simultaneous data transmission. The theoretical analysis results show that compared with the previous approach FFSC, LS approach can reduce the end-to-end delay by 5.13–11.64% and increase the energy efficiency by 14.29–17.53% under the same lifetime with a more balanced energy utilization.

A traffic monitoring stream-based real-time vehicular offence detection approach

ABSTRACTTraffic offences are becoming increasingly serious as traffic volume increases rapidly in large cities, causing serious property damage and threatening public safety. Existing traffic monitoring systems lack the capability of detecting various types of offences in real-time. This paper proposes a novel monitoring stream-based vehicular offence detection algorithm, which discovers various types of offences from high-throughput traffic monitoring stream in real-time. An offence detecting and monitoring system is also designed and implemented. In order to achieve real-time detection, parallel computing techniques are utilized. An optimized data structure, a one producer-multiple consumer model and a re-hash strategy are proposed to reduce the synchronization cost incurred by multiple threads in the parallel implementation. Both real-world data and synthetic data are applied in the experiments. Experimental results demonstrate that the proposed algorithm is able to discover three types of offences from high-throughput traffic monitoring stream in real-time. Scalability is also observed. The experimental results indicate that the proposed system is sufficiently efficient to provide real-time offence detection for major metropolises.

Synchronization on Lie Groups: Coordination of Blind Agents

This paper presents an algorithm for synchronization of blind agents (agents are unable to observe other agents, i.e., no communication) evolving on a connected Lie group G. We employ the method of extremum seeking control for nonlinear dynamical systems defined on connected Riemannian manifolds to achieve synchronization among the agents. In this approach, each agent updates its position on G by only receiving the synchronization cost function. The results are obtained by employing the notion of geodesic dithers for extremum seeking on Riemannian manifolds and their equivalent version on Lie groups and applying Taylor expansion of smooth functions on Riemannian manifolds. Due to geometrical properties of the synchronization set, we employ the method of quotient manifolds to prove the convergence of the proposed algorithm. The obtained results are applied to synchronization problems on SE(3) to demonstrate the efficacy of the proposed algorithm.

Globally exponential synchronization criterion of chaot-ic oscillators using active control

This research paper focuses the globally exponential synchronization between two identical and two nonidentical chaotic oscillators. With the help of Lyapunov direct method and using the active control technique, suitable algebraic conditions are obtained analytically that establish the globally exponential synchronization. The proposed globally exponential synchronization criterion is more general and much less conservative than the previously published works. A comparison, based upon the synchronization speed, cost and quality have been performed with our study of the previously published results. The effect of unknown external disturbances has also been discussed. Numerical simulation results are presented to illustrate the performance and efficiency of this study.

Parallel personalized pagerank on dynamic graphs

Personalized PageRank (PPR) is a well-known proximity measure in graphs. To meet the need for dynamic PPR maintenance, recent works have proposed a local update scheme to support incremental computation. Nevertheless, sequential execution of the scheme is still too slow for highspeed stream processing. Therefore, we are motivated to design a parallel approach for dynamic PPR computation. First, as updates always come in batches, we devise a batch processing method to reduce synchronization cost among every single update and enable more parallelism for iterative parallel execution. Our theoretical analysis shows that the parallel approach has the same asymptotic complexity as the sequential approach. Second, we devise novel optimization techniques to effectively reduce runtime overheads for parallel processes. Experimental evaluation shows that our parallel algorithm can achieve orders of magnitude speedups on GPUs and multi-core CPUs compared with the state-of-the-art sequential algorithm.

Parallelizing Exact and Approximate String Matching via Inclusive Scan on a GPU

In this study, to substantially improve the runtimes of exact and approximate string matching algorithms, we propose a tribrid parallel method for bit-parallel algorithms such as the Shift-Or and Wu-Manber algorithms. Our underlying idea is to interpret bit-parallel algorithms as inclusive-scan operations, which allow these bit-parallel algorithms to run efficiently on a graphics processing unit (GPU); we achieve this speed-up here because inclusive-scan operations not only eliminate duplicate searches between threads but also realize a GPU-friendly memory access pattern that maximizes memory read/write throughput. To realize our ideas, we first define two binary operators and then present a proof regarding the associativity of these operators, which is necessary for the parallelization of the inclusive-scan operations. Finally, we integrate the inclusive-scan scheme into a previous segmentation-based scheme to maximize search throughput, identifying the best tradeoff point between synchronization cost and duplicate work. Through our experiments, we compared our proposed method with previous segmentation-based methods and indexing-based sequence aligners. For online string matching, our proposed method performed 6.7-16.7 times faster than previous methods, achieving a search throughput of up to 1.88 terabits per second (Tbps) on a GeForce GTX TITAN X GPU. We therefore conclude that our proposed method is quite effective for decreasing the runtimes of online string matching of short patterns.

Basker: Parallel sparse LU factorization utilizing hierarchical parallelism and data layouts

Abstract Transient simulation in circuit simulation tools, such as SPICE and Xyce, depend on scalable and robust sparse LU factorizations for efficient numerical simulation of circuits and power grids. As the need for simulations of very large circuits grow, the prevalence of multicore architectures enable us to use shared memory parallel algorithms for such simulations. A parallel factorization is a critical component of such shared memory parallel simulations. We develop a parallel sparse factorization algorithm that can solve problems from circuit simulations efficiently, and map well to architectural features. This new factorization algorithm exposes hierarchical parallelism to accommodate irregular structure that arise in our target problems. It also uses a hierarchical two-dimensional data layout which reduces synchronization costs and maps to memory hierarchy found in multicore processors. We present an OpenMP based implementation of the parallel algorithm in a new multithreaded solver called Basker in the Trilinos framework. We present performance evaluations of Basker on the Intel SandyBridge and Xeon Phi platforms using circuit and power grid matrices taken from the University of Florida sparse matrix collection and from Xyce circuit simulation. Basker achieves a geometric mean speedup of 5.91× on CPU (16 cores) and 7.4× on Xeon Phi (32 cores) relative to state-of-the-art solver KLU. Basker outperforms Intel MKL Pardiso solver (PMKL) by as much as 30× on CPU (16 cores) and 7.5× on Xeon Phi (32 cores) for low fill-in circuit matrices. Furthermore, Basker provides 5.4× speedup on a challenging matrix sequence taken from an actual Xyce simulation.

Phương pháp song song khai phá tập lợi ích cao dựa trên chỉ số hình chiếu

High utility itemsets (HUIs) mining is one of popular problems in data mining. Several parallel and sequential algorithms have been proposed in the literature to solve this problem. All the parallel algorithms to try reduce synchronization cost and caculation global profit of itemsets. In this paper, we present a parallel method for mining HUIs from projection-based indexing to speed up performance and reduce memory requirements. The experimental results show that the performance and number candidate of our algorithm is better than some non parallel algorithms.

New Synchronization Algorithm Based on Delta Synchronization for Compressed Files in the Mobile Cloud Environment

The fast growing market of mobile device adoption and cloud computing has led to exploitation of mobile devices utilizing cloud services. One major chal-lenge facing the usage of mobile devices in the cloud environment is mobile synchronization to the cloud, e.g., synchronizing contacts, text messages, imag-es, and videos. Owing to the expected high volume of traffic and high time complexity required for synchronization, an appropriate synchronization algo-rithm needs to be developed. Delta synchronization is one method of synchro-nizing compressed files that requires uploading the whole file, even when no changes were made or if it was only partially changed. In the present study, we proposed an algorithm, based on Delta synchronization, to solve the problem of synchronizing compressed files under various forms of modification (e.g., not modified, partially modified, or completely modified). To measure the effi-ciency of our proposed algorithm, we compared it to the Dropbox application algorithm. The results demonstrated that our algorithm outperformed the regular Dropbox synchronization mechanism by reducing the synchronization time, cost, and traffic load between clients and the cloud service provider.