Parallel Overhead Research Articles

Flicker Propagation in Power Networks with Hybrid and Parallel Overhead Transmission Lines

Flicker Propagation in Power Networks with Hybrid and Parallel Overhead Transmission Lines

Characterization of Shared-Memory Multi-Core Applications

The multicore processor architectures have been gaining increasing popularity in the recent years. However, many available applications cannot take full advantage of these architectures. Therefore, many researchers have developed several characterization techniques to help programmers understand the behavior of these applications on multicore platforms and to tune them for better efficiency. This paper proposes an on-the-fly, configuration-independent characterization approach for characterizing the inherent characteristics of multicore applications. This approach is fast because it does not depend on the details of any specific machine configuration and does not require repeating the characterization for every target configuration. It just keeps track of memory accesses and the cores that perform these accesses through piping memory traces, on-the-fly, to the analysis tool. We applied this approach to characterize eight applications drawn from SPLASH-2 and PARSEC benchmark suites. This paper presents the inherent characteristics of these applications including memory access instructions, communication characteristics patterns, sharing degree, invalidation degree, communication slack, and communication locality. The results show that two of the studied applications have high parallelization overhead, which are Cholesky and Fluidanimate. The results also indicate that studied applications of SPLASH-2 have higher communication rates than the studied applications of PARSEC and these rates generally increase as the number of used threads increases. Most of the sharing and invalidation occurs in small degrees. However, two of SPLASH-2 applications have significant fraction of communication with high sharing degrees involving four or more threads. Most of the applications have some uniform communication component and the initial thread is generally involved in more communication compared to the other threads.

Tuning the victim selection policy of Intel TBB

The wide adoption of Chip Multiprocessors (CMPs) in almost all ICT segments has triggered a change in the way software needs to be developed. Parallel programming maximizes the performance and energy efficiency of CMPs, but also comes with a new set of challenges. Parallelization overheads can account for sub-linear speedups and can increase the energy consumption of applications. In past experiments we looked at specific operations such as spawning new tasks, dequeuing the task queue and task stealing for Intel TBB. Our results showed that failed steals account for the largest overhead. In this work, we focus on TBB's victim selection policy. We implement a new occupancy-aware policy and we improve the implementation of the pseudo-random policy we proposed in a previous paper. We compare the results of our new policies against an oracle scheme as well as against TBB's random victim selection approach. Our results show improvements in execution times and energy-efficiency of up to 11.23% and 14.72% respectively when compared to TBB's default policy.

SIMD parallel MCMC sampling with applications for big-data Bayesian analytics

Computational intensity and sequential nature of estimation techniques for Bayesian methods in statistics and machine learning, combined with their increasing applications for big data analytics, necessitate both the identification of potential opportunities to parallelize techniques such as Monte Carlo Markov Chain (MCMC) sampling, and the development of general strategies for mapping such parallel algorithms to modern CPUs in order to elicit the performance up the compute-based and/or memory-based hardware limits. Two opportunities for Single-Instruction Multiple-Data (SIMD) parallelization of MCMC sampling for probabilistic graphical models are presented. In exchangeable models with many observations such as Bayesian Generalized Linear Models (GLMs), child-node contributions to the conditional posterior of each node can be calculated concurrently. In undirected graphs with discrete-value nodes, concurrent sampling of conditionally-independent nodes can be transformed into a SIMD form. High-performance libraries with multi-threading and vectorization capabilities can be readily applied to such SIMD opportunities to gain decent speedup, while a series of high-level source-code and runtime modifications provide further performance boost by reducing parallelization overhead and increasing data locality for Non-Uniform Memory Access architectures. For big-data Bayesian GLM graphs, the end-result is a routine for evaluating the conditional posterior and its gradient vector that is 5 times faster than a naive implementation using (built-in) multi-threaded Intel MKL BLAS, and reaches within the striking distance of the memory-bandwidth-induced hardware limit. Using multi-threading for cache-friendly, fine-grained parallelization can outperform coarse-grained alternatives which are often less cache-friendly, a likely scenario in modern predictive analytics workflow such as Hierarchical Bayesian GLM, variable selection, and ensemble regression and classification. The proposed optimization strategies improve the scaling of performance with number of cores and width of vector units (applicable to many-core SIMD processors such as Intel Xeon Phi and Graphic Processing Units), resulting in cost-effectiveness, energy efficiency (‘green computing’), and higher speed on multi-core x86 processors.

Synchronous parallel Kinetic Monte Carlo: Implementation and results for object and lattice approaches

An adaptation of the synchronous parallel Kinetic Monte Carlo (spKMC) algorithm developed by Martinez et al. (2008) to the existing KMC code MMonCa (Martin-Bragado et al. 2013) is presented in this work. Two cases, general enough to provide an idea of the current state-of-the-art in parallel KMC, are presented: Object KMC simulations of the evolution of damage in irradiated iron, and Lattice KMC simulations of epitaxial regrowth of amorphized silicon. The results allow us to state that (a) the parallel overhead is critical, and severely degrades the performance of the simulator when it is comparable to the CPU time consumed per event, (b) the balance between domains is important, but not critical, (c) the algorithm and its implementation are correct and (d) further improvements are needed for spKMC to become a general, all-working solution for KMC simulations.

A Two-Level Multithreaded Delaunay Kernel

This paper presents a fine grain parallel version of the 3D Delaunay Kernel procedure using the OpenMP (Open Multi-Processing) API. A set S = {p1,., pn} of n points is taken as input. S is initially sorted along a space-filling curve so that two points that are close in the insertion order are also close geometrically. The sorted set of point is then divided into M subsets Si, 1 ≤ i ≤ M of equal size n/M. The multithreaded version of the Delaunay kernel inserts M points at a time in the triangulation. OpenMP barriers provide the required synchronization that is needed after each multiple insertion in order to avoid data races. This simple approach exhibits two standard problems of parallel computing: load imbalance and parallel overheads. Those two issues are addressed using a two-level version of the multithreaded Delaunay kernel. Tests show that triangulations of about a billion tetrahedra can be generated on a 32 core machine (Intel Xeon E5-4610 v2 @ 2.30GHz with with 128 GB of memory) in less that 3minutes of wall clock time, with a speedup of 18 compared to the single-threaded implementation.

Implementation of a Parallel XTS Encryption Mode of Operation

Security of data stored inside storage devices is becoming one of the main issues in computer security now. It is known that the most efficient techniques to protect storage devices are using cryptography. Developing newer and more secure encryption algorithms and modes of operation might be critically important to protect these devices since conventional disk encryption algorithms, such as CBC mode, have shown serious security flaws. In this paper, the IEEE XTS encryption mode of operation for storage encryption (P1619 standard) has been implemented using parallel design. A performance comparison between the sequential and parallel algorithms of XTS mode is presented here. Parallel overheads that prevented from achieving perfect linear speedup are measured and minimized. The parallel XTS algorithm has shown a speedup of 1.80, with 90% efficiency, faster than the sequential algorithm. In these simulations, AES is used as encryption algorithm with 256-bit encryption key. Keywords: XTS Mode, Disk Encryption, Encryption Modes, Parallel Processing

Implementation of a parallel XTS encryption mode of operation

Security of data stored inside storage devices is becoming one of the main issues in computer security now. It is known that the most efficient techniques to protect storage devices are using cryptography. Developing newer and more secure encryption algorithms and modes of operation might be critically important to protect these devices since conventional disk encryption algorithms, such as CBC mode, have shown serious security flaws. In this paper, the IEEE XTS encryption mode of operation for storage encryption (P1619 standard) has been implemented using parallel design. A performance comparison between the sequential and parallel algorithms of XTS mode is presented here. Parallel overheads that prevented from achieving perfect linear speedup are measured and minimized. The parallel XTS algorithm has shown a speedup of 1.80, with 90% efficiency, faster than the sequential algorithm. In these simulations, AES is used as encryption algorithm with 256-bit encryption key.

Exploiting Thread-Level Parallelism Based on Balancing Load for Speculative Multithreading

Speculative Multithreading (SpMT) technology is an effective mechanism for automatic parallelization of irregular programs. While speculative parallelization can potentially deliver significant speedup for irregular applications, several speculative parallelization overheads resulting from the factors, especially inter-thread load imbalance, limit these speedups in practice. Most existing thread partitioning methods are mainly based on heuristic rules strategies to generate speculative threads. However, these heuristic rules extracted from people’s experiences cannot estimate quantitatively but qualitatively the overhead. Based on the thorough analysis of the speculative parallelization overhead resulting from inter-thread load imbalance, we propose a novel method to balance the inter-thread load. In this method, we firstly determine a method by which to unroll the loops. Then, we introduce cluster method to search the solution space of speculative threads and finally get the optimal solution. The experimental results show that, the proposed method can effectively reduce the inter-thread load imbalance; and the load imbalance overhead can be effectively reduced by 43.7%. And we can gain 8.8% performance improvement on Olden benchmark suits.

Power flow for general mixed distribution networks

The contemporary concept of Smart Distribution Grid motivates researchers to develop more and more efficient Distribution Management System (DMS) as the basic platform of this concept. This paper deals with power flow as the basic DMS calculation. Procedures for this calculation are usually classified as branch oriented [back/forward sweeping (BFS)] and bus oriented [current injection and Newton/Raphson derivatives]. The first class is favored in this paper. An industry grade generalized BFS (GBFS) procedure is developed for power flow calculation of unbalanced distribution networks. The most complex distribution networks—“irregular” mixed networks—are particularly stressed. In addition, a very simple dealing with inductive and capacitive couplings between parallel overhead lines is presented. It is proven that GBFS is superior in comparison with other power flow procedures tailored for distribution networks—its robustness is at least equal to the robustness of other procedures, but its efficiency is significantly higher. Copyright © 2014 John Wiley & Sons, Ltd.

Optimal Task Mapping for NEMO Model

The climate numerical models require a considerable amount of computing power. The modern parallel architectures provide the needed computing power to perform scientific simulations at acceptable resolutions. However, the efficiency of the exploitation of the parallel architectures by the climate models is often poor. Several factors influence the parallel efficiency such as the parallel overhead due to the communications among concurrent tasks, the memory contention among tasks on the same computing node, the load balancing and the tasks synchronization. The work here described aims at addressing two of the factors influencing the efficiency: the communications and the memory contention. The used approach is based on the optimal mapping of the tasks on the SMP nodes of a parallel cluster. The best mapping can heavily influence the time spent for communications between tasks belonging to the same node either to different nodes. Moreover, if we consider that each parallel task will allocate different amount of memory, the optimal tasks mapping can balance the total amount of main memory allocated on the same node and hence reduce the overall memory contention. The climate model taken into consideration is PELAGOS025 made by coupling the NEMO oceanic model with the BFM biogeochemical model. It has been used in a global configuration with a horizontal resolution of 0.25◦. Three different mapping strategies have been implemented, analyzed and compared with the standard allocation performed by the local scheduler. The parallel architecture used for the evaluation is an IBM iDataPlex with Intel SandyBridge processors located at the CMCC’s Supercomputing Center.

Limits of Parallelism and Boosting in Dim Silicon

Supply-voltage scaling has stagnated in recent technology nodes, leading to so-called dark silicon. To increase overall chip multiprocessor (CMP) performance, it is necessary to improve the energy efficiency of individual tasks so that more tasks can be executed simultaneously within thermal limits. In this article, the authors investigate the limit of voltage scaling together with task parallelization to maintain task completion latency while reducing energy consumption. Additionally, they examine improvements in energy efficiency and parallelism when serial portions of code can be overcome through quickly boosting a core's operating voltage. When accounting for parallelization overheads, minimum task energy is obtained at near-threshold supply voltages across six commercial technology nodes and provides 4× improvement in overall CMP performance. Boosting is most effective when the task is modestly parallelizable but not highly parallelizable.

Effects of pantograph arcing on railway systems with auto transformers

Conducted and radiated electromagnetic interferences on auto transformer (AT) caused by pantograph arcing in electrified railway system are investigated. The overhead lines of electrified railway system can be simplified by three parallel overhead conductor lines. The model of AT in this paper is situated at the end of lines. The propagation of electromagnetic waves along the three conductors is solved by using finite-difference time-domain (FDTD) method. The circuit model for AT is created by state variable equations, and the equations are discretized by backward Euler method. The pantograph arcing is represented by a trapezoidal voltage pulse which has a wide band of frequency just like an arcing has; the rise and fall time are determined by the main frequency. The voltages across the AT windings are computed by the final FDTD iterative equations when all factors are considered.

Finite-element-wise domain decomposition iterative solvers with polynomial preconditioning

A highly scalable finite-element-wise domain decomposition (EDD) iterative solver (with particular emphasis on the flexible GMRES method) based on polynomial preconditioning is discussed in this paper. Compared to alternative methods, finite-element-wise domain decomposition solvers with polynomial preconditioning circumvent the assembly of the matrix, reordering of the degree-of-freedom, redundant computations associated with the interface elements, numerical problems associated with the local preconditioner, and costly global preconditioner construction. A dramatic reduction in parallel overhead is achieved, both in terms of computation and communication results, in a highly scalable solver. This work is implemented using MPI C++and applied to solve static and dynamic structural mechanics problems. The corresponding numerical performance, with emphasis on iterative convergence and parallel scalability, of the proposed approach over the SGI Origin machine is critically assessed.

Modal Domain-Based Modeling of Parallel Transmission Lines With Emphasis on Accurate Representation of Mutual Coupling Effects

Transient and harmonic coupling effects between parallel overhead lines are normally modeled via frequency-dependent traveling-wave-type transmission-line models. Several electromagnetic transient programs rely on a line model based on a constant transformation matrix and modes (FD-line). These line models can, however, give substantial errors in the studies of coupled disturbance from one line to a neighbor line. In this paper, we show that the accuracy of the FD-line in these applications can be greatly improved by representing the line system by independent FD-lines in combination with rational models that take the mutual coupling between the lines into account. A mode-revealing transformation is used for further enhancing the accuracy of the coupling effect. The approach is demonstrated for the simulation of transient coupling between two parallel overhead lines caused by line energization and for assessing the disturbance of a railway signaling system from a neighboring overhead line caused by a transformer inrush current and fault transients.

OpenMP-based parallel transient stability simulation for large-scale power systems

This paper provides an OpenMP-based parallel Very Dishonest Newton (VDHN) algorithm with variable step size, running transient stability simulations (TSA) on multi-core computers. Under the framework of simultaneous solution method of TSA, the step-size control strategy is used according to the local truncation error theory firstly. Then, computation of the generation units, which is the most time-consuming part of the simulation, is dynamically dispatched to several cores using an α dynamic scheduling scheme to obtain workload balancing based on OpenMP. Due to the convergence of Newton-type iterations, an adaptive Jacobian update control strategy is applied to reduce the sequential part of the simulation and the overhead generated by OpenMP. Several large scale test cases verify the validity and practicability of the proposed parallel algorithm, showing that the proposed approach achieves high speed-up and a considerable reduction in parallel overheads.

TBBench: A Micro-Benchmark Suite for Intel Threading Building Blocks

Task-based programming is becoming the state-of-the-art method of choice for extracting the desired performance from multi-core chips. It expresses a program in terms of lightweight logical tasks rather than heavyweight threads. Intel Threading Building Blocks (TBB) is a task-based parallel programming paradigm for multi-core processors. The performance gain of this paradigm depends to a great extent on the efficiency of its parallel constructs. The parallel overheads incurred by parallel constructs determine the ability for creating large-scale parallel programs, especially in the case of fine-grain parallelism. This paper presents a study of TBB parallelization overheads. For this purpose, a TBB micro-benchmarks suite called TBBench has been developed. We use TBBench to evaluate the parallelization overheads of TBB on different multi-core machines and different compilers. We report in detail in this paper on the relative overheads and analyze the running results.

Generalizing Amdahl's Law for Power and Energy

Extending Amdahl's law to identify optimal power-performance configurations requires considering the interactive effects of power, performance, and parallel overhead.

Fast $\ell_1$-SPIRiT Compressed Sensing Parallel Imaging MRI: Scalable Parallel Implementation and Clinically Feasible Runtime

We present l₁-SPIRiT, a simple algorithm for auto calibrating parallel imaging (acPI) and compressed sensing (CS) that permits an efficient implementation with clinically-feasible runtimes. We propose a CS objective function that minimizes cross-channel joint sparsity in the wavelet domain. Our reconstruction minimizes this objective via iterative soft-thresholding, and integrates naturally with iterative self-consistent parallel imaging (SPIRiT). Like many iterative magnetic resonance imaging reconstructions, l₁-SPIRiT's image quality comes at a high computational cost. Excessively long runtimes are a barrier to the clinical use of any reconstruction approach, and thus we discuss our approach to efficiently parallelizing l₁-SPIRiT and to achieving clinically-feasible runtimes. We present parallelizations of l₁-SPIRiT for both multi-GPU systems and multi-core CPUs, and discuss the software optimization and parallelization decisions made in our implementation. The performance of these alternatives depends on the processor architecture, the size of the image matrix, and the number of parallel imaging channels. Fundamentally, achieving fast runtime requires the correct trade-off between cache usage and parallelization overheads. We demonstrate image quality via a case from our clinical experimentation, using a custom 3DFT spoiled gradient echo (SPGR) sequence with up to 8× acceleration via Poisson-disc undersampling in the two phase-encoded directions.

Parallel anisotropic mesh adaptivity with dynamic load balancing for cardiac electrophysiology

Simulations in cardiac electrophysiology generally use very fine meshes and small time steps to resolve highly localized wavefronts. This expense motivates the use of mesh adaptivity, which has been demonstrated to reduce the overall computational load. However, even with mesh adaptivity performing such simulations on a single processor is infeasible. Therefore, the adaptivity algorithm must be parallelised. Rather than modifying the sequential adaptive algorithm, the parallel mesh adaptivity method introduced in this paper focuses on dynamic load balancing in response to the local refinement and coarsening of the mesh. In essence, the mesh partition boundary is perturbed away from mesh regions of high relative error, while also balancing the computational load across processes. The parallel scaling of the method when applied to physiologically realistic heart meshes is shown to be good as long as there are enough mesh nodes to distribute over the available parallel processes. It is shown that the new method is dominated by the cost of the sequential adaptive mesh procedure and that the parallel overhead of inter-process data migration represents only a small fraction of the overall cost.