Parallel Running Time Research Articles

SPARKs: Succinct Parallelizable Arguments of Knowledge

We introduce the notion of aSuccinct Parallelizable Argument of Knowledge(SPARK). This is an argument of knowledge with the following three efficiency properties for computing and proving a (non-deterministic, polynomial time) parallel RAM computation that can be computed in parallel timeTwith at mostpprocessors:—The prover’s (parallel) running time is\( T + \mathrm{poly}\hspace{-2.0pt}\log (T \cdot p) \). (In other words, the prover’s running time is essentiallyTfor large computation times!)—The prover uses at most\( p \cdot \mathrm{poly}\hspace{-2.0pt}\log (T \cdot p) \)processors.—The communication and verifier complexity are both\( \mathrm{poly}\hspace{-2.0pt}\log (T \cdot p) \).The combination of all three is desirable, as it gives a way to leverage a moderate increase in parallelism in favor of near-optimal running time. We emphasize that even a factor two overhead in the prover’s parallel running time is not allowed.Our main contribution is a generic construction of SPARKs from any succinct argument of knowledge where the prover’s parallel running time is\( T \cdot \mathrm{poly}\hspace{-2.0pt}\log (T \cdot p) \)when usingpprocessors, assuming collision-resistant hash functions. When suitably instantiating our construction, we achieve a four-round SPARK foranyparallel RAM computation assuming only collision resistance. Additionally assuming the existence of a succinctnon-interactiveargument of knowledge (SNARK), we construct a non-interactive SPARK that also preserves the space complexity of the underlying computation up to\( \mathrm{poly}\hspace{-2.0pt}\log (T\cdot p) \)factors.We also show the following applications of non-interactive SPARKs. First, they immediately imply delegation protocols with near optimal prover (parallel) running time. This, in turn, gives a way to construct verifiable delay functions (VDFs) from any sequential function. When the sequential function is also memory-hard, this yields the first construction of a memory-hard VDF.

Data stream fusion for accurate quantile tracking and analysis

UDDSketch is a recent algorithm for accurate tracking of quantiles in data streams, derived from the DDSketch algorithm. UDDSketch provides accuracy guarantees covering the full range of quantiles independently of the input distribution and greatly improves the accuracy with regard to DDSketch. In this paper we show how to compress and fuse two or more data streams (or datasets) by leveraging the mergeability of the UDDSketch data summaries. In general, two summaries on two data streams are said to be mergeable if there exists an algorithm that allows combining the two summaries into a single one related to the union of the two datasets, simultaneously preserving the error and size guarantees. The property of mergeability of a sketch enables the parallel and distributed processing of big volume data streams that can be compressed and fused by means of such mergeable data structures. Among the applications strictly related to accurate tracking of quantiles, requiring parallel and/or distributed processing we recall here estimating the latency of a web site, database query optimizers and the need of succinctly summarizing the distribution of values occurring over a sensor network. We prove that UDDSketch is fully mergeable and introduce PUDDSketch, a parallel version of UDDSketch suitable for message-passing based architectures. We formally prove its correctness and compare it to a parallel version of DDSketch, showing through extensive experimental results that our parallel algorithm almost always outperforms the parallel DDSketch algorithm with regard to the overall accuracy in determining the quantiles. Moreover, we also design and implement parallel versions of both the state of the art KLL and REQ sequential algorithms in order to compare and contrast PUDDSketch versus the corresponding parallel algorithms. Our experiments clearly show that PUDDSketch is faster or on par with regard to parallel running time, whilst providing simultaneously greater accuracy.

Theoretically and practically efficient parallel nucleus decomposition

This paper studies the nucleus decomposition problem, which has been shown to be useful in finding dense substructures in graphs. We present a novel parallel algorithm that is efficient both in theory and in practice. Our algorithm achieves a work complexity matching the best sequential algorithm while also having low depth (parallel running time), which significantly improves upon the only existing parallel nucleus decomposition algorithm (Sariyüce et al. , PVLDB 2018). The key to the theoretical efficiency of our algorithm is a new lemma that bounds the amount of work done when peeling cliques from the graph, combined with the use of a theoretically-efficient parallel algorithms for clique listing and bucketing. We introduce several new practical optimizations, including a new multi-level hash table structure to store information on cliques space-efficiently and a technique for traversing this structure cache-efficiently. On a 30-core machine with two-way hyper-threading on real-world graphs, we achieve up to a 55x speedup over the state-of-the-art parallel nucleus decomposition algorithm by Sariyüce et al. , and up to a 40x self-relative parallel speedup. We are able to efficiently compute larger nucleus decompositions than prior work on several million-scale graphs for the first time.

GPU Tensor Cores for Fast Arithmetic Reductions

This article proposes a parallel algorithm for computing the arithmetic reduction of n numbers as a set of matrix-multiply accumulate (MMA) operations that are executed simultaneously by GPU tensor cores. The analysis, assuming tensors of size m x m, shows that the proposed algorithm has a parallel running time of T(n) = 5log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m2</sub> n and a speedup of S = 45log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2m</sub> 2 over a canonical parallel reduction. Experimental performance results on a Tesla V100 GPU show that the tensor-core based approach is energy efficient and runs up to ~ 3:2× and 2× faster than a standard GPU-based reduction and Nvidia's CUB library, respectively, while keeping the numerical error below 1 percent with respect to a double precision CPU reduction. The chained design of the algorithm allows a flexible configuration of GPU thread-blocks and the optimal values found through experimentation agree with the theoretical ones. The results obtained in this work show that GPU tensor cores are relevant not only for Deep Learning or Linear Algebra computations, but also for applications that require the acceleration of large summations.

A precise non-asymptotic complexity analysis of parallel hash functions without tree topology constraints

A recent work shows how we can optimize a tree based mode of operation for a hash function where the sizes of input message blocks and digest are the same, subject to the constraint that the involved tree structure has all its leaves at the same depth. In this work, we show that we can further optimize the running time of such a mode by using a tree having leaves at all its levels. We make the assumption that the input message block has a size a multiple of that of the digest and denote by d the ratio block size over digest size. The running time is evaluated in terms of number of operations performed by the hash function, i.e. the number of calls to its underlying function. It turns out that a digest can be computed in ⌈logd+1(l∕2)⌉+2 evaluations of the underlying function using ⌈l∕2⌉ processors, where l is the number of blocks of the message. Other results of interest are discussed, such as the optimization of the parallel running time for a tree of restricted height.

Performance Evaluation of Parallel Bubble Sort Algorithm on Supercomputer IMAN1

Parallel sorting algorithms order a set of elements USING MULTIPLE processors in order to enhance the performance of sequential sorting algorithms. In general, the performance of sorting algorithms are EVALUATED IN term of algorithm growth rate according to the input size. In this paper, the running time, parallel speedup and parallel efficiency OF PARALLEL bubble sort is evaluated and measured. Message Passing Interface (MPI) IS USED for implementing the parallel version of bubble sort and IMAN1 supercomputer is used to conduct the results. The evaluation results show that parallel bubble sort has better running time as the number of processors increases. On other hand, regarding parallel efficiency, parallel bubble sort algorithm is more efficient to be applied OVER SMALL number of processors

A Recursive Hypergraph Bipartitioning Framework for Reducing Bandwidth and Latency Costs Simultaneously

Intelligent partitioning models are commonly used for efficient parallelization of irregular applications on distributed systems. These models usually aim to minimize a single communication cost metric, which is either related to communication volume or message count. However, both volume- and message-related metrics should be taken into account during partitioning for a more efficient parallelization. There are only a few works that consider both of them and they usually address each in separate phases of a two-phase approach. In this work, we propose a recursive hypergraph bipartitioning framework that reduces the total volume and total message count in a single phase. In this framework, the standard hypergraph models, nets of which already capture the bandwidth cost, are augmented with message nets . The message nets encode the message count so that minimizing conventional cutsize captures the minimization of bandwidth and latency costs together. Our model provides a more accurate representation of the overall communication cost by incorporating both the bandwidth and the latency components into the partitioning objective. The use of the widely-adopted successful recursive bipartitioning framework provides the flexibility of using any existing hypergraph partitioner. The experiments on instances from different domains show that our model on the average achieves up to $52$ percent reduction in total message count and hence results in $29$ percent reduction in parallel running time compared to the model that considers only the total volume.

Analysis of speedups in parallel evolutionary algorithms and [formula omitted] EAs for combinatorial optimization

Evolutionary algorithms are popular heuristics for solving various combinatorial problems as they are easy to apply and often produce good results. Island models parallelize evolution by using different populations, called islands, which are connected by a graph structure as communication topology. Each island periodically communicates copies of good solutions to neighboring islands in a process called migration. We consider the speedup gained by island models in terms of the parallel running time for problems from combinatorial optimization: sorting (as maximization of sortedness), shortest paths and Eulerian cycles. The results show in which settings and up to what degree evolutionary algorithms can be parallelized efficiently. Our results include offspring populations in (1+λ) EAs as a special case. Potential speedups depend on many design choices such as the search operators, representations and fitness functions used on the islands, and also the parameters of the island model. In particular, we show that a natural instance for Eulerian cycles leads to exponential vs. logarithmic speedups, depending on the frequency of migration.

Classifying Imbalanced Data Streams via Dynamic Feature Group Weighting with Importance Sampling.

Data stream classification and imbalanced data learning are two important areas of data mining research. Each has been well studied to date with many interesting algorithms developed. However, only a few approaches reported in literature address the intersection of these two fields due to their complex interplay. In this work, we proposed an importance sampling driven, dynamic feature group weighting framework (DFGW-IS) for classifying data streams of imbalanced distribution. Two components are tightly incorporated into the proposed approach to address the intrinsic characteristics of concept-drifting, imbalanced streaming data. Specifically, the ever-evolving concepts are tackled by a weighted ensemble trained on a set of feature groups with each sub-classifier (i.e. a single classifier or an ensemble) weighed by its discriminative power and stable level. The un-even class distribution, on the other hand, is typically battled by the sub-classifier built in a specific feature group with the underlying distribution rebalanced by the importance sampling technique. We derived the theoretical upper bound for the generalization error of the proposed algorithm. We also studied the empirical performance of our method on a set of benchmark synthetic and real world data, and significant improvement has been achieved over the competing algorithms in terms of standard evaluation metrics and parallel running time. Algorithm implementations and datasets are available upon request.

PANMIN: sequential and parallel global optimization procedures with a variety of options for the local search strategy

We present two sequential and one parallel global optimization codes, that belong to the stochastic class, and an interface routine that enables the use of the Merlin/MCL environment as a non-interactive local optimizer. This interface proved extremely important, since it provides flexibility, effectiveness and robustness to the local search task that is in turn employed by the global procedures. We demonstrate the use of the parallel code to a molecular conformation problem. Program summary Title of program: PANMIN Catalogue identifier: ADSU Program summary URL: http://cpc.cs.qub.ac.uk/summaries/ADSU Program obtainable from: CPC Program Library, Queen's University of Belfast, N. Ireland Computer for which the program is designed and others on which it has been tested: PANMIN is designed for UNIX machines. The parallel code runs on either shared memory architectures or on a distributed system. The code has been tested on a SUN Microsystems ENTERPRISE 450 with four CPUs, and on a 48-node cluster under Linux, with both the GNU g77 and the Portland group compilers. The parallel implementation is based on MPI and has been tested with LAM MPI and MPICH Installation: University of Ioannina, Greece Programming language used: Fortran-77 Memory required to execute with typical data: Approximately O( n 2) words, where n is the number of variables No. of bits in a word: 64 No. of processors used: 1 or many Has the code been vectorised or parallelized?: Parallelized using MPI No. of bytes in distributed program, including test data, etc.: 147163 No. of lines in distributed program, including the test data, etc.: 14366 Distribution format: gzipped tar file Nature of physical problem: A multitude of problems in science and engineering are often reduced to minimizing a function of many variables. There are instances that a local optimum does not correspond to the desired physical solution and hence the search for a better solution is required. Local optimization techniques can be trapped in any local minimum. Global Optimization is then the appropriate tool. For example, solving a non-linear system of equations via optimization, one may encounter many local minima that do not correspond to solutions, i.e. they are far from zero Method of solution: PANMIN is a suite of programs for Global Optimization that take advantage of the Merlin/MCL optimization environment [1,2]. We offer implementations of two algorithms that belong to the stochastic class and use local searches either as intermediate steps or as solution refinement Restrictions on the complexity of the problem: The only restriction is set by the available memory of the hardware configuration. The software can handle bound constrained problems. The Merlin Optimization environment must be installed. Availability of an MPI installation is necessary for executing the parallel code Typical running time: Depending on the objective function

Internal Diffusion-Limited Aggregation: Parallel Algorithms and Complexity

The computational complexity of internal diffusion-limited aggregation (DLA) is examined from both a theoretical and a practical point of view. We show that for two or more dimensions, the problem of predicting the cluster from a given set of paths is complete for the complexity class CC, the subset of P characterized by circuits composed of comparator gates. CC-completeness is believed to imply that, in the worst case, growing a cluster of size n requires polynomial time in n even on a parallel computer. A parallel relaxation algorithm is presented that uses the fact that clusters are nearly spherical to guess the cluster from a given set of paths, and then corrects defects in the guessed cluster through a nonlocal annihilation process. The parallel running time of the relaxation algorithm for two-dimensional internal DLA is studied by simulating it on a serial computer. The numerical results are compatible with a running time that is either polylogarithmic in n or a small power of n. Thus the computational resources needed to grow large clusters are significantly less on average than the worst-case analysis would suggest. For a parallel machine with k processors, we show that random clusters in d dimensions can be generated in $$\mathcal{O}$$ ((n/k+logk)n 2/d ) steps. This is a significant speedup over explicit sequential simulation, which takes $$\mathcal{O}$$ (n 1+2/d ) time on average. Finally, we show that in one dimension internal DLA can be predicted in $$\mathcal{O}$$ (logn) parallel time, and so is in the complexity class NC.

An improved systolic algorithm for the algebraic path problem

We systematically derive a systolic algorithm for the algebraic path problem (APP). We start with an algorithm with (parallel) running time 3 n, but whose data dependencies are not local. The first step in obtaining a systolic algorithm is to localize these dependencies. The standard localization (used by all authors till nor), forces slowdown the algorithm to the point that it takes 5 n−4 time steps. We show that much of this can be avoided, and give a new localization scheme which reduces the time to 4 n−2 (or 4 n−3 if n is odd). We also introduce a new approach to the problem of scheduling such parallel algorithms: we treat the equations defining the algorithm as an executable specification, but execute it on a non standard value domain. Such an abstract interpretation of the program then computes the optimal schedule of the original algorithm. A closed form can be inferred by observing a trace of this execution. We use this approach to obtain optimal schedules for all the algorithms in the paper—local as well as non-local.

Parallel algorithm for analysing activity networks on a tree-structured computer

A parallel algorithm for analysing activity networks is presented that works on a tree-structured parallel computer. The time complexity of the algorithm is 0(n 2/ p + n log p) with p (1 ≤ p ≤ n) processors for an activity network with n events (nodes). It is shown that the cost of this algorithm, which is the product of the parallel running time and the number of processors used, is less than that of each of the existing parallel algorithms for the same problem by at least a factor of n.

Improved processor bounds for combinatorial problems in RNC

Our main result improves the known processor bound by a factor ofn4 (maintaining the expected parallel running time,O(log3n)) for the following important problem:find a perfect matching in a general or in a bipartite graph with n vertices. A solution to that problem is used in parallel algorithms for several combinatorial problems, in particular for the problems of finding i) a (perfect) matching of maximum weight, ii) a maximum cardinality matching, iii) a matching of maximum vertex weight, iv) a maximums-t flow in a digraph with unit edge capacities. Consequently the known algorithms for those problems are substantially improved.

Efficient algorithms for some graph problems on a tree-structured parallel computer

In this paper, we present efficient parallel algorithms for breadth-first and depth-first search of graphs, finding the transitive closure of a directed graph, topological sorting of the nodes of a directed acyclic graph, and all-pairs youngest common ancestors of a directed tree on a tree-structured computer. Assuming that all the graphs and trees under investigation consist of n nodes, these algorithms achieve 0(n log n) time bound except for all-pairs youngest common ancestors problem which requires 0(n 2 log n) time—all with 0(n) processors. In all cases the costs (i.e. (parallel running time × number of processors) products) of our algorithms are better than those of the previously known parallel algorithms, except the last one where the cost is identical to that of the corresponding best known parallel algorithm.

Efficient Boolean Matrix and Graph Algorithms on a Bounded Parallel Computation Model

Two efficient boolean matrix multiplication algorithms on a classical k-bounded parallel computation model are proposed. Assuming that both the boolean matrices are of size (n × n) the time complexity of the first algorithm is O (max(n2/k, na/k)) whereas that for the second algorithm is O(max (n2/k, nβ/k)) for k ≦ n, where α (≦n2) and β ≦n⌈n2⌉) are integers. These algorithms are also applied for solving several important graph problems. The cost (i e parallel running time multiplied by the number of processors used) of each of these parallel algorithms is found better compared to their existing counterparts.

A Simple Parallel Algorithm for the Maximal Independent Set Problem

Two basic design strategies are used to develop a very simple and fast parallel algorithms for the maximal independent set (MIS) problem. The first strategy consists of assigning identical copies of a simple algorithm to small local portions of the problem input. The algorithm is designed so that when the copies are executed in parallel the correct problem output is produced very quickly. A very simple Monte Carlo algorithm for the MIS problem is presented which is based upon this strategy. The second strategy is a general and powerful technique for removing randomization from algorithms. This strategy is used to convert the Monte Carlo algorithm for this MIS problem into a simple deterministic algorithm with the same parallel running time.

Generating combinations in parallel

A parallel algorithm for generating all combinations ofm out ofn items in lexicographic order is presented. The algorithm usesm processors and runs inO(nCm) time. The cost of the algorithm, which is the parallel running time multiplied by the number of processors used, is optimal to within a constant multiplicative factor in view of the Ω(ncm*m) lower bound on the number of operations required to solve this problem using a sequential computer.