Deterministic Parallel Research Articles

The Report on “Deterministic Coin Tossing with Applications to Optimal Parallel List Ranking”

This report summarizes the paper “Deterministic Coin Tossing with Applications to Optimal Parallel List Ranking.” It introduces the DC-ListRank algorithm, a deterministic parallel algorithm for list ranking in distributed computing. The algorithm achieves optimal time complexity and scalability by combining coin tossing with parallel computing. The report covers the background, problem statement, related work, algorithm description, correctness analysis, discussions on strengths and limitations, performance comparisons, and future research directions. The DC-ListRank algorithm represents a significant advancement in parallel list ranking, offering efficiency and practical applications in distributed computing.

A Deterministic Parallel Reduction from Weighted Matroid Intersection Search to Decision

A Deterministic Parallel Reduction from Weighted Matroid Intersection Search to Decision

Computationally Optimized Irradiation Chamber Design for Production of 135Xe in the Washington State University TRIGA Reactor

This work summarizes the radiation transport–based design for a new D2O-moderated ex-core irradiation facility in the Washington State University (WSU) TRIGA reactor for optimization of 135Xe sources used for calibration and quality control testing of Xe gas detection equipment in support of the Comprehensive Test Ban Treaty (CTBT). Three-dimensional (3-D) particle transport analysis characterizing the WSU reactor core using MCNP6.2 (3-D Monte Carlo) and PENTRAN (3-D deterministic parallel SN) form the basis for the computational optimization. Excellent agreement between MCNP6.2 and PENTRAN predictions is observed. A fundamental fuel bundle depletion analysis is applied to enable a more accurate prediction of neutron flux and neutron spectrum distribution, which drives production rates of 135Xe and 133Xe. The results of various model simulations were used to inform recommendations for the final irradiation chamber design, which has been optimized for safe placement in the reactor tank prior to startup and will allow for insertion and rotation of xenon “bean” samples using existing WSU irradiation equipment, while remaining within operational parameters. The irradiation chamber is expected to produce samples that will remain viable for use in CTBT standards applications for durations 70% to 80% longer than samples produced using current procedures. Thus, this design is expected to improve CTBT-related calibrations and performance testing and to support the continued stability of the CTBT monitoring network.

A Deterministic Portable Parallel Pseudo-Random Number Generator for Pattern-Based Programming of Heterogeneous Parallel Systems

SkePU is a pattern-based high-level programming model for transparent program execution on heterogeneous parallel computing systems. A key feature of SkePU is that, in general, the selection of the execution platform for a skeleton-based function call need not be determined statically. On single-node systems, SkePU can select among CPU, multithreaded CPU, single or multi-GPU execution. Many scientific applications use pseudo-random number generators (PRNGs) as part of the computation. In the interest of correctness and debugging, deterministic parallel execution is a desirable property, which however requires a deterministically parallelized pseudo-random number generator. We present the API and implementation of a deterministic, portable parallel PRNG extension to SkePU that is scalable by design and exhibits the same behavior regardless where and with how many resources it is executed. We evaluate it with four probabilistic applications and show that the PRNG enables scalability on both multi-core CPU and GPU resources, and hence supports the universal portability of SkePU code even in the presence of PRNG calls, while source code complexity is reduced.

New graph algorithms via polyhedral techniques

Abstract This article gives a short overview of my dissertation, where new algorithms are given for two fundamental graph problems. We develop novel ways of using linear programming formulations, even exponential-sized ones, to extract structure from problem instances and to guide algorithms in making progress. The first part of the dissertation addresses a benchmark problem in combinatorial optimization: the asymmetric traveling salesman problem (ATSP). It consists in finding the shortest tour that visits all vertices of a given edge-weighted directed graph. A ρ-approximation algorithm for ATSP is one that runs in polynomial time and always produces a tour at most ρ times longer than the shortest tour. Finding such an algorithm with constant ρ had been a long-standing open problem. Here we give such an algorithm. The second part of the dissertation addresses the perfect matching problem. We have known since the 1980s that it has efficient parallel algorithms if the use of randomness is allowed. However, we do not know if randomness is necessary – that is, whether the matching problem is in the class NC. We show that it is in the class quasi-NC. That is, we give a deterministic parallel algorithm that runs in poly-logarithmic time on quasi-polynomially many processors.

Correct program parallelisations

A commonly used approach to develop deterministic parallel programs is to augment a sequential program with compiler directives that indicate which program blocks may potentially be executed in parallel. This paper develops a verification technique to reason about such compiler directives, in particular to show that they do not change the behaviour of the program. Moreover, the verification technique is tool-supported and can be combined with proving functional correctness of the program. To develop our verification technique, we propose a simple intermediate representation (syntax and semantics) that captures the main forms of deterministic parallel programs. This language distinguishes three kinds of basic blocks: parallel, vectorised and sequential blocks, which can be composed using three different composition operators: sequential, parallel and fusion composition. We show how a widely used subset of OpenMP can be encoded into this intermediate representation. Our verification technique builds on the notion of iteration contract to specify the behaviour of basic blocks; we show that if iteration contracts are manually specified for single blocks, then that is sufficient to automatically reason about data race freedom of the composed program. Moreover, we also show that it is sufficient to establish functional correctness on a linearised version of the original program to conclude functional correctness of the parallel program. Finally, we exemplify our approach on an example OpenMP program, and we discuss how tool support is provided.

Deterministic Shuffling Networks to Implement Stochastic Circuits in Parallel

Stochastic computing (SC) in recent years has been defined as a digital computation approach that operates on streams of random bits that represent probability values. SC can perform complex tasks with much smaller hardware footprints compared with conventional binary methods, but previous methods on SC circuits operated on serial bit streams, which leads to high-latency implementations. This article presents a significant improvement over previous work; it provides a deterministic parallel bit shuffling network that can use a simple deterministic thermometer encoding of data, resulting in zero random fluctuation and high accuracy, yet keeping the output bit-stream length constant. We use core “stochastic” logic circuits that do not employ constant coefficients, making them significantly smaller than traditional stochastic logic that use a significant amount of resources to generate such coefficients. Our experiments show that compared with previous SC methods, our method has up to $3\times $ smaller mean absolute error, and better area $\times $ delay and power efficiency. Compared with conventional binary methods, our method is better in terms of area $\times $ delay at 8-bit resolution. It shows better power efficiency ( $40\times $ , $18\times $ , and $8\times $ Gops/W at 8-, 10-, and 12-bit resolutions) compared with conventional binary.

Deterministic parallel fixpoint computation

Abstract interpretation is a general framework for expressing static program analyses. It reduces the problem of extracting properties of a program to computing an approximation of the least fixpoint of a system of equations. The de facto approach for computing this approximation uses a sequential algorithm based on weak topological order (WTO). This paper presents a deterministic parallel algorithm for fixpoint computation by introducing the notion of weak partial order (WPO). We present an algorithm for constructing a WPO in almost-linear time. Finally, we describe Pikos, our deterministic parallel abstract interpreter, which extends the sequential abstract interpreter IKOS. We evaluate the performance and scalability of Pikos on a suite of 1017 C programs. When using 4 cores, Pikos achieves an average speedup of 2.06x over IKOS, with a maximum speedup of 3.63x. When using 16 cores, Pikos achieves a maximum speedup of 10.97x.

A deterministic parallel algorithm for bipartite perfect matching

A fundamental quest in the theory of computing is to understand the power of randomness. It is not known whether every problem with an efficient randomized algorithm also has one that does not use randomness. One of the extensively studied problems under this theme is that of perfect matching. The perfect matching problem has a randomized parallel (NC) algorithm based on the Isolation Lemma of Mulmuley, Vazirani, and Vazirani. It is a long-standing open question whether this algorithm can be derandomized. In this article, we give an almost complete derandomization of the Isolation Lemma for perfect matchings in bipartite graphs. This gives us a deterministic parallel (quasi-NC) algorithm for the bipartite perfect matching problem. Derandomization of the Isolation Lemma means that we deterministically construct a weight assignment so that the minimum weight perfect matching is unique. We present three different ways of doing this construction with a common main idea.

P4-TPG: Accelerating Deterministic Parallel Test Pattern Generation by Preemptive, Proactive, and Preventive Schedulings

According to the prior research, a deterministic parallel test pattern generation (TPG) engine was realized and generated the same test pattern set the serial automatic test pattern generation does during acceleration. However, for retaining the determinism, tremendous idle time is observed when different tasks (either dependent or independent) were synchronized among threads. Therefore, a new deterministic parallel TPG engine called P4-TPG is developed and incorporates preemptive, proactive, and preventive schedulings to further save/reuse the idle time during acceleration. In P4-TPG, preemptive scheduling first modifies the thread flow and brings forward, as many latter tasks as possible, to the idle time. Next, proactive scheduling inserts prospective TPG tasks of unprocessed faults to the remaining idle time and increases the overall utilization of threads. Last, preventive scheduling dynamically skips faults incompatible with the working pattern per thread and shortens the fault list during fault compaction. The experimental results show that P4-TPG not only generates the same test pattern set as the serial TPG does but also achieves averagely $10.36\times $ speedups, is 96.6% better than the prior research, using 12 threads on 18 benchmark circuits.

Concurrency-aware object-oriented programming with roles

Object-oriented Programming has been effective in reducing code complexity in sequential programs, but in current practice, concurrent programs still present a number of challenges. We present here a model of object-oriented programming that identifies concurrent tasks and the relationship between objects and tasks , effectively making objects concurrency-aware . This awareness is formalized in a parallel programming model where every object plays a role in every task (e.g., the readonly role). When an object is shared with a new task, it adapts to the new sharing pattern by changing its roles, and therefore its behavior, i.e., the operations that can be performed with this object. This mechanism can be leveraged to prevent interfering accesses from concurrently executing tasks, and therefore makes parallel execution deterministic. To this end, we present a role-based programming language that includes several novel concepts (role transitions, guarding, slicing) to enable practical, object-oriented deterministic parallel programming. We show that this language can be used to safely implement programs with a range of different parallel patterns. The implementations to 8 widely used programming problems achieve substantial parallel speedups and demonstrate that this approach delivers performance roughly on par with manually synchronized implementations.

Deterministic Parallel Algorithms for Bilinear Objective Functions

Many randomized algorithms can be derandomized efficiently using either the method of conditional expectations or probability spaces with low independence. A series of papers, beginning with work by Luby (1988), showed that in many cases these techniques can be combined to give deterministic parallel (NC) algorithms for a variety of combinatorial optimization problems, with low time- and processor-complexity. We extend and generalize a technique of Luby for efficiently handling bilinear objective functions. One noteworthy application is an NC algorithm for maximal independent set. On a graph G with m edges and n vertices, this takes $${\tilde{O}}(\log ^2 n)$$ time and $$(m + n) n^{o(1)}$$ processors, nearly matching the best randomized parallel algorithms. Other applications include reduced processor counts for algorithms of Berger (SIAM J Comput 26:1188–1207, 1997) for maximum acyclic subgraph and Gale–Berlekamp switching games. This bilinear factorization also gives better algorithms for problems involving discrepancy. An important application of this is to automata-fooling probability spaces, which are the basis of a notable derandomization technique of Sivakumar (In: Proceedings of the 34th Annual ACM Symposium on Theory of Computing (STOC), pp 619–626, 2002). Our method leads to large reduction in processor complexity for a number of derandomization algorithms based on automata-fooling, including set discrepancy and the Johnson–Lindenstrauss Lemma.

Towards robustness of response times: minimising the maximum inter-completion time on parallel machines

Motivated by scheduling practices that require a response to unplanned high-priority jobs as soon as possible without preempting any in-processing jobs, this paper considers a deterministic identical parallel machine scheduling problem to achieve robustness with regard to a worst-case response time. To the best of our knowledge, this paper is the first to study the objective of minimising the maximum inter-completion time, i.e. the maximum time difference between any two consecutive completion times of jobs. For this novel scheduling problem, we first show its NP-hardness, and then propose an integer linear programming formulation and three heuristic approaches. Numerical experiments demonstrate the efficiency of our solution methods.

Computing on many cores

SummaryThis paper presents an alternative method to parallelize programs, better suited to manycore processors than actual operating system–/API‐based approaches like OpenMP and MPI. The method relies on a parallelizing hardware and an adapted programming style. It frees and captures the instruction‐level parallelism (ILP). A many‐core design is presented in which cores are multithreaded and able to fork new threads. The programming style is based on functions. The hardware creates a concurrent thread at each function call. The programming style and the hardware create the conditions to free the ILP, by eliminating the architectural dependences between a call and its continuation after return. We illustrate the method on a sum reduction, a matrix multiplication and a sort. We measure the ILP of the parallel runs and show that it is high enough to feed thousands of cores because it increases with data size. We compare our method to pthread parallelization, showing that (1) our parallel execution is deterministic, (2) our thread management is cheap, (3) our parallelism is implicit, and (4) our method parallelizes functions and loops. Implicit parallelism makes parallel code easy to write and read. Deterministic parallel execution makes parallel code easy to debug.

Guidelines for Reproducibly Building and Simulating Systems Biology Models.

Reproducibility is the cornerstone of the scientific method. However, currently, many systems biology models cannot easily be reproduced. This paper presents methods that address this problem. We analyzed the recent Mycoplasma genitalium whole-cell (WC) model to determine the requirements for reproducible modeling. We determined that reproducible modeling requires both repeatable model building and repeatable simulation. New standards and simulation software tools are needed to enhance and verify the reproducibility of modeling. New standards are needed to explicitly document every data source and assumption, and new deterministic parallel simulation tools are needed to quickly simulate large, complex models. We anticipate that these new standards and software will enable researchers to reproducibly build and simulate more complex models, including WC models.

Deterministic Parallel MaxSAT Solving

Multicore processors are becoming the dominant platform in modern days. As a result, parallel Maximum Satisfiability (MaxSAT) solvers have been developed to exploit this new architecture. However, parallel MaxSAT solvers suffer from non-deterministic behavior, i.e. several runs of the same solver can lead to different solutions. This is a clear downside for applications that require solving the same problem instance more than once. This paper presents the first deterministic parallel MaxSAT solver that ensures reproducibility of results. Experimental results show that the performance of the new deterministic solver is comparable to the corresponding non-deterministic version.Another advantage of using a deterministic solver is the fact that one can easily observe the gains coming from different techniques, since the non-determinism is removed from the solver. For example, sharing learned clauses in parallel MaxSAT is expected to help to further prune the search space and boost the performance of a parallel solver. Yet, so far it has not been made clear which learned clauses should be shared among the different threads. By using the deterministic solver, we present a comparison showing that sharing learned clauses improves the overall performance of parallel MaxSAT solvers.

Parallel Real-Time Scheduling of DAGs

Recently, multi-core processors have become mainstream in processor design. To take full advantage of multi-core processing, computation-intensive real-time systems must exploit intra-task parallelism. In this paper, we address the problem of real-time scheduling for a general model of deterministic parallel tasks, where each task is represented as a directed acyclic graph (DAG) with nodes having arbitrary execution requirements. We prove processor-speed augmentation bounds for both preemptive and non-preemptive real-time scheduling for general DAG tasks on multi-core processors. We first decompose each DAG into sequential tasks with their own release times and deadlines. Then we prove that these decomposed tasks can be scheduled using preemptive global EDF with a resource augmentation bound of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$4$</tex> <mathgraphic fileref="saifullah-ieq1-2297919.gif" graphicformat="GIF"/></formula> . This bound is as good as the best known bound for more restrictive models, and is the first for a general DAG model. We also prove that the decomposition has a resource augmentation bound of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$4$</tex> <mathgraphic fileref="saifullah-ieq2-2297919.gif" graphicformat="GIF"/></formula> plus a constant non-preemption overhead for non-preemptive global EDF scheduling. To our knowledge, this is the first resource augmentation bound for non-preemptive scheduling of parallel tasks. Finally, we evaluate our analytical results through simulations that demonstrate that the derived resource augmentation bounds are safe in practice.

SCCMulti

Tarjan's famous linear time, sequential algorithm for finding the strongly connected components (SCCs) of a graph relies on depth first search, which is inherently sequential. Deterministic parallel algorithms solve this problem in logarithmic time using matrix multiplication techniques, but matrix multiplication requires a large amount of total work. Randomized algorithms based on reachability -- the ability to get from one vertex to another along a directed path -- greatly improve the work bound in the average case. However, these algorithms do not always perform well; for instance, Divide-and-Conquer Strong Components (DCSC), a scalable, divide-and-conquer algorithm, has good expected theoretical limits, but can perform very poorly on graphs for which the maximum reachability of any vertex is small. A related algorithm, MultiPivot, gives very high probability guarantees on the total amount of work for all graphs, but this improvement introduces an overhead that increases the average running time. This work introduces SCCMulti, a multi-pivot improvement of DCSC that offers the same consistency as MultiPivot without the time overhead. We provide experimental results demonstrating SCCMulti's scalability; these results also show that SCCMulti is more consistent than DCSC and is always faster than MultiPivot.

A random walk based methodology for the realistic estimation of radioactivity migration in a porous medium

A model for the migration of radioactivity in a porous medium that optimally combines the best of both the statistical and deterministic approaches is developed. To mimic the fractures and the porous matrix we resort to averaging over random samplings of the fracture length and the branching angle between two consecutive nodes of fractures. For propagation between nodes, certain computational aspects of the deterministic parallel fracture model is adopted.

Composable Parallel Patterns with Intel Cilk Plus

Intel Cilk Plus extends C and C++ to enable writing composable deterministic parallel software that can exploit both the thread and vector parallelism commonly available in modern hardware.