Parallel Discrete Event Simulation Research Articles

A Scalable GVT Estimation Algorithm for PDES: Using Lower Bound of Event-Bulk-Time

Global Virtual Time computation of Parallel Discrete Event Simulation is crucial for conducting fossil collection and detecting the termination of simulation. The triggering condition of GVT computation in typical approaches is generally based on the wall-clock time or logical time intervals. However, the GVT value depends on the timestamps of events rather than the wall-clock time or logical time intervals. Therefore, it is difficult for the existing approaches to select appropriate time intervals to compute the GVT value. In this study, we propose a scalable GVT estimation algorithm based on Lower Bound of Event-Bulk-Time, which triggers the computation of the GVT value according to the number of processed events. In order to calculate the number of transient messages, our algorithm employs Event-Bulk to record the messages sent and received by Logical Processes. To eliminate the performance bottleneck, we adopt an overlapping computation approach to distribute the workload of GVT computation to all worker-threads. We compare our algorithm with the fast asynchronous GVT algorithm using PHOLD benchmark on the shared memory machine. Experimental results indicate that our algorithm has a light overhead and shows higher speedup and accuracy of GVT computation than the fast asynchronous GVT algorithm.

Out-of-Order Parallel Discrete Event Simulation for Transaction Level Models

The validation of system models at the transaction-level typically relies on discrete event (DE) simulation. In order to reduce simulation time, parallel discrete event simulation (PDES) can be used by utilizing multiple cores available on today's host PCs. However, the total order of time imposed by regular DE simulators becomes a bottleneck that severely limits the benefits of parallel simulation. In this paper, we present a new out-of-order (OoO) PDES technique for simulating transaction-level models on multicore hosts. By localizing the simulation time to individual threads and carefully handling events at different times, a system model can be simulated following a partial order of time without loss of accuracy. Subject to advanced static analysis at compile time and table-based decisions at run time, threads can be issued early, reducing the idle time of available cores. Our proposed OoO PDES technique shows high performance gains in simulation speed with only a small increase in compile time. Using six embedded application examples, we also show the speed trade-off for multicore PDES based on different multithreading libraries.

Parallel discrete-event simulation schemes with heterogeneous processing elements.

To understand the effects of nonidentical processing elements (PEs) on parallel discrete-event simulation (PDES) schemes, two stochastic growth models, the restricted solid-on-solid (RSOS) model and the Family model, are investigated by simulations. The RSOS model is the model for the PDES scheme governed by the Kardar-Parisi-Zhang equation (KPZ scheme). The Family model is the model for the scheme governed by the Edwards-Wilkinson equation (EW scheme). Two kinds of distributions for nonidentical PEs are considered. In the first kind computing capacities of PEs are not much different, whereas in the second kind the capacities are extremely widespread. The KPZ scheme on the complex networks shows the synchronizability and scalability regardless of the kinds of PEs. The EW scheme never shows the synchronizability for the random configuration of PEs of the first kind. However, by regularizing the arrangement of PEs of the first kind, the EW scheme is made to show the synchronizability. In contrast, EW scheme never shows the synchronizability for any configuration of PEs of the second kind.

Parallel Discrete Event Simulation for Multi-Core Systems: Analysis and Optimization

Parallel Discrete Event Simulation (PDES) can substantially improve the performance and capacity of simulation, allowing the study of larger, more detailed models, in less time. PDES is a fine-grained parallel application whose performance and scalability is limited by communication latencies. Traditionally, PDES simulation kernels use message passing; often these simulators are written for distributed environments, and shared memory is used to optimize message passing among processes on the same machine. In this paper, we develop, characterize and optimize a thread-based version of a PDES simulator on three representative multi-core platforms. The multi-threaded implementation eliminates multiple message copying and significantly minimizes synchronization delays. We study the performance of the simulator on three hardware platforms: an Intel Core i7 machine, and a 48-core AMD Opteron Magny-Cours system, and a 64-core Tilera TilePro64. We discover that the three platforms encounter substantially different bottlenecks because of their different architectures. We identify these bottlenecks and propose mechanisms to overcome them. Our results show that multi-threaded implementation improves the performance over an MPI-based version by up to a factor of 3 on the Core i7, 1.4 on the AMD Magny-Cours, and 2.8 on the Tilera Tile64.

Partitioned event graph: formalizing LP-based modelling of parallel discrete-event simulation

Logical process (LP) is a modelling paradigm widely used in parallel discrete-event simulation (PDES). However, effective methods for formalizing LP-based modelling of PDES are lacking. This prevents an unambiguous, platform-independent description of LP-based models. We present a formalism named partitioned event graph (PEG) as a solution. PEG extends classical event graph formalism towards a formal specification for LP-based PDES models. We map between PEG and LP-based models, define the structural operational semantics (SOS) in a timed-labelled transition system, and discuss the Wallclock time-based execution. We propose a PEG-based model transformation framework for PDES, which has three model representation phases and distinguishes amongst four kinds of personnel roles. Finally, we present a domain-specific language (DSL) for the PDES of a Lotka–Volterra system and obtain preliminary parallel simulation results using YinHe Simulation Utilities for Parallel Environment (YHSUPE). The case study shows that the PEG-based framework not only effectively transforms a DSL into the LP paradigm, but will also result in efficient parallel simulation on a specific platform. In summary, by setting out specific characteristics for event scheduling and state space partition in the LP paradigm, PEG provides a formal method for model behaviour analysis and cross-platform model transformation.

Discrete Event Execution with One-Sided and Two-Sided GVT Algorithms on 216,000 Processor Cores

Global Virtual Time (GVT) computation is a key determinant of the efficiency and runtime dynamics of Parallel Discrete Event Simulations (PDES), especially on large-scale parallel platforms. Here, three execution modes of a generalized GVT computation algorithm are studied on high-performance parallel computing systems: (1) a synchronous GVT algorithm that affords ease of implementation, (2) an asynchronous GVT algorithm that is more complex to implement but can relieve blocking latencies, and (3) a variant of the asynchronous GVT algorithm to exploit one-sided communication in extant supercomputing platforms. Performance results are presented of implementations of these algorithms on up to 216,000 cores of a Cray XT5 system, exercised on a range of parameters: optimistic and conservative synchronization, fine- to medium-grained event computation, synthetic and nonsynthetic applications, and different lookahead values. Detailed PDES-specific runtime metrics are presented to further the understanding of tightly coupled discrete event dynamics on massively parallel platforms.

Multicore-Oriented Service Optimization of Parallel Discrete Event Simulation

Multicore-Oriented Service Optimization of Parallel Discrete Event Simulation

A unifying framework for specifying DEVS parallel and distributed simulation architectures

DEVS (Discrete Event System Specification) is an approach in the area of modeling and simulation that provides a means of specifying dynamic systems. A variety of DEVS tools have been implemented without a standard developmental guideline across the board, consequently revealing a lack of central frameworks for integrating heterogeneous DEVS simulators. When implementing a DEVS Simulator there are salient concepts that are intuitively defined, such as how events should be processed, what simulation architecture to use, what existing procedures (set of rules/algorithm) can be used, what should be the organizational architecture and so on. The aim of this paper is to propose a theoretical guide in building a DEVS distributed simulation as well as a formalization of underlying concepts to allow symbolic reasoning and automated code synthesis. From a review of existing implementation approaches, we propose a taxonomy of the identified concepts, including some formal definitions as they constitute the essential building blocks of performing Parallel Discrete-Event Simulation by utilizing DEVS. The contribution of this taxonomy and its impact as a unifying framework is that it provides a more systematic understanding of the process of constructing a DEVS simulator. Also, it offers an abstract way for integrating different and heterogeneous DEVS implementation strategies and thus can serve as a contribution to the on-going DEVS standardization efforts.

Modeling and simulation of large-scale social networks using parallel discrete event simulation

The modeling and simulation of social networks is an important approach to better understanding complex social phenomena, especially when the inner structure has remarkable impact on behavior. With the availability of unprecedented data sets, simulating large-scale social networks of millions, or even billions, of entities has become a new challenge. Current simulation environments for social studies are mostly sequential and may not be efficient when social networks grow to a certain size. In order to facilitate large-scale social network modeling and simulation, this paper proposes a framework named SUPE-Net, which is based on a parallel discrete event simulation environment YH-SUPE for massively parallel architectures. The framework is designed as a layered architecture with utilities for network generation, algorithms and agent-based modeling. Distributed adjacency lists are used for graph modeling and a reaction–diffusion paradigm is adapted to model dynamical processes. Experiments are performed using PageRank and the susceptible–infected–recovered (SIR) model on social networks with millions of entities. The results demonstrate that SUPE-Net has achieved a speedup of 12, and increased the event-processing rate by 11%, with good scalability and effectiveness.

An agent-based framework for performance modeling of an optimistic parallel discrete event simulator

Abstract Purpose The performance of an optimistic parallel discrete event simulator (PDES) in terms of the total simulation execution time of an experiment depends on a large set of variables. Many of them have a complex and generally unknown relationship with the simulation execution time. In this paper, we describe an agent-based performance model of a PDES kernel that is typically used to simulate large-sized complex networks on multiple processors or machines. The agent-based paradigm greatly simplifies the modeling of system dynamics by representing a component logical process (LP) as an autonomous agent that interacts with other LPs through event queues and also interacts with its environment which comprises the processor it resides on. Method We model the agents representing the LPs using a “base” class of an LP agent that allows us to use a generic behavioral model of an agent that can be extended further to model more details of LP behavior. The base class focuses only on the details that most likely influence the overall simulation execution time of the experiment. Results We apply this framework to study a local incentive based partitioning algorithm where each LP makes an informed local decision about its assignment to a processor, resulting in a system akin to a self organizing network. The agent-based model allows us to study the overall effect of the local incentive-based cost function on the simulation execution time of the experiment which we consider to be the global performance metric. Conclusion This work demonstrates the utility of agent-based approach in modeling a PDES kernel in order to evaluate the effects of a large number of variable factors such as the LP graph properties, load balancing criteria and others on the total simulation execution time of an experiment.

Advances in Parallel Discrete Event Simulation for Electronic System-Level Design

Editors' notes: The authors target the speeding up of parallel discrete event simulations in transaction-level models.

Improving Communication Patterns for Distributed Cluster-based Individual-oriented Fish School Simulations

Parallel discrete event simulation (PDES) have shown to be an useful paradigm for simulating complex and large-scale models. An individual-oriented approach allows modelers capture complex emerging global behaviors generated by simple local interaction, like observed in self-organized systems. Usually, this type of simulations are highly expensive in terms of computing and communications. One one hand, we can reduce the computing involved in individual interactions by means of developing a robust partitioning method. On the other hand, we have to be able to efficiently handle a huge number of individuals interacting with other individuals stored in memory of remote processors. In this work we will analyze and compare three communication strategies: synchronous and asynchronous message passing (via MPI) and bulk-synchronous parallel (BSP) for our distributed cluster-based individual-oriented fish school simulator. In this type of simulations, the main contributions of our work are: a) we showed that distributed time-driven simulations do not always improve the performance when using synchronous communication strategies, b) we show asynchronous communications strategies are more efficient. In addition, we have verified that the bulk-synchronous parallel method is a scalable.

Supports for transparent object-migration in PDES systems

It is well known that Parallel Discrete Event Simulation systems may suffer, in terms of delivered performance, from imbalance of the computational load. In case of conservative synchronization we may experience CPU under-utilization and/or excessive communication overhead. On the other hand, for the optimistic paradigm we may even have rollback thrashing effects, with a consequent reduction of the percentage of productive (ie not rolled back) work carried out. This paper presents the design of a global memory management architecture supporting application-transparent migration of simulation objects whose state is scattered across dynamically allocated memory chunks. Our approach is based on a non-intrusive background protocol that provides each instance of the simulation kernel with information on the current mapping of the virtual address space of all the other instances. Dynamic memory requests by the application layer are then locally mapped onto virtual-address ranges that maximize the likelihood of being portable onto the address space of a remote kernel instance. In this way, independently of the load-balancing trigger (or policy), we maximize the likelihood that a desirable migration across a specific couple of kernels can actually take place due to compliance of the corresponding source/destination address spaces. We have integrated the global memory manager within the ROme OpTimistic Simulator (ROOT-Sim), namely a run-time environment based on the optimistic synchronization paradigm which automatically and transparently parallelizes the execution of event-handler-based simulation programs conforming to ANSI-C. Further, we provide a contribution in the direction of widening load-balancing schemes for optimistic simulation systems by defining migration triggers and selection policies for the objects to be migrated on the basis of memory usage patterns. An experimental assessment of the architecture and of memory-oriented load balancing is also provided.

Synchronization methods in parallel and distributed discrete-event simulation

This work attempts to provide insight into the problem of executing discrete event simulation in a distributed fashion. The article serves as the state of the art in Parallel Discrete-Event Simulation (PDES) by surveying existing algorithms and analyzing the merits and drawbacks of various techniques. We discuss the main characteristics of existing synchronization methods for parallel and distributed discrete event simulation. The two major categories of synchronization protocols, namely conservative and optimistic, are introduced and various approaches within each category are presented. We also present the latest efforts towards PDES on emerging platforms such as heterogeneous multicore processors, Web services, as well as Grid and Cloud environment.

A novel parallelization technique for DEVS simulation of continuous and hybrid systems

In this paper, we introduce a novel parallelization technique for Discrete Event System Specification (DEVS) simulation of continuous and hybrid systems. Here, like in most parallel discrete event simulation methodologies, the models are first split into several sub-models which are then concurrently simulated on different processors. In order to avoid the cost of the global synchronization of all processes, the simulation time of each sub-model is locally synchronized in a real-time fashion with a scaled version of physical time, which implicitly synchronizes all sub-models. The new methodology, coined Scaled Real-Time Synchronization (SRTS), does not ensure a perfect synchronization in its implementation. However, under certain conditions, the synchronization error introduced only provokes bounded numerical errors in the simulation results. SRTS uses the same physical time-scaling parameter throughout the entire simulation. We also developed an adaptive version of the methodology (Adaptive-SRTS) where this parameter automatically evolves during the simulation according to the workload. We implemented the SRTS and Adaptive-SRTS techniques in PowerDEVS, a DEVS simulation tool, under a real-time operating system called the Real-Time Application Interface. We tested their performance by simulating three large-scale models, obtaining in all cases a considerable speedup.

Core frequency adjustment to optimize Time Warp on many-core processors

Time Warp synchronized parallel discrete event simulators are organized to operate asynchronously and aggressively without explicit synchronization between the concurrently executing simulation objects. In place of an explicit synchronization mechanism, the concurrent simulators implement an independent but common virtual clock model and a rollback/recovery mechanism to restore causal order when out-of-order events are detected. When the critical path of execution of the simulation is balanced across this parallel threads of execution, this can result in a highly effective, lightweight synchronization mechanism to implement parallel simulation. However, imbalances in the workload across the threads can result in excessive rollback in some threads and slowed progress of the critical path. On small shared memory multi-core systems, a lowest time-stamp scheduling policy can effectively balance the workload. However, on larger many-core chips, conventional load balancing and workload migration will once again become necessary. Fortunately, emerging many-core chips contain some interesting features that can potentially be exploited to improve the performance of parallel simulations. In particular, the recently developed Intel Single-chip Cloud Computer (SCC) provides mechanisms for the runtime control of the frequency and voltage settings of the chip. Furthermore, the frequency and voltage settings are independently set within different regions (called islands) of the chip. Thus, in a Time Warp simulation, one could increase the frequency of the cores executing threads on the critical path (those experiencing infrequent rollback) and decrease the frequency of the cores executing threads off the critical path (those experiencing excessive rollback). This paper investigates the run-time control and adjustment of core frequency in some contemporary x86 multi-core processors to identify the platforms that can support the exploration of dynamic run-time control of core frequency settings. The results show that while all multi-core processors have software controllable core frequency modulation capabilities, they are generally not fully independent as the system comes under load and are therefore unsuitable for these studies. Fortunately, one processor, the AMD X6 line, provides software control for core frequencies that can be fixed (by software) even as the system operates under load.

Efficient Master/Worker Parallel Discrete Event Simulation on Metacomputing Systems

The master/worker (MW) paradigm can be used as an approach to parallel discrete event simulation (PDES) on metacomputing systems. MW PDES applications incur overheads not found in conventional PDES executions executing on tightly coupled machines. We introduce four optimization techniques in MW PDES systems on public resource and desktop grid infrastructures. Work unit caching, pipelined state updates, expedited message delivery, and adaptive work unit scheduling mechanisms in the context of MW PDES are described. These optimizations provide significant performance benefits when used in tandem. We present results showing that an optimized MW PDES system using these techniques can exhibit performance comparable to a traditional PDES system for queueing network and particle physics simulation applications while providing execution capability across metacomputing systems.

A hierarchical parallel discrete event simulation kernel for multicore platform

The development of CPU has stepped into the era of multicore. Current parallel simulation kernel utilizes multicore resource by multi-process, which leads to inefficiency on communication and synchronization. To fulfill this gap, we proposed a HPSK (hierarchical parallel simulation kernel) model, which schedules logical processes and executes events in parallel with multithread paradigm. Based on this model, three key algorithms were proposed to support high performance: (1) An event management algorithm was proposed to improve the efficiency of creation and release of events. It uses a lock-free creation and asynchronous commitment mechanism to decouple the relationship between threads, hence reduce the overhead of locks. (2) A pointer-based communication algorithm was proposed to improve efficiency of communication between threads. It uses a buffer mechanism to avoid interrupting the execution of target thread. The target thread will read events from the buffers when it needs. By using ring-structure buffers, synchronization between sending and receiving of threads can be annihilated. (3) An approximate method was proposed to compute LBTS (Lower Bound on Time Stamp). It uses an asynchronous mechanism to eliminate disturbing of thread execution and a two-level filter mechanism to reduce redundant LBTS computation. A series of experiments with a modified phold model show that HPSK can achieve good performance for applications on different conditions. It can run 8× faster than μsik when event locality and lookahead is low.

Load sharing for optimistic parallel simulations on multi core machines

Parallel Discrete Event Simulation (PDES) is based on the partitioning of the simulation model into distinct Logical Processes (LPs), each one modeling a portion of the entire system, which are allowed to execute simulation events concurrently. This allows exploiting parallel computing architectures to speedup model execution, and to make very large models tractable. In this article we cope with the optimistic approach to PDES, where LPs are allowed to concurrently process their events in a speculative fashion, and rollback/ recovery techniques are used to guarantee state consistency in case of causality violations along the speculative execution path. Particularly, we present an innovative load sharing approach targeted at optimizing resource usage for fruitful simulation work when running an optimistic PDES environment on top of multi-processor/multi-core machines. Beyond providing the load sharing model, we also define a load sharing oriented architectural scheme, based on a symmetric multi-threaded organization of the simulation platform. Finally, we present a real implementation of the load sharing architecture within the open source ROme OpTimistic Simulator (ROOT-Sim) package. Experimental data for an assessment of both viability and effectiveness of our proposal are presented as well.

Behavior-aware Probabilistic Synchronization in Parallel Simulations and the Influence of the Simulation Model

Efficient event scheduling and synchronization constitutes an essential part of high-performance parallel discrete event simulation. Traditional synchronization approaches like conservative and optimistic synchronization focus on a simple scheduling paradigm based on a primitive set of rules. However, we argue that a sophisticated synchronization algorithm considering event interactions can remarkably improve the performance of the simulation. In this article, we discuss three different heuristics which analyze those dependencies online and speed up the simulation by a factor of up to 6.5. Further, we analyze the ability of this paradigm to automatically detect available parallelism by applying it to scenarios featuring different degrees of available parallelism.