Parallel Discrete Event Simulation Systems Research Articles

Transparently Mixing Undo Logs and Software Reversibility for State Recovery in Optimistic PDES

The Time Warp synchronization protocol for Parallel Discrete Event Simulation (PDES) is universally considered a viable solution to exploit the intrinsic simulation model parallelism and to provide model execution speedup. Yet it leads the PDES system to execute events in an order that may generate causal inconsistencies that need to be recovered via rollback , which requires restoration of a previous (consistent) simulation state whenever a causality violation is detected. The rollback operation is so critical for the performance of a Time Warp system that it has been extensively studied in the literature for decades to find approaches suitable to optimize it. The proposed solutions can be roughly classified as based on either checkpointing or reverse computing . In this article, we explore the practical design and implementation of a fully new approach based on the runtime generation of so-called undo code blocks , which are blocks of instructions implementing the reverse memory side effects generated by the forward execution of the events. However, this is not done by recomputing the original values to be restored, as instead it occurs in reverse computing schemes. Hence, the philosophy undo code blocks rely on is similar in spirit to that of undo-logs (as a form of checkpointing). Nevertheless, they are not data logs (as instead checkpoints are); rather, they are logs of instructions. Our proposal is fully transparent, thanks to the reliance on static software instrumentation (targeting the x86 architecture and Linux systems). Also, as we show, it can be combined with classical checkpointing to further improve the runtime behavior of the state recoverability support as a function of the workload. We also present experimental results related to our implementation, which is released as free software and fully integrated into the open source ROOT-Sim package. Experimental data support the viability and effectiveness of our proposal.

Transparent Speculative Parallelization of Discrete Event Simulation Applications Using Global Variables

Parallelizing (compute-intensive) discrete event simulation (DES) applications is a classical approach for speeding up their execution and for making very large/complex simulation models tractable. This has been historically achieved via parallel DES (PDES) techniques, which are based on partitioning the simulation model into distinct simulation objects (somehow resembling objects in classical object-oriented programming), whose states are disjoint, which are executed concurrently and rely on explicit event-exchange (or event-scheduling) primitives as the means to support mutual dependencies and notification of their state updates. With this approach, the application developer is necessarily forced to reason about state separation across the objects, thus being not allowed to rely on shared information, such as global variables, within the application code. This implicitly leads to the shift of the user-exposed programming model to one where sequential-style global variable accesses within the application code are not allowed. In this article we remove this limitation by providing support for managing global variables in the context of DES code developed in ANSI-C, which gets automatically parallelized. Particularly, we focus on speculative (also termed optimistic) PDES systems that run on top of multi-core machines, where simulation objects can concurrently process their events with no guarantee of causal consistency and actual violations of causality rules are recovered through rollback/recovery schemes. In compliance with the nature of speculative processing, in our proposal global variables are transparently mapped to multi-versions, so as to avoid any form of safety predicate verification upon their updates. Consistency is ensured via the introduction of a new rollback/recovery scheme based on detecting global variables' reads on non-correct versions. At the same time, efficiency in the execution is guaranteed by managing multi-version variables' lists via non-blocking algorithms. Furthermore, the whole approach is fully transparent, being it based on automatized instrumentation of the application software (particularly ELF objects). Hence the programmer is exposed to the classical (and easy to code) sequential-style programming scheme while accessing any global variable. An experimental assessment of our proposal, based on a suite of case study applications, run on top of an off-the-shelf Linux machine equipped with 32 CPU-cores and 64 GB of RAM, is also presented.

Supports for transparent object-migration in PDES systems

It is well known that Parallel Discrete Event Simulation systems may suffer, in terms of delivered performance, from imbalance of the computational load. In case of conservative synchronization we may experience CPU under-utilization and/or excessive communication overhead. On the other hand, for the optimistic paradigm we may even have rollback thrashing effects, with a consequent reduction of the percentage of productive (ie not rolled back) work carried out. This paper presents the design of a global memory management architecture supporting application-transparent migration of simulation objects whose state is scattered across dynamically allocated memory chunks. Our approach is based on a non-intrusive background protocol that provides each instance of the simulation kernel with information on the current mapping of the virtual address space of all the other instances. Dynamic memory requests by the application layer are then locally mapped onto virtual-address ranges that maximize the likelihood of being portable onto the address space of a remote kernel instance. In this way, independently of the load-balancing trigger (or policy), we maximize the likelihood that a desirable migration across a specific couple of kernels can actually take place due to compliance of the corresponding source/destination address spaces. We have integrated the global memory manager within the ROme OpTimistic Simulator (ROOT-Sim), namely a run-time environment based on the optimistic synchronization paradigm which automatically and transparently parallelizes the execution of event-handler-based simulation programs conforming to ANSI-C. Further, we provide a contribution in the direction of widening load-balancing schemes for optimistic simulation systems by defining migration triggers and selection policies for the objects to be migrated on the basis of memory usage patterns. An experimental assessment of the architecture and of memory-oriented load balancing is also provided.

Efficient Master/Worker Parallel Discrete Event Simulation on Metacomputing Systems

The master/worker (MW) paradigm can be used as an approach to parallel discrete event simulation (PDES) on metacomputing systems. MW PDES applications incur overheads not found in conventional PDES executions executing on tightly coupled machines. We introduce four optimization techniques in MW PDES systems on public resource and desktop grid infrastructures. Work unit caching, pipelined state updates, expedited message delivery, and adaptive work unit scheduling mechanisms in the context of MW PDES are described. These optimizations provide significant performance benefits when used in tandem. We present results showing that an optimized MW PDES system using these techniques can exhibit performance comparable to a traditional PDES system for queueing network and particle physics simulation applications while providing execution capability across metacomputing systems.

Parallel discrete event simulation on desktop grid computing infrastructures

In contrast to traditional Parallel Discrete Event Simulation (PDES) systems using dedicated machines, an alternative master/worker approach for high throughput execution on shared computing platforms is proposed. Master/worker systems offer potential advantages over traditional PDES systems such as better support for heterogeneous machines, client-driven load balancing, fault tolerance, and execution on non-dedicated machines. We examine feasibility of executing PDES codes on master/worker systems. We characterise application properties that are important for efficient execution and define suitable performance metrics. The Aurora Parallel and Distributed Simulation System utilising the master/worker approach is described along with performance measurements in executing conservative PDES codes.

A geographically distributed enterprise simulation system

Sandia National Laboratories has the role of system integrator in the US Department of Energy’s Nuclear Weapons Complex (NWC) and the responsibility for maintaining the nuclear stockpile. Maintenance is a complex task that involves a great number of geographically distributed functions, and a variety of analysis tools and models are used to plan operations, assess capabilities and guide political decisions. We are developing a framework for modeling the NWC unde interactive human control. The framework consists of a simulation system whose users and components are distributed across the entire United States. The purpose of the system is to support decisions about questions ranging from low-level operational matters involving only a sub-component of the complex to high-level policy issues requiring simulation of the entire complex. The system integrates multiple domain-specific legacy models and policy models and allows simulation at varying levels of detail. To provide such a framework, we have had to address a number of technical problems, whose solution is the focus of this paper. The three most fundamental problems are • to provide geographically distributed, synchronized human participation in the simulation; • to provide distributed ownership and security in such a way that the facility hosting a domain-specific model retains all rights to provide or deny a user access to its information; and • to integrate legacy models written in a variety of tools or languages and executed on different platforms. Key features of our approach are the use of the Java language and the coordination of the simulation through a new parallel discrete-event simulation system.

Visualizing parallel simulations that execute in network computing environments

Parallel discrete-event simulation systems (PDES) are used to simulate large-scale applications such as modeling telecommunication networks, transportation grids, and battlefield scenarios. While a large amount of PDES research has focused on employing multiprocessors and multicomputers, the use of networks of workstations interconnected through Ethernet or ATM has evolved into a popular effective platform for PDES. Nonetheless, the development of efficient PDES systems in network computing environments is not without obstacles that severely degrade simulator performance. To better understand how these factors degrade performance as well as develop new algorithms to mitigate them, we investigate the use of graphical visualization to provide insight into performance evaluation and simulator execution. We began with a general-purpose network computing visualization system, PVaniM, and used it to investigate the execution of an advanced version of Time Warp, called Georgia Tech Time Warp (GTW), which executes in network computing environments. Because PDES systems such as GTW are essentially middleware that support their own applications, we soon realized these systems require their own middleware-specific visualization support. To this end we have extended PVaniM into a new system, called PVaniM–GTW by adding middleware-specific views. Our experiences with PVaniM–GTW indicates that these enhancements enable one to better satisfy the needs of PDES middleware than general-purpose visualization systems while also not requiring the development of application-specific visualizations by the end-user.

Design and Implementation of DynamicLoad Balancing Algorithms for Rollback Reductionin Optimistic PDES

In an optimistic parallel simulation, logical processes (Ips) proceed with their computation without any constraints. However, if the computing requirements of different lps are not balanced or if the processors are not homogeneous, some lps may lag behind in simulation time while others surge forward. In other words, if the simulation clocks of different lps are not progressing at the same rate, cascading rollbacks may occur nullifying the potential benefit of an optimistic parallel discrete event simulation (PDES). Hence it is necessary to balance the computational load on different lps in such a way that their local simulation clocks advance almost at the same rate. In this paper, we propose two algorithms for dynamic load balancing which reduce the number of rollbacks in an optimistic PDES system. Our first algorithm is based on the load transfer mechanism between lps; while the second algorithm, based on the principle of evolutionary strategy, migrates logical processes between several pairs of physical processors. We have implemented both of these algorithms on a cluster of heterogeneous workstations and studied their performance. The experimental results show that the algorithm based on the load transfer is effective when the grain size is greater than 10 milliseconds. The algorithm based on the process migration yields good performance only for grain sizes of 20 milliseconds or larger. In both of these cases the speed up ranges mostly between and 2 using four processors.

Transparent incremental state saving in time warp parallel discrete event simulation

Many systems rely on the ability to rollback (or restore) parts of the system state to undo or recover from undesired or erroneous computations. Examples of such systems include fault tolerant systems with checkpointing, editors with undo capabilities, transaction and data base systems and optimistically synchronized parallel and distributed simulations. An essential part of such systems is the state saving mechanism. It should not only allow efficient state saving, but also support efficient state restoration in case of roll back. Furthermore, it is often a requirement that this mechanism is transparent to the user. In this paper we present a method to implement a transparent incremental state saving mechanism in an optimistically synchronized parallel discrete event simulation system based on the Time Warp mechanism. The usefulness of this approach is demonstrated by simulations of large, detailed, realistic FCA and a DCA-like cellular phone systems.

Asynchronous parallel discrete event simulation

Complex models may have model components distributed over a network and generally require significant execution times. The field of parallel and distributed simulation has grown over the past fifteen years to accommodate the need of simulating the complex models using a distributed versus sequential method. In particular, asynchronous parallel discrete event simulation (PDES) has been widely studied, and yet we envision greater acceptance of this methodology as more readers are exposed to PDES introductions that carefully integrate real-world applications. With this in mind, we present two key methodologies (conservative and optimistic) which have been adopted as solutions to PDES systems. We discuss PDES terminology and methodology under the umbrella of the personal communications services application.

A fast asynchronous GVT algorithm for shared memory multiprocessor architectures

The computation of Global Virtual Time is of fundamental importance in Time Warp based Parallel Discrete Event Simulation Systems. Shared memory multiprocessor architectures can support interprocess communication with much smaller overheads than distributed memory systems. This paper presents a new, completely asynchronous, Gvt algorithm which provides very fast and accurate Gvt estimation with significantly lower overhead than previous approaches. The algorithm presented is able to support more efficient memory management, termination, and other global control mechanisms. The Gvt algorithm described enables any Time Warp entity to compute Gvt at any time without slowing down other entities, in particular, those executing on the critical path. Experimental results are presented for a shared memory Time Warp system that employs a two tiered distributed memory management scheme. The proof of the correctness and the accuracy of the algorithm are also presented. Finally, some suggestions on possible further optimization of the implementation are given.

PORTS: a parallel, optimistic, real-time simulator

This paper describes issues concerning the design of an optimistic parallel discrete event simulation system that executes in environments that impose real-time constraints on the simulator's execution. Two key problems must be addressed by such a system. First the timing characteristics of the parallel simulator must be sufficiently predictable to allow one to guarantee that real-time deadlines for completing simulation computations will be met. Second, the optimistic computation must be able to interact with its surrounding environment with as little latency as possible, necessitating rapid commitment of I/O operations. To address the first question, we show that optimistic simulators that never send incorrect messages (sometimes called “aggressive-no-risk” simulators) provide sufficient predictability to allow traditional schedulability analysis techniques commonly used in real-time systems to be applied. We show that incremental state saving techniques introduce sufficient unpredictability that they are not well-suited for real-time environments. We observe that the traditional “lowest timestamp first” scheduling policy used in many optimistic parallel simulation systems is an optimal (in the real-time sense) scheduling algorithm when event timestamps and real-time deadlines are the same. Finally, to address the question for rapid commitment of I/O operations, we utilize a continuous GVT computation scheme for shared-memory multiprocessors where a new value of GVT is computed after processing each event in the simulation. These ideas are incorporated in a parallel, optimistic, real-time simulation system called PORTS. Initial performance measurements of the shared-memory based PORTS system executing on a Kendall Square Research multiprocessor are presented. Initial performance results are encouraging, demonstrating that PORTS achieves performance approaching that of a conventional Time Warp system for the benchmark programs that were tested.