Optimistic Parallel Simulation Research Articles

Managing Pending Events in Sequential and Parallel Simulations Using Three-tier Heap and Two-tier Ladder Queue

Performance of sequential and parallel Discrete Event Simulations (DES) is strongly influenced by the data structure used for managing and processing pending events. Accordingly, we propose and evaluate the effectiveness of our multi-tiered (two- and three-tier) data structures and our Two-tier Ladder Queue, for both sequential and optimistic parallel simulations on distributed memory platforms. Our experiments compare the performance of our data structures against a performance-tuned version of the Ladder Queue, which has been shown to outperform many other data structures for DES. The core simulation-based empirical assessments are in C++ and are based on 2,500 configurations of well-established PHOLD and PCS benchmarks. In addition, we use an Avian Influenza Epidemic Model (AIM) for experimental analyses. We have conducted experiments on two computing clusters with different hardware to ensure our results are reproducible. Moreover, to fully establish the robustness of our analysis and data structures, we have also implemented pertinent queues in Java and verified consistent, reproducible performance characteristics. Collectively, our analyses show that our three-tier heap and two-tier ladder queue outperform the Ladder Queue by 60× in some simulations, particularly those with higher concurrency per Logical Process (LP), in both sequential and Time Warp synchronized parallel simulations.

Load sharing for optimistic parallel simulations on multi core machines

Parallel Discrete Event Simulation (PDES) is based on the partitioning of the simulation model into distinct Logical Processes (LPs), each one modeling a portion of the entire system, which are allowed to execute simulation events concurrently. This allows exploiting parallel computing architectures to speedup model execution, and to make very large models tractable. In this article we cope with the optimistic approach to PDES, where LPs are allowed to concurrently process their events in a speculative fashion, and rollback/ recovery techniques are used to guarantee state consistency in case of causality violations along the speculative execution path. Particularly, we present an innovative load sharing approach targeted at optimizing resource usage for fruitful simulation work when running an optimistic PDES environment on top of multi-processor/multi-core machines. Beyond providing the load sharing model, we also define a load sharing oriented architectural scheme, based on a symmetric multi-threaded organization of the simulation platform. Finally, we present a real implementation of the load sharing architecture within the open source ROme OpTimistic Simulator (ROOT-Sim) package. Experimental data for an assessment of both viability and effectiveness of our proposal are presented as well.

Modeling and optimization of non-blocking checkpointing for optimistic simulation on myrinet clusters

Checkpointing-and-Communication Library (CCL) is a recently developed software which implements CPU offloaded, non-blocking checkpointing functionalities in support of optimistic parallel simulation on myrinet clusters. This is achieved by exploiting data transfer capabilities provided by a programmable DMA engine on board of myrinet network cards. Re-synchronization between CPU and DMA activities must sometimes be employed for several reasons, such as the maintenance of data consistency, thus adding overhead to (otherwise CPU cost-free) non-blocking checkpoint operations. In this paper we present a detailed cost model for non-blocking checkpointing and derive a performance effective re-synchronization semantic which we call minimum cost re-synchronization. With this semantic, an occurrence of re-synchronization either commits an on-going DMA based checkpoint operation (causing suspension of CPU activities) or aborts the operation (with possible increase in the expected rollback cost due to a reduced amount of committed checkpoints) on the basis of a minimum overhead expectation evaluated through the cost model.

Large-scale network simulation techniques

Simulation of large-scale networks remains to be a challenge, although various network simulators are in place. In this paper, we identify fundamental issues for large-scale networks simulation, and porpose new techniques that address them. First, we exploit optimistic parallel simulation techniques to enable fast execution on inexpensive hyper-threaded, multiprocessor systems. Second, we provide a compact, light-weight implementation framework that greatly reduces the amount of state required to simulate large-scale network models. Based on the proposed techniques, we provide sample simulation models for two networking protocols: TCP and OSPF. We implement these models in a simulation environment ROSSNet, which is an extension to the previously developed optimistic simulator ROSS. We perform validation experoments for TCP and OSPF and present performance reuslts of our techniques by simulating OSPF and TCP on a large and realistic topology, such as AT&T's US network based on rocketfuel data. The end result of these innovations is that we are able to simulate million node network tolopgies using inexpensive commercial off-the-shelf hyper-threaded multiprocessor systems consuming less than 1.4 GB of RAM in total .

Nonblocking checkpointing for optimistic parallel simulation: description and an implementation

Describes a nonblocking checkpointing mode in support of optimistic parallel discrete event simulation. This mode allows real concurrency in the execution of state saving and other simulation specific operations (e.g, event list update, event execution) with the aim of removing the cost of recording state information from the completion time of the parallel simulation application. We present an implementation of a C library supporting nonblocking checkpointing on a myrinet based cluster, which demonstrates the practical viability of this checkpointing mode on standard off-the-shelf hardware. By the results of an empirical study on classical parameterized synthetic benchmarks, we show that, except for the case of minimal state granularity applications, nonblocking checkpointing allows improvement of the speed of the parallel execution, as compared to commonly adopted, optimized checkpointing methods based on the classical blocking mode. A performance study for the case of a personal communication system (PCS) simulation is additionally reported to point out the benefits from nonblocking checkpointing for a real world application.

Optimistic parallel simulation of a large-scale view storage system

In this paper, we present the design and implementation of a complex view storage system model. Here, a hierarchy of view storage servers are connected to an array of client-side local disks. The term view refers to the output or result of a query made on the part of an application that is executing on a client machine. These queries can to be arbitrarily complex and formulated using SQL. The goal of this system is to reduce the turnaround time of queries by exploiting locality both at the local disk level as well as between clients and servers prior to making the request to the highest level database server. This model has been designed for execution with an optimistic simulation engine. One of the primary drawbacks of this parallel synchronization mechanism has been high overheads due to state-saving. We attack this problem by implementing the model using reverse computation. Here, the event processing routines are made reversible, which avoids having incrementally state-saving values that can be reverse computed, such as ++ and −−. Destructive assignments of the form a= b are saved using a swap operation, which precludes the need for additional state space. In our performance study of this application, we find that speedups range from 1.5 to over 5 on four processors. Super-linear speedups are attributed to a slow memory subsystem and the increased availability of level-1 and level-2 cache when moving to a larger number of processors.

Parallel shared-memory simulator performance for large ATM networks

A performance comparison between an optimistic and a conservative parallel simulation kernel is presented. Performance of the parallel kernels is also compared to a central-event-list sequential kernel. A spectrum of ATM network and traffic scenarios representative of those used by ATM networking researchers are used for the comparison. Experiments are conducted with a cell-level ATM network simulator and an 18-processor SGI PowerChallenge shared-memory multiprocessor. The results show the performance advantages of parallel simulation ove r sequential simulation for ATM networks. Speedups of 4-5 relative to a fast sequential kernel are achieved on 16 processors for several large irregular ATM benchmark scenarios and the optimistic kernel achieves 2 to 5 times speedup on all 7 benchmarks. However, the relative performance of the two parallel simulation kernels is dependent on the size of the ATM network, the number of traffic sources, and the traffic source types used in the simulation. For some benchmarks the best single point performance is provided by the conservative kernel even on a single processor. Unfortunately, the conservative kernel performance is susceptible to small changes in the modeling code and is outperformed by the optimistic kernel on 5 of the 7 benchmarks. The optimistic parallel simulation kernel thus provides most robust performance, but its speedup is limited by the overheads of its implementation, which make it approximately half the speed of the sequential kernel on one processor.

Efficient optimistic parallel simulations using reverse computation

In optimistic parallel simulations, state-saving techniques have traditionally been used to realize rollback. In this article, we propose reverse computation as an alternative approach, and compare its execution performance against that of state-saving. Using compiler techniques, we describe an approach to automatically generate reversible computations, and to optimize them to reap the performance benefits of reverse computation transparently. For certain fine-grain models, such as queuing network models, we show that reverse computation can yield significant improvement in execution speed coupled with significant reduction in memory utilization, as compared to traditional state-saving. On sample models using reverse computation, we observe as much as a six-fold improvement in execution speed over traditional state-saving.

Performance analysis of optimistic parallel simulations with limited rolled back events

Using the parallel simulation approach is not always the best choice. There are situations in which the sequential approach works better, in terms of simulation effectiveness. Indeed, the optimistic parallel simulation time consists of two basic components: the forward execution time, and the rollback mechanism time-overhead. It is convenient to parallelize a simulation until the latter component is not predominant with respect to the former one. The breakeven point depends on the nature of the model to simulate, on the characteristics of the simulation platform, and on the choice of tuning parameters such as the number of processors and the checkpoint interval. This paper deals with an evaluation of the performances of the sequential and parallel approach, in case the optimistic parallel method is used. An analytical model is introduced to study the sensitivity of the parallel simulation effectiveness to the forward event time, in case each event is rolled back no more than once. The model is first experimentally validated and then used to determine the breakeven point between the parallel and sequential approach.

Design and Implementation of Dynamic Load Balancing Algorithms for Rollback Reduction in Optimistic PDES

In an optimistic parallel simulation, logical processes (Ips) proceed with their computation without any constraints. However, if the computing requirements of different lps are not balanced or if the processors are not homogeneous, some lps may lag behind in simulation time while others surge forward. In other words, if the simulation clocks of different lps are not progressing at the same rate, cascading rollbacks may occur nullifying the potential benefit of an optimistic parallel discrete event simulation (PDES). Hence it is necessary to balance the computational load on different lps in such a way that their local simulation clocks advance almost at the same rate. In this paper, we propose two algorithms for dynamic load balancing which reduce the number of rollbacks in an optimistic PDES system. Our first algorithm is based on the load transfer mechanism between lps; while the second algorithm, based on the principle of evolutionary strategy, migrates logical processes between several pairs of physical processors. We have implemented both of these algorithms on a cluster of heterogeneous workstations and studied their performance. The experimental results show that the algorithm based on the load transfer is effective when the grain size is greater than 10 milliseconds. The algorithm based on the process migration yields good performance only for grain sizes of 20 milliseconds or larger. In both of these cases the speed up ranges mostly between and 2 using four processors.

Optimistic parallel simulation of reliable multicast protocols

Parallel discrete-event simulation offers the promise of harnessing the computational power of multiple processors in order to reduce the time needed for simulation-based performance studies. In this paper, we investigate the use of optimistic parallel simulation techniques in simulating reliable multicast communication network protocols. Through empirical studies (using the TeD simulation programming language, the Georgia Tech time warp simulator, and a 12-processor SGI Challenge), we find that these parallelized simulations can run noticeably faster than a uniprocessor simulation and, in a number of cases, can make effective use of parallel resources. These results are somewhat surprising because reliable multicast protocols require considerable communication (and hence synchronization) among different network entities.

PORTS: a parallel, optimistic, real-time simulator

This paper describes issues concerning the design of an optimistic parallel discrete event simulation system that executes in environments that impose real-time constraints on the simulator's execution. Two key problems must be addressed by such a system. First the timing characteristics of the parallel simulator must be sufficiently predictable to allow one to guarantee that real-time deadlines for completing simulation computations will be met. Second, the optimistic computation must be able to interact with its surrounding environment with as little latency as possible, necessitating rapid commitment of I/O operations. To address the first question, we show that optimistic simulators that never send incorrect messages (sometimes called “aggressive-no-risk” simulators) provide sufficient predictability to allow traditional schedulability analysis techniques commonly used in real-time systems to be applied. We show that incremental state saving techniques introduce sufficient unpredictability that they are not well-suited for real-time environments. We observe that the traditional “lowest timestamp first” scheduling policy used in many optimistic parallel simulation systems is an optimal (in the real-time sense) scheduling algorithm when event timestamps and real-time deadlines are the same. Finally, to address the question for rapid commitment of I/O operations, we utilize a continuous GVT computation scheme for shared-memory multiprocessors where a new value of GVT is computed after processing each event in the simulation. These ideas are incorporated in a parallel, optimistic, real-time simulation system called PORTS. Initial performance measurements of the shared-memory based PORTS system executing on a Kendall Square Research multiprocessor are presented. Initial performance results are encouraging, demonstrating that PORTS achieves performance approaching that of a conventional Time Warp system for the benchmark programs that were tested.

Determining the global progress of parallel simulation with FIFO communication property

In an optimistic parallel simulation, the global progress (i.e., the global virtual time or GVT) must be computed from time to time. We present a simple and efficient algorithm for computing the GVT in a distributed environment with FIFO communication delays. The accuracy and the complexity of the algorithm are discussed.

Memory management algorithms for optimistic parallel simulation

This paper studies the space complexity of an optimistic parallel simulation protocol called Time Warp. We evaluate four Time Warp memory management algorithms: fossil collection, message sendback, cancelback and artificial rollback. We identify two criteria in designing Time Warp memory management algorithms. Criterion 1 tests if a memory management algorithm ensures that Time Warp simulation always stops (either completes or terminates when memory is exhausted). If an algorithm does not satisfy this criterion, then the simulation may be trapped in an infinite loop. Criterion 2 tests if a memory management algorithm is independent of processor parameters (e.g., number of processors available for the parallel simulation, processor speed and interprocessor communication costs). We show that if an algorithm satisfies this second criterion, then the amount of memory consumed by Time Warp simulation is bounded by the amount consumed by sequential simulation. For algorithms that do not have full control of uncommitted objects (e.g., fossil collection and message sendback), Criterion 2 is not satisfied in general. For algorithms that have full control of uncommitted objects (e.g., cancelback and artificial rollback), special treatments are necessary to satisfy Criterion 1 (i.e., to ensure that the algorithms do not cancel future objects such that global virtual time never advances).

Optimistic Parallel Simulation of Continuous Time Markov Chains Using Uniformization

In an earlier paper we described how uniformization can be used as the basis of a conservative parallel simulation algorithm for simulating a continuous time Markov chain (CTMC). The fundamental notion is that uniformization permits the calculation (in advance of actually running the simulation) of instants where processors will synchronize, achieving much lower synchronization overhead than is usually attributed to conservative methods. In this paper we extend the idea further, showing how to use uniformization in the context of an optimistic parallel simulation to reduce the frequency of state-saving, schedule intelligently, and eliminate the Global Virtual Time (GVT) calculation. We demonstrate the efficiency of the method by implementation on a 16-processor Intel iPSC/2 and on 256 processors of the Intel Touchstone Delta.

A fixed point approach to parallel discrete event simulation

Discrete event simulation is viewed as solving a fixed point problem whose unknowns are infinite histories or streams of event and time information. Stream domains provide two notions of convergence, which correspond to the usual categorization of simulation methods. Metric convergence leads to optimistic parallel simulation (the classic event list mechanism turns out to be a specialization), and convergence in the sense of partial orders leads to conservative parallel simulation.