Optimistic Simulation Research Articles

Fast Distributed Simulation of Sensor Networks Using Optimistic Synchronization

Network simulation is an important tool for testing and evaluating wireless sensor network applications. Parallel simulation strategies improve the scalability of these tools. However, achieving high performance depends on reducing the synchronization overhead among simulation processes. In this paper, we present an optimistic simulation algorithm with support for backtracking and re-execution. The algorithm reduces the number of synchronization cycles to the number of transmissions in the network under test. We implement SnapSim, an extension to the popular Avrora simulator, based on this algorithm. The experimental results show that our prototype system improves the performance of Avrora by 2 to 10 times for typical network-centric sensor network applications, and up to three orders of magnitude for applications that use the radio infrequently. We also implement a distributed version of SnapSim, D-SnapSim, which runs on a cluster. The experimental results show that D-SnapSim further improves the performance of SnapSim by up to 10 times for applications that use the radio frequently.

Supports for transparent object-migration in PDES systems

It is well known that Parallel Discrete Event Simulation systems may suffer, in terms of delivered performance, from imbalance of the computational load. In case of conservative synchronization we may experience CPU under-utilization and/or excessive communication overhead. On the other hand, for the optimistic paradigm we may even have rollback thrashing effects, with a consequent reduction of the percentage of productive (ie not rolled back) work carried out. This paper presents the design of a global memory management architecture supporting application-transparent migration of simulation objects whose state is scattered across dynamically allocated memory chunks. Our approach is based on a non-intrusive background protocol that provides each instance of the simulation kernel with information on the current mapping of the virtual address space of all the other instances. Dynamic memory requests by the application layer are then locally mapped onto virtual-address ranges that maximize the likelihood of being portable onto the address space of a remote kernel instance. In this way, independently of the load-balancing trigger (or policy), we maximize the likelihood that a desirable migration across a specific couple of kernels can actually take place due to compliance of the corresponding source/destination address spaces. We have integrated the global memory manager within the ROme OpTimistic Simulator (ROOT-Sim), namely a run-time environment based on the optimistic synchronization paradigm which automatically and transparently parallelizes the execution of event-handler-based simulation programs conforming to ANSI-C. Further, we provide a contribution in the direction of widening load-balancing schemes for optimistic simulation systems by defining migration triggers and selection policies for the objects to be migrated on the basis of memory usage patterns. An experimental assessment of the architecture and of memory-oriented load balancing is also provided.

Model and simulation of exascale communication networks

Exascale supercomputers will have millions or even hundreds of millions of processing cores and the potential for nearly billion-way parallelism. Exascale compute and data storage architectures will be critically dependent on the interconnection network. The most popular interconnection network for current and future supercomputer systems is the torus (eg, k-ary, n-cube). This paper focuses on the modelling and simulation of ultra-large-scale torus networks using Rensselaer's Optimistic Simulator System. We compare real communication delays between our model and the actual torus network from Blue Gene/L using 2048 processors. Our performance experiments demonstrate the ability to simulate million-node to billion-node torus networks. The torus network model for a 16-million-node configuration shows a high degree of strong scaling when going from 1024 cores to 32 768 cores on Blue Gene/L, with a peak event-rate of nearly 5 billion events per second. We also demonstrate the performance of our torus network model configured with 1 billion nodes on both Blue Gene/L and Blue Gene/P systems. The observed best event rate at 128 K cores is 12.36 billion per second on Blue Gene/P.

Load sharing for optimistic parallel simulations on multi core machines

Parallel Discrete Event Simulation (PDES) is based on the partitioning of the simulation model into distinct Logical Processes (LPs), each one modeling a portion of the entire system, which are allowed to execute simulation events concurrently. This allows exploiting parallel computing architectures to speedup model execution, and to make very large models tractable. In this article we cope with the optimistic approach to PDES, where LPs are allowed to concurrently process their events in a speculative fashion, and rollback/ recovery techniques are used to guarantee state consistency in case of causality violations along the speculative execution path. Particularly, we present an innovative load sharing approach targeted at optimizing resource usage for fruitful simulation work when running an optimistic PDES environment on top of multi-processor/multi-core machines. Beyond providing the load sharing model, we also define a load sharing oriented architectural scheme, based on a symmetric multi-threaded organization of the simulation platform. Finally, we present a real implementation of the load sharing architecture within the open source ROme OpTimistic Simulator (ROOT-Sim) package. Experimental data for an assessment of both viability and effectiveness of our proposal are presented as well.

Fine-Grained Parallel and Distributed Spatial Stochastic Simulation of Biological Reactions

Fine-Grained Parallel and Distributed Spatial Stochastic Simulation of Biological Reactions

Incorporating Nonlinear Relationships in Microarray Missing Value Imputation

Microarray gene expression data often contain missing values. Accurate estimation of the missing values is important for downstream data analyses that require complete data. Nonlinear relationships between gene expression levels have not been well-utilized in missing value imputation. We propose an imputation scheme based on nonlinear dependencies between genes. By simulations based on real microarray data, we show that incorporating nonlinear relationships could improve the accuracy of missing value imputation, both in terms of normalized root-mean-squared error and in terms of the preservation of the list of significant genes in statistical testing. In addition, we studied the impact of artificial dependencies introduced by data normalization on the simulation results. Our results suggest that methods relying on global correlation structures may yield overly optimistic simulation results when the data have been subjected to row (gene)-wise mean removal.

Time synchronization mechanism in Agent-based distributed simulation

The time synchronization mechanism is one of the key problems on Agent-based distributed simulation research.The time synchronization mechanism of Agent-based distributed simulation environment(ADSimE) — conservative mechanism and optimistic mechanism were introduced.That is,the conservative mechanism was used within the node of the platform,while optimistic mechanism was used between the nodes.This kind of time synchronization mechanism could promote the performance of the time advancing in the simulation platform.The principles of the two mechanisms were also introduced.Finally,the key process of the design and running for conservative simulation Agent and optimistic simulation Agent were described in detail.

On the construction of committed consistent global states in optimistic simulation

In this paper, we study how to reuse checkpoints taken in an uncorrelated manner during the forward execution phase in an optimistic simulation system in order to construct global consistent snapshots which are also committed, namely the logical time they refer to is lower than the current Global-Virtual-Time (GVT). Specifically, we present a heuristic-based mechanism relying on update operations applied to local committed checkpoints of the logical processes so to eliminate mutual dependencies among the final achieved state values. The mechanism is lightweight since it does not require any form of (distributed) coordination to determine which are the checkpoint update operations to be locally performed for each logical process. At the same time it is likely to reduce the amount of checkpoint update operations required to realign the consistent global state exactly to the current GVT, taken as the reference time for the snapshot. Our proposal can support, in a performance effective manner, termination detection schemes based on global predicates evaluated on a committed and consistent global snapshot, which represent an alternative as relevant as classical termination check only relying on the current GVT value.

Reversible discrete event formulation and optimistic parallel execution of vehicular traffic models

Vehicular traffic simulations are useful in applications such as emergency planning and traffic management, for rapid response and resilience. Here, a parallel traffic simulation approach is presented that reduces the time for simulating emergency vehicular traffic scenarios. We use a reverse computation-based optimistic execution approach to parallel execution of microscopic, vehicular-level models of traffic. The unique aspects of this effort are: exploration of optimistic simulation of vehicular traffic; addressing the related reverse computation challenges; achieving absolute, as opposed to self-relative, speedup. The design, development and performance study of the parallel simulation system is presented, demonstrating excellent sequential and parallel performance. A speed up of nearly 20 on 32 processors is observed on a vehicular network of 65,000 intersections and 13 million vehicles.

On constructing optimistic simulation algorithms for the discrete event system specification

This article describes a Time Warp simulation algorithm for discrete event models that are described in terms of the Discrete Event System Specification (DEVS). The article shows how the total state transition and total output function of a DEVS atomic model can be transformed into an event processing procedure for a logical process. A specific Time Warp algorithm is constructed around this logical process, and it is shown that the algorithm correctly simulates a DEVS coupled model that consists entirely of interacting atomic models. The simulation algorithm is presented abstractly; it is intended to provide a basis for implementing efficient and scalable parallel algorithms that correctly simulate DEVS models.

Parallel Environment for DEVS and Cell-DEVS Models

Discrete Event System Specification (DEVS) is a sound formalism to describe generic dynamic systems in a hierarchical and modular way. Cell-DEVS is a DEVS-based formalism intended to model compleX physical systems as cell spaces. This work presents new techniques for eXecuting DEVS and Cell-DEVS models in parallel and distributed environments based on the WARPED kernel, an implementation of the Time Warp protocol. The optimistic simulator PCD++, built as a new simulation engine for CD++, is a toolkit that implements the DEVS and Cell-DEVS formalisms. We redesign algorithms in CD++ to carry out optimistic simulations using a non-hierarchical approach that reduces the communication overhead. The message-passing organization is analyzed using a high-level abstraction referred to as wall clock time slice. We propose a two-level user-controlled state-saving mechanism to achieve efficient and fleXible state saving at runtime. Various optimization strategies are applied to PCD++ and their effects are analyzed quantitatively, including a risk-free message type-based state-saving strategy to reduce the number of states saved during the simulation significantly, and a one log file per node strategy to break the bottleneck caused by file I/O operations. It is shown that PCD++ markedly outperforms other alternatives and considerable speedups can be achieved in parallel and distributed simulations.

Multiprogrammed non-blocking checkpoints in support of optimistic simulation on myrinet clusters

CCL (checkpointing and communication library) is a software layer in support of optimistic parallel discrete event simulation (PDES) on myrinet-based COTS clusters. Beyond classical low latency message delivery functionalities, this library implements CPU offloaded, non-blocking (asynchronous) checkpointing functionalities based on data transfer capabilities provided by a programmable DMA engine on board of myrinet network cards. These functionalities are unique since optimistic simulation systems conventionally rely on checkpointing implemented as a synchronous, CPU-based data copy. Releases of CCL up to v2.4 only support monoprogrammed non-blocking checkpoints. This forces re-synchronization between CPU and DMA activities, which is a potential source of overhead, each time a new checkpoint request must be issued at the simulation application level while the last issued one is still being carried out by the DMA engine. In this paper we present a redesigned release of CCL (v3.0) that, exploiting hardware capabilities of more advanced myrinet clusters, supports multiprogrammed non-blocking checkpoints. The multiprogrammed approach allows higher degree of concurrency between checkpointing and other simulation specific operations carried out by the CPU, with benefits on performance. We also report the results of the experimental evaluation of those benefits for the case of a Personal Communication System (PCS) simulation application, selected as a real world test-bed.

Distributed simulation of modular time Petri nets: An approach and a case study exploiting temporal uncertainty

This paper proposes an approach to modular modelling and simulation of complex time-critical systems. The modelling language is represented by Merlin and Farber's Time Petri Nets (TPNs) augmented with inhibitor arcs and modular constructs borrowed from the Petri Net Markup Language (PNML) interchange format. Analysis techniques depend on Temporal Uncertainty Time Warp (TUTW), a time warp algorithm capable of exploiting temporal uncertainty in general optimistic simulations over a networked context. A key feature of the approach is the fact that TPN models naturally exhibit a certain degree of temporal uncertainty which the TUTW control engine can exploit to achieve good speedup without a loss in the accuracy of the simulation results. The developed TUTW/TPN kernel is demonstrated by modelling and simulation of a real-time system example.

Modeling and optimization of non-blocking checkpointing for optimistic simulation on myrinet clusters

Checkpointing-and-Communication Library (CCL) is a recently developed software which implements CPU offloaded, non-blocking checkpointing functionalities in support of optimistic parallel simulation on myrinet clusters. This is achieved by exploiting data transfer capabilities provided by a programmable DMA engine on board of myrinet network cards. Re-synchronization between CPU and DMA activities must sometimes be employed for several reasons, such as the maintenance of data consistency, thus adding overhead to (otherwise CPU cost-free) non-blocking checkpoint operations. In this paper we present a detailed cost model for non-blocking checkpointing and derive a performance effective re-synchronization semantic which we call minimum cost re-synchronization. With this semantic, an occurrence of re-synchronization either commits an on-going DMA based checkpoint operation (causing suspension of CPU activities) or aborts the operation (with possible increase in the expected rollback cost due to a reduced amount of committed checkpoints) on the basis of a minimum overhead expectation evaluated through the cost model.

An Approach for the Unified Time Management Mechanism for HLA

The High Level Architecture (HLA) is the IEEE 1516 standard to model and design a distributive simulation environment. It establishes a common distributed simulation framework, which facilitates the interoperability and reuse of simulation components. However, some HLA services are very low level and difficult to use, especially when a simulation is designed with a particular time synchronization mechanism. This article describes an agent interface, called Smart Time Management (STM). The capabilities of the STM include the following: taking over the event’s time-stamp tagging work, maintaining a lookahead value, and unifying different time advance approaches provided by the HLA runtime infrastructure. In addition, it adopts the time warp mechanism for optimistic simulation. In conclusion, the STM presents a unified and scalable middle layer to allow the user to construct an HLA federation with a unanimous Time Management interface when solving the synchronization issue.

Smart Time Management—the unified time synchronization interface for the distributed simulation

Distributed simulation is a widely studied technique for the networked virtual environment. Among existing technologies, High Level Architecture (HLA) establishes a common distributed simulation framework that facilitates the interoperability and reuse of simulation components. However, some HLA services are very low-level and difficult to use, especially when a simulation is designed using a particular time synchronization mechanism. This paper describes an agent interface called Smart Time Management (STM). STM is used to unify the time management services of the time-stepped, event-driven, and optimistic time advancement in HLA specification. The capabilities of the STM include the followings: taking over events timestamp tagging work, maintaining a look-ahead value, and unifying different time advance approaches provided by the HLA Run Time Infrastructure (RTI). In addition, it adopts the time warp mechanism for optimistic simulation. In summary, STM presents a unified and scalable middle layer to allow the user to construct an HLA federation with a unanimous time management interface when solving the synchronization issue. The presented middle layer enables the user to deploy the conservative and optimistic synchronization mechanisms in a unanimous way.

Temporal Uncertainty Time Warp: An Implementation Based on Java and ActorFoundry

This article describes temporal uncertainty time warp (TUTW), a distributed control engine designed for exploitation of temporal uncertainty (TU) in general optimistic simulations. Novel in TUTW is an event model in which time intervals are attached to events instead of classical punctual or “precise” time stamps. All of this complies with system specifications where the occurrence time of events cannot be known with complete certainty. TUTW is able to take advantage of TU by resolving events in such a way as to reduce the number of rollbacks. The simulator performance can thus be improved without necessarily compromising the accuracy of the results. An agent-based implementation of TUTW was achieved that enables simulations to be carried out over the Internet. The implementation depends on Java and the ActorFoundry middleware. The implementation is totally portable and simplifies configuration and control of a distributed simulation virtual computer that includes heterogeneous computing platforms. The article reports performance and accuracy results of TUTW applied to a large QNET simulation model.

Large-scale network simulation techniques

Simulation of large-scale networks remains to be a challenge, although various network simulators are in place. In this paper, we identify fundamental issues for large-scale networks simulation, and porpose new techniques that address them. First, we exploit optimistic parallel simulation techniques to enable fast execution on inexpensive hyper-threaded, multiprocessor systems. Second, we provide a compact, light-weight implementation framework that greatly reduces the amount of state required to simulate large-scale network models. Based on the proposed techniques, we provide sample simulation models for two networking protocols: TCP and OSPF. We implement these models in a simulation environment ROSSNet, which is an extension to the previously developed optimistic simulator ROSS. We perform validation experoments for TCP and OSPF and present performance reuslts of our techniques by simulating OSPF and TCP on a large and realistic topology, such as AT&T's US network based on rocketfuel data. The end result of these innovations is that we are able to simulate million node network tolopgies using inexpensive commercial off-the-shelf hyper-threaded multiprocessor systems consuming less than 1.4 GB of RAM in total .

Optimistic parallel simulation of a large-scale view storage system

In this paper, we present the design and implementation of a complex view storage system model. Here, a hierarchy of view storage servers are connected to an array of client-side local disks. The term view refers to the output or result of a query made on the part of an application that is executing on a client machine. These queries can to be arbitrarily complex and formulated using SQL. The goal of this system is to reduce the turnaround time of queries by exploiting locality both at the local disk level as well as between clients and servers prior to making the request to the highest level database server. This model has been designed for execution with an optimistic simulation engine. One of the primary drawbacks of this parallel synchronization mechanism has been high overheads due to state-saving. We attack this problem by implementing the model using reverse computation. Here, the event processing routines are made reversible, which avoids having incrementally state-saving values that can be reverse computed, such as ++ and −−. Destructive assignments of the form a= b are saved using a swap operation, which precludes the need for additional state space. In our performance study of this application, we find that speedups range from 1.5 to over 5 on four processors. Super-linear speedups are attributed to a slow memory subsystem and the increased availability of level-1 and level-2 cache when moving to a larger number of processors.

ROSS: A high-performance, low-memory, modular Time Warp system

In this paper, we introduce a new Time Warp system called ROSS: Rensselaer’ s optimistic simulation system. ROSS is an extremely modular kernel that is capable of achieving event rates as high as 1,250,000 events per second when simulating a wireless telephone network model (PCS) on a quad processor PC server. In a head-to-head comparison, we observe that ROSS out performs the Georgia Tech Time Warp (GTW) system by up to 180% on a quad processor PC server and up to 200% on the SGI Origin 2000. ROSS only requires a small constant amount of memory buffers greater than the amount needed by the sequential simulation for a constant number of processors. ROSS demonstrates for the first time that stable, highly efficient execution using little memory above what the sequential model would require is possible for low-event granularity simulation models. The driving force behind these high-performance and low-memory utilization results is the coupling of an efficient pointer-based implementation framework, Fujimoto's fast GVT algorithm for shared memory multiprocessors, reverse computation and the introduction of kernel processes ( KPs). KPs lower fossil collection overheads by aggregating processed event lists. This aspect allows fossil collection to be done with greater frequency, thus lowering the overall memory necessary to sustain stable, efficient parallel execution. These characteristics make ROSS an ideal system for use in large-scale networking simulation models. The principle conclusion drawn from this study is that the performance of an optimistic simulator is largely determined by its memory usage.