Alignment Properties in Ordinal Comparison of DiscreteEvent Dynamic Systems

Abstract

In the design and optimization of discrete event dynamic systems, it is often necessary to order alternative designs based on their relative performance, i.e., to rank them from best to worst. In this paper, alignment of observed performance orders with true orders is considered and properties of the alignment are investigated. Spearman‘s rank correlation coefficient is a measure of agreement between the observed performance orders and the true ones. It is shown that Spearman‘s coefficient converges exponentially in the simulation time or observation time, which gives a strong evidence of the efficiency of order comparison for discrete event dynamic systems. In the context of simulation, the effect of simulation dependence on the alignment is also discussed. It is found that neither independent simulation nor the scheme of common random numbers (CRN), a popular scheme for variance reduction, can yield dominant performance. Finally, numerical examples based on a networking optimization problem are provided to illustrate the convergence of Spearman‘s coefficient. In these examples, the standard clock (SC) simulation technique provides much faster convergence than either independent simulations or CRN simulations. Both the SC and CRN methods use the same random number sequence to drive many events in parallel; however, under SC the events driving the parallel experiments are all identical, whereas under CRN they may be different.