Naive Simulation Research Articles

Completion for Multiple Reduction Orderings

We present a completion procedure (called MKB) that works for multiple reduction orderings. Given equations and a set of reduction orderings, the procedure simulates a computation performed by the parallel processes each of which executes the standard completion procedure (KB) with one of the given orderings. To gain efficiency, however, we develop new inference rules working on objects called nodes, which are data structures consisting of a pair s : t of terms associated with the information to show which processes contain the rule s → t (or t → s) and which processes contain the equation s ↔ t. The idea is based on the observation that some inferences made in different processes are often closely related, so we can design inference rules that simulate these inferences all in a single operation. Our experiments show that MKB is significantly more efficient than the naive simulation of parallel execution of KB procedures, when the number of reduction orderings is large enough. We also present an extension of this technique to the unfailing completion for multiple reduction orderings, which is useful in various areas of automated reasoning, including equational theorem proving.

A new representation for a renewal-theoretic constant appearing in asymptotic approximations of large deviations

The probability that a stochastic process with negative drift exceed a value a often has a renewal-theoretic approximation as $a \to \infty$. Except for a process of iid random variables, this approximation involves a constant which is not amenable to analytic calculation. Naive simulation of this constant has the drawback of necessitating a choice of finite a, thereby hurting assessment of the precision of a Monte Carlo simulation estimate, as the effect of the discrepancy between a and $\infty$ is usually difficult to evaluate. Here we suggest a new way of representing the constant. Our approach enables simulation of the constant with prescribed accuracy. We exemplify our approach by working out the details of a sequential power one hypothesis testing problem of whether a sequence of observations is iid standard normal against the alternative that the sequence is AR(1). Monte Carlo results are reported.

Detection of feigned mental disorders on the personality assessment inventory: a discriminant analysis.

Psychological assessment with multiscale inventories is largely dependent on the honesty and forthrightness of those persons evaluated. We investigated the effectiveness of the Personality Assessment Inventory (PAI) in detecting participants feigning three specific disorders: schizophrenia, major depression, and generalized anxiety disorder. With a simulation design, we tested the PAI validity scales on 166 naive (undergraduates with minimal preparation) and 80 sophisticated (doctoral psychology students with 1 week preparation) participants. We compared their results to persons with the designated disorders: schizophrenia (n = 45), major depression (n = 136), and generalized anxiety disorder (n = 40). Although moderately effective with naive simulators, the validity scales evidenced only modest positive predictive power with their sophisticated counterparts. Therefore, we performed a two-stage discriminant analysis that yielded a moderately high hit rate (> 80%) that was maintained in the cross-validation sample, irrespective of the feigned disorder or the sophistication of the simulators.

Potentially unlimited variance reduction in importance sampling of Markov chains

We consider the application of importance sampling in steady-state simulations of finite Markov chains. We show that, for a large class of performance measures, there is a choice of the alternative transition matrix for which the ratio of the variance of the importance sampling estimator to the variance of the naive simulation estimator converges to zero as the sample path length goes to infinity. Obtaining this ‘optimal’ transition matrix involves computing the performance measure of interest, so the optimal matrix cannot be computed in precisely those situations where simulation is required to estimate steady-state performance. However, our results show that alternative transition matrices of the form Q = P + E/T, where P is the original transition matrix and T is the sample path length, can be expected to provide good results. Moreover, we provide an iterative algorithm for obtaining alternative transition matrices of this form that converge to the optimal matrix as the number of iterations increases, and present an example that shows that spending some computer time iterating this algorithm and then conducting the simulation with the resulting alternative transition matrix may provide considerable variance reduction when compared to naive simulation.

Potentially unlimited variance reduction in importance sampling of Markov chains

We consider the application of importance sampling in steady-state simulations of finite Markov chains. We show that, for a large class of performance measures, there is a choice of the alternative transition matrix for which the ratio of the variance of the importance sampling estimator to the variance of the naive simulation estimator converges to zero as the sample path length goes to infinity. Obtaining this ‘optimal’ transition matrix involves computing the performance measure of interest, so the optimal matrix cannot be computed in precisely those situations where simulation is required to estimate steady-state performance. However, our results show that alternative transition matrices of the form Q = P + E/T, where P is the original transition matrix and T is the sample path length, can be expected to provide good results. Moreover, we provide an iterative algorithm for obtaining alternative transition matrices of this form that converge to the optimal matrix as the number of iterations increases, and present an example that shows that spending some computer time iterating this algorithm and then conducting the simulation with the resulting alternative transition matrix may provide considerable variance reduction when compared to naive simulation.

The detection of exaggerated or simulated memory disorder by neuropsychological methods

Some neuropsychological tests devised to detect exaggeration and simulation of memory disorder are reviewed with reference to models of memory and our current knowledge of the behaviour of amnesic patients. These tests are based on the assumption that naive simulators or exaggerators do not know either that amnesic patients can perform them normally, or near normally, or that subjects can only fail them by deliberate suppression of the correct response. It is concluded that several such tests show promise and are worthy of further development.

Asymptotics of Likelihood Ratio Derivative Estimators in Simulations of Highly Reliable Markovian Systems

We discuss the estimation of derivatives of a performance measure using the likelihood ratio method in simulations of highly reliable Markovian systems. We compare the difficulties of estimating the performance measure and of estimating its partial derivatives with respect to component failure rates as the component failure rates tend to 0 and the component repair rates remain fixed. We first consider the case when the quantities are estimated using naive simulation; i.e., when no variance reduction technique is used. In particular, we prove that in the limit, some of the partial derivatives can be estimated as accurately as the performance measure itself. This result is of particular interest in light of the somewhat pessimistic empirical results others have obtained when applying the likelihood ratio method to other types of systems. However, the result only holds for certain partial derivatives of the performance measure when using naive simulation. More specifically, we can estimate a certain partial derivative with the same relative accuracy as the performance measure if the partial derivative is associated with a component either having one of the largest failure rates or whose failure can trigger a failure transition on one of the “most likely paths to failure.” Also, we develop a simple criterion to determine which partial derivatives will satisfy either of these properties. In particular, we can identify these derivatives using a sensitivity measure which can be calculated for each type of component. We also examine the limiting behavior of the estimates of the performance measure and its derivatives which are obtained when an importance sampling scheme known as balanced failure biasing is used. In particular, we show that the estimates of all derivatives can be improved. In contrast to the situation that arose when using naive simulation, we prove that in the limit, all derivatives can be estimated as accurately as the performance measure when balanced failure biasing is employed. Finally, we formalize the notion of a “most likely path to failure” in the setting of highly reliable Markovian systems. We accomplish this by proving a conditional limit theorem for the distribution of the sample paths leading to a system failure, given that a system failure occurs before the system returns to the state with all components operational. We use this result to establish our other results.

Importance Sampling for the Simulation of Highly Reliable Markovian Systems

In this paper we investigate importance sampling techniques for the simulation of Markovian systems with highly reliable components. The need for simulation arises because the state space of such systems is typically huge, making numerical computation inefficient. Naive simulation is inefficient due to the rarity of the system failure events. Failure biasing is a useful importance sampling technique for the simulation of such systems. However, until now, this technique has been largely heuristic. We present a mathematical framework for the study of failure biasing. Using this framework we derive variance reduction results which explain the orders of magnitude of variance reduction obtained in practice. We show that in many cases the magnitude of the variance reduction is such that the relative errors of the estimates remain bounded as the failure rates of components tend to zero. We also prove that the failure biasing heuristic in its original form may not give bounded relative error for a large class of systems and that a modification of the heuristic works for the general case. The theoretical results in this paper agree with experiments on the subject which have been reported in a previous paper.

SADDLEPOINT METHODS FOR BOOTSTRAP CONFIDENCE BANDS IN NONPARAMETRIC REGRESSION

SummaryBootstrap techniques have been used to construct confidence bands in nonparametric regression problems (Härdle & Bowman, 1988). Yet the required simulation is generally computationally intensive and therefore makes it difficult to conduct further investigations. In this paper, two saddlepoint methods are considered as alternatives to the naive simulation procedure. Some improvements to Härdle & Bowman's bootstrap method are suggested. The improvements are numerically verified using these efficient and accurate analytic methods.

Inclusion-Exclusion-Bonferroni Identities and Inequalities for Discrete Tube-Like Problems via Euler Characteristics

Improvements to the classical inclusion-exclusion identity are developed. There are two main results: an abstract combinatoric result and a concrete geometric result. In the abstract result conditions are given which guarantee the existence of a depth $d + 1$ identity or inequality for the indicator function of a union of a finite collection of events, that is, an expression which is a linear combination of indicator functions of at most $(d + 1)$-fold intersections of the events. Such an identity or inequality can be integrated with respect to any probability measure to yield a probability identity or inequality. Connections are given to previous work on Bonferroni-type inequalities. The concrete result says that there is a depth $d + 1$ identity for the union of finitely many balls in $d$-dimensional Euclidean space. With a single correction term this result also holds in the $d$-dimensional sphere. These results form the basis for a discrete theory of tubes, which up to now has been continuous in nature. The spherical result is used to give a simulation method for finding critical probabilities for multiple-comparisons procedures, and a computer program implementing the method is described. Numerical results are presented which demonstrate that in the tails of the distribution probability estimates based on the method tend to exhibit less variability than estimates based on naive simulation.

Simple stochastic networks: Some problems and procedures

Abstract : The paper provides an analysis of 'stochastic networks,' that is networks whose underlying link times are expressed as random variables. Analytic and Monte Carlo methods for finding the distribution function (or parameters thereof) of the maximal time through such a network are discussed. The first section, which considers analytical procedures, emphasizes networks whose link times are given by probability functions from an exponential family. It is shown that the Markov property of the exponential distribution may be utilized to simplify analytic computations. In the second section Monte Carlo techniques are described which provide useful information on project completion time distributions. It is argued that these easily applied techniques are more economical (fewer computations are required) than naive simulation. Throughout the paper, we suggest a blending of the analytical and simulation procedures into an efficient overall method for studying stochastic networks. (Author)