Discrete Time Optimal Stopping Problems Research Articles

Optimal Condition-Based Mission Abort Decisions

Failures of safety-critical mission-based systems, such as aircraft and submarines, could result in significant losses and damage. To enhance the survivability of such systems, their missions may be aborted if the failure risk becomes too high. We investigate such mission abort policies under a completely observed two-stage degradation process that progresses stochastically from “normal” to “defective” to “failure.” Mission abort decisions are considered as a function of the duration of the defective stage. This mission abort problem is formulated as a discrete-time optimal stopping problem with the goal of minimizing the expected total cost of mission failure and system failure. In addition to deriving some structural properties, we also numerically evaluate several intuitive heuristic policies. Finally, a joint optimization problem is formulated to simultaneously identify the optimal mission abort policy and the optimal investment to delay system deterioration.

The Optimal Equilibrium for Time-Inconsistent Stopping Problems---The Discrete-Time Case

We study an infinite-horizon discrete-time optimal stopping problem under non-exponential discounting. A new method, which we call the iterative approach, is developed to find subgame perfect Nash equilibria. When the discount function induces decreasing impatience, we establish the existence of an equilibrium through fixed-point iterations. Moreover, we show that there exists a unique optimal equilibrium, which generates larger value than any other equilibrium does at all times. To the best of our knowledge, this is the first time a dominating subgame perfect Nash equilibrium is shown to exist in the literature of time-inconsistency.

Characterizing the Structure of the Optimal Stopping Policies Arising in Operations

This paper studies a stochastic model of optimal stopping processes that arise frequently in operational problems (e.g., when a manager needs to determine an optimal epoch to stop a process). For such problems, we propose an effective method that characterizes the structure of the optimal stopping policy for the class of discrete-time optimal stopping problems. Using the method, we also provide a set of metatheorems that characterize when a threshold or control-band type stopping policy is optimal. We show that our proposed method can characterize the structure of the optimal policy for some stopping problems for which conventional methods fail to do so. Our method also simplifies the analysis of some existing results. In addition, the metatheorems help identify sufficient conditions that yield simple optimal policies when such policies are not generally optimal. We show the aforementioned benefits of our method by applying it to several optimal stopping problems frequently encountered, for example, in operations, marketing, finance and economics literature. We remark that structural results make an optimal-stopping policy easier to follow, describe, compute and hence implement. They also help understand how a stopping policy should respond to changes in the operational environment. In addition, structural results are critical for the development of efficient algorithms to solve optimal stopping problems numerically.

Discrete Time Optimal Stopping Problems with Fractional Rewards

In this paper, to the first, we give the general formulation of a discrete time optimal stopping problem under the criterion of a fractional reward. Using the standard mathematical argument in the theory of mathematical programming, we prove the existence of optimal stopping times for this problem. Furthermore we discuss ε-optimal stopping times, finite stopping times, and Markov case, respectively. To the next, we give the formulation of new stopping problems which combined the discrete time multiple stopping problem and the criterion of a fractional reward, and then prove the existence of optimal stopping times for this problem.

In This Issue

In This Issue

On the rates of convergence of simulation-based optimization algorithms for optimal stopping problems

In this paper, we study simulation-based optimization algorithms for solving discrete time optimal stopping problems. Using large deviation theory for the increments of empirical processes, we derive optimal convergence rates for the value function estimate and show that they cannot be improved in general. The rates derived provide a guide to the choice of the number of simulated paths needed in optimization step, which is crucial for the good performance of any simulation-based optimization algorithm. Finally, we present a numerical example of solving optimal stopping problem arising in finance that illustrates our theoretical findings.

On the Robustness of the Snell Envelope

We analyze the robustness properties of the Snell envelope backward evolution equation for the discrete time optimal stopping problem. We consider a series of approximation schemes, including cut–off-type approximations, Euler discretization schemes, interpolation models, quantization tree models, and the stochastic mesh method of Broadie and Glasserman. In each situation, we provide nonasymptotic convergence estimates, including $\mathbb{L}_p$-mean error bounds and exponential concentration inequalities. We deduce these estimates from a single and general robustness property of Snell envelope semigroups. In particular, this analysis allows us to recover existing convergence results for the quantization tree method and to improve significantly the rates of convergence obtained for the stochastic mesh estimator of Broadie and Glasserman. In the second part of the article, we propose a new approach based on a genealogical tree approximation model of the reference Markov process in terms of a neutral-type genetic model. In contrast to Broadie–Glasserman Monte Carlo models, the computational cost of this new stochastic approximation is linear in the number of particles. Some simulation results are provided and confirm the interest of this new algorithm.

Best choice from the planar Poisson process

Various best-choice problems related to the planar homogeneous Poisson process in a finite or semi-infinite rectangle are studied. The analysis is largely based on the properties of the one-dimensional box-area process associated with the sequence of records. We prove a series of distributional identities involving exponential and uniform random variables, and give a resolution to the Petruccelli–Porosinski–Samuels paradox on the coincidence of asymptotic values in certain discrete-time optimal stopping problems.

A quantization algorithm for solving multidimensional discrete-time optimal stopping problems

A new grid method for computing the Snell envelope of a function of an $\mathbb{R}^d$-valued simulatable Markov chain $(X_k)_{0\lambda \leq k\lambda \leq n}$ is proposed. (This is a typical nonlinear problem that cannot be solved by the standard Monte Carlo method.) Every $X_k$ is replaced by a `quantized approximation' $\widehat{X}_k$ taking its values in a grid $\Gamma_k$ of size $N_k$. The $n$ grids and their trans\-ition probability matrices form a discrete tree on which a pseudo-Snell envelope is devised by mimicking the regular dynamic programming formula. Using the quantization theory of random vectors, we show the existence of a set of optimal grids, given the total number $N$ of elementary $\mathbb{R}^d$-valued quantizers. A recursive stochastic gradient algorithm, based on simulations of $(X_k)_{0\lambda \leq k \lambda \leq n}$, yields these optimal grids and their transition probability matrices. Some a priori error estimates based on the $L^p$-quantization errors $\|X_k-\widehat X_k\|_{_p}$ are established. These results are applied to the computation of the Snell envelope of a diffusion approximated by its (Gaussian) Euler scheme. We apply these result to provide a discretization scheme for reflected backward stochastic differential equations. Finally, a numerical experiment is carried out on a two-dimensional American option pricing problem.

Optimal stopping problem with finite-period reservation

This paper presents a discrete-time optimal stopping problem with a finite planning horizon. For a search cost s, a random offer, w∼ F( w), will be found for each time. Offers appearing subsequently are allowed to be not only accepted or passed up but also “reserved” for recall later. To reserve an offer with value w, a reserving cost r( w) is incurred with each reserved offer expiring k periods after. The objective is to maximize the expected discounted net profit with the provision that an offer must be accepted. The major finding is that an offer reserved during the search process must not be accepted prior to its maturity of reservation, however, it may be accepted on the maturity.

Optimal Stopping Problem with Controlled Recall

This paper deals with the following discrete-time optimal stopping problem. For fixed search costs, a random offer, w ~ F(w), will be found for each time. This offer is either accepted, rejected, or “reserved” for recall later. The reserving cost for any offer depends on its value, regardless of how long the offer is reserved. The objective is to maximize the expected discounted net profit, provided that an offer must be accepted. The major finding is that no previously reserved offer should be accepted prior to the deadline of the search process.