Optimal Stopping Strategy Research Articles

Robust Multiple Stopping—A Duality Approach

We develop a method to solve, theoretically and numerically, general optimal stopping problems. Our general setting allows for multiple exercise rights—that is, optimal multiple stopping—for a robust evaluation that accounts for model uncertainty with a dominated family of priors and for general reward processes driven by multidimensional jump-diffusions. Our approach relies on first establishing robust martingale dual representation results for the multiple stopping problem that satisfy appealing almost sure pathwise optimality properties. Next, we exploit these theoretical results to develop upper and lower bounds that, as we formally show, not only converge to the true solution asymptotically, but also constitute genuine prelimiting upper and lower bounds. We illustrate the applicability of our approach in a few examples and analyze the impact of model uncertainty on optimal multiple stopping strategies. Funding: R. J. A. Laeven received financial support from the Netherlands Organization for Scientific Research (NWO) [Grants NWO-Vidi and NWO-Vici]. J. G. M. Schoenmakers received financial support from the Deutsche Forschungsgemeinschaft Excellence Cluster Math+ Berlin [Project AA4-2]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/moor.2021.0237 .

Minimizing Energy Usage and Makespan of Elevator Operation in Rush Hour Using Multi-Objective Variable Neighborhood Strategy Adaptive Search with a Mobile Application

The purpose of this study is to address two major issues: (1) the spread of epidemics such as COVID-19 due to long waiting times caused by a large number of waiting for customers, and (2) excessive energy consumption resulting from the elevator patterns used by various customers. The first issue is addressed through the development of a mobile application, while the second issue is tackled by implementing two strategies: (1) determining optimal stopping strategies for elevators based on registered passengers and (2) assigning passengers to elevators in a way that minimizes the number of floors the elevators need to stop at. The mobile application serves as an input parameter for the optimization toolbox, which employs the exact method and multi-objective variable neighborhood strategy adaptive search (M-VaNSAS) to find the optimal plan for passenger assignment and elevator scheduling. The proposed method, which adopts an even-odd floor strategy, outperforms the currently practiced procedure and leads to a 42.44% reduction in waiting time and a 29.61% reduction in energy consumption. Computational results confirmed the effectiveness of the proposed approach.

New class of optimal multiple stopping times problems

This paper is devoted to study a new discounted nonlinear optimal multiple stopping times problem with discounted factor β > 0 and infinite horizon. Under the right continuity of the underlying process, we show that the problem can be reduced to a sequence of ordinary optimal stopping problems. Next in the Markovian case, we characterise the value function of the problem in terms of β-excessive functions. Finally, in the special case of a diffusion process, we give explicit expressions for the value function of the problem as well as the optimal stopping strategy. As an explicit example in finance, we apply our theoretical results to manage a new generalised swing contract which gives its holder n rights to mark the price X of a stock, where the payment is only allowed at the final exercise time rather than at each exercise time as in the classical swing contact.

Stoptions: Representations and Applications

We introduce a new derivative security called a stoption. After paying an upfront premium, the owner of a stoption accrues realized price changes in some underlying security until the exposure is stopped by the owner. Upon stopping, the reward is the sum of all of the previous price changes plus a deterministic amount which can vary with the stopping time. Stoptions are finite-lived and hence must be stopped at or before a fixed maturity date. We propose a particular discrete-time probabilistic model for the underlying's price changes and then determine the optimal stopping strategy and stoption premium for that model in closed-form. We also present an application to DVA (debit valuation adjustment) under full collateralization.

Online reinforcement learning for adaptive interference coordination

AbstractHeterogeneous networks (HetNets), in which small cells overlay macro cells, are a cost‐effective approach to increase the capacity of cellular networks. However, HetNets have raised new issues related to cell association and interference management. In particular, the optimal configuration of interference coordination (IC) parameters is a challenging task because it depends on multiple stochastic processes such as the locations of the users, the traffic demands, or the strength of the received signals. This work proposes a self‐optimization algorithm capable of finding the optimal configuration in an operating network. We address the problem using a reinforcement learning (RL) approach, in which the actions are the IC configurations, whose performances are initially unknown. The main difficulty is that, due to the variable network conditions, the performance of each action may change over time. Our proposal is based on two main elements: the sequential exploration of subsets of actions (exploration regions), and an optimal stopping (OS) strategy for deciding when to end current exploration and start a new one. For our algorithm, referred to as local exploration with optimal stopping (LEOS), we provide theoretical bounds on its long‐term regret per sample and its convergence time. We compare LEOS to state‐of‐the‐art learning algorithms based on multiarmed bandits and policy gradient RL. Considering different changing rates in the network conditions, our numerical results show that LEOS outperforms the first alternative by 22%, and the second one by 48% in terms of average regret per sample.

On optimal stopping strategies for text recognition in a video stream as an application of a monotone sequential decision model

The paper describes the problem of stopping the text field recognition process in a video stream, which is a novel problem, particularly relevant to real-time mobile document recognition systems. A decision-theoretic framework for this problem is provided, and similarities with existing stopping rule problems are explored. Following the theoretical works on monotone stopping rule problems, a strategy is proposed based on thresholding the estimation of the expected difference between consequent recognition results. The efficiency of this strategy is evaluated on an openly accessible dataset. The results show that this method outperforms the previously published methods based on identical results cluster size thresholding. Notes on future work include incorporation of recognition result confidence estimations in the proposed model and more precise evaluation of the observation cost.

Optimal stopping of Brownian motion with broken drift

AbstractWe solve an optimal stopping problem where the underlying diffusion is Brownian motion on with a positive piecewise constant drift changing at zero. It is assumed that the drift μ1 on the negative side is smaller than the drift μ2 on the positive side. The main observation is that if , then there exist values of the discounting parameter for which it is not optimal to stop in the vicinity of zero where the drift changes. However, when the discounting gets larger, the stopping region becomes connected and contains zero. This is in contrast with results concerning optimal stopping of skew Brownian motion where the skew point is for all values of the discounting parameter in the continuation region in case the skew parameter is bigger than 1/2. In all cases, a practical implementation of these optimal stopping strategies, based on hitting times of boundaries of continuation regions, would require access to the highest‐possible frequency data which is consistent with the continuous‐time modeling framework.

Discretionary stopping of stochastic differential equations with generalised drift

We consider the problem of optimally stopping a general one-dimensional stochastic differential equation (SDE) with generalised drift over an infinite time horizon. First, we derive a complete characterisation of the solution to this problem in terms of vari- ational inequalities. In particular, we prove that the problem’s value function is the difference of two convex functions and satisfies an appropriate variational inequality in the sense of distributions. We also establish a verification theorem that is the strongest one possible because it involves only the optimal stopping problem’s data. Next, we derive the complete explicit solution to the problem that arises when the state process is a skew geometric Brownian motion and the reward function is the one of a financial call option. In this case, we show that the optimal stopping strategy can take sev- eral qualitatively different forms, depending on parameter values. Furthermore, the explicit solution to this special case shows that the so-called “principle of smooth fit” does not hold in general for optimal stopping problems involving solutions to SDEs with generalised drift.

An Optimal Stopping Approach to Cell Selection in 5G Networks

Initial cell search and selection is one of the first few essential steps that a mobile device must perform to access a mobile network. The traditional maximum received power based initial cell selection may result in unbalanced loads across the cells in multi-tier heterogeneous 5G networks with millimeter wave. The objective of this work is to improve the initial cell search and selection scheme to achieve a more balanced load distribution in 5G networks. We propose to augment the existing pure maximum received power based scheme with a new load factor broadcast as part of system information. To study the theoretical performance of the proposed initial cell search and selection scheme, we formulate a throughput optimization problem and solve it by using optimal stopping theory. We characterize the throughput optimal stopping strategy and the attained maximum throughput. We prove that the proposed initial cell search and selection scheme is throughput optimal with a carefully optimized connection threshold. The optimal connection threshold and maximum throughput can be found by solving a fixed point equation. We further develop an iterative algorithm to compute the solution to the fixed point equation and prove the convergence of the proposed iterative algorithm. We also provide simulation results to demonstrate the analytical results and obtain insights. Our analysis and results show that the proposed initial cell search and selection scheme can help achieve a more balanced load distribution in 5G networks.

Optimal Wireless Service Within Average Delay

Service provision over wireless networks is a key performance criterion, yet an onerous task due to the stochastic nature of channels, transmission overhead, and service delay requirements. This paper considers services differentiated by preference without benefit or penalty for early or delayed completion, instead a benefit exists for maintaining the average delay of completed services below a predefined limit. The average delay is constrained at each network instance via joint optimization of probing order and transmission decisions, resulting in a nonlinear expanding-then-collapsing optimization constraint. A permutation in the transmission order does not impact the sum delay but can impact the average delay. Therefore, finding the optimal order is paramount for proper service provision. A particular greedy ordering is shown to reduce the set of feasible solutions without losing global optimality. Practical solutions are proposed via stochastic programming formulation and an optimal stopping strategy that does not use a priori knowledge. The stopping strategy results in a set of thresholds, which can be practically computed a priori and implemented in a decentralized manner. Simulations show that our proposed Dantzig inspired stopping strategy achieves near optimal sum service value with higher throughput than a stochastic programming formulation.

A sufficient stochastic maximum principle for a kind of recursive optimal control problem with obstacle constraint

In this paper, we study a kind of recursive optimal control problem whose utility functional is described by the solution of a reflected backward stochastic differential equation (BSDE). We obtain a sufficient stochastic maximum principle of optimal controls. Moreover, a mixed optimal control problem is considered to illustrate the application of our theoretical result and the optimal control and stopping strategy are given.

Maximizing survival time in a random walk on an interval

ABSTRACTA gambler buys N tokens that enable him to play N rounds of the following game. A symmetric random walk on a discrete interval { − r, …, r} starts from the point 0. The gambler knows only the number of steps made so far, but is unaware of the current position of the walk. Once the walk hits one of the barriers − r or r for the first time in the current round, the round ends with no payoff. The gambler can start a new round by inserting a new token, if there are any tokens left. The gambler can end the game at any time getting the payoff equal to the number of steps made in the current round. We find the optimal stopping strategy for this game and calculate the expected payoff once the optimal strategy is applied.

Numerical scheme for Dynkin games under model uncertainty

We introduce an efficient numerical scheme for continuous time Dynkin games under model uncertainty. We use the Skorokhod embedding in order to construct recombining tree approximations. This technique allows us to determine convergence rates and to construct numerically optimal stopping strategies. We apply our method to several examples of game options.

SELLING AT THE ULTIMATE MAXIMUM IN A REGIME-SWITCHING MODEL

This paper deals with optimal prediction in a regime-switching model driven by a continuous-time Markov chain. We extend existing results for geometric Brownian motion by deriving optimal stopping strategies that depend on the current regime state and prove a number of continuity properties relating to optimal value and boundary functions. Our approach replaces the use of closed form expressions, which are not available in our setting, with PDE arguments that also simplify the approach of [du Toit & Peskir (2009) Selling a stock at the ultimate maximum, Annals of Applied Probability 19 (3), 983–1014.] in the classical Brownian case.

Technical Note—Path-Dependent and Randomized Strategies in Barberis’ Casino Gambling Model

We consider the dynamic casino gambling model initially proposed by Barberis (2012) and study the optimal stopping strategy of a precommitting gambler with cumulative prospect theory (CPT) preferences. We illustrate how the strategies computed in Barberis (2012) [Barberis N (2012) A model of casino gambling. Management Sci. 58(1): 35–51.] can be strictly improved by reviewing the betting history or by tossing an independent coin, and we explain that the improvement generated by using randomized strategies results from the lack of quasi-convexity of CPT preferences. Moreover, we show that any path-dependent strategy is equivalent to a randomization of path-independent strategies.

Lower Bounds for Bruss’ Odds Problem with Multiple Stoppings

This paper addresses Bruss’ odds problem with multiple stopping chances. A decision maker sequentially observes a sequence of independent 0/1 (failure/success) random variables to correctly predict the last success with multiple stopping chances. First, we give a nontrivial lower bound of the probability of win (obtaining the last success) for the problem with m-stoppings. Next, we show that the asymptotic value for each classical secretary problem with multiple stoppings attains our lower bound. Finally, we prove a conjecture on the classical secretary problem, which gives a connection between the probability of win and the threshold values of the optimal stopping strategy.

Optimal Liquidation of an Asset under Drift Uncertainty

We study a problem of finding an optimal stopping strategy to liquidate an asset with unknown drift. Taking a Bayesian approach, we model the initial beliefs of an individual about the drift by all ...

Design of limited-stop bus service with capacity constraint and stochastic travel time

This paper develops a mathematical model for the optimal stopping design of limited-stop bus service, which allows each bus vehicle to skip some stops. To better reflect the reality, this paper considers the vehicle capacity and stochastic travel time. Also, vehicles are all allowed to skip stops whereas any stop is not allowed to be skipped by two consecutive vehicles. A hybrid artificial bee colony (ABC) and Monte Carlo method is developed to solve the optimal stopping strategy. Finally, the model and solution method are validated by a numerical example, and a sensitivity analysis is performed on the passenger demand.

Optimal Traffic Scheduling Between Roadside Units in Vehicular Delay-Tolerant Networks

This paper investigates the problem of traffic scheduling between roadside units (RSUs) in a vehicular delay-tolerant network. A source RSU needs the help of passing-by vehicles to forward its traffic to a destination RSU. Costs are associated with energy consumption and information loss. When a vehicle arrives, the source RSU needs to decide whether to stop, i.e., transmit its traffic to the vehicle, or skip this vehicle and continue to wait for other vehicles. The source RSU's objective is to achieve the minimal rate of cost. In this paper, we theoretically derive an optimal stopping strategy of the source RSU. Simulation results are presented to show the effectiveness of the derived strategy.

Optimal Casino Betting: Why Lucky Coin and Good Memory are Important

We consider the dynamic casino gambling model initially proposed by Barberis [Manage. Sci., 2012, 58, 35-51] and study the optimal stopping strategy of a pre-committing gambler with cumulative prospect theory (CPT) preferences. We illustrate how the strategies computed in Barberis [2012] can be strictly improved by reviewing the betting history or by tossing an independent coin, and we explain that the improvement generated by using randomized strategies results from the lack of quasi-convexity of CPT preferences. Moreover, we show that any path-dependent strategy is equivalent to a randomization of path-independent strategies.