Optimal Sequential Allocation Research Articles

An optimal design in a two-stage ethical allocation based on U-statistics

The present article is related to a comparison between two competing treatments, say A and B in the context of a Phase III clinical trial. This comparison is made through a U-statistic estimator of the functional , where X and Y represent the continuous responses on treatments A and B respectively. Suppose the subjects are coming sequentially to a clinic and they are to be treated either by A or B. The recruitment of the subjects is done by following a two-stage ethical allocation design with an optimal sequential allocation at the first stage. Here a sequential sampling scheme with equal randomization is adopted to optimize the stage 1 sample size for a given the total sample size of the trial. Then, a response-adaptive randomization rule is employed at the second-stage with a view to achieve ethical gain measured in terms of the proportion of allocations on the better treatment. The motivation of the proposed two stage design is to assign a larger number of subjects to the more promising treatment. Here we study various asymptotic results associated with the proposed design and compare its performance with a single stage equal allocation design. Also, the proposed procedure is illustrated through a real-life data set on carcinoma in the oropharynx.

Kullback–Leibler upper confidence bounds for optimal sequential allocation

We consider optimal sequential allocation in the context of the so-called stochastic multi-armed bandit model. We describe a generic index policy, in the sense of Gittins [J. R. Stat. Soc. Ser. B Stat. Methodol. 41 (1979) 148-177], based on upper confidence bounds of the arm payoffs computed using the Kullback-Leibler divergence. We consider two classes of distributions for which instances of this general idea are analyzed: the kl-UCB algorithm is designed for one-parameter exponential families and the empirical KL-UCB algorithm for bounded and finitely supported distributions. Our main contribution is a unified finite-time analysis of the regret of these algorithms that asymptotically matches the lower bounds of Lai and Robbins [Adv. in Appl. Math. 6 (1985) 4-22] and Burnetas and Katehakis [Adv. in Appl. Math. 17 (1996) 122-142], respectively. We also investigate the behavior of these algorithms when used with general bounded rewards, showing in particular that they provide significant improvements over the state-of-the-art.

A Proof of the Bomber Problem's Spend-It-All Conjecture

The Bomber Problem concerns optimal sequential allocation of partially effective ammunition x while under attack from enemies arriving according to a Poisson process over a time interval of length t. In the doubly continuous setting, in certain regions of (x, t)-space we are able to solve the integral equation defining the optimal survival probability and find the optimal allocation function K(x, t) exactly in these regions. As a consequence, we complete the proof of the “spend-it-all” conjecture of Bartroff et al. (2010b), which gives the boundary of the region where K(x, t) = x.

Optimal sequential allocation with imperfect feedback information

A given number of bullets will be fired sequentially in an attempt to destroy as many targets as possible from a fixed number of targets. The probability of destroying a target at each shot is known. After each shot, there is a report on the state for the target; destroyed or intact. The reports are subject to the usual two types of errors and the probabilities of making these errors are also known. This paper shows that the myopic decision strategy that picks the next target to be the one with the highest intact posterior probability is the optimal strategy.

Optimal sequential allocation with imperfect feedback information

A given number of bullets will be fired sequentially in an attempt to destroy as many targets as possible from a fixed number of targets. The probability of destroying a target at each shot is known. After each shot, there is a report on the state for the target; destroyed or intact. The reports are subject to the usual two types of errors and the probabilities of making these errors are also known. This paper shows that the myopic decision strategy that picks the next target to be the one with the highest intact posterior probability is the optimal strategy.

Dynamic allocation policies for the finite horizon one armed bandit problem

The unknown performance of a new experiment is to be evaluated and compared with that of an existing one over a finite horizon. The explicit structure of an optimal sequential allocation policy is obtained under pertinent reward/loss functions, when the experiments are characterized by random variables with distributions from the one parameter exponential family.

AN ANALYSIS OF A STOCHASTIC RESOURCE ALLOCATION MODEL WITH VARIOUS UTILITY FUNCTIONS

This paper deals with stochastic models of an optimal sequential allocation of resources between consumption and production. We obtain the following results by means of the theory of martingales. For a model of resource allocation we establish the dynamic programming equation and show that a supermartingale characterizes the composition of the model and clear the composition of the optimality, representing the sufficient condition for an allocation to be optimal via a martingale. Further, we show the following three results as the application of the above result. (1) For the Kennedy's model we give another proof for the fact on an optimal allocation. (2) For a model with a convex utility function, we represent an optimal allocation via a supelmartingale. (3) For a model with a logarithmic utility function, we obtain an explicit optimal allocation.