Sequential experimental design and response optimisation

Abstract

We consider the situation where one wants to maximise a functionf(θ,x) with respect tox, with θ unknown and estimated from observationsy k . This may correspond to the case of a regression model, where one observesy k =f(θ,x k )+ε k , with ε k some random error, or to the Bernoulli case wherey k ∈{0, 1}, with Pr[y k =1|θ,x k |=f(θ,x k ). Special attention is given to sequences given by\(x_{k + 1} = \arg \max _x f(\hat \theta ^k ,x) + \alpha _k d_k (x)\), with\(\hat \theta ^k \) an estimated value of θ obtained from (x1, y1),...,(x k ,y k ) andd k (x) a penalty for poor estimation. Approximately optimal rules are suggested in the linear regression case with a finite horizon, where one wants to maximize ∑ N i=1 w i f(θ, x i ) with {w i } a weighting sequence. Various examples are presented, with a comparison with a Polya urn design and an up-and-down method for a binary response problem.