The psi-marginal adaptive method: How to give nuisance parameters the attention they deserve (no more, no less)

Abstract

Adaptive testing methods serve to maximize the information gained regarding the values of the parameters of a psychometric function (PF). Such methods typically target only one or two ("threshold" and "slope") of the PF's four parameters while assuming fixed values for the "nuisance" parameters ("guess rate" and "lapse rate"). Here I propose the "psi-marginal" adaptive method, which can target nuisance parameters but only when this is the most optimal manner in which to reduce uncertainty in the value of the parameters of primary interest. The method is based on Kontsevich and Tyler's (1999) psi-method. However, in the proposed method a posterior distribution defined across all parameters of unknown value is maintained. Each of the parameters is specified either as a parameter of primary interest whose estimation should be optimized or as a nuisance parameter whose estimation should be subservient to the estimation of the parameters of primary interest. Critically, selection of stimulus intensities is based on the expected information gain in the marginal posterior distribution defined across the parameters of interest only. The appeal of this method is that it will target nuisance parameters adaptively and only when doing so maximizes the expected information gain regarding the values of the parameters of interest. Simulations indicate that treating the lapse rate as a nuisance parameter in the psi-marginal method results in smaller bias and higher precision in threshold and slope estimates compared to the original psi method. The method is highly flexible and various other uses are discussed.