Application of Monte-Carlo simulations for validating adaptive psychophysical procedures

Abstract

In psychoacoustics, data collection from human listeners is commonly done by adaptively varying the stimulus parameters according to the data collected from preceding trials during an auditory task. Compared to the method of constant stimuli, the adaptive procedures allow the adjustment of stimuli for individual listeners to achieve a desired performance range. The adaptive procedures may be designed to estimate performance thresholds, psychometric functions, or the parameters of psychophysical models. The algorithms within these procedures that govern the selection of stimulus parameters may be based on heuristics or Bayesian active learning, and they may differ in terms of computational complexity, accuracy, reliability, stability, and rate of convergence. Choosing an appropriate adaptive procedure for a specific application can be assisted by Monte-Carlo simulations, in which candidate procedures are run on simulated listeners whose responses are governed by ground truth psychometric functions or psychophysical models. The estimated model parameters can be then compared to the ground-truth parameters to evaluate the performance of each adaptive procedure. In this presentation, an introduction of this approach will be provided with examples from psychoacoustics. The analysis techniques that address accuracy, reliability, stability, and rate of convergence will be discussed.