Revealing multisensory benefit with diffusion modeling

Abstract

Multisensory information can benefit perceptual, memory, and decision-making processes. These benefits commonly manifest in superior detection and discrimination of multisensory stimuli, as well as improved perception and subsequent memory of unisensory representation of an object previously encoded in a multisensory context. However, the vast majority of studies to date analyze accuracy, sensitivity and/or reaction time data independently to compare multisensory and unisensory conditions. Considering the well-established speed-accuracy trade-off, we asked whether some multisensory benefits go unnoticed when measured using traditional methods that do not take both reaction time and accuracy into account simultaneously, and whether an approach combining them can more reliably characterize and quantify the broad extent of multisensory interactions across perception and cognition. While drift diffusion models have been previously shown to be effective in addressing the speed-accuracy trade-off and providing a reliable and accurate measure of multisensory benefits, one impediment of this approach is the requirement of a large number of trials to estimate model parameters and to characterize effects. This may be prohibitive in many experimental paradigms. Several model variants attempt to reduce the required number of trials, either by averaging across participants or limiting the search space for the parameters. Here, we employed a hierarchical drift diffusion model, that utilizes Bayesian priors, allowing parameter estimation with smaller sample sizes while still making subject-specific parameter estimates. We analyzed data in perceptual detection and discrimination tasks across multiple sensory combinations, to investigate if the diffusion model would provide a sensitive and reliable measure of multisensory benefits. Results indicate that across visual, auditory and tactile modality combinations, the diffusion model was either as or more sensitive than traditional accuracy, sensitivity, or reaction time measures, and was the only measure that consistently detected multisensory benefits in a statistically significant fashion. We recommend the use of diffusion modeling approaches when assessing the outcomes of multisensory experiments, especially as they become more computationally efficient.