Audio-haptic intermodal coupling for comparative search tasks

Abstract

Existing works on audio-haptic coupling mainly exploit the spatial complementarity between haptic and audio channels, for local and distant information respectively (e.g., target search in 2D/3D space). This work explores a new coupling strategy in which users can simultaneously compare two different values in order to deduce if they present equivalent or different intensities. Based on suitable rendering functions, we propose to render the reference value on the audio channel and the explored function (variable intensity) on the haptic channel. The use of a tempo parameter in both sensorial signals creates a close connection between the two channels and enables an intuitive and efficient comparison of the two values. Experimental evaluation shows the benefits of the proposed approach. However, we observed that a selection of users tended to underestimate extreme values (high and low). This sensorial comparison strategy will be used in future investigations to improve the search of target scores during molecular docking tasks.