Abstract
Attention supports our urge to forage on social cues. Under certain circumstances, we spend the majority of time scrutinising people, markedly their eyes and faces, and spotting persons that are talking. To account for such behaviour, this article develops a computational model for the deployment of gaze within a multimodal landscape, namely a conversational scene. Gaze dynamics is derived in a principled way by reformulating attention deployment as a stochastic foraging problem. Model simulation experiments on a publicly available dataset of eye-tracked subjects are presented. Results show that the simulated scan paths exhibit similar trends of eye movements of human observers watching and listening to conversational clips in a free-viewing condition.
Highlights
Consider a clip displaying social interactions, in particular a conversational clip: the chief concern of this article is to model the deployment of attention through gaze by a human subject who is viewing and listening to the clip.Why should this research problem be relevant beyond its merits?One straightforward reason lies in the classic data mining hurdle
The rationale behind experiments is to figure out whether simulated behaviours are characterised by statistical properties that are significantly close to those featured by human subjects who have been eye-tracked while watching conversational videos
Any model can be considered adequate if model-generated scan paths could have been generated by human observers while attending to the same audio-visual stimuli
Summary
Consider a clip displaying social interactions, in particular a conversational clip (audio and video): the chief concern of this article is to model the deployment of attention through gaze by a human subject who is viewing and listening to the clip. The free-viewing task given to subjects allows for dynamically inferring the history of their ‘‘internal’’ selection goals as captured by the resulting attentive gaze behaviour. 5) Different from the current propensity towards endto-end approaches, the model-based behavior of gaze deployment provides an explainable account This is important if the approach is to be used in a subject’s mining context (for example, inferring socially-aware psychological traits of the perceiver or atypical development in the appraisal of social cues). The recent theoretical perspectives on active/attentive sensing [36] promote the idea that the ultimate objective of the active sensing loop (Steps 1-3) should be to maximise via exploration the long term total rewards and to gain additional knowledge about the environment This endeavour recalls that of animals foraging for food. A recent work by Tavakoli et al [74] directly learn the end-to-end mapping for the multi-modal saliency prediction by using a deep neural network instead of relying on a sampling scheme and multiple feature maps
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
