A Wizard of Oz Study Exploring How Agreement/Disagreement Nonverbal Cues Enhance Social Interactions for Individuals Who Are Blind

Abstract

Given their visual nature, nonverbal social cues, such as facial and head movements, are largely inaccessible to individuals who are blind, limiting the information gleaned during interactions. While social assistive aids have explored some nonverbal cues, such as detecting and communicating facial expressions, relatively few nonverbal cues have been explored. A thorough and systematic study has yet to investigate the importance and usefulness of many nonverbal social cues for individuals who are blind. This work takes this first step by beginning to explore the nonverbal cue of agreement/disagreement as indicated by head/body movements including head nod, head shake, leaning forward and leaning backward. To facilitate the investigation of the usefulness of nonverbal cues for individuals who are blind, we propose the use of a Wizard of Oz experiment to rapidly evaluate nonverbal communications using existing technologies rather than building new and complete systems. We first explore the usefulness of agreement/disagreement nonverbal cues using our existing Social Interaction Assistant platform in which most of the seemingly automated processes were manually performed by a wizard without the knowledge of participants. We conducted an experiment with 11 individuals who are blind or visually impaired involving one-on-one interactions with trained interviewers. Results show the potential of agreement/disagreement nonverbal cues within the social interactions of individuals who are blind.