Effects of emotional feedback on children, using different modalities

Abstract

The potential for involving emotions in human–technology interaction has received considerable attention in many fields of design, as it has many implications in the design of educational applications to improve learner–computer interaction. This study attempted to create emotionally supportive learning environments by adding emotional feedback through cues from vocal and facial expression. Faces and voices are powerful sources of emotional information through visual and auditory modalities, the two main components of multimedia. Two quasi-experiments were conducted to investigate the impact of visual and auditory emotional feedback on the instruction of school-age children with regard to their perceived social presence, intrinsic motivation, and performance during the learning process. The independent variables in the first experiment were speech and static facial expression. A total of 173 sixth graders participated in the experiments. Results of the first experiment demonstrated that children preferred the auditory modality of speech to the visual modality of facial expression. In the second experiment, the independent variables were speech and dynamic facial expression. The findings revealed no significant differences between speech and dynamic facial expression, indicating that emotional feedback through audio displays or visual displays could function equally in the social perceptions of children, and help to motivate them in learning activities. Dynamic facial expression and speech work alternately. The findings of this study may help designers to create emotionally supportive learning environments for children, using emotional feedback through speech and facial expression.