Exploring Intuitive Visuo-Tactile Interaction Design for Culture Education: A Chinese-Chess-Based Case Study

Abstract

Gamified and immersive educational interventions have proven to be powerful motivators to encourage participation in unattractive activities, and human–computer interaction (HCI) is crucial for the successful development of such educational interventions. However, existing interaction methods suffer from either visuo-tactile inconsistency or unfriendly interaction methods, resulting in an unenjoyable user experience. Hence, in this article, the visuo-tactile interaction design based on Norman’s design principles (visibility, feedback, constraint, consistency, affordance, and mapping) and spatial augmented reality (SAR) for intuitive cultural education is proposed for the first time, and an application for Chinese chess, SARChess, is implemented as a case study. Players interact with tangible chess pieces rather than through a digital interface, and the visual output and tangible input are integrated seamlessly to achieve high-level visuo-tactile consistency. Furthermore, a mixed investigation method based on pretest–posttest design, subjective questionnaires, and theme-based interviews was designed and performed to evaluate the user experience, learning performance, and player motivation of SARChess from both objective and subjective perspectives. The results show that SARChess has a significantly better user experience, learning performance, and player motivation than conventional chess and mouse-interaction-based chess, and the unique gameplay and visual cues in SARChess make it more engaging to participants. Additionally, this study finds and summarizes some design implications for applying design principles to traditional cultural education applications.