AirPiano: Enhancing music playing experience in virtual reality with mid-air haptic feedback

Abstract

We present AirPiano, an enhanced music playing system to provide touchable experiences in HMD-based virtual reality with mid-air haptic feedback. AirPiano allows users to enjoy enriched virtual piano-playing experiences with touchable keys in the air. We implement two haptic rendering schemes to mimic the resisting force of piano keys using ultrasonic vibrations. Constant Feedback is designed to provide short and intense feedback whereas Adaptive Feedback is designed to follow the changes in feedback from a real keypress. A user study is conducted using three simple musical pieces. Results show that adding mid-air haptic rendering can significantly improve the user experience on the virtual piano. Adaptive Feedback further increases clarity, reality, enjoyment, and user satisfaction, when compared to circumstances in which no haptic feedback is being generated. Our design of AirPiano can shed light on the introduction of a touchable musical playing system in virtual reality with mid-air haptic feedback.