Abstract
In this paper, an interactive learning experience is proposed, aiming to involve museum visitors in a personalized experience of the transmittal of cultural knowledge in an active and creative way. The proposed system, called HapticSOUND, consists of three subsystems: (a) the Information, where visitors are informed about the traditional musical instruments; (b) the Entertainment, where visitors are entertained by playing serious games to virtually assemble traditional musical instruments by a set of 3D objects; and (c) the Interaction, where visitors interact with a digital musical instrument which is an exact 3D-printed replica of a traditional musical instrument, where cameras have been placed to capture user gestures and machine learning algorithms have been implemented for gesture recognition. The museum visitor can interact with the lifelike replica to tactilely and aurally explore the instrument’s abilities, producing sounds guided by the system and receiving real-time visual and audio feedback. Emphasis is given to the Interaction Subsystem, where a pilot study was conducted to evaluate the usability of the subsystem. Preliminary results were promising since the usability was satisfactory, indicating that it is an innovative approach that utilizes sensorimotor learning and machine learning techniques in the context of playing sounds based on real-time gesture and fingering recognition.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.