A brain-computer interface with gamification in the Metaverse

Abstract

This study contributes to our understanding of the Metaverse by presenting a case study of the implementation of brain-computer interface supported game-based engagement in a Virtual Environment (VE). In VE, individuals can communicate with anyone, anywhere, anytime, without any limits. This situation will increase the barrier-free living standards of disabled people in a more accessible environment. A virtual world of well-being awaits these individuals, primarily through gamified applications thanks to Brain-Computer Interfaces. Virtual environments in the Metaverse can be infinitely large, but the user's movement in a virtual reality (VR) environment is constrained by the natural environment. Locomotion has become a popular motion interface as it allows for full exploration of VE. In this study, the teleport method from locomotion methods was used. To teleport, the user selects the intended location using brain signals before being instantly transported to that location. Brain signals are decomposed into alpha, beta, and gamma bands. The features of each band signal in Time, frequency, and time-frequency domains are extracted. In this proposed method, the highest performance of binary classification was obtained in the frequency domain and the Alpha band. Signals in the alpha band were tested in the domains Time, Frequency, and Time-Frequency. Teleport operations are faster with Time and more stable with the frequency domain. However, the Hilbert-Huang Transform (HHT) method used in the Time-Frequency domain could not respond adequately to real-time applications. All these analyses were experienced in the Erzurum Virtual Tour case study, which was prepared to promote cultural heritage with the gamification method.