Real-Time Adjustment of Tracking Offsets Through a Brain-Computer Interface for Weight Perception in Virtual Reality

Abstract

Provision of the perception of pseudo-weight through tracking offsets in virtual reality (VR) allows users to estimate the weight of virtual objects. However, the contribution of the user's real-time perception to such illusions is unknown. Here, we focus on this issue using a brain-computer interface (BCI), through which the user's perception of the weight of virtual objects can be detected in real-time and used to adjust the tracking offset in a closed loop. We first trained a computational model with electroencephalography (EEG) data by asking users to imagine lifting a heavy or a light ball. With this model, the user's perception of the object weight could be detected through the BCI in real-time to adjust the tracking offset, thereby enabling further generation of a more realistic visual sensation. Then, we evaluated the effects of the BCI tracking offset on the perception of the weights of three virtual objects used to simulate real objects, namely, tennis, billiard, and bowling balls. Our results showed that the BCI tracking offset could assist participants in generating perceived weights for virtual objects in VR. We further showed that our approach can provide weight perception through real-time adjustment of the tracking offset, which might be useful for new virtual objects that appear suddenly in the virtual environment. Additionally, most participants (78%) preferred this BCI tracking offset system for weight perception. This article provides the first quantification of weight perception for a virtual object with a BCI that can be used to adjust the tracking offset in real-time for pseudo-weight perception in VR.