A VR-based BCI interactive system for UAV swarm control

Abstract

The traditional Unmanned Aerial Vehicle (UAV) swarm control mainly adopts the ground station method, which is too fixed, and the interaction is difficult to meet the high dynamic task requirements. There is an urgent need for new interaction methods to integrate the advantages of human thinking in dealing with uncertain problems. Nevertheless, brain-computer interface(BCI) technology is directly controlled by thoughts, one of the most promising next-generation human–computer interaction technologies. Therefore, in this study, we innovatively applied the BCI system based on Virtual Reality (VR) to the group UAV and realized a novel and intelligent group control method, which proposes new ideas and paradigms for the control of swarm UAVs in the future. Specifically, this study takes a quadcopter as an example. A modular and extensible multi-quadcopter system was created, and then a visual stimulation 3D VR scene system with a digital twin function was established. On this basis, the BCI system based on the Stable state visual evoked potential (SSVEP) paradigm was adopted for the swarm control of the quadcopter. The experimental results show that the formation control of multi-quadcopter is successfully realized by the subjects using the proposed VR-based BCI interactive system, with an accuracy rate of 90% and a good performance in information transmission rate. In addition, the immersive VR twin system established one-to-one for EEG signal acquisition allows subjects to have a better experience.