Virtual Reality for Immersive Human Machine Teaming with Vehicles

Abstract

We present developments in constructing a 3D environment and integrating a virtual reality headset in our Project Aquaticus platform. We designed Project Aquaticus to examine the interactions between human-robot teammate trust, cognitive load, and perceived robot intelligence levels while they compete in games of capture the flag on the water. Further, this platform will allows us to study human learning of tactical judgment under a variety of robot capabilities. To enable human-machine teaming (HMT), we created a testbed where humans operate motorized kayaks while the robots are autonomous catamaran-style surface vehicles. MOOS-IvP provides autonomy for the robots. After receiving an order from a human, the autonomous teammates can perform tasks conducive to capturing the flag, such as defending or attacking a flag. In the Project Aquaticus simulation, the humans control their virtual vehicle with a joystick and communicate with their robots via radio. Our current simulation is not engaging or realistic for participants because it presents a top-down, omniscient view of the field. This fully observable representation of the world is well suited for managing operations from the shore and teaching new players game mechanics and strategies; however, it does not accurately reflect the limited and almost chaotic view of the world a participant experiences while in their motorized kayak on the water. We present creating a 3D visualization through Unity that users experience through a virtual reality headset. Such a system allows us to perform experiments without the need for a significant investment in on-water experiment resources while also permitting us to gather data year-round through the cold winter months.