Virtual Makerspaces: Merging AR/VR/MR to Enable Remote Collaborations in Physical Maker Activities

Abstract

We present a mixed-reality system for remote collaborations, where collaborators can discuss, explore, create and learn about 3D physical objects. The system combines Hololens augmented reality, 3D Kinect cameras, PC and virtual reality interfaces, into a virtual space that hosts remote collaborators, and physical & virtual objects. When talented people have access to fabrication tools and expertise, incredible inventions can be manifested to solve local and global problems. Because of this, the “maker movement” is a growing phenomenon, manifested through the increased number of maker spaces in many affluent communities. However, many talented individuals lack access to resources in their local communities, and collaboration opportunities with remote experts are wasted due to the limitations of current teleconferencing systems. We present a mixed-reality (MR) system for enabling remote collaborations in the context of maker activities, that allows groups of students and instructors to discuss, explore, create and learn about physical objects. The system combines augmented reality (AR) headsets, 3D cameras, PC and virtual reality (VR) interfaces, into a virtual space that contains multiple remote students, instructors, physical and virtual objects. Remote students can see a real-time 3D scan of the on-site user's physical environment, and the virtual avatars of other students. The system can support learning and exploration by showing virtual overlays on real objects (ex: showing a physical robot's sensor data or internal circuitry) while responding to real-time manipulation of physical objects by the on-site user; and can support design activities by allowing remote and local participants to annotate physical objects with virtual drawings and virtual models. This platform is being developed as an open-source project, and we are currently building applications with the intention to deploy in hybrid makerspace classrooms involving on-site and remote students.