Mixed reality and digital twins for astronaut training

Abstract

Astronaut training is complex due to its specific infrastructure and equipment requirements, which are often costly. Immersive technologies can simulate entire environments, allowing for interactive user experiences. Previous research has confirmed the effectiveness of immersive technologies in training and learning. However, simulations alone are insufficient; participants must also engage with vehicles, machinery, and spacecraft systems to understand their impact on the environment and adjacent objects. This study develops four scenarios within three modules, inspired by NASA and ESA astronaut training programs. The first module presents a theoretical scenario using mixed reality (MR), incorporating topics from ESA's training modules. The second module offers two practical scenarios: an ISS emergency simulation that includes Hohmann transfer and circular motion concepts, and a spacewalk for repair tasks. In these scenarios, digital twins of the ISS propulsion and navigation systems, as well as a spacesuit, were created. The third module simulates a spacecraft launch, utilizing 3D models from SpaceX's Falcon Heavy and digital twins of its propulsion and navigation systems. Participants interacted with the digital twins and scenarios, generating data that was stored and analyzed against the ISS dataset from the Jet Propulsion Laboratory (JPL) and telemetry from the SpaceX Falcon Heavy launch. In this work's initial phase, the main objective was to assess whether digital twins could be integrated with immersive technologies for effective training. A mean squared error metric was employed to compare the digital twins with the physical object data, confirming the alignment of the developed digital twins with the actual systems, thereby validating their utility for astronaut training.