Are you Seeing what I'm Seeing?: Perceptual Issues with Digital Twins in Virtual Reality

Abstract

With the availability of data and visualization capabilities, digital twins have grown in use cases and entered the virtual reality (VR) realm. Utilizing VR enhances the value of digital twins by enabling training, testing, and refinement of a system, through support of a user's workflow when interacting with the system. However, VR is not fully representative of real-world environments, particularly in areas such as visual perception, proprioception, vestibular response, and perceptions of information. The simulation and training domain has learned which ways VR can benefit users and which ways it falls short, allowing a rich domain of information for the digital twin world to leverage. While many of these differences can be mitigated through targeted environmental design or hardware selection, these differences in perceptual capabilities and feedback negatively impact the validity of VR digital twins. These impacts could range from user discomfort, injury, negative transfer to the real world, and poor design decisions based on unrepresentative digital twins. This paper will discuss the current use cases of digital twins in VR that center on human interaction, common perceptual concerns in VR uncovered in simulation research, and how these issues could impact a digital twin's representativeness. As the applicability of digital twins in VR expands, mitigations for perceptual issues in VR and open areas for improvement in VR for digital twins are presented.