IDTVR: A Novel Cloud Framework for an Interactive Digital Twin in Virtual Reality

Abstract

Digital Twins (DTs) and Virtual Reality (VR) have recently gained significant popularity. The term digital twin refers to a virtual representation of a real-world environment. DTs are known for providing intuitive, accurate, and real-time visualizations of complex physical systems. In this paper, we present a novel, cloud-based framework for creating digital twin environments using VR within a cloud environment. We demonstrate our framework with two real-world case studies inspired by clinical insights obtained from experts in post-stroke rehabilitation. In the case of rehabilitation, sessions within virtual scenarios mimicking patients’ everyday life may significantly improve their rehabilitation quality. Inspired by biofeedback, we used a digital twin in VR to allow patients to enhance hand stability using real-time data streaming. Our second case study demonstrated the utility of our approach for allowing patients to more comfortably navigate within their homes by generating a virtual twin of an apartment. Our work is designed to transcend the boundary between reality and virtuality by letting users interact directly with the real and virtual worlds simultaneously. To achieve real-time synchronization between the real and the virtual environments, we established an infrastructure blueprint that features efficient sensor data collection, streaming, processing, and storage in a secure cloud environment. Data may then be accessed from any site, directly from the cloud, to update the virtual environment. In the end, we show that our infrastructure is generalizable across clinically-relevant tasks. We hope to extend our work to other areas that would benefit from a digital twin-based VR space.