Sensory augmentation for subsea robot teleoperation

Abstract

Subsea construction operations heavily rely on remotely operated vehicles (ROV). Due to the dynamics of the subsea environment such as uncertain turbulences, affected visibility and interference with subsea ecosystems, control of subsea ROV is challenging for human operators, especially for construction professionals who have never been exposed to such an environment. The traditional visual feedback method can hardly provide direct and intuitive cues about workplace uncertainties, especially the flow conditions, which makes the ROV operation a work with a high mental load and high training barrier. To augment human sensation of the ROV and workplace status, this research proposes a hierarchical intuitive control method based on Virtual Reality (VR) and haptic simulators. A distributed sensor system that allows a flexible add-on sensor package for subsea environmental data collection is applied to the ROVs to collect hydrodynamic data of subsea workplaces. Then, a Digital Twins (DT) module receives and integrates all data to drive a simulation approach for data augmentation. Last, multi-level sensory feedback methods, including far-field augmented 3D visual feedback, near-field haptic suit tactile feedback, and micro-field haptic suit turbulence feedback are generated and sent to human operators through haptic devices. Compared to previous studies, this system reconstructs the realistic subsea environment with detailed hydrodynamic features in VR digital twin. Whole-body level haptic feedback is realized based on hydrodynamic information. This proposed system ensures an immersive awareness of the proximity conditions and predictions of potential damage and status changes during ROV operations. As a result, a less-trained operator can pilot the ROV based on intuition to maximize the performance and avoid potential mistakes.