Remote Collaboration in a Complex Environment

Abstract

A remote collaboration system simplifies a collaboration task among multiple users at a distance. It allows local users see the real world with virtual guidance/annotations (provided by remote experts) seamlessly rendering over target objects in the real world, so they can easily follow the remote guidance to complete complicated tasks. An effective collaboration system often utilizes (1) computer vision algorithms (e.g. 3D reconstruction, and visual tracking and localization algorithms) to analyze objects and scenes in the real world, as well as (2) augmented reality (AR) to accurately superimpose virtual guidance/annotations over the target object in a real environment. However, some complex working environments with deformable linear objects (DLO), e.g. wire harness, prevents remote experts from easily annotating objects in the video, virtual scenes, or clear non-verbal communication due to the difficulty of DLO tracking. In this project, we proposed a remote guidance system, which is able to identify and track multiple DLOs, starting with minimal user input, so it allows remote experts to provide local users with simple and clear instructions/guidance in a complex environment.