Teleoperated Service Robot with an Immersive Mixed Reality Interface

Angelica Nakayama,Daniel Ruelas,Ernesto Bribiesca,Jesus Savage

doi:10.15622/ia.20.6.1

Abstract

Teleoperated service robots can perform more complex and precise tasks as they combine robot skills and human expertise. Communication between the operator and the robot is essential for remote operation and strongly affects system efficiency. Immersive interfaces are being used to enhance teleoperation experience. However, latency or time delay can impair the performance of the robot operation. Since remote visualization involves transmitting a large amount of video data, the challenge is to decrease communication instability. Then, an efficient teleoperation system must have a suitable operation interface capable of visualizing the remote environment, controlling the robot, and having a fast response time. This work presents the development of a service robot teleoperation system with an immersive mixed reality operation interface where the operator can visualize the real remote environment or a virtual 3D environment representing it. The virtual environment aims to reduce the latency on communication by reducing the amount of information sent over the network and improve user experience. The robot can perform navigation and simple tasks autonomously or change to the teleoperated mode for more complex tasks. The system was developed using ROS, UNITY 3D, and sockets to be exported with ease to different platforms. The experiments suggest that having an immersive operation interface provides improved usability for the operator. The latency appears to improve when using the virtual environment. The user experience seems to benefit from the use of mixed reality techniques; this may lead to the broader use of teleoperated service robot systems.

Highlights

Service robots can perform more complex and precise tasks by harnessing human experience through teleoperation
The interface tests were divided into three types: real mode only, virtual mode only, and mode mixing freely.In all tests, users had to wear the Head-Mounted Displays (HMD) and grab the joystick
Once the users got used to controlling the robot with the joystick, they were able to navigate the robot through real space and visualize the environment according to the robot’s new position

Summary

Introduction

Service robots can perform more complex and precise tasks by harnessing human experience through teleoperation. Teleoperation systems are intended to provide technical means to perform the desired task in a remote environment [1]. In an efficient teleoperation system, the operator must be ensured that the desired task is adequately performed in the remote environment. To this end, teleoperation systems must overcome a series of barriers such as distance, and time delay, among others. The relationship between the operator and the remote environment strongly affects the system efficiency. This means that system performance can be vastly improved by allowing the operator to understand the interaction with the remote environment intuitively and

Objectives

Results

Discussion

Conclusion