Remote Driving Control With Real-Time Video Streaming Over Wireless Networks: Design and Evaluation

Abstract

There is currently a large gap between current AI based autonomous-driving cars and fully autonomous cars. In such situation, remote control of cars can be a unique solution to fill the gap. Human assistance through remote driving can provide more flexibility and intelligence than a single artificial intelligence. Furthermore, remote driving enables valuable operational data to be obtained, thus laying the groundwork for gradually providing autonomous driving performance in the future. Remote driving of a vehicle is to drive a vehicle remotely via communication infrastructure. Latency in communication technology has a significant impact on the real-time control of the vehicles as well as responding to unexpected situations. For remotely driven vehicles, real-time transmission of data and images is particularly important. The latency or delay between a vehicle and the controller depends on the communication methods and transport protocols of the video streaming. Furthermore, the control or driving performance also depends on the speed of the vehicle. Therefore, in this paper, we explore the impact of different communication methods of video streaming and vehicle “speed” on the performance of remote driving.We design a vehicle remote driving system based on ROS (Robot Operating System) with ROS as the core communication architecture to realize remote control of vehicles. The video stream is transmitted using three different streaming methods such as ROS multi-computer communication, TCP protocol, and UDP protocol. To be specific, we implement a simple remote driving system for a “model” car and let multiple people drive the car through the remote driving system at different speeds. Then, we analyze how the drivers perform in terms of whether the vehicle gets off the track while driving.We quantify the performance of remote driving at different speeds and with different protocols of video transmission methods and perform comparative analysis. We find that “UDP protocol based video streaming” to achieve 720P video streaming with a latency of less than 50ms, which is helpful for further research on remote driving. Furthermore, to evaluate the impact of the video streaming approach on remote driving, we conduct some experiments by “directly” observing the vehicle with eyes instead of using video streaming. In this way, we remove the latency of video streaming. The experiment results show that the speed of the vehicle and the video streaming methods have a significant impact on the driving performance and demonstrate that the UDP protocol-based video streaming method is better suited for remote driving. The results imply that remote driving should be used in a low-speed environment rather than at high speed.