A Hardware-Accelerated System for High Resolution Real-Time Screen Sharing

Abstract

Establishing an interactive screen sharing system that supports ultra high resolution (such as 4k) is challenging, with latency and frame rate playing important roles in user experience. The screen frame needs to be compressed efficiently without consuming extensive computational resources. We present a hardware-accelerated system for real-time screen sharing, which decreases encoding workload by exploiting content redundancies between successive screen frames. We propose a multiple codec approach that utilizes several encoders with H.264 Advanced Video Coding (H.264/AVC) of different input sizes, creating savings in encoding time by selecting the appropriate one for updated screen content. An optimized metadata processing method is proposed as well. Small but distant updates within a frame can be split into independent frames for more efficient compression, which is also beneficial for interactive latency. In the evaluation, the proposed system takes less encoding time than general single codec implementation in common screen sharing scenarios. Measurement for latency shows that the end-to-end latency for 4K resolution screen sharing is only about 17–25 ms, which makes the proposed system suitable for various applications in local wired and wireless connections.