Design and implementation of parallel video combiner architecture for multi-user video conferencing at ultra-high definition resolution

Abstract

In software driven multi-user video conferencing systems, consumer progression towards ultra-high definition (i.e., 8 k) resolution introduces new challenges to the video combination process in sustaining smooth combined video playback at high frame rates. As such, this paper analyzes the underlying architecture of a conventional video combiner to identify the performance impact of combined video frame rates at ultra-high definition resolutions. Then, two straightforward parallel video combination architectures using software application threads, namely PVC-1 and PVC-2, are studied as a benchmark. In PVC-1, the number of application threads tallies with the number of client videos to be combined, which improves playback performance at the expense of inconsistencies in inset client frame rates within a combined video. PVC-2 includes a synchronizer to address these inconsistencies, but exhibits marginal performance gains. To address the aforementioned problems, a balanced workload parallel video combiner architecture is proposed, namely PVC-3. In this architecture, a balanced workload management algorithm stitches client videos into an ultra-high definition combined frame, based on the number of available logical processors on a multi-core processor. This method improves frame rate performance and sustains consistent client frame rates within a combined video. Experimental results suggest that PVC-3 is superior over PVC-2 and achieves a frame rate performance gain of 27 % against a conventional video combiner for a combined video of 16 clients (each at a resolution of 720p) and zero standard deviation in combined frame rate consistency.