IDirector: An Intelligent Directing System for Live Broadcast

Abstract

Live sports broadcasting is the live coverage of sports (e.g., a soccer match) as a television program, on various types of broadcasting media (e.g., television or internet). Directing such live sports broadcast is cost-expensive and demands experienced sports directors with sufficient broadcasting skills. In this paper, we demonstrate an end-to-end intelligent system for live sports broadcasting, namely iDirector, which aims to mimic the human-in-loop live broadcasting process by aggregating the input multi-camera video streaming into the final output program video (PGM video) for audience. We construct this system as an event-driven pipeline with three modules: video decoder, video analyzer, and broadcasting controller. Specifically, given the multi-view video streaming captured from cameras placing in the stadium, video decoder module first decodes the input video streaming into a series of frames and clips. Next, video analyzer performs multiple pre-learned models in parallel for frame- and clip-level content understanding (e.g., events localization and highlight detection). Based on all the analytic results across frames and clips, with only 30 seconds looking ahead, broadcasting controller automatically produces the broadcast videos via camera view switch, playback and slow-motion. When some high-profile events (e.g., free kick) happen, broadcasting controller will render visual effects on PGM video to enhance audiences' entertained pleasure.