Multiple streams synchronization in collaborative multimedia systems

Abstract

With the recent increase of the communication bandwidth and processor power, new types of applications have emerged. Among them, there are multimedia application, in which users are able to control, combine, and manipulate different types of media, such as text, sound, video, computer graphics, and animation. A key requirement in any multimedia application is to synchronize the delivery of various media streams to the user. To achieve this, the sender has to provide the temporal relations between the streams as they are captured. Since the receiver uses this information in streams presentation, its accuracy is very important. Our main contribution is to provide a suit of synchronization algorithms for audio, video and X-windows streams that work correctly in the presence of load variations. First, we propose a mechanism for assigning a correct synchronization specification to media units that takes into account the workload variation at the sender; although this issue is critical, it has been largely ignored in previous work. Second, for detecting the skew between the streams, we propose a synchronization condition that works in the general case of streams having different media unit durations. Based on this condition, we develop an adaptive lip-synchronization algorithm. By estimating the display time of video frames, our algorithm is robust and stable in the presence of both network and workstation load. To synchronize the X-windows stream with the audio/video stream we propose a novel approach that combines dropping X packets with delaying the X client. Finally, we extend our algorithms to a distributed environment. We do this by proposing (1) a mechanism for extracting the synchronization information from mixed audio streams, and (2) a lightweight mechanism to achieve global clock synchronization.