Advancing multimedia: application sharing, latency measurements and user-created services

Abstract

Online collaboration tools exist and have been used since the early days of the Internet. Asynchronous tools such as wikis and discussion boards and real-time tools such as instant messaging and voice conferencing have been the only viable collaboration solutions up until recently, due to the low bandwidth between participants. With the increasing bandwidth in computer networks, multimedia collaboration such as application sharing and video conferencing have become feasible. Application and desktop sharing allows sharing of any application with one or more people over the Internet. The participants receive the screen-view of the shared application from the server. Their mouse and keyboard events are delivered and regenerated at the server. Application and desktop sharing enables collaborative work, software tutoring, and e-learning over the Internet. I have developed a high performance application and desktop sharing system called BASS which is efficient, reliable, independent of the operating system, scales well via heterogeneous multicast, supports all applications, and features true application sharing. Most of the time an application sharing session requires audio and video conferencing to be more useful. High quality video conferencing requires a fair amount of bandwidth and unfortunately Internet bandwidth of home users is still limited and shared by more than one application and user. Therefore, I measured the performance of popular video conferencing applications under congestion to understand whether they are flexible enough to adapt to fluctuating and limited bandwidth conditions. In particular, I analyzed how Skype, Windows Live Messenger, Eyebeam and X-Lite react to changes in available bandwidth, presence of HTTP and BitTorrent traffic and wireless packet losses. To perform these measurements more effectively, I have also developed vDelay, a novel tool for measuring the capture-to-display latency (CDL) and frame rate of real-time video conferencing sessions. vDelay enables developers and testers to measure the CDL and frame rate of any video conferencing application without modifying the source code. Further, it does not require any specialized hardware. I have used vDelay to measure the CDL and frame rate of popular video chat applications including Skype, Windows Live Messenger, and GMail video chat. vDelay can also be used to measure the CDL and frame rate of these applications in the presence of bandwidth variations. The results from the performance study showed that existing products, such as Skype, adapt to bandwidth fluctuations fairly well and can differentiate wireless and congestion-based packet losses. Therefore, rather than trying to improve video conferencing tools, I changed my focus to end-user created communication-related services to increase the utility of existing stand alone Internet services, devices in the physical world, communication and online social networks. I have developed SECE (Sense Everything, Control Everything), a new language and its supporting software infrastructure for user created services. SECE allows non-technical end-users to create services that combine communication, social networks, presence, calendaring, location and devices in the physical world. SECE is an event-driven system that uses a natural-English-like language to trigger action scripts. Users associate actions with events and when an event happens its associated action is executed. Presence updates, social network updates, incoming calls, email, calendar and time events, sensor inputs and location updates can trigger rules. SECE retrieves all this information from multiple sources to personalize services and to adapt them to changes in the users context and preferences. Actions can control the delivery of email, change the handling of phone calls, update social network status and set the state of actuators such aslights, thermostats and electrical appliance.