Video as a high-level data type

Abstract

Object-oriented application frameworks are among the most advanced tools available to software engineers for the development and maintenance of applications with sophisticated interactive user interfaces. These frameworks provide text and graphics as high-level data types but still lack support for motion video or animation, which demand high screen update rates and multi-threading. We identified the screen update model `drawing by invalidation' (or `damage repair'), which is widely used in frameworks as well as other user interface toolkits, as a major obstacle to video integration. We generalized the model to remove this obstacle and changed an application framework's internal architecture accordingly. The adapted framework together with a newly developed video class hierarchy now allows a programmer to easily place video objects in any shape and number seamlessly integrated with other visual objects in any user interface element. For instance, video objects may appear in scrollable views, as possibly overlapping building blocks in graphics editors, as characters in text editors, and as items in list or pop-up menus. The classes in the hierarchy factor out common functionality to ease the development of customized video objects. For instance, the setup of and communication with a separate thread of control as well as the implementation of the new drawing model can be inherited and reused in subclasses.