Intelligent Multimedia Presentation Research Articles

TeleMorph: A Fuzzy Logic Approach to Network-Aware Transmoding in Mobile Intelligent Multimedia Presentation Systems

Mobile intelligent multimedia presentation systems are limited by various constraints including mobile network characteristics, mobile device capabilities, and user preferences. Those presentation systems which incorporate remotely stored multimedia rely on the bandwidth which is available during actual content transmission on the connecting mobile network. One approach to deal with this is to transcode content, thus reducing its data rate requirement, although this technique is inherently limited by the lowest acceptable quality of that media element. Alternatively, content can be transmoded to different modalities with lower bandwidth requirements. TeleMorph, a cross-modality adaptation control platform is detailed in this paper. The main premise of TeleMorph is that cross-modality adaptation decisions in mobile presentation systems must occur with primary consideration for bandwidth fluctuations. TeleMorph has been implemented as a network-aware fuzzy inference system that controls cross-modality adaptations between multimodal audio-video, audio-images, and images-text presentations, as well as unimodal audio and text presentations. Initially, a brief introduction to Intelligent Multimedia and Mobile Intelligent Multimedia is given, and related systems discussed. TeleTuras, a tourist information application implemented as a testbed for TeleMorph, is utilized for objective and subjective evaluations through its integration of six disparate test scenarios that incorporate various media types and qualities. Positive results for each evaluation technique, based on specific metrics, are also presented. In addition, future work on TeleMorph is also detailed.

Bandwidth determined transmoding through fuzzy logic in mobile intelligent multimedia presentation systems

Mobile intelligent multimedia presentation systems are subject to various resource constraints including mobile network characteristics, mobile device capabilities and user preferences. Those presentation systems which incorporate remote multimedia content accessed across HTTP (Hyper Text Transfer Protocol) or RTP (Real Time Transfer Protocol) protocols are particularly reliant on the capabilities of the connecting mobile network (i.e. minimum, average and maximum bandwidth) and in particular on the real-time constraints (i.e. currently available bandwidth, packet loss, bit error rate, latency) which prevail during actual content transmission. One approach to address this is to scale content, thus reducing its datarate requirement, although this technique is inherently limited by the lowest acceptable quality of that media element. Alternatively, content can be converted from one modality to another with a lower resource requirement. TeleMorph, a cross-modality adaptation control platform is detailed here. Initially a brief introduction to Intelligent Multimedia and to Mobile Intelligent Multimedia is given, and key systems discussed. The main premise of TeleMorph is that cross-modality adaptations in mobile presentation systems must be controlled in a manner which gives primary consideration to bandwidth fluctuations as well as the constraints listed above. The current prototype of TeleMorph, which uses a fuzzy inference system to control cross-modality adaptations between video and audio, is described. Particular focus is given to the fuzzy inputs, fuzzy control rules and fuzzy outputs which have been utilised in decision making. TeleTuras, a tourist information application which has been implemented as a testbed for TeleMorph, gives promising evaluation results based on multimedia and bandwidth specific test scenarios. TeleMorph is related to other approaches in the area of Mobile Intelligent Multimedia Presentation Systems. TeleMorph differs from other approaches in that it focuses specifically on the challenges posed by controlling bandwidth determined cross-modality adaptations in a mobile network environment. Future work on TeleMorph's output presentation composition will incorporate images and text also, thus allowing for extended adaptation between video, audio, images and text, as well as multimodal combinations of these media elements.

AutoBrief: an experimental system for the automatic generation of briefings in integrated text and information graphics

This paper describes AutoBrief, an experimental intelligent multimedia presentation system that generates presentations in text and information graphics in the domain of transportation scheduling. Acting as an intelligent assistant, AutoBrief creates a presentation to communicate its analysis of alternative schedules. In addition, the multimedia presentation facilitates data exploration through its complex information visualizations and support for direct manipulation of presentation elements. AutoBrief's research contributions include (1) a design enabling a new human–computer interaction style in which intelligent multimedia presentation objects (textual or graphic) can be used by the audience in direct manipulation operations for data exploration, (2) an application-independent approach to multimedia generation based on the representation of communicative goals suitable for both generation of text and of complex information graphics, and (3) an application-independent approach to intelligent graphic design based upon communicative goals. This retrospective overview paper, aimed at a multidisciplinary audience from the fields of human–computer interaction and natural language generation, presents AutoBrief's design and design rationale.

TELEMORPH: BANDWIDTH-DETERMINED MOBILE MULTIMODAL PRESENTATION

This article presents the initial stages of research at the University of Ulster into a mobile intelligent multimedia presentation system called TeleMorph. TeleMorph aims to dynamically generate multimedia presentations using output modalities that are determined by the bandwidth available on a mobile device’s wireless connection. To demonstrate the effectiveness of this research, TeleTuras, a tourist information guide for the city of Derry, will implement the solution provided by TeleMorph, thus demonstrating its effectiveness. This article does not focus on the multimodal content composition but rather concentrates on the motivation for and issues surrounding such intelligent tourist systems.

Standard reference model for intelligent multimedia presentation systems

Standard reference model for intelligent multimedia presentation systems

Case studies in intelligent multimedia presentation design systems in terms of the standard reference model: The IMMPS project, the PreGen system, and the StrMP system

Case studies in intelligent multimedia presentation design systems in terms of the standard reference model: The IMMPS project, the PreGen system, and the StrMP system

Case studies in intelligent multimedia presentation design systems in terms of the standard reference model: the IMMPS project, the PreGen system, and the StrMP system

This paper contributes to a Standard Reference Model (RM) for intelligent multimedia presentation systems. In the past few years, we have developed several intelligent presentation systems at the Multimedia Information NEtwork (MINE) Lab at Tamkang University. We describe these systems in terms of architecture and function of components. The first system, Intelligent MultiMedia Presentation System (IMMPS), allows a presentation designer to construct knowledge rules reflecting the background of individual audiences. Therefore, the generated presentation is able to learn from the audiences, which results in different presentation for each individual. The second system (PreGen) uses inference rules for the automatic generation of multimedia presentations. An ICON programming technique is also used in the PreGen project. Another system for designing structured multimedia presentations using data flow/control flow diagrams is also discussed. In order to run multimedia presentations generated by different systems, we are developing a multimedia abstract machine, which is a time Petri net based software simulator for running multimedia assembly instructions.

Describing the MMI 2 demonstrators with the IMMPS reference model

The architecture of the MMI demonstrators is presented, firstly as described by the authors and then in terms of the Reference Model for Intelligent Multimedia Presentation Systems (IMMPS). The analysis shows that the presentation pipeline of the MMI 2 architecture can be readily described in terms of the Reference Model, without either significant parts of the Reference Model being unused, or parts of the architecture remaining undescribed. The Reference Model would have to be extended to accommodate the input components of the MMI 2 architecture, but these are explicitly outside its scope. The exercise has shown some minor limitations in the coordination and layout control in the MMI 2 architecture which were not evident before.

Defining a taxonomy of output modalities from an HCI perspective

The paper addresses an issue that must be resolved to produce a scientifically sound and practically useful reference model for intelligent multimedia presentation systems (IMP systems), namely tha...

Integrating the Amsterdam hypermedia model with the standard reference model for intelligent multimedia presentation systems

The standard reference model (SRM) for intelligent multimedia presentation systems describes a framework for the automatic generation of multimedia presentations. This framework, however, lacks an explicit document model of the presentation being generated. The Amsterdam hypermedia model (AHM) describes the document features of a hypermedia presentation explicitly. We take the AHM and use it as a basis for describing in detail the stages of generating a hypermedia presentation within the SRM framework, which we summarise in a table. By doing so, the responsibilities of the individual SRM layers become more apparent.

Relating the primitive hierarchy of the PREMO standard to the standard reference model for intelligent multimedia presentation systems

The need for a suitable classification of media types arises for several reasons when building or comparing multimedia systems. Within an Intelligent Multimedia Presentation Systems (IMMPS), it is necessary to formulate and encode design knowledge for decision making on the appropriate medium in which to present information and for the generation of the presentation. It is also required in order to specify interfaces to and between system components which will be employed to run a generated presentation before the user's eyes. This task is reflected in the Standard Reference Model (SRM, see this volume) for IMMPS by the Presentation Display Layer. However, the SRM does not instantiate this layer in detail, but instead refers to the Presentation Environment for Multimedia Objects (PREMO) ISO/IEC standard which provides a reference model for a presentation runtime environment for multimedia. PREMO already contains a set of basic structures, the so-called PREMO Primitive Hierarchy, to describe different media types. Thus the question arises, as to how far the PREMO Primitive Hierarchy could serve as a media classification for the SRM in general. In particular, this would support consistency between the design and presentation layers of the SRM if PREMO were used to instantiate the presentation layer. In the current paper, we first point to a number of typical problems with generating classifications of media types. We then provide a brief introduction to PREMO and its Primitive Hierarchy. Finally, the benefits and costs of using the PREMO Primitive Hierarchy for the SRM are discussed.

Intelligent multimedia layout: A reference architecture for the constraint-based spatial layout of multimedia presentations

With increases in the complexity and dynamics of multimedia information communicated by current applications, there arises a corresponding need towards a standard technology for intelligent multimedia interfaces. In this article, we address those components of an intelligent multimedia presentation system (IMMPS) which deal with the design and the realization of spatial layout. We treat multimedia layout as a multidimensional constraint problem and propose a reference architecture for a general-purpose intelligent multimedia layout manager (IMMLM) that is based on a dedicated constraint solver kernel.

An analysis of COMET and MAGIC using the standard reference model for intelligent multimedia presentation systems

We provide an overview of two multimedia generation systems developed at Columbia University, COMET and MAGIC, and analyze them using the standard reference model for intelligent multimedia presentation systems. COMET (Coordinated Multimedia Explanation Testbed) emphasizes the coordinated generation of text and 3D graphics in a military maintenance and repair documentation domain. It features support for automated determination of what information to present, assignment of information to appropriate media and generation of textual cross-references to the structure and content of generated graphics, including the automated redesign of graphics to make possible better cross-references. MAGIC (Multimedia Abstract Generation for Intensive Care) generates presentations that use temporal media, including speech and animation, to explain the status of hospital patients. It extends the media coordination approach used in COMET to include a multi-stage negotiation process that solves temporal constraints imposed by individual media generators to determine when and for how long actions should occur in each medium.

Defining a taxonomy of output modalities from an HCI perspective

The paper addresses an issue that must be resolved to produce a scientifically sound and practically useful reference model for intelligent multimedia presentation systems (IMP systems), namely that of providing, from the point of view of Human-Computer Interaction (HCI), a systematic understanding of the types of output information to be presented by IMP systems. The term ‘medium’, as it is used in the context of multimedia systems, is too coarse-grained for distinguishing between different types of output information. The paper introduces the notion of (representational) ‘modalities’ to enable sufficiently fine-grained distinctions to be made. For the term itself to be meaningful, ‘multimodal’ presentations must be composed of unimodal representations. In the approach presented, unimodal representations are defined from a small number of basic properties whose combinations specify the ‘generic’ level of a taxonomy of unimodal output modalities. Additional basic property distinctions serve to generate the more fine-grained ‘atomic’ and ‘sub-atomic’ levels in a hierarchical fashion. The taxonomy is set up with the aim of satisfying four basic requirements, viz. completeness, orthogonality, relevance and intuitiveness. A concluding discussion illustrates the practical use of the taxonomy.

Using constraint propagation in MAGPIE: a multi-agent approach

Multimodal presentation planning must solve the constraints imposed by the structure of the discourse. In this paper, we discuss a multi-agent implementation of the design and realization layers of the Standard Reference Model (SRM) for Intelligent Multimedia Presentation Systems (IMMPS), where design constraints for multimodal presentations are distributed over a number of agents. Thus, a constraint propagation mechanism is required so that the distributed constraints can be satisfied cooperatively by these agents.

The standard reference model in the AIMI and TEXPLAN systems

In this article, we describe the presentation components of two systems created at The MITRE Corporation: An Intelligent Multimedia Interface (AIMI) and the Textual Explanation PLANner (TEXPLAN) system. We then describe and analyze these two systems using the Standard Reference Model (SRM) for Intelligent Multimedia Presentation Systems (IMMPS) [M. Bordegoni, G. Faconti, S. Feiner, M. Maybury, T. Rist, S. Ruggieri, P. Trahanias, M. Wilson, A standard reference model for intelligent multimedia presentation systems, Computer Standards and Interfaces, 18 (6,7) 477–496].

Adaptation in IMMPS as a decision making process

Intelligent Multimedia Presentation Systems (IMMPS) are characterised by their capability to dynamically adapt at run-time, based on the state of several aspects of user-computer interaction, such as the user profile, task and information characteristics, etc. In this paper, we treat adaptation as a decision making process, and propose the employment of established techniques from the domain of Decision Making to assist this process. It is argued that, following this approach, the adaptation process can be easily modified, customised and re-used in different application domains and for different user interaction requirements. The designer is enabled to incorporate different design strategies, which can be dynamically and automatically modified, according to the run-time requirements of the system. The application of the proposed approach is exemplified in the media/modalities allocation process.

A standard reference model for intelligent multimedia presentation systems

This article summarizes the main results of a joint endeavor towards a standard reference model (SRM) for intelligent multimedia presentation systems (IMMPSs). After a brief motivation, we give basic definitions for media terms and presentation systems. The core of this contribution is a generic reference architecture that reflects an implementation-independent view of the processes required for the generation of multimedia presentations. The reference architecture is described in terms of layers, components, and knowledge servers. Our SRM focuses on the functions assigned to the layers and components, rather than on the methods or communication protocols that may be employed to realize this functionality. Finally, we point to some possible extensions of the reference model.

Supporting the reuse of multimedia presentations

We developed an intelligent multimedia presentation design system earlier. Based on the system, we now develop a multimedia database system that supports the reuse of multimedia presentations and resources. The database consists of two layers: the frame layer and the resource layer. A frame group in the first layer is the basic presentation unit to be reused. A resource group in the second layer is a collection of resources to be used by a frame or a frame group. In these two layers, we use four types of links in the database to group frames and/or resources. Also, multimedia resources are associated with a number of properties. Using these properties, we allow the user to search for resources via a database browser. The database is built on the top of a commercial object-oriented database server and run under a windowing system.