Format-independent Adaptation Research Articles

Format-independent and metadata-driven media resource adaptation using semantic web technologies

Adaptation of media resources is an emerging field due to the growing amount of multimedia content on the one hand and an increasing diversity in usage environments on the other hand. Furthermore, to deal with a plethora of coding and metadata formats, format-independent adaptation systems are important. In this paper, we present a new format-independent adaptation system. The proposed adaptation system relies on a model that takes into account the structural metadata, semantic metadata, and scalability information of media bitstreams. The model is implemented using the web ontology language. Existing coding formats are mapped to the structural part of the model, while existing metadata standards can be linked to the semantic part of the model. Our new adaptation technique, which is called RDF-driven content adaptation, is based on executing SPARQL Protocol and RDF Query Language queries over instances of the model for media bitstreams. Using different criteria, RDF-driven content adaptation is compared to other adaptation techniques. Next to real-time execution times, RDF-driven content adaptation provides a high abstraction level for the definition of adaptations and allows a seamless integration with existing semantic metadata standards.

NinSuna: a fully integrated platform for format-independent multimedia content adaptation and delivery using Semantic Web technologies

The current multimedia landscape is characterized by a significant heterogeneity in terms of coding and delivery formats, usage environments, and user preferences. The main contribution of this paper is a discussion of the design and functioning of a fully integrated platform for multimedia adaptation and delivery, called NinSuna. This platform is able to efficiently deal with the aforementioned heterogeneity in the present-day multimedia ecosystem, thanks to the use of format-agnostic adaptation engines (i.e., engines independent of the underlying coding format) and format-agnostic packaging engines (i.e., engines independent of the underlying delivery format). Moreover, NinSuna also provides a seamless integration between metadata standards and adaptation processes. Both our format-independent adaptation and packaging techniques rely on a model for multimedia bitstreams, describing the structural, semantic, and scalability properties of these multimedia streams. News sequences were used as a test case for our platform, enabling the user to select news fragments matching his/her specific interests and usage environment characteristics.

Design options and comparison of in-network H.264/SVC adaptation

This paper explores design options and evaluates implementations of in-network, RTP/RTSP based adaptation MANEs (Media Aware Network Elements) for H.264/SVC content streaming. The obvious technique to be employed by such an adaptation MANE is to perform SVC specific bitstream extraction or truncation. Another mechanism that can be used is description (metadata) driven, coding format independent adaptation based on generic Bitstream Syntax Descriptions (gBSD), as specified within MPEG-21 Digital Item Adaptation (DIA). Adaptation MANE architectures for both approaches are developed and presented, implemented in end-to-end streaming/adaptation prototype systems, and experimentally evaluated and compared. For the gBSD based solution, open issues like the granularity of bitstream descriptions and of bitstream adaptation, metadata overhead, metadata packetization and transport options, and error resilience in case of metadata losses, are addressed. The experimental results indicate that a simple SVC specific adaptation MANE does clearly outperform the gBSD based adaptation variants. Yet, the conceptual advantages of the description driven approach, like coding format independence and flexibility, may outweigh the performance drawbacks in specific applications.

GBFlavor: a new tool for fast and automatic generation of generic bitstream syntax descriptions

The efficient adaptation of scalable media resources is a major point of interest, due to today's tremendous heterogeneity in terms of end-user terminals, network technologies, and coding formats. In order to create a media resource adaptation engine supporting current and future coding formats, a generic (i.e., format-independent) solution is needed. One way to realize this goal is to rely on automatically created textual descriptions of the high-level syntax of binary media resources. MPEG-21 generic Bitstream Syntax Schema (gBS Schema) is a tool that is part of the MPEG-21 Multimedia Framework. It enables the use of generic Bitstream Syntax Descriptions (gBSDs), i.e., textual descriptions in XML, to steer the adaptation of a binary media resource, using format-independent adaptation logic. The major contribution of this paper is the introduction of gBFlavor. It is a novel solution for the automatic and format-agnostic generation of gBSDs. gBFlavor offers the possibility to automatically create a format-specific parser that is able to produce a gBSD, taking as input a particular media resource compliant to the coding format described by the parser. This paper provides an overview of the gBFlavor language, which allows describing the high-level structure of a coding format. The overall functioning of a gBFlavor-enabled adaptation framework is discussed as well. Performance results for two scalable coding formats, in particular H.264/AVC Scalable Video Coding and JPEG2000, show that our proposed solution outperforms existing techniques in terms of execution speed.

An optimized MPEG-21 BSDL framework for the adaptation of scalable bitstreams

A format-agnostic framework for content adaptation allows reaching a maximum number of users in heterogeneous multimedia environments. Such a framework typically relies on the use of scalable bitstreams. In this paper, we investigate the use of bitstreams compliant with the scalable extension of the H.264/MPEG-4 AVC standard in a format-independent framework for content adaptation. These bitstreams are scalable along the temporal, spatial, and SNR axis. To adapt these bitstreams, a format-independent adaptation engine is employed, driven by the MPEG-21 Bitstream Syntax Description Language (BSDL). MPEG-21 BSDL is a specification that allows generating high-level XML descriptions of the structure of a scalable bitstream. As such, the complexity of the adaptation of scalable bitstreams can be moved to the XML domain. Unfortunately, the current version of MPEG-21 BSDL cannot be used to describe the structure of large video bitstreams because the bitstream parsing process is characterized by an increasing memory consumption and a decreasing description generation speed. Therefore, in this paper, we describe a number of extensions to the MPEG-21 BSDL specification that make it possible to optimize the processing of bitstreams. Moreover, we also introduce a number of additional extensions necessary to describe the structure of scalable H.264/AVC bitstreams. Our performance analysis demonstrates that our extensions enable the bitstream parsing process to translate the structure of the scalable bitstreams into an XML document multiple times faster. Further, a constant and low memory consumption is obtained during the bitstream parsing process.

A framework for fully format-independent adaptation of scalable bit streams

This paper presents a framework for modeling and adaptation of arbitrary scalable multimedia bit streams in a manner that is fully format agnostic ("universal transcoding"). It is entitled structured scalable metaformats (SSM), and is based on establishing a universal model for all scalable bit streams, which in turn allows a compact specification of the set of allowed adaptations, and also how the bit stream adaptation is performed. In order to enable format-agnostic adaptation, two elements are standardized: 1) metadata associated with a scalable bit stream conveying the parameters of its SSM model, and how it is to be manipulated to obtain various adapted versions, as well as information that allows making appropriate adaptation decisions using only the model parameters and 2) a specification of outbound network and terminal constraints, for real-time decisions based on both user preferences and network conditions. By interpreting the descriptions, a universal adaptation engine can adapt the content to suit the specified needs and preferences of recipients, without knowledge of the specifics of the content, its encoding and encryption. With universal adaptation, different adaptation infrastructures are no longer needed for different types of scalable media, eliminating the high costs of traditional transcoding. Many of the SSM concepts have been adopted into the MPEG-21 Part 7 standard entitled Digital Item Adaptation.

MPEG-21 digital item adaptation: enabling universal multimedia access

The access devices of today are becoming increasingly sophisticated. Thanks to multimedia, communication is much widespread and therefore more powerful. However, we face a serious problem of heterogeneity in our terminals, in our networks, and in the people who ultimately consume and interact with the information presented to them. In this article, we focus on Part 7 of the MPEG-21 standard (ISO/IEC 21000-7), which we refer to as Digital Item Adaptation (DIA). At the time of this writing, the DIA specification is at the penultimate stage of Final Committee Draft (2003); final approval is scheduled for December 2003. The general DIA concept is that Digital Item is subject to both a resource adaptation and a descriptor adaptation engine, which together produce the adapted Digital Item. Note that the standard specifies only the tools that assist with the adaptation process, not the adaptation engines themselves. Digital Item Adaptation (DIA) specifies the following natural environment description tools: location and time and audiovisual environment. The DIA offers a rich set of tools to assist with the adaptation of Digital Items. It offers standardized tools for the description of usage environments, tools to create high-level descriptions of the bitstream syntax to achieve format-independent adaptation, tools that assist in making tradeoffs between feasible adaptation operations and constraints, tools that enable low-complexity adaptation of metadata, and tools for session mobility. Moving forward, the MPEG-21 committee is considering amendments to the specification-for instance, tools that provide further assistance with modality conversion and tools that relate more specifically to the adaptation of audio and graphics media. Furthermore, also being actively considered is how to express the rights that a User has to perform adaptation and how this expression fits into a system that governs those rights.