Digital Item Adaptation Research Articles

Adaptive delivery and handoff management framework for multimedia session mobility

With the increasing use of small portable devices and wireless networks, a trend to support computing on the move has emerged. This trend is referred to as ‘anytime/anywhere computing’ for mobility support. In particular, multimedia service with heterogeneous devices, location-free access and user-context awareness is still a very active and evolving field of research in broad areas. In this paper, a movable multimedia framework, namely UMOST (Ubiquitous Middleware Of Streaming Technology), is proposed for multimedia session mobility.It provides various levels of quality of service management, as well as seamless user-level handoffs using MPEG-21 Digital Item Adaptation (DIA).

Combining MPEG Tools to Generate Video Summaries Adapted to the Terminal and Network

Moving picture experts group (MPEG) standards provide tools for a broad range of purposes, covering from coding to metadata description tools. In this paper, the combined use of tools from different MPEG standards is described in the context of a video summarization application. The main objective of the framework is the efficient generation of summaries, integrated with their adaptation to the user's terminal and network. The MPEG-4 Scalable Video Coding specification is used for fast adaptation and summary bitstream generation. MPEG-21 digital item adaptation tools are used to describe metadata related to the user's terminal and network. MPEG-7 tools are used to describe the summary. Finally, the framework is compared with alternative approaches (variations and transcoding), in terms of efficiency, rate-distortion performance and other aspects.

MPEG-21 및 H.264/AVC SVC 기반 동적 비디오 적응 방법

근래의 이종 네트워크 환경에서 비디오를 안정되게 전송하기 위해서는 네트워크 특성, 단말기 성능, 사용자 선호도 등의 소비 환경을 고려한 비디오 적응 방법이 요구되며, 비디오 부호화에서의 스케일러빌러티는 그 요구조건을 만족시키기 위한 좋은 해결책이다. 본 논문에서는 H.264/AVC SVC 스트림을 MPEG-21 디지털 아이템 적응(Digital Item Adaptation, DIA) 도구를 이용하여 네트워크 환경, 단말의 성능, 사용자 선호도에 따라 동적(dynamic)으로 비디오 콘텐츠를 적응시키는 방법을 제안한다. 실험에서는 이종망 네트워크 환경을 모델링하여 대역폭이 최대 62%의 변화가 있는 경우에도 제안한 방법으로 지연이 거의 없이 실시간으로 비트율을 적응할 수 있음을 보인다. Seamless streaming of multimedia content that ensures Quality of Service over heterogeneous networks has been a desire for many multimedia services, for which the multimedia contents should be adapted to usage environments such as network characteristics, terminal capabilities, and user preferences. Scalability in video coding is a good feature to meet the requirement of heterogeneous networks. In this paper, we propose a dynamic adaptation scheme of H.264/AVC SVC bit-stream using the MPEG-21 Digital Item Adaptation (DIA) tool. MPEG-21 DIA framework provides systematic solutions in choosing an adaptation operation to given conditions and supports interoperable video adaptation. The experiment results show that the proposed adaptation scheme provides QoS-enabled delivery and consumption of SVC with time-varying constraints of network, terminal, and user preference, in a robust and efficient way. In particular, the proposed adaptation scheme is proved to work well with very low delay under the condition that the variation rate of the given network bandwidth is upto 62%.

Content on demand video adaptation based on MPEG-21 digital item adaptation

One of the major objectives in multimedia research is to provide pervasive access and personalized use of multimedia information. Pervasive access of video data implies the access of cognitive and affective aspects of video content. Personalized use requires the services satisfy individual user's needs on video content. This article attempts to provide a content-on-demand (CoD) video adaptation solution by considering users' preference on cognitive content and affective content for video media in general, sports video and movies in particular. In this article, CoD video adaptation system is developed to support users' decision in selecting their content of interest and adaptively deliver video source by selecting relevant content and dropping frames while considering network conditions. First, video contents are annotated by the description schemes (DSs) provided by MPEG-7 multimedia description schemes (MDSs). Then, to achieve a generic adaptation solution, the adaptation is developed following MPEG-21 Digital Item Adaptation (DIA) framework. We study the MPEG-21 reference software on XML generation and develop our own system for CoD video adaptation in three steps: (1) the content information is parsed from MPEG-7 annotation XML file together with bitstream to generate generic Bitstream Syntax Description (gBSD); (2) Users' preference, network characteristic and adaptation QoS (AQoS) are considered for making adaptation decision; (3) adaptation engine automatically parses adaptation decisions and gBSD to achieve adaptation. Unlike most existing adaptation work, the system adapts the content of interest in the video stream according to users' preference. We implement the above-mentioned MPEG-7 and MPEG-21 standards and provide a generic video adaptation solution. Adaptation based on gBSD avoids complex video computation. Thirty students from various departments were invited to assess the system and their responses have been positive.

A Service Oriented Architecture for Personalized Universal Media Access

Multimedia streaming means delivering continuous data to a plethora of client devices. Besides the actual data transport, this also needs a high degree of content adaptation respecting the end users’ needs given by content preferences, transcoding constraints, and device capabilities. Such adaptations can be performed in many ways, usually on the media server. However, when it comes to content editing, like mixing in subtitles or picture-in-picture composition, relying on third party service providers may be necessary. For economic reasons this should be done in a service-oriented way, because a lot of adaptation modules can be reused within different adaptation workflows. Although service-oriented architectures have become widely accepted in the Web community, the multimedia environment is still dominated by monolithic systems. The main reason is the insufficient support for working with continuous data: generally the suitability of Web services for handling complex data types and state-full applications is still limited. In this paper we discuss extensions of Web service frameworks, and present a first implementation of a service-oriented framework for media streaming and digital item adaptation. The focus lies on the technical realization of the services. Our experimental results show the practicality of the actual deployment of service-oriented multimedia frameworks.

Intelligent multimedia engines for multimedia content adaptation

Current end user devices are not able to display all kind of multimedia data. However, this does not conform to the vision of universal multimedia access (UMA). Server side resource adaptation can solve some of the problems caused by heterogeneity because the multimedia resources are transformed sufficiently such that they fit the consumer's preferences and usage environment before they are sent to client. MPEG-21 is the ISO/IEC standard that defines a normative open framework for multimedia delivery and consumption concerning all parties in the delivery and consumption chain. However, the implementation of an intelligent adaptation engine (AE) capable of determining the required transformations is intentionally left to the tool vendors. An AE, based on the MPEG-21 digital item adaptation (DIA) framework, has an adaptation decision-taking engine (ADTE). In this paper, we present research efforts on ADTEs and consider how intelligent ADTEs can be designed and developed.

An interoperable delivery framework for scalable media resources

In this article an interoperable framework for the delivery of scalable media resources (e.g., in the standardized scalable video coding format) is presented. The framework provides support for video on demand as well as multicast streaming, and performs efficient, generic, and interoperable adaptation of streamed content based on MPEG- 21 Digital Item Adaptation. The server as well as the clients of the streaming framework implement the MPEG Extensible Middleware and utilize the MPEG Query Format for querying the available media resources. The framework has been fully integrated into the VLC media player. The architecture for both VoD and multicast is presented in detail. Finally, a comparison in terms of performance of the generic MPEG-21 metadata-based adaptation approach to an SVC-specific adaptation approach is provided.

An Objective Perceptual Quality-Based ADTE for Adapting Mobile SVC Video Content

In this paper, we propose an Adaptation Decision-Taking Engine (ADTE) that targets the delivery of scalable video content in mobile usage environments. Our ADTE design relies on an objective perceptual quality metric in order to achieve video adaptation according to human visual perception, thus allowing to maximize the Quality of Service (QoS). To describe the characteristics of a particular usage environment, as well as the properties of the scalable video content, MPEG-21 Digital Item Adaptation (DIA) is used. Our experimental results show that the proposed ADTE design provides video content with a higher subjective quality than an ADTE using the conventional maximum-bit-allocation method.

Using MPEG-21 for cross-layer multimedia content adaptation

This paper presents a cross-layer model— formulated using interoperable description formats—for the adaptation of scalable H.264/MPEG-4 AVC (i.e., SVC) content in a video streaming system operating on a Wireless LAN access network without QoS mechanisms. SVC content adaptation on the server takes place on the application layer using an adaptation process compliant with the MPEG-21 Digital Item Adaptation (DIA) standard, based on input comprised of MPEG-21 DIA descriptions of content and usage environment parameters. The latter descriptions integrate information from different layers, e.g., device characteristics and packet loss rate, in an attempt to increase the interoperability of this cross-layer model, thus making it applicable to other models. For the sake of deriving model parameters, performance measurements from two wireless access point models were taken in account. Throughout the investigation it emerged that the behavior of the system strongly depends on the access point. Therefore, we investigated the use of end-to-end-based rate control algorithms for steering the content adaptation. Simulations of rate adaptation algorithms were subsequently performed, leading to the conclusion that a TFRC-based adaptation technique (TCP-Friendly Rate Control) performs quite well in adapting to limited bandwidth and varying network conditions. In the paper we demonstrate how TFRC-based content adaptation can be realized using MPEG-21 tools.

이동 환경을 위한 적응형 다시점 비디오 서비스 프레임워크

In this paper, we propose an adaptive multi-view video service framework suitable for mobile environments. The proposed framework generates intermediate views in near-realtime and overcomes the limitations of mobile services by adapting the multi-view video according to the processing capability of a mobile device as well as the user characteristics of a client. By implementing the most of adaptation processes at the server side, the load on a client can be reduced. H.264/AVC is adopted as a compression scheme. The framework could provide an interactive service with efficient video service to a mobile client. For this, we present a multi-view video DIA (Digital Item Adaptation) that adapts the multi-view video according to the MPEG-21 DIA multimedia framework. Experimental results show that our proposed system can support a frame rate of 13 fps for 320{\times}240 video and reduce the time of generating an intermediate view by 20 % compared with a conventional 3D projection method.

Design options and comparison of in-network H.264/SVC adaptation

This paper explores design options and evaluates implementations of in-network, RTP/RTSP based adaptation MANEs (Media Aware Network Elements) for H.264/SVC content streaming. The obvious technique to be employed by such an adaptation MANE is to perform SVC specific bitstream extraction or truncation. Another mechanism that can be used is description (metadata) driven, coding format independent adaptation based on generic Bitstream Syntax Descriptions (gBSD), as specified within MPEG-21 Digital Item Adaptation (DIA). Adaptation MANE architectures for both approaches are developed and presented, implemented in end-to-end streaming/adaptation prototype systems, and experimentally evaluated and compared. For the gBSD based solution, open issues like the granularity of bitstream descriptions and of bitstream adaptation, metadata overhead, metadata packetization and transport options, and error resilience in case of metadata losses, are addressed. The experimental results indicate that a simple SVC specific adaptation MANE does clearly outperform the gBSD based adaptation variants. Yet, the conceptual advantages of the description driven approach, like coding format independence and flexibility, may outweigh the performance drawbacks in specific applications.

Real-Time System for Adaptive Video Streaming Based on SVC

This paper presents the integration of scalable video coding (SVC) into a generic platform for multimedia adaptation. The platform provides a full MPEG-21 chain including server, adaptation nodes, and clients. An efficient adaptation framework using SVC and MPEG-21 digital item adaptation (DIA) is integrated and it is shown that SVC can seamlessly be adapted using DIA. For protection of packet losses in an error prone environment an unequal erasure protection scheme for SVC is provided. The platform includes a real-time SVC encoder capable of encoding CIF video with a QCIF base layer and fine grain scalable quality refinement at 12.5 fps on off-the-shelf high-end PCs. The reported quality degradation due to the optimization of the encoding algorithm is below 0.6 dB for the tested sequences.

Media Accessibility for Low-Vision Users in the MPEG-21 Multimedia Framework

Ethical and legal requirements have made accessibility a crucial feature in any information systems. This paper presents a content adaptation framework, based on the MPEG-21 standard, to help low-vision users have better accessibility to visual contents. We first present an overview of MPEG-21 Digital Item Adaptation (DIA) and the low-vision description tool which enables interoperable content adaptation. This description tool lists seven low-vision symptoms, namely loss of fine detail, lack of contrast, central vision loss, peripheral vision loss, hemianopia, light sensitivity, and need of light. Then we propose a systematic contrast-enhancement method to improve the content visibility for low-vision users, focusing on the first two symptoms. The effectiveness of the low-vision description tool and our adaptation framework is verified by some experiments with an adaptation test-bed. The major advantages of the proposed approach include 1) support of a wide range of low-vision conditions, and 2) customized content adaptation to specific characteristics of each user.

Integrating Multi-view Stereoscopic Transmission System into MPEG-21 DIA (Digital Item Adaptation)

In general multi-view system, all the view sequences acquired at the server are transmitted to the client. However, this kind of system requires high processing power of the server as well as the client, thus it is posing a difficulty in practical applications. To overcome this problem, a relatively simple method is to transmit only two view-sequences requested by the client in order to deliver a stereoscopic video. In this system, effective communication between the server and the client is one of important aspects. Therefore, we propose an efficient multi-view system that transmits two view-sequences according to user's request. The view selection process is integrated into MPEG-21 DIA (Digital Item Adaptation) so that our system is compatible to MPEG-21 multimedia framework. Furthermore, multi-view descriptors related to multi-view camera and systems are newly introduced. The syntax of the descriptions and their elements is represented in XML (extensible Markup Language) schema. Intermediate view reconstruction (IVR) is used to reduce such discomfort with excessive disparity. Furthermore, IVR is useful for smooth transition between two stereoscopic view sequences. Finally, through the implementation of testbed, we can show the valuables and possibilities of our system.

Preference-driven personalization for flexible digital item adaptation

The delivery of multimedia content often needs the adaptation of the content in order to satisfy user constraints. With the Digital Item Adaptation part, the MPEG-21 standard already defines a useful frame-work to handle this task. However, in modern service-oriented architectures the functionality of adaptation is split over several services. Hence, the central instantiation of a suitable service chain needs to tackle a complex multi-objective optimization problem. In this problem between content choice and possible adaptations the current preference model in the MPEG-7/21 standard still lacks expressiveness. In the course of this paper we demonstrate this shortcoming and how the integration of more powerful models can ease the instantiation problem. Furthermore we explain how to efficiently evaluate preference trade-offs by evaluating skyline queries as currently investigated in the field of information systems. As a running example we use preference-based content adaptation in a typical media streaming application with Web services as basic modules. The contribution of our framework is to enable a central coordinator to instantiate an executable service composition chain by integrating all needed Web services to adapt the multimedia content in the best possible fashion in the sense of Pareto optimality.

A Framework for Utility-Based Multimedia Adaptation

Content adaptation is an important issue of multimedia frameworks in order to achieve universal multimedia access (UMA), i.e., to enable consumption of multimedia content independently of the given resource limitations, terminal capabilities, and user preferences. The digital item adaptation (DIA) standard, one of the core specifications of the MPEG-21 framework, supports content adaptation considering a wide range of networks, devices, and user preferences. Most adaptive multimedia frameworks targeting the UMA vision do not consider utility aspects in their adaptation decisions. This paper focuses on a generic semantic-based audio-visual utility model for DIA that aims to enhance the multimedia experience for the user. Our proposed model is able to take the semantics and the perceptual features of the content as well as the users' specific utility aspects into account. Based on a detailed analysis of these constraints, we will show how the model reacts on individual input data. For choosing the best adaptation decision considering resource limitations on client and server sides as well as network characteristics, we evaluate four algorithms for performing this adaptation decision taking task. We will discuss results according to some use case scenarios

Dynamic and interoperable adaptation of SVC for QoS-enabled streaming

Seamless streaming of multimedia content that ensures Quality of Service (QoS) over heterogeneous networks has been a desire for many multimedia services, for which the multimedia contents should be adapted to usage environments such as network characteristics, terminal capabilities, and user preferences. Scalability in video coding is a good feature to meet the requirement of heterogeneous networks. Currently, the Joint Video Team (JVT) of the ISO/IEC Moving Picture Experts Group (MPEG) and the ITU-T Video Coding Experts Group (VCEG) is standardizing a new scalable video coding standardization, called as SVC, which will become an extension to H.264/MPEG-4 AVC. In this paper, we propose a dynamic adaptation scheme of SVC bit-stream using the MPEG-21 Digital Item Adaptation (DIA) tool. MPEG-21 DIA framework provides systematic solutions in choosing the optimal adaptation operation to given conditions and supports interoperable video adaptation. The experiment results showed that the proposed adaptation scheme provides QoS-enabled delivery and consumption of SVC with time-varying constraints of network, terminal, and user preference, in a robust and efficient way. In addition, session mobility, in which a session is movable from one type of device to another type of device, was performed using switching spatial layer of SVC stream'.

Development of QoS-Aware Ubiquitous Content Access (UCA) Testbed

In this paper, a test bed of QoS-aware ubiquitous content access (UCA), which allows a user to access and consume multimedia content through convergent networks of broadcasting and communications (WLAN/CDMA) using diverse devices in a seamless and transparent way, is designed and implemented based on the adaptation of scalable and non-scalable content. Especially, the MPEG-21 digital item adaptation (DIA) is employed to find a systematic solution in choosing an optimal adaptation operation under given usage environments in an interoperable way. Furthermore, scalable video coding (SVC) technology is adopted to provide flexible adaptation in spatial, temporal, and quality dimensions in real-time

A standard‐based context sensitive service architecture for digital object dissemination

AbstractThis paper describes a context sensitive service architecture for digital object dissemination based on OpenURL and MPEG‐21 Digital Item Adaptation (DIA). This architecture entails representing service requests using the OpenURL ContextObjects and conveying the ContextObject Representations with the OpenURL Transport. The rich expressiveness of the Usage Environment Description (UED) tool defined in the MPEG‐21 DIA standard can then be incorporated into the OpenURL ContextObject by means of OpenURL extensibility mechanism, and the MPEG‐21 DIA adaptation engine can be used to construct highly customizable and adaptive context sensitive dissemination services for institutional repositories.

Dynamic multimedia stream adaptation and rate control for heterogeneous networks

Dynamic adaptation of multimedia content is seen as an important feature of next generation networks and pervasive systems enabling terminals and applications to adapt to changes in e.g. context, access network, and available Quality-of-Service (QoS) due to mobility of users, devices or sessions. We present the architecture of a multimedia stream adaptation service which enables communication between terminals having heterogeneous hardware and software capabilities and served by heterogeneous networks. The service runs on special content adaptation nodes which can be placed at any location within the network. The flexible structure of our architecture allows using a variety of different adaptation engines. A generic transcoding engine is used to change the codec of streams. An MPEG-21 Digital Item Adaptation (DIA) based transformation engine allows adjusting the data rate of scalable media streams. An intelligent decision-taking engine implements adaptive flow control which takes into account current network QoS parameters and congestion information. Measurements demonstrate the quality gains achieved through adaptive congestion control mechanisms under conditions typical for a heterogeneous network.