Multimedia Description Schemes Research Articles

User Edited Contents 생성을 위한 동영상 메타데이터 스키마 설계 및 저작 도구 구현

In this paper, we design new video metadata schema for searching video segments to create UEC (User Edited Contents). The proposed video metadata schema employs hierarchically structured units of 'Title-Event-Place(Scene)-Shot', and defines the fields of the semantic information as structured form in each segment unit. Since this video metadata schema is defined by analyzing the structure of existing UECs and by experimenting the tagging and searching the video segment units for creating the UECs, it helps the users to search useful video segments for UEC easily than MPEG-7 MDS (Multimedia Description Scheme) which is a general purpose international standard for video metadata schema.

온톨로지 기반의 멀티미디어 콘텐츠 검색 시스템

최근 멀티미디어 기술과 네트워크의 발전으로 멀티미디어 콘텐츠의 생성은 빠르게 늘어가고 있다. 그러나 이를 찾고 활용하는 기술은 부족하다. 이러한 문제를 보완하고 해결하기 위해 멀티미디어 콘텐츠를 위한 표준안들이 많이 있으나 멀티미디어 데이터 형식과 상호운용성을 표현하기에는 부족하다. 본 논문에서는 멀티미디어 콘텐츠를 상호 운용할 수 있도록 하는 온톨로지 기반의 콘텐츠 검색 시스템을 제시한다. 온톨로지는 MPEG-7의 MDS(Multimedia Description Scheme)와 같은 스키마 구조에 온톨로지 구축을 위한 규칙을 제시하여 구성한다. 이러한 온톨로지를 바탕으로 멀티미디어 데이터가 가지고 있는 연관성을 확장하여 의미론적 검색 시스템을 구축한다. With the development of multimedia and network technology, the production of multimedia contents is rapidly increasing. Meanwhile, the technology to search and use the contents is still insufficient. There are standards for multimedia contents to address the problem, but they cannot fully support diverse multimedia data types or ensure their interoperability. In this paper, an ontology-based content search system is proposed to ensure the interoperability of multimedia contents. The ontology is configured by presenting the rules for it using the schema structure of the multimedia description scheme (MDS) of MPEG-7. Based on this ontology, This paper extend multimedia relationship based on ontology, thus established the semantic retrieval system.

Content on demand video adaptation based on MPEG-21 digital item adaptation

One of the major objectives in multimedia research is to provide pervasive access and personalized use of multimedia information. Pervasive access of video data implies the access of cognitive and affective aspects of video content. Personalized use requires the services satisfy individual user's needs on video content. This article attempts to provide a content-on-demand (CoD) video adaptation solution by considering users' preference on cognitive content and affective content for video media in general, sports video and movies in particular. In this article, CoD video adaptation system is developed to support users' decision in selecting their content of interest and adaptively deliver video source by selecting relevant content and dropping frames while considering network conditions. First, video contents are annotated by the description schemes (DSs) provided by MPEG-7 multimedia description schemes (MDSs). Then, to achieve a generic adaptation solution, the adaptation is developed following MPEG-21 Digital Item Adaptation (DIA) framework. We study the MPEG-21 reference software on XML generation and develop our own system for CoD video adaptation in three steps: (1) the content information is parsed from MPEG-7 annotation XML file together with bitstream to generate generic Bitstream Syntax Description (gBSD); (2) Users' preference, network characteristic and adaptation QoS (AQoS) are considered for making adaptation decision; (3) adaptation engine automatically parses adaptation decisions and gBSD to achieve adaptation. Unlike most existing adaptation work, the system adapts the content of interest in the video stream according to users' preference. We implement the above-mentioned MPEG-7 and MPEG-21 standards and provide a generic video adaptation solution. Adaptation based on gBSD avoids complex video computation. Thirty students from various departments were invited to assess the system and their responses have been positive.

Efficient multimedia query-by-content from mobile devices

The phenomenal growth in multimedia content has lead to the development of a variety of multimedia description schemes, which can be used to facilitate querying of multimedia databases. In the increasingly mobile environment of today, multimedia query formats need to be applicable to mobile devices, which, compared to desktop PCs, have specific limitations such as small screen size, limited memory and processing power and high bandwidth cost. As a potential solution to multimedia querying in mobile environments, this paper introduces two concepts: query streaming and its application as targeted browsing. Targeted browsing is a technique for multimedia query-by-content designed especially for mobile devices while query streaming is a method for continually updating a query by sending additional terms to an existing query. This paper describes an implementation of query streaming that combines the Multimedia Query Format (MQF) (a standard communication language for querying multimedia databases) with Fragment Request Units (FRU) and Fragment Update Units (FUU) (which provide a standard way of randomly accessing fragments of XML documents). For efficient compression of the multimedia query XML files, the use of binary compression using MPEG BiM is proposed and a number of use case scenarios are examined. Results show that the proposed solution to provide a significant reduction in the file size required to perform multimedia querying.

MPEG‐7 tools for Universal Multimedia Access

AbstractUniversal Multimedia Access (UMA) deals with seamless access to once‐only‐created content via any kind of terminal and any kind of network connectivity, which implies that the content should be adapted in order to fit a variety of terminal and network characteristics, as well as user preferences. The MPEG‐7 standard offers some support for UMA within its section on Multimedia Description Schemes (MDS). Within the standard, several groups of tools serve this purpose. For instance, the Navigation and Access Tools provide some Description Schemes that allow the description of adapted content variations and summaries and allow for preprocessed content versions. Some support is also found in the Content Metadata Tools (Media and Usage Tools), for real‐time ease in creation of online content versions and in limited support for session description, which is completed in MPEG‐21.

Automatic user preference learning for personalized electronic program guide applications

AbstractIn this article, we introduce a user preference model contained in the User Interaction Tools Clause of the MPEG‐7 Multimedia Description Schemes, which is described by a UserPreferences description scheme (DS) and a UsageHistory description scheme (DS). Then we propose a user preference learning algorithm by using a Bayesian network to which weighted usage history data on multimedia consumption is taken as input. Our user preference learning algorithm adopts a dynamic learning method for learning real‐time changes in a user's preferences from content consumption history data by weighting these choices in time. Finally, we address a user preference–based television program recommendation system on the basis of the user preference learning algorithm and show experimental results for a large set of realistic usage‐history data of watched television programs. The experimental results suggest that our automatic user reference learning method is well suited for a personalized electronic program guide (EPG) application.

An MPEG-7 scheme for semantic content modelling and filtering of digital video

Part 5 of the MPEG-7 standard specifies Multimedia Description Schemes (MDS); that is, the format multimedia content models should conform to in order to ensure interoperability across multiple platforms and applications. However, the standard does not specify how the content or the associated model may be filtered. This paper proposes an MPEG-7 scheme which can be deployed for digital video content modelling and filtering. The proposed scheme, COSMOS-7, produces rich and multi-faceted semantic content models and supports a content-based filtering approach that only analyses content relating directly to the preferred content requirements of the user. We present details of the scheme, front-end systems used for content modelling and filtering and experiences with a number of users.

Multimodal Semantic Analysis and Annotation for Basketball Video

This paper presents a new multiple-modality method for extracting semantic information from basketball video. The visual, motion, and audio information are extracted from video to first generate some low-level video segmentation and classification. Domain knowledge is further exploited for detecting interesting events in the basketball video. For video, both visual and motion prediction information are utilized for and scene boundary detection algorithm; this will be followed by scene classification. For audio, audio keysounds are sets of specific audio sounds related to semantic events and a classification method based on hidden Markov model (HMM) is used for audio keysound identification. Subsequently, by analyzing the multimodal information, the positions of potential semantic events, such as foul and shot at the basket, are located with additional domain knowledge. Finally, a video annotation is generated according to MPEG-7 multimedia description schemes (MDSs). Experimental results demonstrate the effectiveness of the proposed method.

Agent-Based Intelligent Multimedia Broadcasting within MPEG-21 Multimedia Framework

It is expected that an intelligent broadcasting service (IBS) will be able to provide broadcast programs based on user preference and program-associated information (metadata) in order to assist users in easy navigation of the program content being broadcast. In this way, users will be able to access program content anytime, anywhere, and in the manner they wish. This type of IBS will be a basis for future broadcasting services such as customized broadcasting or personal casting. In this paper, we introduce an agent-based multimedia broadcasting framework using the Foundation for Intelligent Physical Agents (FIPA) and MPEG-7 technologies within MPEG-21. We use a FIPA implementation called FIPA open source as a platform for exchanging user preferences and program information as FIPA messages between a server and its clients. The user preference is modeled as the User Preference description scheme in MPEG-7 multimedia description schemes. We discuss a framework structure and implementation for the IBS.

MPEG-7 based description schemes for multi-level video content classification

MPEG-7 has emerged as the standard for multimedia data content description for efficiently describing multimedia content. In this context, its primary goal is to provide flexible and effective searching and retrieval of multimedia resources. Most of the earlier work on MPEG-7 description schemes (DSs) and descriptors (Ds) focuses on the description of a single multimedia document, whereas MPEG-7 can be further exploited to support more advances implementations under multimedia database systems. Therefore, it is important to reconsider issues related to high level multimedia modeling and representation, in the light of the MPEG-7 perspective. In this paper, we propose a high level multimedia representation and description scheme based on multi-level video modeling and semantic video classification. The proposed multi-level multimedia representation and DSs are expected to support more effective video content indexing and accessing operations. The presented DSs and Ds are further described by using the XML Schema language, which has been adopted as the basis of the Description Definition Language (DDL) of the MPEG-7 standard.

MPEG-7 multimedia description schemes

MPEG-7 multimedia description schemes (MDSs) are metadata structures for describing and annotating audio-visual (AV) content. The description schemes (DSs) provide a standardized way of describing in XML the important concepts related to AV content description and content management in order to facilitate searching, indexing, filtering, and access. The DSs are defined using the MPEG-7 description definition language, which is based on the XML Schema language, and are instantiated as documents or streams. The resulting descriptions can be expressed in a textual form (i.e., human readable XML for editing, searching, filtering) or compressed binary form (i.e., for storage or transmission). In this paper, we provide an overview of the MPEG-7 MDSs and describe their targeted functionality and use in multimedia applications.

Object-based multimedia content description schemes and applications for MPEG-7

In this paper, we describe description schemes (DSs) for image, video, multimedia, home media, and archive content proposed to the MPEG-7 standard. MPEG-7 aims to create a multimedia content description standard in order to facilitate various multimedia searching and filtering applications. During the design process, special care was taken to provide simple but powerful structures that represent generic multimedia data. We use the extensible markup language (XML) to illustrate and exemplify the proposed DSs because of its interoperability and flexibility advantages. The main components of the image, video, and multimedia description schemes are object, feature classification, object hierarchy, entity-relation graph, code downloading, multi-abstraction levels, and modality transcoding. The home media description instantiates the former DSs proposing the 6-W semantic features for objects, and 1-P physical and 6-W semantic object hierarchies. The archive description scheme aims to describe collections of multimedia documents, whereas the former DSs only aim at individual multimedia documents. In the archive description scheme, the content of an archive is represented using multiple hierarchies of clusters, which may be related by entity-relation graphs. The hierarchy is a specific case of entity-relation graph using a containment relation. We explicitly include the hierarchy structure in our DSs because it is a natural way of defining composite objects, a more efficient structure for retrieval, and the representation structure used in MPEG-4. We demonstrate the feasibility and the efficiency of our description schemes by presenting applications that already use the proposed structures or will greatly benefit from their use. These applications are the visual apprentice, the AMOS-search system, a multimedia broadcast news browser, a storytelling system, and an image meta-search engine, MetaSEEk.