MPEG-4 Standard Research Articles

Understanding Timelines Within MPEG Standards

Nowadays, media content can be delivered via diverse broadband and broadcast technologies. Although these different technologies have somehow become rivals, their coordinated usage and convergence, by leveraging of their strengths and complementary characteristics, can bring many benefits to both operators and customers. For example, broadcast TV content can be augmented by on-demand broadband media content to provide enriched and personalized services, such as multi-view TV, audio language selection, and inclusion of real-time web feeds. A piece of evidence is the recent Hybrid Broadcast Broadband TV (HbbTV) standard, which aims at harmonizing the delivery and consumption of (hybrid) broadcast and broadband TV content. A key challenge in these emerging scenarios is the synchronization between the involved media streams, which can be originated by the same or different sources, and delivered via the same or different technologies. To enable synchronized (hybrid) media delivery services, some mechanisms providing timelines at the source side are necessary to accurately time align the involved media streams at the receiver-side. This paper provides a comprehensive review of how clock references (timing) and timestamps (time) are conveyed and interpreted when using the most widespread delivery technologies, such as DVB, RTP/RTCP and MPEG standards (e.g., MPEG-2, MPEG-4, MPEG-DASH, and MMT). It is particularly focused on the format, resolution, frequency, and the position within the bitstream of the fields conveying timing information, as well as on the involved components and packetization aspects. Finally, it provides a survey of proofs of concepts making use of these synchronization related mechanisms. This complete and thorough source of information can be very useful for scholars and practitioners interested in media services with synchronization demands.

A Novel Compression Technique for Multi-Camera Nodes through Directional Correlation

A major fraction of multimedia stream contents tends to be redundant and leads to wastage of storage capacity and channel bandwidth. In order to eliminate surplus data, standard video compression algorithms exploit spatial and temporal correlation present in video sequence. However, in case of a multisensor network, intersensor statistical redundancy is the most significant factor in acquiring efficient link utilization as well as making resultant findings valuable to the end user. In this paper, an extension to our previously proposed scheme has been presented to accomplish performance goals of a multisensor environment. Standard MPEG codec has been used to accomplish distributed motion compensation in prespecified directions known as directional correlation. Video frame correlation has been estimated locally at the camera node as well as across different nodes, defined as node communication strategies. Further, receiver feedback assists in quality control after reconstitution by decoder assessment. Results estimated have been analyzed for saving ratios and multimedia quality. Results analysis illustrates increased gains in frame quality and compression saving, achieved through reducing node displacement from the reference node [Formula: see text].

Fast and Efficient Indexing and Similarity Searching in 2D/3D Image Databases

D-index is among the most efficient similarity search indexes, its performance dramatically depends on the choice of the ρ parameter. In This paper we propose a new criteria and a technique that ensures a good choice of D-index ρ parameter for reducing the D-index searching cost with an acceptable construction cost. We present our results upon two 2D/3D image databases; the Amsterdam Library of Images ALOI-1000 consists of 72000 color images of views and COIL-100 database consists of 7200 color images of views. We have described each view by combination of three well established descriptors from the MPEG-7 standard: CSD, SCD and EHD, and we have used the recommended distance measures by MPEG-7 to compare views. The results obtained prove the search efficiency of the proposed method against the original D-index and the sequential method

A complete framework to manage multimedia ontologies in digital ecosystems

The managing of large amounts of data in digital ecosystems and in big data environments requires the use of formal models and techniques to represent and organise information. In this paper, we describe a novel framework to manage multimedia ontologies using a knowledge engineering approach. The proposed methodology is based on multimedia ontologies organised following a formal model. Our ontologies use multimedia data and linguistic properties to bridge the gap between the target semantic classes and the available low-level multimedia descriptors. The multimedia features are automatically extracted using algorithms based on MPEG-7 standard. In our approach, multimedia ontologies are used to annotate and categorise images. The informative image content is annotated with semantic information extracted from our ontologies and the categories are dynamically built by means of a general knowledge base. The multimedia ontologies are exposed by means of web services and a peer-to-peer (P2P) architecture to allow their reuse and sharing. A complete use case example is presented and experimental results show the efficiency of our approach in the annotation and classification tasks using a combination of textual and visual components.

Unobtrusive emotion sensing and interpretation in smart environment

Currently, a particular focus of human centered technology is in expanding traditional contextual sensing and smart processing capabilities of ubiquitous systems exploiting user's affective and emotional states to develop more natural communication between computing artefacts and users. This paper presents a smart environment of Web services that has been developed to integrate and manage different existing and new emotion sensing applications, which working together provide tracking and recognition of human affective state in real time. In addition, two emotion interpreters based on the proposed 6-FACS and Distance models have been developed. Both models operate with encoded facial deformations described either in terms of Ekman's Action Units or Facial Animation Parameters of MPEG-4 standards. Fuzzy inference system based on reasoning model implemented in a knowledge base has been used for quantitative measurement and recognition of three-level intensity of basic and non-prototypical facial expressions. Designed frameworks integrated to smart environment have been tested in order to evaluate capability of the proposed models to extract and classify facial expressions providing precision of interpretation of basic emotions in range of 65--96% and non-prototypical emotions in range of 55--65%. The conducted tests confirm that such basic as non-prototypical expressions may be composed by other basic emotions establishing in this way the concordance between existing psychological models of emotions and Ekman's model traditionally used by affective computing applications.

Enhancing the Interoperability of Multimedia Learning Objects Based on the Ontology Mapping

This article addresses the interoperability between the semantic learning platforms and the educational resources banks, more precisely between the LOM and MPEG-7 standards. LOM is a set of metadata associated with e-learning content, while MPEG-7 is a standard for describing multimedia content. The use of educational resources has become an essential component to meet the learning needs. Given the multimedia nature of these resources, such use causes problems in the interoperability of multimedia learning objects in e-Learning environments, indexing and retrieval of digital resources. Faced with these problems, we propose a new approach for the multimedia learning objects by using the ontology mapping between the LOM and MPEG-7 ontologies.

An Object-Level Feature Representation Model for the Multi-target Retrieval of Remote Sensing Images

To address the problem of multi-target retrieval (MTR) of remote sensing images, this study proposes a new object-level feature representation model. The model provides an enhanced application image representation that improves the efficiency of MTR. Generating the model in our scheme includes processes, such as object-oriented image segmentation, feature parameter calculation, and symbolic image database construction. The proposed model uses the spatial representation method of the extended nine-direction lower-triangular (9DLT) matrix to combine spatial relationships among objects, and organizes the image features according to MPEG-7 standards. A similarity metric method is proposed that improves the precision of similarity retrieval. Our method provides a trade-off strategy that supports flexible matching on the target features, or the spatial relationship between the query target and the image database. We implement this retrieval framework on a dataset of remote sensing images. Experimental results show that the proposed model achieves competitive and high-retrieval precision.

The Research of 3D Mouth Animation Synthesis Driven by Continuous Chinese Text Based on Mpeg-4

A 3D human facial model was made based on the MPEG-4 standard. A method of synthesis of mouth animation based on a entire Chinese sentence was put forward, according to the influence of Chinese pronunciation rules such as co-articulation sufficiently. The method computes the extent of influence between current phoneme and the one next to it, and constructs the continuous mouth-shape sequence. Then filter the phoneme which has little effect on the last and the next one when pronouncing continuously. The 3D facial model was driven to synthesize smooth and realistic animation. After the experiments, it is proved that the method has a good versatility and reliability, which can be used on different kinds of 3D models. At the same time, the method can effectively improve the problem of co-articulation in continuous Chinese pronunciation, which makes the synthesis of mouth animation more smooth, more fluent, and more close to real pronunciation of people.

A Hierarchical Security Model for Multimedia Big Data

In this paper, the author proposes a hierarchical security model (HSM) to enhance security assurance for multimedia big data. It provides role hierarchy management and security roles/rules administration by seamlessly integrating the role-based access control (RBAC) with the object-oriented concept, spatio-temporal constraints, and multimedia standard MPEG-7. As a result, it can deal with challenging and unique security requirements in the multimedia big data environment. First, it supports multilayer access control so different access permission can be conveniently set for various multimedia elements such as visual/audio objects or segments in a multimedia data stream when needed. Second, the spatio-temporal constraints are modeled for access control purpose. Finally, its security processing is efficient to handle high data volume and rapid data arrival rate.

Reconfigurable media coding: An overview

This paper provides an overview of the rationale of the Reconfigurable Media Coding framework developed by MPEG standardization committee to overcome the limits of traditional ways of providing decoder specifications. Such framework is an extension of the Reconfigurable Video coding framework now encompassing also 3D Graphics coding standard. The idea of this approach is to specify decoders using an actor dataflow based representation consisting of self-contained processing units (coding tools) connected altogether and communicating by explicitly exchanging data. Such representation provides a specification for which several properties of the algorithms interesting for codec implementations are explicitly exposed and can be used for exploring different implementation objectives.

Parser description-based bitstream parser generation for MPEG RMC framework

In this paper, we present a bitstream syntax description scheme that enables the automatic generation of a bitstream parser in the reconfigurable media coding (RMC) framework. In the RMC framework, a standard video decoder can be defined with a set of functional units (FUs) pre-defined in a tool library and a decoder description containing the interconnections between the FUs and the bitstream parser description. The FUs in the tool library are codec-independent in that any FU can be reused. In contrast, the decoder description is codec-specific. In particular, the bitstream parser description is required for the generation of the bitstream parser FU in the RMC framework. There have been several works on bitstream parser description and parser FU generation; however, conventional methods have mainly focused on built-in parser generation during the design time, rather than on the automatic generation of the parser FU in the run-time from the description. As a response to this problem, we propose a bitstream parser description format and a run-time parser generation mechanism based on a generic parser FU. The proposed language aims to support run-time parser FU generation by describing three types of information: control flow, external functions, and parser interfaces. The control flow description is described based on the finite state machine concept, making the translation of the BSD in run-time easy. Throughout the paper, we show how the proposed scheme is better suited for the MPEG RMC framework than the existing description formats. We also discuss the implementation of the generic parser FU (GPFU) to show that the run-time parser generation can be done efficiently with the proposed description format. A simulation result using the generic parser FU based on the existing MPEG standards (MPEG-4 SP and SC3DMC) is provided as an existential proof.

MPEG Reconfigurable Video Coding: From specification to a reconfigurable implementation

This paper demonstrates that it is possible to produce automatic, reconfigurable, and portable implementations of multimedia decoders onto platforms with the help of the MPEG Reconfigurable Video Coding (RVC) standard. MPEG RVC is a new formalism standardized by the MPEG consortium used to specify multimedia decoders. It produces visual representations of decoder reference software, with the help of graphs that connect several coding tools from MPEG standards. The approach developed in this paper draws on Dataflow Process Networks to produce a Minimal and Canonical Representation (MCR) of MPEG RVC specifications. The MCR makes it possible to form automatic and reconfigurable implementations of decoders which can match any actual platforms. The contribution is demonstrated on one case study where a generic decoder needs to process a multimedia content with the help of the RVC specification of the decoder required to process it. The overall approach is tested on two decoders from MPEG, namely MPEG-4 part 2 Simple Profile and MPEG-4 part 10 Constrained Baseline Profile. The results validate the following benefits on the MCR of decoders: compact representation, low overhead induced by its compilation, reconfiguration and multi-core abilities.

Distributed Content Semantic Organization and Access Control with EuterpeFS

Files have been the most common information containers for many decades and, as such, they have been established as the natural way for users to access their data. With the growth of distributed systems, the distributed file systems have emerged as a constantly active research area, with various solutions that cover a vast variety of requirements. EuterpeFS is a distributed file system that emphasizes in the efficient semantic organization of the distributed content and its protection, based on MPEG standards. Its layered, fine grained and standards based, design makes EuterpeFS an easily adaptable distributed file system that can work with different publish- subscribe protocols, security and semantic technologies; even with other distributed file systems. The contributions of EuterpeFS can be summarized in the following aspects: ．Semantic organization of content where the directory structured is dictated by ontologies. ．Content protection using MPEG-21 licensing technologies allowing protecting content in a dynamic distributed environment, where users are arbitrarily entering and leaving the system. ．Implementation based on standard MPEG-M API enabling easy integration and interoperation between different implementations of particular components.

Coding B-Frames of Color Videos with Fuzzy Transforms

We use a new method based on discrete fuzzy transforms for coding/decoding frames of color videos in which we determine dynamically the GOP sequences. Frames can be differentiated into intraframes, predictive frames, and bidirectional frames, and we consider particular frames, called Δ-frames (resp., R-frames), for coding P-frames (resp., B-frames) by using two similarity measures based on Lukasiewicz -norm; moreover, a preprocessing phase is proposed to determine similarity thresholds for classifying the above types of frame. The proposed method provides acceptable results in terms of quality of the reconstructed videos to a certain extent if compared with classical-based F-transforms method and the standard MPEG-4.

Automated Content Metadata Extraction Services Based on MPEG Standards

This paper is concerned with the generation, acquisition, standardized representation and transport of video metadata. The use of MPEG standards in the design and development of interoperable media architectures and web services is discussed. A high-level discussion of several algorithms for metadata extraction is presented. Some architectural and algorithmic issues encountered when designing services for real-time processing of video streams, as opposed to traditional offline media processing, are addressed. A prototype real-time video analysis system for generating MPEG-7 Audiovisual Description Profile from MPEG-2 transport stream encapsulated video is presented. Such a capability can enable a range of new services such as content-based personalization of live broadcasts given that the MPEG-7 based data models fit in well with specifications for advanced television services such as TV-Anytime and Alliance for Telecommunications Industry Solutions IPTV Interoperability Forum.

Multi-modal user interaction method based on gaze tracking and gesture recognition

This paper presents a gaze tracking technology which provides a convenient human–centric interface for multimedia consumption without any wearable device. It enables a user to interact with various multimedia on a large display in distance by tracking user movement and acquiring high resolution eye images. This paper also presents a gesture recognition technology which is helpful to interact with scene descriptions in terms of controlling and rendering scene objects. It is based on Hidden Markov Model and CRF using a commercial depth sensor. And then, this paper shows a collaboration method with those new sensors and MPEG standards in order to achieve interoperability among interactive applications, new user interaction devices and users.

Image retrieval using edge based shape similarity with multiresolution enhanced orthogonal polynomials model

Since shape is one of the important low level features of any Content Based Image Retrieval (CBIR) system, this paper proposes a new edge based shape feature representation method with multiresolution enhanced orthogonal polynomials model and morphological operations for effective image retrieval. In the proposed method, initially the orthogonal polynomials model coefficients are computed and reordered into multiresolution subband like structure. Edge image is then obtained by utilizing the two level adaptive thresholds and local maxima of the gradient in horizontal, vertical, diagonal and anti-diagonal directions. The approximate shape boundary of the image is recovered with morphological operations. Then the Pseudo Zernike moment based global shape features, which are invariant to basic geometric transformations, are extracted. The obtained features are termed as global shape feature vector and are used for retrieving similar images with Canberra distance metric. The efficiency of the proposed method is experimented on a subset of standard Corel, Yale and MPEG-7 databases and the results are compared with existing techniques.

Trademark image retrieval using an integrated shape descriptor

Trademarks are distinctive visual symbols with high reputational value, due to the perception of quality and innovation associated with them. They are important reputational assets used as a marketing tool to convey a certain assurance of quality, innovation, and the standards, which the manufacturer seeks to maintain. This motivates the need for trademark protection by providing a solution to prevent infringement. This problem can be addressed by developing retrieval systems capable of comparing the visual similarity of trademarks. This paper contributes to the research in this field by proposing an innovative trademark retrieval technique with improved retrieval performance due to the integration of global and local descriptors. The global descriptor employed is the Zernike moment’s coefficients. The local descriptor is the edge-gradient co-occurrence matrix, derived from the contour information that is considered very important in human perception of visual similarity. The proposed retrieval technique is tested using the standard MPEG-7 shape database of 1400 images and the MPEG-7 trademark database of 3260 images. The results show 5% precision/recall improvement in the case of the MPEG-7 shape database, as well as 2.35% Bull’s eye score improvement and 19.8% NMRR score improvement for the 10 randomly selected trademarks from the MPEG-7 trademarks database.

Removal of Blocking Artifacts using Various Filtering Techniques

processing is the method for processing various kinds of images, processed images can be stored conveniently and transmission of such kind of images from one place to another place becomes easy to the user. By using image compression method we are able to represent the image with lesser number of bits. By using image compression technique we can reduce the bandwidth and the volume of the data to be transmitted. Different types of images can be processed and we can reduce blocking artifacts up to acceptable level. Some standard techniques MPEG and JPEG are used in image processing field. Some other types of images like GIF, JPG and PNG are used. Lossless compression is used for medical image compression because whole information is necessary for analysis; even a single bit loss is not bearable. Lossy image compression technique is used in photographic images because loss of bits is bearable. When we compress any kind of image by lossy type of image compression technique then there will be loss of bits, and when we want to recover such kind of image then we face problem of blurring of images, which is sometimes called as the annoying problem of blocking artifacts. To recover original image back we can use decompression followed by the different filtering methods. We can compare these filtering techniques on the basis of the MATLAB simulation results. Various Parameters can be compared like PSNR, MSE and BER. General Terms Image compression, Redundancy, compression factor, blocking artifacts, filtering techniques, removing noise et. al.

A New Method for Content-Based Image Retrieval via Subsets of Key Contours Fragments

In this paper, a novel method is proposed for solving an open problem of shape matching in content-based image retrieval. In order to prepare for matching two images, our novel method uses an improved segment method to get more accurate contours. And then we find a new way to extract the key contour fragments, so that a shape can be presented as many key contour fragments in this way. It is an effective way to do analysis in image contour. In addition, a similar measure method is proposed to get the correlation rate of two images via subsets of key contour fragments. Finally, experiment results show that our new method can get more robustness than existing methods and achieve a superior matching effect in the standard shape databases MPEG-7.