Multimedia Processing Systems Research Articles

A New Comparative Study of Dimensionality Reduction Methods in Large-Scale Image Retrieval

Indexing images by content is one of the most used computer vision methods, where various techniques are used to extract visual characteristics from images. The deluge of data surrounding us, due the high use of social and diverse media acquisition systems, has created a major challenge for classical multimedia processing systems. This problem is referred to as the ‘curse of dimensionality’. In the literature, several methods have been used to decrease the high dimension of features, including principal component analysis (PCA) and locality sensitive hashing (LSH). Some methods, such as VA-File or binary tree, can be used to accelerate the search phase. In this paper, we propose an efficient approach that exploits three particular methods, those being PCA and LSH for dimensionality reduction, and the VA-File method to accelerate the search phase. This combined approach is fast and can be used for high dimensionality features. Indeed, our method consists of three phases: (1) image indexing within SIFT and SURF algorithms, (2) compressing the data using LSH and PCA, and (3) finally launching the image retrieval process, which is accelerated by using a VA-File approach.

Salient Object Detection by Spatiotemporal and Semantic Features in Real-Time Video Processing Systems

Object detection is significant for event analysis in various intelligent multimedia processing systems. Although there have been many studies conducting research in this area, effective and efficient object detection methods for video sequences are still much desired. In this article, we investigate salient object detection in real-time multimedia processing systems. Considering the intrinsic relationship between top-down and bottom-up saliency features, we present a new effective method for video salient object detection based on deep semantic and spatiotemporal cues. After extracting top-down semantic features for object perception by a 2-D convolutional network, we concatenate them with bottom-up spatiotemporal cues for motion perception extracted by a 3-D convolutional network. In order to combine these features effectively, we feed them into a 3-D deconvolutional network for feature-sharing learning between semantic features and spatiotemporal cues for the final saliency prediction. Additionally, we propose a novel Gaussian-like loss function with an L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -norm regularization term for parameter learning. Experimental results show that the proposed salient object detection approach performs better in terms of both effectiveness and efficiency for video sequences compared with the state-of-the-art models.

Shape Adaptable Medical Multimedia Processing

Introduction:Electronic medical healthcare systems are becoming the backbone of health organizations over the world. The huge amounts of medical multimedia produced by these systems especially images and videos are transmitted by the computer networks that connect these systems. The variability in the shape and texture of transferred medical multimedia data needs adaptable procedures to process these data efficiently. In other words, these procedures must adjust automatically based on the shape of region of interests in the medical multimedia images to cope with fast changes in the healthcare environments. In this paper, we have proposed shape adaptable watermarking approaches for medical multimedia processing systems. The medical images generated by X-rays, MRI and CT modalities are used in our experiments to test proposed approaches. In addition, these approaches were tested under different kinds of signal processing and geometric attacks. The comparative comparison of our proposed approaches with state-of-art approaches proved the superiority and capability of our approaches to adjust the number of selected subands of medical cover image to embed and extract the hospital watermark logos.Background and Objective:The development of an adjustable approach to process medical multimedia signals for healthcare system. The aim of this research is to select adaptably the number of subands of cover image to hide the information of hospital logo watermark inside them such that embedded watermarks can resist different kinds of attacks.Method:The proposed adjustable approach consists of suband selection method, criterion, embedding and extraction procedures, DWT transform, attacks, evaluation metrics,etc.Results &amp; Conclusion:It provides robust and adjustable method to embed and extract watermark logo at different resolution levels of cover medical image and uses with images of different sizes and modalities.

Editorial Introduction to the Special Issue on Multimedia Big Data for Extreme Events

The four papers in this special section examine the use of multimedia Big Data for leveraging big events such as natural disasters. Leveraging technological advances and using Big Data approaches for information management and processing in extreme events have attracted much research attention. Multimedia is increasingly becoming the most important and valuable source for insight and information on extreme events. It covers from everyone’s experiences to everything happening in the world. There will be lots of multimedia big data in extreme events—surveillance video, disaster images, social media, voice and video, to name a few, only if their volumes grow to the extent that the traditional multimedia processing and analysis systems may not handle effectively. As such, multimedia big data in extreme events are spurring on tremendous amounts of research and development of related technologies and applications. The papers in this issue is intended to provide a premier forum for researchers working on multimedia big data for extreme events to present their recent research results. It also provides an important opportunity for multidisciplinary studies connecting multimedia computing to disaster information management.

Unequal error protection codes derived from SEC‐DED codes

Error correction codes are commonly used to protect the data stored in memories from errors. Among the codes used, single error correction double error detection (SEC-DED) codes are probably the most common due to their simplicity. In some applications, the importance of the bits is different, being some of them critical while others can tolerate some errors. This is the case, for example, in some multimedia and signal processing systems. For those applications, unequal error protection (UEP) codes that provide different protection for different bits can be used. In many cases, the bits that require extra protection are only a few. Therefore, it would be convenient to use a traditional code extended to provide additional protection for a few bits. A simple method to derive UEP codes from SEC-DED codes is presented. The proposed UEP codes protect a few bits against double errors and behave as SEC-DED codes for the rest. Encoding and decoding complexity is only slightly larger than that of an SEC-DED code and the implementation is a simple modification of the SEC-DED implementation.

Modeling and Analysis of Image Dependence and Its Implications for Energy Savings in Error Tolerant Image Processing

We present an analysis of the relationship between input images and energy consumption in error tolerant image processing. Under aggressive voltage scaling, the output image quality of image processing depends on input images for two reasons: 1) error tolerance among images is naturally disparate in terms of perceptual image quality assessment, and 2) the error rate under aggressive voltage scaling varies by input image types. Based on both effects, the supply voltage can be optimized for a given quality requirement so as to achieve ultralow power/energy dissipation. Our analysis demonstrates the significance of the accurate delay estimation, which depends on not only combinational inputs but also the previous state of the logic. We present a new sequential model for accurate error estimation. Based on the model, our experimental results demonstrate that different input image types lead to very different output quality. We also present the effect of process variation on the relationship between input image and output quality. The dependence of energy consumption on input images provides a new perspective for low-power multimedia and image processing system design.

Statistical Performance Analysis and Estimation for Parallel Multimedia Processing

When parallelizing complex multimedia processing on multiple processors, the stochastic timing behavior should be carefully studied. Although there are already many papers on the performance analysis of stochastic parallel system, they are not targeted on multimedia processing. In this paper, first we study H.264/AVC encoder (running on x86) and QSDPCM encoder (running on TI TMS32C62) to characterize important aspects of the stochastic timing behavior in complex multimedia processing applications. It is shown that the variation and correlation are indeed very significant. In order to make systematic analysis feasible, we apply Stochastic Timed Marked Graph (STMG) as a formal model to capture essential timing related behaviors of parallel multimedia processing systems. Then, we show how the local timing variations and correlations interact and propagate to the global timing behavior; from this we conclude general parallelization guidelines. Furthermore, we develop an analytical performance estimation technique to derive the probability distribution of timing behavior for parallel multimedia processing systems that have correlated stochastic timing behaviors inside. The estimation technique is based on principal component analysis and approximations.

A framework for efficient and rapid development of cross-platform audio applications

In this article, we present CLAM, a C++ software framework, that offers a complete development and research platform for the audio and music domain. It offers an abstract model for audio systems and includes a repository of processing algorithms and data types as well as all the necessary tools for audio and control input/output. The framework offers tools that enable the exploitation of all these features to easily build cross-platform applications or rapid prototypes for media processing algorithms and systems. Furthermore, included ready-to-use applications can be used for tasks such as audio analysis/synthesis, plug-in development, feature extraction or metadata annotation. CLAM represents a step forward over other similar existing environments in the multimedia domain. Nevertheless, it also shares models and constructs with many of those. These commonalities are expressed in the form of a metamodel for multimedia processing systems and a design pattern language.

A Domain-Specific Metamodel for Multimedia Processing Systems

In this paper, we introduce 4MPS, a metamodel for multimedia processing systems. The goal of 4MPS is to offer a generic system metamodel that can be instantiated to describe any multimedia processing design. The metamodel combines the advantages of the object-oriented paradigm and metamodeling techniques with system engineering principles and graphical models of computation. 4MPS is based on the classification of multimedia processing objects into two main categories: Processing objects that operate on data and controls, and Data objects that passively hold media content. Processing objects encapsulate a method or algorithm. They also include support for synchronous data processing and asynchronous event-driven Controls as well as a configuration mechanism and an explicit life cycle state model. Data input to and output from Processing objects is done through Ports. Data objects offer a homogeneous interface to media data, and support for metaobject-like facilities such as reflection and serialization. The metamodel can be expressed in the language of graphical models of computation such as the Dataflow Networks and presents a comprehensive conceptual framework for media signal processing applications. 4MPS has its practical validation in several existing environments, including the author's CLAM framework.

Design of power-efficient pipelined truncated multipliers with various output precision

An energy-efficient multiplier is very desirable for multimedia and digital signal processing systems. In many of these systems, the effective dynamic range of input operands for multipliers is generally limited to a small range and the case with maximum range seldom occurs. In addition, the output products of multipliers are usually rounded or truncated to avoid growth in word size. Based on these features, a low-power signed pipelined truncated multiplier is proposed that can dynamically detect multiple combinations of input ranges and deactivate a large amount of the unnecessary transitions in non-effective ranges to reduce the power consumption. Moreover, the proposed multiplier can trade output precision against power consumption so as to further reduce power consumption. Experimental results show that the proposed multiplier consumes up to 90% less power than the conventional standard multiplier while still maintaining an acceptable output precision and quality.

Memory Size Computation for Real-Time Multimedia Applications Based on Polyhedral Decomposition

In real-time multimedia processing systems a very large part of the power consumption is due to the data storage and data transfer. Moreover, the area cost is often largely dominated by the memory modules. In deriving an optimized (for area and/or power) memory architecture, memory size computation is an important step in the exploration of the possible algorithmic specifications of multimedia applications. This paper presents a novel non-scalar approach for computing exactly the memory size in real-time multimedia algorithms. This methodology uses both algebraic techniques specific to the data-flow analysis used in modern compilers and, also, more recent advances in the theory of polyhedra. In contrast with all the previous works which are only estimation methods, this approach performs exact memory computations even for applications significantly large in terms of the code size, number of scalars, and number of array references.

Customer value creation through customer-as-innovator approach: a case study of video processing LSI development

Changes in the business process brought about by the Customer-as-Innovator approach in some fields of product and service development have been making progress in recent years. This approach has the potential to realise dramatic improvements in development cost cuts, development times, and production volume for customers, developers, and producers. It may also enable significant reforms in existing value chains. This paper describes a case study involving the development of a multimedia video processing system LSI, positioned as a key semiconductor device in information networks of the future. This paper examines and analyses how advances in system LSI toolkits and other areas have been bringing about a shift in the system LSI development process from developers and producers (order recipients) to customers (order issuers).

IJADE surveillant—an intelligent multi-resolution composite neuro-oscillatory agent-based surveillance system

Due to the rapid development of technology, especially in the field of Internet systems, there is an increasing demand both for intelligent, mobile and autonomous systems and for the usage and conveyance of multi-media information through cyberspace. In this paper, we propose an innovative, intelligent multi-agent based model, namely intelligent Java Agent Development Environment (iJADE), to provide an intelligent agent-based platform in the e-commerce environment. In addition to the facilities found in contemporary agent development platforms, which focus on the autonomy and mobility of multi-agents, iJADE provides an intelligent layer (known as the “Conscious Layer”) to support the implementation of various AI functionalities in order to produce “smart” agents. From an implementation point of view, we introduce an intelligent, multi-media processing system known as “ iJADE Surveillant”—an intelligent multi-resolution composite neuro-oscillatory agent-based surveillance system—which is based on the integration of the following modules. (a) An automatic coarse-to-fine figure-ground scene segmentation module using the Composite Neuro-Oscillatory Wavelet-based model. (b) An automatic human face detection and extraction module using an Active Contour Model with facial “landmarks” vectors. (c) Invariant human face identification based on the Elastic Graph Dynamic Link Model. To conform to the current (and future) multi-media system standards, all of iJADE Surveillant is implemented using the MPEG-7 system framework—with comprehensive Description Schemes, feature descriptors and a model framework. From an experimental point of view, a scene gallery of over 6000 color scene images is used to test the automatic scene segmentation. One hundred distinct human subjects (with over 1020 tested scenes) are used to test the intelligent human face identification. An overall correct (invariant) facial recognition rate of over 90% is attained. We hope that the implementation of the iJADE Surveillant can provide an invariant and higher-order intelligent object (pattern) encoding, searching and identification solution for future MPEG-7 applications.

KAIST image computing system (KICS): A parallel architecture for real-time multimedia data processing

An efficient parallel architecture is proposed for high-performance multimedia data processing using multiple multimedia video processors (MVP; TMS320C80), which are fully programmable general digital signal processors (DSP). This paper describes several requirements for a multimedia data processing system and the system architecture of an image computing system called the KAIST Image Computing System (KICS). The performance of the KICS is evaluated in terms of its I/O bandwidth and the execution time for some image processing functions. An application of the KICS to the real-time Moving Picture Expert Group 2 (MPEG-2) encoder is introduced. The programmability and the high-speed data-access capability of the KICS are its most important features as a high-performance system for real-time multimedia data processing.

VLSI implementations of image and video multimedia processing systems

An overview of very large scale integrated (VLSI) implementations of multimedia processing systems is given with particular emphasis on architectures for image and video processing. Alternative design approaches are discussed for dedicated image and video processing circuits and for programmable multimedia processors. Current design examples of dedicated and programmable architectures are reviewed, and the techniques employed to improve the performance for multimedia processing are therein identified. Future trends in multimedia processing systems are anticipated with respect to current developments in emerging image and video multimedia applications.