Multimedia Information Retrieval System Research Articles

“The most passionate cover I’ve seen”: emotional information in fan-created U2 music videos

Purpose– The purpose of this paper is to explore how both producers and consumers of user-created music videos on YouTube communicate emotional information.Design/methodology/approach– In total, 150 filmic documents containing fan-generated versions of U2’s “Song for Someone” were purposively collected. The author used discourse analysis to understand the types of videos created, the communication of emotional information from both the producers and the consumers, the social construction of emotion in the filmic documents, and elements of intertextuality that represented emotion.Findings– Fans created videos containing cover versions, original versions of the song with new visual content, and tutorials about how to play the song. Producers of cover versions communicated emotional information, especially tenderness, through facial expression, their surroundings, and corresponding musical elements. Producers’ visual content expressed emotion through meaningful photographs and sad stories. Producers’ descriptions revealed emotion as well. Emotions were individually experienced and socially constructed. Consumers conveyed emotion through likes, dislikes, and expressive positive comments. Intertextuality communicated passion for U2 through tour references, paraphernalia displays, band photographs, imitating the band, and musical mashups.Practical implications– Information science can work towards a new generation of multimedia information retrieval systems that incorporate emotion in order to help users discover documents in meaningful ways that move beyond keyword and bibliographic searches.Originality/value– This is one of the earliest research papers in the area of emotional information retrieval (EmIR).

Late Semantic Fusion Approaches for Multimedia Information Retrieval with Automatic Tag Generation

retrieval in general and content based image retrieval in particular are well known research fields in information retrieval management. An image contains several types of visual information which are difficult to extract and combine manually by humans. The main goal of this paper is to show multimedia information retrieval task using the combination of textual pre-filtering and image re-ranking. The combination of textual and visual techniques and retrieval processes used to develop the multimedia information retrieval system by which we solves the problem of the semantic gap of the given query. Five late semantic fusion approaches are used for text based and content based image retrieval of any dataset. The logistic regression relevance feedback algorithm is used to determine the similarity between the images from the dataset to the query.

Optimization of Filter Combination for Query by Humming

In a multimedia information retrieval system, the response time to a query is crucial for user experience. For a practical QBH system, developers not only need to devise an effective method of melody match, but also need to consider the response speed to a query. Although the index-based methods can effectively reduce the latency, to meet the strict requirements in most real situations where, for example, the database contains hundreds of thousands of songs, further efforts are needed. To strike a balance between response speed and retrieval accuracy, this paper presents an optimal design for filter combination (FC) applied in a query by humming (QBH) system. In the design of FC, we first make use of a greedy algorithm to sort filters and then group filters to reduce the total number of levels of filters. To further accelerate the speed of retrieval and present results within a time limit, we adopt the dynamic programming to optimize the confidence threshold in each filter. The proposed methodology can greatly accelerate the retrieval process with the cost of sacrificing the least accuracy. A four-level filtering QBH system is evaluated on a large-scale database of 100,000 MIDI files. The experimental results demonstrate the feasibility of the proposed method.

Multi region based image retrieval system

Multimedia information retrieval systems continue to be an active research area in the world of huge and voluminous data. The paramount challenge is to translate or convert a visual query from a human and find similar images or videos in large digital collection. In this paper, a technique of region based image retrieval, a branch of Content Based Image Retrieval, is proposed. The proposed model does not need prior knowledge or full semantic understanding of image content. It identifies significant regions in an image based on feature-based attention model which mimic viewer’s attention. The Curvelet Transform in combination with colour descriptors are used to represent each significant region in an image. Experimental results are analysed and compared with the state-of-the-art Region Based Image Retrieval Technique.

Multimedia Information Retrieval Based on Late Semantic Fusion Approaches: Experiments on a Wikipedia Image Collection

Main goal of this work is to show the improvement of using a textual pre-filtering combined with an image re-ranking in a Multimedia Information Retrieval task. The defined three step-based retrieval processes and a well-selected combination of visual and textual techniques help the developed Multimedia Information Retrieval System to overcome the semantic gap in a given query. In the paper, five different late semantic fusion approaches are discussed and experimented in a realistic scenario for multimedia retrieval like the one provided by the publicly available ImageCLEF Wikipedia Collection.

Personalized Multimedia Information Retrieval based on User Profile Mining

This paper focuses on how to retrieve personalized multimedia information based on user interest which can be mined from user profile. After analyzing the related works, a general structure of the personalized multimedia information retrieval system is given, which combines online module and offline module. Firstly, we collect a large-sale of photos from multimedia information sharing websites. Then, we record the information of the users who upload the multimedia information. For a given user, we save his history data which could describe the multimedia data. Secondly, the relationship between contents of multimedia data and semantic information is analyzed and then the user interest model is constructed by a modified LDA model which can integrate all the influencing factors in the task of multimedia information retrieval. Thirdly, the query distributions of all the topics can be estimated by the proposed modified LDA model. Thirdly, based on the above offline computing process, the online personalized multimedia information ranking algorithm is given which utilize the user interest model and the query word. Fourthly, multimedia information retrieval results are obtained using the proposed personalized multimedia information ranking algorithm. Finally, performance evaluation is conducted by a series of experiments to test the performance of the proposed algorithm compared with other methods on different datasets.

Group Delay Based Methods for Speaker Segregation and its Application in Multimedia Information Retrieval

A novel method of single channel speaker segregation using the group delay cross correlation function is proposed in this paper. The group delay function, which is the negative derivative of the phase spectrum, yields robust spectral estimates. Hence the group delay spectral estimates are first computed over frequency sub-bands after passing the speech signal through a bank of filters. The filter bank spacing is based on a multi-pitch algorithm that computes the pitch estimates of the competing speakers. An affinity matrix is then computed from the group delay spectral estimates of each frequency sub-band. This affinity matrix represents the correlations of the different sub-bands in the mixed broadband speech signal. The grouping of correlated harmonics present in the mixed speech signal is then carried out by using a new iterative graph cut method. The signals are reconstructed from the respective harmonic groups which represent individual speakers in the mixed speech signal. Spectrographic masks are then applied on the reconstructed signals to refine their perceptual quality. The quality of separated speech is evaluated using several objective and subjective criteria. Experiments on multi-speaker automatic speech recognition are conducted using mixed speech data from the GRID corpus. A cell phone based multimedia information retrieval system (MIRS) for multi-source meeting environments are also developed.

A System based on Adaptive Background Subtraction Approach for Moving Object Detection and Tracking in Videos

Video surveillance systems are based on video and image processing research areas in the scope of computer science. Video processing covers various methods which are used to browse the changes in existing scene for specific video. Nowadays, video processing is one of the important areas of computer science. Two-dimensional videos are used to apply various segmentation and object detection and tracking processes which exists in multimedia content-based indexing, information retrieval, visual and distributed cross-camera surveillance systems, people tracking, traffic tracking and similar applications. Background subtraction (BS) approach is a frequently used method for moving object detection and tracking. In the literature, there exist similar methods for this issue. In this research study, it is proposed to provide a more efficient method which is an addition to existing methods. According to model which is produced by using adaptive background subtraction (ABS), an object detection and tracking system’s software is implemented in computer environment. The performance of developed system is tested via experimental works with related video datasets. The experimental results and discussion are given in the study

New grand challenge for multimedia information retrieval: bridging the utility gap

The needs and expectations regarding multimedia content access have grown rapidly with the fast development of multimedia technology and the explosion of multimedia content around us. This imposed high demands on the level of sophistication of multimedia information retrieval (MIR) solutions. Although the potential to develop the MIR technology that meets such high demands has also rapidly grown over the years, we are not there yet with adequate solutions. This paper states that a significant step forward could become possible if the MIR field moves towards a utility-centered research focus. There, the criteria related to utility should be deployed to help us bridge the critical remaining gap that is in front of us—the utility gap, the gap between the expected and de facto usefuleness of MIR systems. Utility criteria reach beyond the objective relevance of MIR results to also consider their informativeness and how helpful they are for user’s further actions. Bridging the utility gap can therefore be seen as the next grand challenge in the MIR research field. To pursue this challenge, we propose a utility-by-design approach, by which utility is targeted explicitly and embedded deep in the foundations of MIR solutions. The paper will first motivate this new MIR grand challenge and position it with respect to the current efforts in the field. Then, some possibilities for realizing the utility-by-design approach will be highlighted and translated into a number of recommended research directions.

A study of awareness in multimedia search

Awareness of another’s activity is an important aspect of facilitating collaboration between users, enabling an “understanding of the activities of others” ( Dourish & Bellotti, 1992). In this paper we investigate the role of awareness and its effect on search performance and behaviour in collaborative multimedia retrieval. We focus on the scenario where two users are searching at the same time on the same task, and via an interface, can see the activity of the other user. The main research question asks: does awareness of another searcher aid a user when carrying out a multimedia search session? To encourage awareness, an experimental study was designed where two users were asked to compete to find as many relevant video shots as possible under different awareness conditions. These were individual search (no awareness), Mutual awareness (where both users could see the other’s search screen), and unbalanced awareness (where one user is able to see the other’s screen, but not vice-versa). Twelve pairs of users were recruited, and the four worst performing TRECVID 2006 search topics were used as search tasks, under four different awareness conditions. We present the results of this study, followed by a discussion of the implications for multimedia information retrieval systems.

Multimedia data mining and retrieval for multimedia databases using associations and correlations

With the explosion in the complexity and amount of pervasive multimedia data, there are high demands of multimedia services and applications in various areas for people to easily access and distribute multimedia data. Facing with abundance multimedia resources but inefficient and rather old-fashioned keyword-based information retrieval approaches, a content-based multimedia information retrieval (CBMIR) system is required to (i) reduce the dimension space for storage saving and computation reduction; (ii) advance multimedia learning methods to accurately identify target semantics for bridging the semantics between low-level/mid-level features and high-level semantics; and (iii) effectively search media content for dynamical media delivery and enable the extensive applications to be media-type driven.

Fast Retrieval Algorithm for Earth Mover's Distance Using EMD Lower Bounds and a Skipping Algorithm

The earth mover's distance (EMD) is a measure of the distance between two distributions, and it has been widely used in multimedia information retrieval systems, in particular, in content-based image retrieval systems. When the EMD is applied to image problems based on color or texture, the EMD reflects the human perceptual similarities. However, its computations are too expensive to use in large-scale databases. In order to achieve efficient computation of the EMD during query processing, we have developed “fastEMD,” a library for high-speed feature-based similarity retrievals in large databases. This paper introduces techniques that are used in the implementation of the fastEMD and performs extensive experiments to demonstrate its efficiency.

Multimedia retrieval at INEX 2007

Structured document retrieval from XML documents allows for the retrieval of XML document fragments, i.e., XML elements or passages, that contain relevant information. The main INEX Ad Hoc task focuses on text-based XML retrieval. The INEX Multimedia track considers types of media other than text and its objective is to exploit the XML structure that provides a logical level at which multimedia objects are connected, in order to improve the retrieval performance of an XML-driven multimedia information retrieval system. To this end, it provides an evaluation platform for the retrieval of multimedia documents and document fragments. In addition, it creates a discussion forum where the participating groups can exchange their ideas on different aspects of the multimedia XML retrieval task.

Multimedia retrieval at INEX 2006

Structured document retrieval allows for the retrieval of document fragments, i.e. XML elements, containing relevant information. The main INEX Ad Hoc track focuses on text-based XML element retrieval. Although text is dominantly present in most XML document collections, other types of media can also be found. Existing research on multimedia information retrieval has shown that it is far from trivial to determine the combined relevance of a document that contains several multimedia objects. The objective of the INEX multimedia track is to exploit the XML structure that provides a logical level at which multimedia objects are connected, to improve the retrieval performance of an XML-driven multimedia information retrieval system.

Intelligent Indexing and Semantic Retrieval of Multimodal Documents

Finding useful information from large multimodal document collections such as the WWW without encountering numerous false positives poses a challenge to multimedia information retrieval systems (MM...

Multimedia search and retrieval: new concepts, system implementation, and application

We first present new concepts applicable to the design of multimedia search and retrieval schemes in general, and to MPEG-7 in particular, the multimedia description standard in progress. Raw multimedia data is assumed to exist in the form of programs that typically consist of a combination of media types such as visual, audio, and text. We partition each such media stream into smaller units based on actual physical events. These physical events within each media stream can then be effectively indexed for retrieval. The concept of logical events is introduced next; we define logical events as those that can provide different views of the content as may be desired by a user. Such events usually result from either the correlation of events that cross different media types, of by merging recursively chosen events from a lower level within each media type. We then address the related issue of how to develop a practical multimedia information retrieval system that exploits the aforementioned concepts of physical and logical events as well as other aspects such as storage, representation and indexing to enable efficient search, retrieval, and browsing. Finally, we implement the proposed concepts and solutions within a multimedia system that addresses a real application, effective browsing of broadcast news, and evaluate its performance.

Design issues of multimedia information indexing and retrieval systems

Multimedia information indexing and retrieval systems are required to manage and use ever increasing multimedia information effectively and efficiently. This paper first provides an over-view of general capabilities and architecture of multimedia information indexing and retrieval systems, and then discusses a number of main design issues including data modelling, user interface, feature extraction, indexing, feature similarity measures and quality of service guarantees.

MITOMAP: an update on the status of the human mitochondrial genome database.

We have continued to develop MITOMAP, a comprehensive database for the human mitochondrial DNA (mtDNA). MITOMAP uses the mtDNA sequence as the unifying element for bringing together information on mitochondrial genome structure and function, pathogenic mutations and their clinical characteristics, population associated variation and gene-gene interactions. As increasingly larger regions of the human genome are sequenced and characterized, the need for integrating such information will grow. Consequently, MITOMAP not only provides a valuable reference for the mitochondrial biologist, it will also provide a model for the development of comprehensive, multi-media information storage and retrieval systems for other components of the human genome. This paper is an update of the changes which have occurred to MITOMAP over the past year.

Structuring collaborative information-seeking dialogues

Conversational approaches to human—computer collaboration have so far mostly been employed for the design of natural language interfaces. It is claimed, however, that the conversational interaction model described in the paper can also feasibly be applied to graphical and multimodal interactions. The model comprises two interrelated parts: first, the description of local discourse structures and functional interrelations between dialogue acts to capture (local) conversational tactics, and second, the description of global structures by so-called dialogue scripts which are related to (global) information-seeking strategies. The application of the model in a multi-media information retrieval system is described in the conclusions to the paper.

A WWW interface to the OMNIS/Myriad literature retrieval engine

Cataloguing and searching procedures in traditional library systems are expensive, time-consuming and often incomplete. OMNIS is a novel multimedia information retrieval system for the administration of documents in libraries and offices. Using the fulltext database system Myriad combined with scanning and OCR technologies it offers the disclosing, archiving and searching functions at drastically reduced costs with much more precision. Documents may contain page images, full-length PostScript or other medial information and offer the user a much better insight into documents. At Technische Universität München a considerable number of computer science documents have been made searchable by a simple fulltext query language. To make this document retrieval system available to WWW clients the OMNIS document access function was implemented as OMNIS-WWW server which is already in operation. This paper contains the substantial features of OMNIS (especially its searching function) and discusses the concepts and the implementation of its WWW server. The development of OMNIS is promoted by DFG (Deutsche Forschungsgemeinschaft) and DFN-Verein (Deutsches Forschungsnetz) [6].