Visual Information Retrieval Research Articles

Weighted indexing of TV sports news videos

The paper proposes measures for weighted indexing of sports news videos. The content-based analyses of sports news videos lead to the classification of frames or shots into sports categories. A set of sports categories reported in a given news video can be used as a video representation in visual information retrieval system. However, such an approach does not take into account how many sports events of a given category have been reported and how long these events have been presented in news for televiewers. Weighting of sports categories in a video representation reflecting their importance in a given video or in a whole video data base would be desirable. The effects of applying the proposed measures have been demonstrated in a test video collection. The experiments and evaluations performed on this collection have also shown that we do not need to apply perfect content-based analyses to ensure proper weighted indexing of sports news videos. It is sufficient to recognize the content of only some frames and to determine the number of shots, scenes or pseudo-scenes detected in temporal aggregation process, or even only the number of events of a given sports category in a sports news video being indexed.

Evaluating multimodal relevance feedback techniques for medical image retrieval

Medical image retrieval can assist physicians in finding information supporting their diagnosis and fulfilling information needs. Systems that allow searching for medical images need to provide tools for quick and easy navigation and query refinement as the time available for information search is often short. Relevance feedback is a powerful tool in information retrieval. This study evaluates relevance feedback techniques with regard to the content they use. A novel relevance feedback technique that uses both text and visual information of the results is proposed. The two information modalities from the image examples are fused either at the feature level using the Rocchio algorithm or at the query list fusion step using a common late fusion rule. Results using the ImageCLEF 2012 benchmark database for medical image retrieval show the potential of relevance feedback techniques in medical image retrieval. The mean average precision (mAP) is used as the evaluation metric and the proposed method outperforms commonly-used methods. The baseline without feedback reached 16 % whereas the relevance feedback with 20 images reached up to 26.35 % with three steps and when using 100 images up to 34.87 % in four steps. Most improvements occur in the first two steps of relevance feedback and then results start to become relatively flat. This might also be due to only using positive feedback as negative feeback often also improves results after more steps. The effect of relevance feedback in automatically spelling corrected and translated queries is investigated as well. Results without mistakes were better than spell-corrected results but the spelling correction more than double results over non-corrected retrieval. Multimodal relevance feedback has shown to be able to help visual medical information retrieval. Next steps include integrating semantics into relevance feedback techniques to benefit from the structured knowledge of ontologies and experimenting on the fusion of text and visual information.

Alzheimer's disease diagnosis on structural MR images using circular harmonic functions descriptors on hippocampus and posterior cingulate cortex

Recently, several pattern recognition methods have been proposed to automatically discriminate between patients with and without Alzheimer's disease using different imaging modalities: sMRI, fMRI, PET and SPECT. Classical approaches in visual information retrieval have been successfully used for analysis of structural MRI brain images. In this paper, we use the visual indexing framework and pattern recognition analysis based on structural MRI data to discriminate three classes of subjects: normal controls (NC), mild cognitive impairment (MCI) and Alzheimer's disease (AD). The approach uses the circular harmonic functions (CHFs) to extract local features from the most involved areas in the disease: hippocampus and posterior cingulate cortex (PCC) in each slice in all three brain projections. The features are quantized using the Bag-of-Visual-Words approach to build one signature by brain (subject). This yields a transformation of a full 3D image of brain ROIs into a 1D signature, a histogram of quantized features. To reduce the dimensionality of the signature, we use the PCA technique. Support vector machines classifiers are then applied to classify groups. The experiments were conducted on a subset of ADNI dataset and applied to the “Bordeaux-3City” dataset. The results showed that our approach achieves respectively for ADNI dataset and “Bordeaux-3City” dataset; for AD vs NC classification, an accuracy of 83.77% and 78%, a specificity of 88.2% and 80.4% and a sensitivity of 79.09% and 74.7%. For NC vs MCI classification we achieved for the ADNI datasets an accuracy of 69.45%, a specificity of 74.8% and a sensitivity of 62.52%. For the most challenging classification task (AD vs MCI), we reached an accuracy of 62.07%, a specificity of 75.15% and a sensitivity of 49.02%. The use of PCC visual features description improves classification results by more than 5% compared to the use of hippocampus features only. Our approach is automatic, less time-consuming and does not require the intervention of the clinician during the disease diagnosis.

Learning Deep Generative Models

Building intelligent systems that are capable of extracting high-level representations from high-dimensional sensory data lies at the core of solving many artificial intelligence–related tasks, including object recognition, speech perception, and language understanding. Theoretical and biological arguments strongly suggest that building such systems requires models with deep architectures that involve many layers of nonlinear processing. In this article, we review several popular deep learning models, including deep belief networks and deep Boltzmann machines. We show that (a) these deep generative models, which contain many layers of latent variables and millions of parameters, can be learned efficiently, and (b) the learned high-level feature representations can be successfully applied in many application domains, including visual object recognition, information retrieval, classification, and regression tasks.

IJIREEICE - Electrical, Electronics, Instrumentation and Control

Content Based Visual Information Retrieval is one of the most cardinal and crucial aspect in the category of digital image processing. The significant mechanism that has been lucidly highlighted here is the fast multiresolution image querying using Color Coherence Vector (CCV) along with the refinement of color histogram. However, the retrieval speed and accuracy can be enhanced using color characteristics and mathematical model despite the collection of innumerable images in dataset. Color feature extraction and coherency estimation assist in flawless and swiftest image querying according to the matched template (matrices) of the color histogram image with maximum precision.

Classification of Alzheimer’s disease subjects from MRI using hippocampal visual features

Indexing and classification tools for Content Based Visual Information Retrieval (CBVIR) have been penetrating the universe of medical image analysis. They have been recently investigated for Alzheimer's disease (AD) diagnosis. This is a normal knowledge diffusion process, when methodologies developed for multimedia mining penetrate a new application area. The latter brings its own specificities requiring an adjustment of methodologies on the basis of domain knowledge. In this paper, we develop an automatic classification framework for AD recognition in structural Magnetic Resonance Images (MRI). The main contribution of this work consists in considering visual features from the most involved region in AD (hippocampal area) and in using a late fusion to increase precision results. Our approach has been first evaluated on the baseline MR images of 218 subjects from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database and then tested on a 3T weighted contrast MRI obtained from a subsample of a large French epidemiological study: Bordeaux dataset. The experimental results show that our classification of patients with AD versus NC (Normal Control) subjects achieves the accuracies of 87 % and 85 % for ADNI subset and Bordeaux dataset respectively. For the most challenging group of subjects with the Mild Cognitive Impairment (MCI), we reach accuracies of 78.22 % and 72.23 % for MCI versus NC and MCI versus AD respectively on ADNI. The late fusion scheme improves classification results by 9 % in average for these three categories. Results demonstrate very promising classification performance and simplicity compared to the state-of-the-art volumetric AD diagnosis methods.

Visual representation of information as communicative practice

Anyone who has clarified a thought or prompted a response during a conversation by drawing a picture has exploited the potential of image making to convey information. Images are increasingly ubiquitous in daily communication due to advances in visually enabled information and communication technologies (ICT), such as information visualization applications, image retrieval systems, and virtual collaborative work tools. Although images are often used in social contexts, information science research concerned with the visual representation of information typically focuses on the image artifact and system building. To learn more about image making as a form of social interaction and as a form of information practice, a qualitative study examined face‐to‐face conversations involving the creation of ad hoc visualizations (i.e., “napkin drawings”). Interactional sociolinguistic concepts of conversational involvement and coordination guided multimodal analysis of video‐recorded interactions that included spontaneous drawing. Findings show patterns in communicative activities associated with the visual representation of information. Furthermore, the activity of mark making contributes to the maintenance of conversational involvement in ways that are not always evident in the drawn artifact. This research has implications for the design and evaluation of visually enabled virtual collaboration environments, visual information extraction and retrieval systems, and data visualization tools.

Computing Polynomial Segmentation through Radial Surface Representation

The Visual Information Retrieval (VIR) area requires robust implementations achieved trough mathematical representations for images or data sets. The implementation of a mathematical modeling goes from the corpus image selection, an appropriate descriptor method, a segmentation approach and the similarity metric implementation whose are treated as VIR elements. The goal of this research is to found an appropriate modeling to explain how its items can be represented to achieve a better performance in VIR implementations. A direct method is tested with a subspace arrangement approach. The General Principal Component Analysis (GPCA) is modified inside its segmentation process. Initially, a corpus data sample is tested, the descriptor of RGB colors is implemented to obtain a three dimensional description of image data. Then a selection of radial basis function is achieved to improve the similarity metric implemented. It is concluded that a better performance can be achieved applying powerful extraction methods in visual image retrieval (VIR) designs based in a mathematical formulation. The results lead to design VIR systems with high level of performance based in radial basis functions and polynomial segmentations for handling data sets.

VIRTUE: A visual tool for information retrieval performance evaluation and failure analysis

Objective: Information Retrieval (IR) is strongly rooted in experimentation where new and better ways to measure and interpret the behavior of a system are key to scientific advancement. This paper presents an innovative visualization environment: Visual Information Retrieval Tool for Upfront Evaluation (VIRTUE), which eases and makes more effective the experimental evaluation process.Methods: VIRTUE supports and improves performance analysis and failure analysis.Performance analysis: VIRTUE offers interactive visualizations based on well-known IR metrics allowing us to explore system performances and to easily grasp the main problems of the system.Failure analysis: VIRTUE develops visual features and interaction, allowing researchers and developers to easily spot critical regions of a ranking and grasp possible causes of a failure.Results: VIRTUE was validated through a user study involving IR experts. The study reports on (a) the scientific relevance and innovation and (b) the comprehensibility and efficacy of the visualizations.Conclusion: VIRTUE eases the interaction with experimental results, supports users in the evaluation process and reduces the user effort.Practice: VIRTUE will be used by IR analysts to analyze and understand experimental results.Implications: VIRTUE improves the state-of-the-art in the evaluation practice and integrates visualization and IR research fields in an innovative way.

A comprehensive review of significant researches on content based indexing and retrieval of visual information

Developments in multimedia technologies have paved way for the storage of huge collections of video documents on computer systems. It is essential to design tools for content-based access to the documents, so as to allow an efficient exploitation of these collections. Content based analysis provides a flexible and powerfulway to access video data when compared with the other traditional video analysis techniques. The area of content based video indexing and retrieval (CBVIR), focusing on automating the indexing, retrieval and management of video, has attracted extensive research in the last decade. CBVIR is a lively area of research with enduring acknowledgments from several domains. Herein a vital assessment of contemporary researches associated with the content-based indexing and retrieval of visual information. In this paper, we present an extensive review of significant researches on CBVIR. Concise description of content based video analysis along with the techniques associated with the content based video indexing and retrieval is presented.

Designing for specialized visual domains

Purpose – The purpose of this paper is to investigate the organization and design of a real-life digital collection for the field of Book Arts. This paper also examines the challenges in building digital libraries for visual domains with specialized user pools. Design/methodology/approach – A qualitative study was conducted. An online survey asked participants from the field of Book Arts to provide free-text descriptions of randomized visual representatives taken from the collection using open-ended question and answer boxes. Themes extracted from the survey were coded and presented using basic statistical measures. Findings – The results of this study identified a set of specialized access points for items that were selected as being significant for Book Arts and perhaps other areas in the Fine Arts. Results informed retrieval of visual information for this domain along with user interface design and evaluation strategies. In addition, the survey used in this study demonstrated the ability to collect longer, more in-depth answers and category terms alike from the participants. Originality/value – This study was necessary for the practical purpose of organizing a digital collection of the Book Arts, a unique collection. While similar studies have been conducted, even for collections deemed to be from the “Fine Arts”, previous studies do not agree on what is important for organizing digital collections including varying opinions of certain bibliographic access points and visual representation.

Research on Agricultural Products Quality Control Based on Computer Vision Information Technology

In 2012, the ministry of agriculture released data showed that the import and export trade deficit of China's agricultural products had reached 49190000000 US dollars. China is a large agricultural country, agricultural crop occupies world front row, but export transactions accounted for the total trading lower percentage, the main reason is the quality testing technology that does not pass, detection technology is relatively backward, in this background, this paper puts forward a kind of advanced agricultural products quality control technology - computer visual information retrieval control technology. On the basis of the two-order upwind and linear interpolation theory, the use of image segmentation technology can successfully establish a computer vision inspection of agricultural products to mathematical model, and the establishment of the interpolation functions. Finally this paper is based on peanut detection as an example, the 50 cases of peanut quality are detected by using computer vision nondestructive technology testing, and the use of MATLAB software carries out mathematical statistics for its quality situation, the test results and the artificial inspection results are compared to verify the reliability of computer vision quality detection.

Retrieval of high-dimensional visual data: current state, trends and challenges ahead

Information retrieval algorithms have changed the way we manage and use various data sources, such as images, music or multimedia collections. First, free text information of documents from varying sources became accessible in addition to structured data in databases, initially for exact search and then for more probabilistic models. Novel approaches enable content-based visual search of images using computerized image analysis making visual image content searchable without requiring high quality manual annotations. Other multimedia data followed such as video and music retrieval, sometimes based on techniques such as extracting objects and classifying genre. 3D (surface) objects and solid textures have also been produced in quickly increasing quantities, for example in medical tomographic imaging. For these two types of 3D information sources, systems have become available to characterize the objects or textures and search for similar visual content in large databases. With 3D moving sequences (i.e., 4D), in particular medical imaging, even higher-dimensional data have become available for analysis and retrieval and currently present many multimedia retrieval challenges. This article systematically reviews current techniques in various fields of 3D and 4D visual information retrieval and analyses the currently dominating application areas. The employed techniques are analysed and regrouped to highlight similarities and complementarities among them in order to guide the choice of optimal approaches for new 3D and 4D retrieval problems. Opportunities for future applications conclude the article. 3D or higher-dimensional visual information retrieval is expected to grow quickly in the coming years and in this respect this article can serve as a basis for designing new applications.

Fixation-dependent memory for natural scenes: An experimental test of scanpath theory.

Many modern theories propose that perceptual information is represented by the sensorimotor activity elicited by the original stimulus. Scanpath theory (Noton & Stark, 1971) predicts that reinstating a sequence of eye fixations will help an observer recognize a previously seen image. However, the only studies to investigate this are correlational ones based on calculating scanpath similarity. We therefore describe a series of 5 experiments that constrain the fixations during encoding or recognition of images in order to manipulate scanpath similarity. Participants encoded a set of images and later had to recognize those that they had seen. They spontaneously selected regions that they had fixated during encoding (Experiment 1), and this was a predictor of recognition accuracy. Yoking the parts of the image available at recognition to the encoded scanpath led to better memory performance than randomly selected image regions (Experiment 2), and this could not be explained by the spatial distribution of locations (Experiment 3). However, there was no recognition advantage for re-viewing one's own fixations versus someone else's (Experiment 4) or for retaining their serial order (Experiment 5). Therefore, although it is beneficial to look at encoded regions, there is no evidence that scanpaths are stored or that scanpath recapitulation is functional in scene memory. This paradigm provides a controlled way of studying the integration of scene content, spatial structure, and oculomotor signals, with consequences for the perception, representation, and retrieval of visual information.

Renal DMSA scan morphology analysis using undecimated wavelet transform and isocontours

Signal and image processing tools open up opportunities to develop modules that can be interesting for supporting clinical decisions. In this paper, the aim is to develop an algorithm for providing better insight into renal scintigraphy scans using sparse transform such as discrete wavelet transform and isocontours. Renal 99mTc–DMSA scintigraphy scans are widely accepted as the most sensitive method for evaluating renal lesions and parenchymal damage like renal scars. An approach for improving DMSA scan analysis and regions of interest is presented from the aspect of visual inspection and relevant morphology information retrieval. For the purpose of testing archived scans are considered.

Content-Based Management Service for Medical Videos

Development of health information technology has had a dramatic impact to improve the efficiency and quality of medical care. Developing interoperable health information systems for healthcare providers has the potential to improve the quality and equitability of patient-centered healthcare. In this article, we describe an automated content-based medical video analysis and management service that provides convenience and ease in accessing the relevant medical video content without sequential scanning. The system facilitates effective temporal video segmentation and content-based visual information retrieval that enable a more reliable understanding of medical video content. The system is implemented as a Web- and mobile-based service and has the potential to offer a knowledge-sharing platform for the purpose of efficient medical video content access.

3D CBIR with sparse coding for image-guided neurosurgery

This research takes an application-specific approach to investigate, extend and implement the state of the art in the fields of both visual information retrieval and machine learning, bridging the gap between theoretical models and real world applications. During an image-guided neurosurgery, path planning remains the foremost and hence the most important step to perform an operation and ensures the maximum resection of an intended target and minimum sacrifice of health tissues. In this investigation, the technique of content-based image retrieval (CBIR) coupled with machine learning algorithms are exploited in designing a computer aided path planning system (CAP) to assist junior doctors in planning surgical paths while sustaining the highest precision. Specifically, after evaluation of approaches of sparse coding and K-means in constructing a codebook, the model of sparse codes of 3D SIFT has been furthered and thereafter employed for retrieving, The novelty of this work lies in the fact that not only the existing algorithms for 2D images have been successfully extended into 3D space, leading to promising results, but also the application of CBIR that is mainly in a research realm, to a clinical sector can be achieved by the integration with machine learning techniques. Comparison with the other four popular existing methods is also conducted, which demonstrates that with the implementation of sparse coding, all methods give better retrieval results than without while constituting the codebook, implying the significant contribution of machine learning techniques.

Medida de similitud basada en saliencia

The ubiquitous growth of multimedia production is causing the creation of new visual information retrieval paradigms. One of the most relevant among them is that represented by Visual Information Retrieval Systems (VIRS), where a common task is ordering a set images according to their similarity to a given one. In this work a new proposal for evaluating similarity between two images is introduced; both images are represented by respective feature vectors, and the perceptual cue used to generate the similarity measure is saliency, a concept thoroughly known in Psychology. New methodologies for quantifying saliency of feature values, for combining them during a comparison process and, eventually, to weight that feature attending to the result of the combination, are introduced as well. The results for the evaluation of this similarity measure in an image based content retrieval task are presented, as well as their comparison with those obtained using euclidean distance in the same task. Both are validated by volunteers who labelled the retrieved sets.

All‐visual retrieval: How people search and respond to an affect‐driven visual information retrieval system

AbstractThe design of information retrieval (IR) systems must respond to the goals, intentionality and prior domain knowledge of the users. During focus groups conducted, end users complained that instead of looking for specific‐items, they might be interested in a spectrum of concepts – all things related to something or all things of a particular color. To respond to these needs, an entirely visuals‐driven information retrieval system project was developed using a test‐bed of copyright‐free images reflecting monographs, graphics, and work collections. In the absence of such IR systems, not much is known about how users will interact with a visuals‐only retrieval system. This poster describes the project in general and its usage to explore (a) how users interact with graphic‐only retrieval for exploring traditional and non‐traditional access points and (b) how the affective component impacts the use of such systems. Findings based on the study will help shed light on research based on visual information systems and user behavior when interacting with such systems. The findings will be useful both in designing systems that respond to user needs, and add to prior research in information seeking and retrieval.

Visual similarity issues in face recognition

The paper discusses several issues of visual similarity in face detection and recognition. Using a straightforward concept of keypoint correspondences, a method is proposed to formalise the subjective impressions of |similar faces|, |similar eyes|, |similar chins|, etc. The method exploits the mechanism of affine near-duplicate fragment detection originally proposed for visual information retrieval. It is shown that using such a method, a simple and relatively reliable face detection/identification systems can be build without any model (or training) of human faces, which can work with images containing multiple faces shown on random backgrounds. Additionally, it is proposed how the same approach can be used to optimise databases of face images and to identify individuals who are at higher risks of mistaken face identification.