Visual Information Retrieval Systems Research Articles

Exploring the context of visual information seeking

PurposeInformation seeking is an interactive behaviour of the end users with information systems, which occurs in a real environment known as context. Context affects information-seeking behaviour in many different ways. The purpose of this paper is to investigate the factors that potentially constitute the context of visual information seeking.Design/methodology/approachThe authors used a Straussian version of grounded theory, a qualitative approach, to conduct the study. Using a purposive sampling method, 28 subjects participated in the study. The data were analysed using open, axial and selective coding in MAXQDA software.FindingsThe contextual factors influencing visual information seeking were classified into seven categories, including: “user characteristics”, “general search features”, “visual search features”, “display of results”, “accessibility of results”, “task type” and “environmental factors”.Practical/implicationsThis study contributes to a better understanding of how people conduct searches in and interact with visual search interfaces. Results have important implications for the designers of information retrieval systems.Originality/valueThis paper is among the pioneer studies investigating contextual factors influencing information seeking in visual information retrieval systems.

VisIRR

In this article, we present an interactive visual information retrieval and recommendation system, called VisIRR, for large-scale document discovery. VisIRR effectively combines the paradigms of (1) a passive pull through query processes for retrieval and (2) an active push that recommends items of potential interest to users based on their preferences. Equipped with an efficient dynamic query interface against a large-scale corpus, VisIRR organizes the retrieved documents into high-level topics and visualizes them in a 2D space, representing the relationships among the topics along with their keyword summary. In addition, based on interactive personalized preference feedback with regard to documents, VisIRR provides document recommendations from the entire corpus, which are beyond the retrieved sets. Such recommended documents are visualized in the same space as the retrieved documents, so that users can seamlessly analyze both existing and newly recommended ones. This article presents novel computational methods, which make these integrated representations and fast interactions possible for a large-scale document corpus. We illustrate how the system works by providing detailed usage scenarios. Additionally, we present preliminary user study results for evaluating the effectiveness of the system.

Automatic and online setting of similarity thresholds in content-based visual information retrieval problems

Several information recovery systems use functions to determine similarity among objects in a collection. Such functions require a similarity threshold, from which it becomes possible to decide on the similarity between two given objects. Thus, depending on its value, the results returned by systems in a search may be satisfactory or not. However, the definition of similarity thresholds is difficult because it depends on several factors. Typically, specialists fix a threshold value for a given system, which is used in all searches. However, an expert-defined value is quite costly and not always possible. Therefore, this study proposes an approach for automatic and online estimation of the similarity threshold value, to be specifically used by content-based visual information retrieval system (image and video) search engines. The experimental results obtained with the proposed approach prove rather promising. For example, for one of the case studies, the performance of the proposed approach achieved 99.5 % efficiency in comparison with that obtained by a specialist using an empirical similarity threshold. Moreover, such automated approach becomes more scalable and less costly.

Weighted indexing of TV sports news videos

The paper proposes measures for weighted indexing of sports news videos. The content-based analyses of sports news videos lead to the classification of frames or shots into sports categories. A set of sports categories reported in a given news video can be used as a video representation in visual information retrieval system. However, such an approach does not take into account how many sports events of a given category have been reported and how long these events have been presented in news for televiewers. Weighting of sports categories in a video representation reflecting their importance in a given video or in a whole video data base would be desirable. The effects of applying the proposed measures have been demonstrated in a test video collection. The experiments and evaluations performed on this collection have also shown that we do not need to apply perfect content-based analyses to ensure proper weighted indexing of sports news videos. It is sufficient to recognize the content of only some frames and to determine the number of shots, scenes or pseudo-scenes detected in temporal aggregation process, or even only the number of events of a given sports category in a sports news video being indexed.

Browse to search, visualize to explore: Who needs an alternative information retrieving model?

This article presents the results of the evaluation of SCIViS – a visualization-based scientific information retrieval (IR) system. SCIViS is based on the logic of concept maps and enables the visualization of relationships among descriptors and authors of scientific papers. It creates an interactive interface between the user’s cognitive space and document information space. The system was evaluated by the group of 138 psychology students which have performed a variety of search tasks, using both the classic text-based and the visualization based IR systems. The SCIViS model has proved to be effective and intuitive. Participants were more efficient using the visual IR system, particularly when performing tasks requiring modifications of the initial query and finding alternative keywords. User responses indicated that system’s speed and ease of use are the most important attributes of the overall assessment. They also revealed the impact of users’ previous experience with IR systems on users’ satisfaction and perception of usefulness. Previous experience may be regarded both as the rate of success in performing search tasks, as well as the familiarity with popular search engines. The later one seems to be an important factor in modeling users’ information seeking behavior and their attitudes toward alternative IR models.

Visual representation of information as communicative practice

Anyone who has clarified a thought or prompted a response during a conversation by drawing a picture has exploited the potential of image making to convey information. Images are increasingly ubiquitous in daily communication due to advances in visually enabled information and communication technologies (ICT), such as information visualization applications, image retrieval systems, and virtual collaborative work tools. Although images are often used in social contexts, information science research concerned with the visual representation of information typically focuses on the image artifact and system building. To learn more about image making as a form of social interaction and as a form of information practice, a qualitative study examined face‐to‐face conversations involving the creation of ad hoc visualizations (i.e., “napkin drawings”). Interactional sociolinguistic concepts of conversational involvement and coordination guided multimodal analysis of video‐recorded interactions that included spontaneous drawing. Findings show patterns in communicative activities associated with the visual representation of information. Furthermore, the activity of mark making contributes to the maintenance of conversational involvement in ways that are not always evident in the drawn artifact. This research has implications for the design and evaluation of visually enabled virtual collaboration environments, visual information extraction and retrieval systems, and data visualization tools.

Computing Polynomial Segmentation through Radial Surface Representation

The Visual Information Retrieval (VIR) area requires robust implementations achieved trough mathematical representations for images or data sets. The implementation of a mathematical modeling goes from the corpus image selection, an appropriate descriptor method, a segmentation approach and the similarity metric implementation whose are treated as VIR elements. The goal of this research is to found an appropriate modeling to explain how its items can be represented to achieve a better performance in VIR implementations. A direct method is tested with a subspace arrangement approach. The General Principal Component Analysis (GPCA) is modified inside its segmentation process. Initially, a corpus data sample is tested, the descriptor of RGB colors is implemented to obtain a three dimensional description of image data. Then a selection of radial basis function is achieved to improve the similarity metric implemented. It is concluded that a better performance can be achieved applying powerful extraction methods in visual image retrieval (VIR) designs based in a mathematical formulation. The results lead to design VIR systems with high level of performance based in radial basis functions and polynomial segmentations for handling data sets.

Medida de similitud basada en saliencia

The ubiquitous growth of multimedia production is causing the creation of new visual information retrieval paradigms. One of the most relevant among them is that represented by Visual Information Retrieval Systems (VIRS), where a common task is ordering a set images according to their similarity to a given one. In this work a new proposal for evaluating similarity between two images is introduced; both images are represented by respective feature vectors, and the perceptual cue used to generate the similarity measure is saliency, a concept thoroughly known in Psychology. New methodologies for quantifying saliency of feature values, for combining them during a comparison process and, eventually, to weight that feature attending to the result of the combination, are introduced as well. The results for the evaluation of this similarity measure in an image based content retrieval task are presented, as well as their comparison with those obtained using euclidean distance in the same task. Both are validated by volunteers who labelled the retrieved sets.

All‐visual retrieval: How people search and respond to an affect‐driven visual information retrieval system

AbstractThe design of information retrieval (IR) systems must respond to the goals, intentionality and prior domain knowledge of the users. During focus groups conducted, end users complained that instead of looking for specific‐items, they might be interested in a spectrum of concepts – all things related to something or all things of a particular color. To respond to these needs, an entirely visuals‐driven information retrieval system project was developed using a test‐bed of copyright‐free images reflecting monographs, graphics, and work collections. In the absence of such IR systems, not much is known about how users will interact with a visuals‐only retrieval system. This poster describes the project in general and its usage to explore (a) how users interact with graphic‐only retrieval for exploring traditional and non‐traditional access points and (b) how the affective component impacts the use of such systems. Findings based on the study will help shed light on research based on visual information systems and user behavior when interacting with such systems. The findings will be useful both in designing systems that respond to user needs, and add to prior research in information seeking and retrieval.

Evaluation of content-based image descriptors by statistical methods

Evaluation of visual information retrieval systems is usually performed by executing test queries and computing recall- and precision-like measures based on predefined media collections and ground truth information. This process is complex and time consuming. For the evaluation of feature transformations (transformation of visual media objects to feature vectors) it would be desirable to have simpler methods available as well. In this paper we introduce a supplementary evaluation procedure for features that is founded on statistical data analysis. A second novelty is that we make use of the existing visual MPEG-7 descriptors to judge the characteristics of feature transformations. The proposed procedure is divided into four steps: (1) feature extraction, (2) merging with MPEG-7 data and normalisation, (3) statistical data analysis and (4) visualisation and interpretation. Three types of statistical methods are used for evaluation: (1) univariate description (moments, etc.), (2) identification of similarities between feature elements (e.g. cluster analysis) and (3) identification of dependencies between variables (e.g. factor analysis). Statistical analysis provides beneficial insights into the structure of features that can be exploited for feature redesign. Application and advantages of the proposed approach are shown in a number of toy examples.

Image retrieval: Benchmarking visual information indexing and retrieval systems

Image retrieval: Benchmarking visual information indexing and retrieval systems

Utilizing shape retrieval in sketch synthesis

This article describes SR-Sketch, a sketch creation tool that can act both as a front-end visual query module to visual information retrieval systems and as an aid tool for fast image composition. The system allows the user to draw shapes on the computer screen using the mouse cursor. At any time during this process the user can query a shape database for similar shapes and then select the ones s/he thinks are more relevant. The system can then automatically align and replace user-drawn objects with the chosen database shapes in the user sketch. For any match between a sketch shape and a database shape, the application can provide a visual explanation of how and why two shapes are considered similar. Evaluation results show that the system achieves favorable results with respect to noise tolerance, speed and reliability. SR-Sketch is freely available for download and experimentation.

Distributed content-based visual information retrieval system on peer-to-peer networks

With the recent advances of distributed computing, the limitation of information retrieval from a centralized image collection can be removed by allowing distributed image data sources to interact with each other for data storage sharing and information retrieval. In this article, we present our design and implementation of DISCOVIR: DIStributed COntent-based Visual Information Retrieval system using the Peer-to-Peer (P2P) Network. We describe the system architecture and detail the interactions among various system modules. Specifically, we propose a Firework Query Model for distributed information retrieval, which aims to reduce the network traffic of query passing in the network. We carry out experiments to show the distributed image retrieval system and the Firework information retrieval algorithm. The results show that the algorithm reduces network traffic while increases searching performance.

A Survey of Rotation Invariant Texture Classification Methods

In this survey, the advances in rotation invariant texture classification techniques are discussed. The overall evolution of this niche from nascent to reasonably mature field is studied. It is very difficult to include all the works in such a paper; an attempt is made to include representative works of major methods. Markov Random Field based schemes are computationally involved. Methods based on wavelet and multichannel filtering are used for rotation invariant texture classification. Visual information retrieval system requires a robust and computationally less involved rotation invariant texture features. This area would be pursued vigorously in near future.

Visual information retrieval system via content-based approach

In this paper a content-based image retrieval method that can search large image databases efficiently by color, texture, and shape content is proposed. Quantized RGB histograms and the dominant triple (hue, saturation, and value), which are extracted from quantized HSV joint histogram in the local image region, are used for representing global/local color information in the image. Entropy and maximum entry from co-occurrence matrices are used for texture information and edge angle histogram is used for representing shape information. Relevance feedback approach, which has coupled proposed features, is used for obtaining better retrieval accuracy. A new indexing method that supports fast retrieval in large image databases is also presented. Tree structures constructed by k-means algorithm, along with the idea of triangle inequality, eliminate candidate images for similarity calculation between query image and each database image. We find that the proposed method reduces calculation up to average 92.2 percent of the images from direct comparison.

Testing visual information retrieval methodologies case study: Comparative analysis of textual, icon, graphical, and “spring” displays

AbstractAlthough many different visual information retrieval systems have been proposed, few have been tested, and where testing has been performed, results were often inconclusive. Further, there is very little evidence of benchmarking systems against a common standard. An approach for testing novel interfaces is proposed that uses bottom‐up, stepwise testing to allow evaluation of a visualization, itself, rather than restricting evaluation to the system instantiating it. This approach not only makes it easier to control variables, but the tests are also easier to perform. The methodology will be presented through a case study, where a new visualization technique is compared to more traditional ways of presenting data.

Facial Image Indexing in Multimedia Databases

Pictures and video sequences showing human faces are of high importance in content-based retrieval systems, and consequently face detection has been established as an important tool in the framework of many multimedia applications like indexing, scene classification and news summarisation. In this work, we combine skin colour and shape features with template matching in an efficient way for the purpose of facial image indexing. We propose an adaptive two-dimensional Gaussian model of the skin colour distribution whose parameters are re-estimated based on the current image or frame, reducing generalisation problems. Masked areas obtained from skin colour detection are processed using morphological tools and assessed using global shape features. The verification stage is based on a template matching variation providing robust detection. Facial images and video sequences are indexed according to the number of included faces, their average colour components and their scale, leading to new types of content-based retrieval criteria in query-by-example frameworks. Experimental results have shown that the proposed implementation combines efficiency, robustness and speed, and could be easily embedded in generic visual information retrieval systems or video databases.

Visualization in information retrieval: a three-level analysis

Visualization in information retrieval (IR) is a rapidly emerging area in which human-computer interaction (HCI) is adapted to the design of interfaces for IR systems. There are numerous proposals for the development of new visual IR systems. In parallel with the advent of advanced software and hardware, the attempts to design visualization-based IR systems are stimulated with the promise that information visualization applies visual processing to abstract information. Under this rapid development in the research area, there are few attempts to conceptualize this new field of study. This paper intends to classify the works of visualization for IR system design in a systematic way, by providing a mosaic view of the emerging area of visualization in IR system design. The paper proposes a three-level approach to analyzing visualization in IR system design: (i) component level analysis, (ii) technique level analysis and (iii) interaction level analysis. This three-level approach helps us to understand the state of visualization for the design of IR systems in multiple ways. The three-level approach also enables us in further serious investigation in this emerging field that has been created by the intersection of visualization and IR, by allowing us to look at different layers of visualization in IR that have both technical and theoretical concerns.

Semantics in visual information retrieval

A compositional approach increases the level of representation that can be automatically extracted and used in a visual information retrieval system. Visual information at the perceptual level is aggregated according to a set of rules. These rules reflect the specific context and transform perceptual words into phrases capturing pictorial content at a higher, and closer to the human, semantic level.

Conference Notes - 1996: Foundations of Advanced Information Visualization for Visual Information (Retrieval) Systems

Journal of the American Society for Information ScienceVolume 50, Issue 9 p. 835-837 Miscellaneous Conference Notes—1996: Foundations of Advanced Information Visualization for Visual Information (Retrieval) Systems Mark Rorvig, rorvig@lis.unt.edu School of Library and Information Sciences, University of North Texas, P.O. Box 13796, Denton, Texas 76203-6796Search for more papers by this authorMatthias Hemmje, hemmje@darmstadt.gmd.de German National Research Center for Information Technology (GMD), Integrated Publication and Information Systems Institute, Dolivostr. 15, D-64293 Darmstadt, GermanySearch for more papers by this author Mark Rorvig, rorvig@lis.unt.edu School of Library and Information Sciences, University of North Texas, P.O. Box 13796, Denton, Texas 76203-6796Search for more papers by this authorMatthias Hemmje, hemmje@darmstadt.gmd.de German National Research Center for Information Technology (GMD), Integrated Publication and Information Systems Institute, Dolivostr. 15, D-64293 Darmstadt, GermanySearch for more papers by this author First published: 22 June 1999 https://doi.org/10.1002/(SICI)1097-4571(1999)50:9<835::AID-ASI12>3.0.CO;2-CCitations: 1AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onEmailFacebookTwitterLinked InRedditWechat No abstract is available for this article.Citing Literature Volume50, Issue91999Pages 835-837 RelatedInformation