Image Retrieval Applications Research Articles

Robust perceptual color image hashing using randomized hypercomplex matrix factorizations

Compact representations of color image and video content allow efficient search, retrieval and storage of this content over the Internet and online repositories. However, most of these representations neither take into account the inherent correlation nor the perceptual redundancy of the color information. In this paper, we propose a perceptual hash representation for color images using robust image features. These features, most dominant singular vectors extracted using the quaternion singular value decomposition (QSVD) of pseudorandomly selected overlapping image blocks, are efficiently used for color image search and retrieval applications. Their robustness is guaranteed by the underlying singular vectors. The motivation behind our work is twofold: 1) the ability of the QSVD algorithm to provide the best low-rank approximation of color images in the Frobenius norm sense and 2) compact representations to handle the color components as a single entity. The QSVD algorithm leads to proper modeling of possible geometric attacks as an independent and identically-distributed (i.i.d) quaternionic random noise on the singular vectors. Such modeling simplifies the hash code detector design. Hash code robustness against geometric attacks is evaluated over a large set of test color images where the proposed scheme outperforms existing factorization-based hashing algorithms in terms of lower miss and false alarm probabilities by orders of magnitude. Finally, the improved robustness and performance does not come at the expense of increased computational complexity which is another salient feature of the proposed hashing scheme.

A comparative study of hash based approximate nearest neighbor learning and its application in image retrieval

Plenty of data are available due to the growth of digital technology that creates a high expectation in retrieving the relevant images, accurately and efficiently for a given query image. For searching the relevant images efficiently for the Large Scale dataset, the searching algorithm should have fast access capability. The existing Exact Nearest Neighbor search performs in linear time and so it takes more time as both the dataset and data dimension increases. As a remedy to provide sub-linear/logarithmic time complexity, usage of Approximate Nearest Neighbor (ANN) algorithms is emerging at a rapid rate. This paper discusses about the importance of ANN and their general classification; the different categories involved in Learning to Hash has been analyzed with their pros and cons; different bit assignment types and methods to minimize the Quantization Errors have been reviewed along with its merits and demerits. Therefore, it serves to increase the efficiency of the Image Retrieval process in Large Scale.

Large-scale retrieval for medical image analytics: A comprehensive review.

Over the past decades, medical image analytics was greatly facilitated by the explosion of digital imaging techniques, where huge amounts of medical images were produced with ever-increasing quality and diversity. However, conventional methods for analyzing medical images have achieved limited success, as they are not capable to tackle the huge amount of image data. In this paper, we review state-of-the-art approaches for large-scale medical image analysis, which are mainly based on recent advances in computer vision, machine learning and information retrieval. Specifically, we first present the general pipeline of large-scale retrieval, summarize the challenges/opportunities of medical image analytics on a large-scale. Then, we provide a comprehensive review of algorithms and techniques relevant to major processes in the pipeline, including feature representation, feature indexing, searching, etc. On the basis of existing work, we introduce the evaluation protocols and multiple applications of large-scale medical image retrieval, with a variety of exploratory and diagnostic scenarios. Finally, we discuss future directions of large-scale retrieval, which can further improve the performance of medical image analysis.

Multi-View Nonparametric Discriminant Analysis for Image Retrieval and Recognition

A novel multi-view nonparametric discriminant analysis method is proposed for the application of cross-modal image retrieval and zero-shot recognition. We exploit the class boundary structure and discrepancy information of the available views in order to formulate an optimization criterion, which is automatically adjusted to the multi-view class structures. The proposed method allows for multiple projection directions, by relaxing the Gaussian distribution assumption of related methods. The experiments demonstrate that the proposed method can achieve superior results comparing to several existing methods.

SiNC: Saliency-injected neural codes for representation and efficient retrieval of medical radiographs.

Medical image collections contain a wealth of information which can assist radiologists and medical experts in diagnosis and disease detection for making well-informed decisions. However, this objective can only be realized if efficient access is provided to semantically relevant cases from the ever-growing medical image repositories. In this paper, we present an efficient method for representing medical images by incorporating visual saliency and deep features obtained from a fine-tuned convolutional neural network (CNN) pre-trained on natural images. Saliency detector is employed to automatically identify regions of interest like tumors, fractures, and calcified spots in images prior to feature extraction. Neuronal activation features termed as neural codes from different CNN layers are comprehensively studied to identify most appropriate features for representing radiographs. This study revealed that neural codes from the last fully connected layer of the fine-tuned CNN are found to be the most suitable for representing medical images. The neural codes extracted from the entire image and salient part of the image are fused to obtain the saliency-injected neural codes (SiNC) descriptor which is used for indexing and retrieval. Finally, locality sensitive hashing techniques are applied on the SiNC descriptor to acquire short binary codes for allowing efficient retrieval in large scale image collections. Comprehensive experimental evaluations on the radiology images dataset reveal that the proposed framework achieves high retrieval accuracy and efficiency for scalable image retrieval applications and compares favorably with existing approaches.

Fast uniform content-based satellite image registration using the scale-invariant feature transform descriptor

Content-based satellite image registration is a difficult issue in the fields of remote sensing and image processing. The difficulty is more significant in the case of matching multisource remote sensing images which suffer from illumination, rotation, and source differences. The scale-invariant feature transform (SIFT) algorithm has been used successfully in satellite image registration problems. Also, many researchers have applied a local SIFT descriptor to improve the image retrieval process. Despite its robustness, this algorithm has some difficulties with the quality and quantity of the extracted local feature points in multisource remote sensing. Furthermore, high dimensionality of the local features extracted by SIFT results in time-consuming computational processes alongside high storage requirements for saving the relevant information, which are important factors in content-based image retrieval (CBIR) applications. In this paper, a novel method is introduced to transform the local SIFT features to global features for multisource remote sensing. The quality and quantity of SIFT local features have been enhanced by applying contrast equalization on images in a pre-processing stage. Considering the local features of each image in the reference database as a separate class, linear discriminant analysis (LDA) is used to transform the local features to global features while reducing dimensionality of the feature space. This will also significantly reduce the computational time and storage required. Applying the trained kernel on verification data and mapping them showed a successful retrieval rate of 91.67% for test feature points.

Emotion-Based Classification and Indexing for Wallpaper and Textile

This study, based on human emotions and visual impression, develops a novel framework of classification and indexing for wallpaper and textiles. This method allows users to obtain a number of similar images that can be corresponded to a specific emotion by indexing through a reference image or an emotional keyword. In addition, a predefined color–emotion model is applied to deal with the transference between emotions and colors in the paper. Besides color and emotion, the other significant feature for indexing is texture. Therefore, two features—the main colors (the representative colors) and the foreground complexity of a color image—are adopted in the method. The foreground complexity (a pattern complexity) is also called the texture of the pattern in an image. Another contribution of this study is the new algorithms of Touch Four Sides (TFS) and Touch Up Sides (TUS), which can aid in extracting an accurate background and foreground for color images. The potential applications of this study can support non-professionals in finding suitable color-combinations based on emotions for many applications with the transference between emotions and colors, and to imitate the professional operation of the color matching such as interior design, product design, advertising design, image retrieval and other relative applications.

Content‐based image retrieval based on binary signatures cluster graph

AbstractIn this paper, we approach a method of clustering binary signature of image in order to create a clustering graph structure for building the content‐based image retrieval. First, the paper presents the segmentation method based on low‐level visual features including colour and texture of image. On the basis of segmented image, the paper creates binary signature to describe location, colour, and shape of interest objects. In order to match similar images, the paper presents a similarity measure between the images based on binary signature. From that, the paper proposes the method of clustering binary signature to quickly query similar images. At the same time, the graph data structure is built using the partition cluster technique and the rules of binary signatures' distribution of images. On the basis of data structure, we propose a graph creation algorithm, a cluster splitting/merging algorithm, and a similarity image retrieval algorithm. To illustrate the proposed theory, we build an image retrieval application and assess the experimental results on the image datasets including COREL (1,000 images), CBIR images (1,344 images), WANG (10,800 images), MSRDI (15,720 images), and ImageCLEF (20,000 images).

Salient Object Subitizing

We study the problem of Salient Object Subitizing, i.e. predicting the existence and the number of salient objects in an image using holistic cues. This task is inspired by the ability of people to quickly and accurately identify the number of items within the subitizing range (1-4). To this end, we present a salient object subitizing image dataset of about 14K everyday images which are annotated using an online crowdsourcing marketplace. We show that using an end-to-end trained Convolutional Neural Network (CNN) model, we achieve prediction accuracy comparable to human performance in identifying images with zero or one salient object. For images with multiple salient objects, our model also provides significantly better than chance performance without requiring any localization process. Moreover, we propose a method to improve the training of the CNN subitizing model by leveraging synthetic images. In experiments, we demonstrate the accuracy and generalizability of our CNN subitizing model and its applications in salient object detection and image retrieval.

Codebook Guided Feature-Preserving for Recognition-Oriented Image Retargeting.

Traditional image resizing methods, such as uniform scaling and content-aware image retargeting, are designed to preserve the visually salient contents of an image while resizing it. In this paper, we propose a novel image resizing approach called recognition-oriented image retargeting. Its goal is to preserve the distinctive local features for recognition instead of the traditional visual saliency during resizing. Moreover, we also apply our approach to image matching and image retrieval applications to verify its performance. Meanwhile, using our approach to these applications is able to solve some of the challenging problems in their fields. In image matching application, we find that our approach shows promising preservation of local feature descriptors. In image retrieval task, extensive experiments on Oxford5K, Holidays, Paris, and Flickr100k data sets demonstrate that our approach consistently outperforms other image retargeting methods by large margins in the aspects of retrieval precision and query bits.

Fitting‐based optimisation for image visual salient object detection

To overcome some major problems with traditional saliency evaluation metrics, full‐reference image quality assessment (IQA) metrics, which have similar but stricter objectives, are used. Inspired by the root mean absolute error, the authors propose a fitting‐based optimisation method for salient object detection algorithms. Their algorithm analyses the quantitative relationship between saliency and ground truth values, and uses the derived relationship to fit the saliency values to the original saliency maps. This ensures that the resulting images, which are composed of fitted values, are closer to the ground truth. The proposed algorithm first computes the statistics of the ground truth and saliency maps computed by each salient object detection algorithm. These statistics are used to compute the parameters of four fitting models, which generally agree with the characteristics of the statistical data. For a new saliency map, they use the fitting model with the computed parameters to obtain the fitted saliency values, which are confined to the range [0, 255]. Finally, they evaluate their saliency optimisation algorithm using traditional evaluation metrics, IQA metrics, and a content‐based image retrieval application. The results show that the proposed approach improves the quality of the optimised saliency maps.

Measuring and Predicting Tag Importance for Image Retrieval.

Textual data such as tags, sentence descriptions are combined with visual cues to reduce the semantic gap for image retrieval applications in today's Multimodal Image Retrieval (MIR) systems. However, all tags are treated as equally important in these systems, which may result in misalignment between visual and textual modalities during MIR training. This will further lead to degenerated retrieval performance at query time. To address this issue, we investigate the problem of tag importance prediction, where the goal is to automatically predict the tag importance and use it in image retrieval. To achieve this, we first propose a method to measure the relative importance of object and scene tags from image sentence descriptions. Using this as the ground truth, we present a tag importance prediction model to jointly exploit visual, semantic and context cues. The Structural Support Vector Machine (SSVM) formulation is adopted to ensure efficient training of the prediction model. Then, the Canonical Correlation Analysis (CCA) is employed to learn the relation between the image visual feature and tag importance to obtain robust retrieval performance. Experimental results on three real-world datasets show a significant performance improvement of the proposed MIR with Tag Importance Prediction (MIR/TIP) system over other MIR systems.

Deep Hashing Based Fusing Index Method for Large-Scale Image Retrieval

Hashing has been widely deployed to perform the Approximate Nearest Neighbor (ANN) search for the large-scale image retrieval to solve the problem of storage and retrieval efficiency. Recently, deep hashing methods have been proposed to perform the simultaneous feature learning and the hash code learning with deep neural networks. Even though deep hashing has shown the better performance than traditional hashing methods with handcrafted features, the learned compact hash code from one deep hashing network may not provide the full representation of an image. In this paper, we propose a novel hashing indexing method, called the Deep Hashing based Fusing Index (DHFI), to generate a more compact hash code which has stronger expression ability and distinction capability. In our method, we train two different architecture’s deep hashing subnetworks and fuse the hash codes generated by the two subnetworks together to unify images. Experiments on two real datasets show that our method can outperform state-of-the-art image retrieval applications.

Heterogeneous Manifold Ranking for Image Retrieval

Graph-based ranking models, such as manifold ranking (MR), have been widely used in various image retrieval applications. To further improve such models, a current trend is to fuse the ranking results from multiple feature sets. Most of existing methods mainly concentrate on fusing the homogeneous feature sets derived from a single information channel, like the multiple modalities of image visual content, but little is known in fusing such heterogeneous feature sets derived from multiple information channels as the click-through data associated with images and their visual content. The primary challenge is how to effectively exploit the complementary properties of the heterogeneous feature sets. Another tough issue is the low-quality nature of the click-through data, which makes the exploration of such complementary properties more difficult. In this paper, we propose a heterogeneous MR (HMR) model, in which a couple of graphs built on the click and visual feature sets are fused to simultaneously encode the image ranking results. Specifically, our HMR model applies different solutions to fuse the heterogeneous feature sets in terms of whether the relevance feedback mechanism is available or not. In addition, we develop a click refinement technique to address the noiseness and sparseness problems inherent in the click-through data. Concretely, it prunes the inaccurate clicks from the click-through data using a neighbor voting strategy, and then enriches the pruned data with novel yet accurate clicks based on a novel collaborative filtering (CF) approach, which is devised by integrating the merits of three popularly used CF methods, thus called Tri-CF algorithm. Extensive experiments on the tasks of click refinement and image retrieval demonstrate the superior performance of the proposed algorithms over several representative methods, especially when the click-through data is highly noisy and sparse.

Noise reduction for pattern-matching algorithms

Many of the data analysis algorithms that base their analysis on pattern occurrences tend to use objective assessment measures at one point or another. In many cases, especially in multimedia research, these objective measures were originally developed for the purpose of mimicking subjective assessments to automate the assessment pipeline. Using such measures is understandable when the user is a human subject but becomes arguable when there is an intermediate user in the form of an analysis algorithm. We present here a multidimensional noise reduction scheme that cleans the data from the perspective of the algorithmic system. The proposed scheme is then applied to applications of image coding and content-based image retrieval. Although the noise reduction adversely affects the objective scales, we show that it actually enhances the performance of the analysis algorithm. For instance, the percentage retrieval precision of tiger images was 3.5-fold better than the non-enhanced system. This precision enhancement is accompanied by a 55% reduction in retrieval time on average and further reduction in space costs.

Manifold Preserving: An Intrinsic Approach for Semisupervised Distance Metric Learning.

In this paper, we address the semisupervised distance metric learning problem and its applications in classification and image retrieval. First, we formulate a semisupervised distance metric learning model by considering the metric information of inner classes and interclasses. In this model, an adaptive parameter is designed to balance the inner metrics and intermetrics by using data structure. Second, we convert the model to a minimization problem whose variable is symmetric positive-definite matrix. Third, in implementation, we deduce an intrinsic steepest descent method, which assures that the metric matrix is strictly symmetric positive-definite at each iteration, with the manifold structure of the symmetric positive-definite matrix manifold. Finally, we test the proposed algorithm on conventional data sets, and compare it with other four representative methods. The numerical results validate that the proposed method significantly improves the classification with the same computational efficiency.

Integration of global and local features based on Hybrid Similarity Matching Scheme for Medical Image Retrieval System

Similarity measure is a challenging task in Content-Based Medical Image Retrieval (CBMIR) systems and the matching scheme is designed to improve the retrieval performance. However, there are several major shortcomings with conventional approaches for a matching scheme which can extensively affect their application of Medical Image Retrieval (MIR). To overcome the issues, in this paper a Multi-Level Matching (MLM) method for MIR using hybrid feature similarity is proposed. Here, images are represented by multi-level features including local level and global level. The Colour and Edge Directivity Descriptor (CEDD) is used as a colour and edge-based descriptor. Speeded-Up Robust Features (SURF) and Local Binary Pattern (LBP) are used as a local descriptor. The hybrid of both global and local features yields enhanced retrieval accuracy, which is analysed over collected image databases. From the experiment, the proposed method achieves better accuracy value about 92%, which is higher than other methods.

Noise reduction for pattern-matching algorithms

Many of the data analysis algorithms that base their analysis on pattern occurrences tend to use objective assessment measures at one point or another. In many cases, especially in multimedia research, these objective measures were originally developed for the purpose of mimicking subjective assessments to automate the assessment pipeline. Using such measures is understandable when the user is a human subject but becomes arguable when there is an intermediate user in the form of an analysis algorithm. We present here a multidimensional noise reduction scheme that cleans the data from the perspective of the algorithmic system. The proposed scheme is then applied to applications of image coding and content-based image retrieval. Although the noise reduction adversely affects the objective scales, we show that it actually enhances the performance of the analysis algorithm. For instance, the percentage retrieval precision of tiger images was 3.5-fold better than the non-enhanced system. This precision enhancement is accompanied by a 55% reduction in retrieval time on average and further reduction in space costs.

The Colour Similarity in the Application and Research of Image Retrieval of Tianmen blue calico

Aiming at the digital information gap of Tianmen blue calico, which is the intangible cultural heritage of Hubei Province, this paper analyzes the technical feasibility of the content-based digital image retrieval system of blue calico, drawing the conclusion that efficient retrieval of images can be achieved by calculating the vector similarity of colour features. Four types of similar distances of blue calico digital images are calculated using the OpenCV Library. Analyzing and selecting the standards of calculation show a good foundation for the Tianmen blue calico digital image retrieval system based on content retrieval.

Quantization Selection of Colour Histogram Bins to Categorize the Colour Appearance of Landscape Paintings for Image Retrieval

In the world of today, most images are digitized and kept in digital libraries for better organization and management. With the growth of information and communication technology, collection holders such as museums or cultural institutions have been increasingly interested in making their collections available anytime and anywhere for any Image Retrieval (IR) activities such as browsing and searching. In a colour image retrieval application, images retrieved by users are accomplished according to their specifications on what they want or acquire, which could be based upon so many concepts. We suggest an approach to categorize the colour appearances of whole scene landscape painting images based on human colour perception. The colour features in the image are represented using a colour histogram. We then find the suitable quantization bins that can be used to generate optimum colour histograms for all categories of colour appearances, which is selected based on theHarmonic Mean of the precision and recall, also known as F-Score percentage higher saturated value. Colour appearance attributes in the CIELab colour model (L-Lightness, a and b are colour-opponent dimension) are used to generate colour appearance feature vectors namely the saturation metric, lightness metric and multicoloured metric. For the categorizations, we use the Nearest Neighbour (NN) method to detect the classes by using the predefined colour appearance descriptor measures and the pre-set thresholds. The experimental results show that the quantization of CIELab colour model into 11 uniformly bins for each component had achieved the optimum result for all colour appearances categories.