Problem Of Image Retrieval Research Articles

Intramodal consistency in triplet-based cross-modal learning for image retrieval

Cross-modal retrieval requires building a common latent space that captures and correlates information from different data modalities, usually images and texts. Cross-modal training based on the triplet loss with hard negative mining is a state-of-the-art technique to address this problem. This paper shows that such approach is not always effective in handling intra-modal similarities. Specifically, we found that this method can lead to inconsistent similarity orderings in the latent space, where intra-modal pairs with unknown ground-truth similarity are ranked higher than cross-modal pairs representing the same concept. To address this problem, we propose two novel loss functions that leverage intra-modal similarity constraints available in a training triplet but not used by the original formulation. Additionally, this paper explores the application of this framework to unsupervised image retrieval problems, where cross-modal training can provide the supervisory signals that are otherwise missing in the absence of category labels. Up to our knowledge, we are the first to evaluate cross-modal training for intra-modal retrieval without labels. We present comprehensive experiments on MS-COCO and Flickr30k, demonstrating the advantages and limitations of the proposed methods in cross-modal and intra-modal retrieval tasks in terms of performance and novelty measures. We also conduct a case study on the ROCO dataset to assess the performance of our method on medical images and present an ablation study on one of our approaches to understanding the impact of the different components of the proposed loss function. Our code is publicly available on GitHub https://github.com/MariodotR/FullHN.git.

Performance analysis of image retrieval system using deep learning techniques

ABSTRACT The image retrieval is the process of retrieving the relevant images to the query image with minimal searching time in internet. The problem of the conventional Content-Based Image Retrieval (CBIR) system is that they produce retrieval results for either colour images or grey scale images alone. Moreover, the CBIR system is more complex which consumes more time period for producing the significant retrieval results. These problems are overcome through the proposed methodologies stated in this work. In this paper, the General Image (GI) and Medical Image (MI) are retrieved using deep learning architecture. The proposed system is designed with feature computation module, Retrieval Convolutional Neural Network (RETCNN) module, and Distance computation algorithm. The distance computation algorithm is used to compute the distances between the query image and the images in the datasets and produces the retrieval results. The average precision and recall for the proposed RETCNN-based CBIRS is 98.98% and 99.15% respectively for GI category, and the average precision and recall for the proposed RETCNN-based CBIRS are 99.04% and 98.89% respectively for MI category. The significance of these experimental results is used to produce the higher image retrieval rate of the proposed system.

Location retrieval using qualitative place signatures of visible landmarks

Location retrieval based on visual information is to retrieve the location of an agent (e.g. human, robot) or the area they see by comparing their observations with a certain representation of the environment. Existing methods generally treat the problem as a content-based image retrieval problem and have demonstrated promising results in terms of localization accuracy. However, these methods are challenging to scale up due to the volume of reference data involved; and the image descriptions might not be easily understandable/communicable for humans to describe surroundings. Considering that humans often use less precise but easily produced qualitative spatial language and high-level semantic landmarks when describing an environment, a coarse-to-fine qualitative location retrieval method is proposed in this work to quickly narrow down the initial location of an agent by exploiting the available information in large-scale open data. This approach describes and indexes a location/place using the perceived qualitative spatial relations between ordered pairs of co-visible landmarks from the perspective of viewers, termed as ‘qualitative place signatures’ (QPS). The usability and effectiveness of the proposed method were evaluated using openly available datasets, together with simulated observations by considering different types perception errors.

Zero-Shot Sketch-Based Image Retrieval Using StyleGen and Stacked Siamese Neural Networks.

Sketch-based image retrieval (SBIR) refers to a sub-class of content-based image retrieval problems where the input queries are ambiguous sketches and the retrieval repository is a database of natural images. In the zero-shot setup of SBIR, the query sketches are drawn from classes that do not match any of those that were used in model building. The SBIR task is extremely challenging as it is a cross-domain retrieval problem, unlike content-based image retrieval problems because sketches and images have a huge domain gap. In this work, we propose an elegant retrieval methodology, StyleGen, for generating fake candidate images that match the domain of the repository images, thus reducing the domain gap for retrieval tasks. The retrieval methodology makes use of a two-stage neural network architecture known as the stacked Siamese network, which is known to provide outstanding retrieval performance without losing the generalizability of the approach. Experimental studies on the image sketch datasets TU-Berlin Extended and Sketchy Extended, evaluated using the mean average precision (mAP) metric, demonstrate a marked performance improvement compared to the current state-of-the-art approaches in the domain.

Semantic Image Retrieval Analysis Based on Deep Learning and Singular Value Decomposition

The exponential growth in the total quantity of digital images has necessitated the development of systems that are capable of retrieving these images. Content-based image retrieval is a technique used to get images from a database. The user provides a query image, and the system retrieves those photos from the database that are most similar to the query image. The image retrieval problem pertains to the task of locating digital photographs inside extensive datasets. Image retrieval researchers are transitioning from the use of keywords to the utilization of low-level characteristics and semantic features. The push for semantic features arises from the issue of subjective and time-consuming keywords, as well as the limitation of low-level characteristics in capturing high-level concepts that users have in mind. The main goal of this study is to examine how convolutional neural networks can be used to acquire advanced visual features. These high-level feature descriptors have the potential to be the most effective compared to the handcrafted feature descriptors in terms of image representation, which would result in improved image retrieval performance. The (CBIR-VGGSVD) model is an ideal solution for content-based image retrieval that is based on the VGG-16 algorithm and uses the Singular Value Decomposition (SVD) technique. The suggested model incorporates the VGG-16 model for the purpose of extracting features from both the query images and the images kept in the database. Afterwards, the dimensionality of the features retrieved from the VGG-16 model is reduced using SVD. Then, we compare the query photographs to the dataset images using the cosine metric to see how similar they are. When all is said and done, images that share a high degree of similarity will be successfully extracted from the dataset. A validation of the retrieval performance of the CBIR-VGGSVD model is performed using the Corel-1K dataset. When the VGG-16 standard model is the sole one used, the implementation will produce an average precision of 0.864. On the other hand, when the CBIR-VGGSVD model is utilized, this average precision is revealed to be (0.948). The findings of the retrieval ensured that the CBIR-VGGSVD model provided an improvement in performance on the test pictures that were utilized, surpassing the performance of the most recent approaches.

Multiple-environment Self-adaptive Network for aerial-view geo-localization

Multiple-environment Self-adaptive Network for aerial-view geo-localization

Study on the development strategy of information integration of packaging design and pattern design under visual communication course

Abstract In this paper, the importance of modern packaging design and pattern design and the application of pattern design in modern packaging design are the first two aspects of achieving the development of information integration of packaging design and pattern design, and the visual design based on Markov random field is proposed for the image retrieval and classification problems in traditional model packaging design and pattern design. Wavelet transforms, multi-scale analysis of image texture features and region merging conditions in visual design using GBIS algorithm, and image retrieval in pattern design are examined and analyzed. The results show that the model trained with both loss functions has the best retrieval top-k accuracy on both datasets, reaching 54.3% on the DARN dataset and 3.9% accuracy on Deep Fashion. This paper demonstrates the model’s effectiveness for modifying and retrieving pattern design image attributes.

CMLocate: A cross‐modal automatic visual geo‐localization framework for a natural environment without GNSS information

AbstractIn this paper, a new approach to visual geo‐localization for natural environments is proposed. The digital elevation model (DEM) data in virtual space is rendered and construct a panoramic skyline database is constructed. By combining the skyline database with real‐world image data (used as the “queries” to be localized), visual geo‐localization is treated as a cross‐modal image retrieval problem for panoramic skyline images, creating a unique new visual geo‐localization benchmark for the natural environment. Specifically, the semantic segmentation model named LineNet is proposed, for skyline extractions from query images, which has proven to be robust to a variety of complex natural environments. On the aforementioned benchmarks, the fully automatic method is elaborated for large‐scale cross‐modal localization using panoramic skyline images. Finally, the compound index is delicately designed to reduce the storage space of the positioning global descriptors and improve the retrieval efficiency. Moreover, the proposed method is proven to outperform most state‐of‐the‐art methods.

Collaborative Semantic Communication for Edge Inference

We study the collaborative image retrieval problem at the wireless edge, where multiple edge devices capture images of the same object from different angles and locations, which are then used jointly to retrieve similar images at the edge server over a shared multiple access channel (MAC). We propose two novel deep learning-based joint source and channel coding (JSCC) schemes for the task over both additive white Gaussian noise (AWGN) and Rayleigh slow fading channels, with the aim of maximizing the retrieval accuracy under a total bandwidth constraint. The proposed schemes are evaluated on a wide range of channel signal-to-noise ratios (SNRs), and shown to outperform the single-device JSCC and the separation-based multiple-access benchmarks. We also propose a channel state information-aware JSCC scheme with attention modules to enable our method to adapt to varying channel conditions.

Correspondence-Free Domain Alignment for Unsupervised Cross-Domain Image Retrieval

Cross-domain image retrieval aims at retrieving images across different domains to excavate cross-domain classificatory or correspondence relationships. This paper studies a less-touched problem of cross-domain image retrieval, i.e., unsupervised cross-domain image retrieval, considering the following practical assumptions: (i) no correspondence relationship, and (ii) no category annotations. It is challenging to align and bridge distinct domains without cross-domain correspondence. To tackle the challenge, we present a novel Correspondence-free Domain Alignment (CoDA) method to effectively eliminate the cross-domain gap through In-domain Self-matching Supervision (ISS) and Cross-domain Classifier Alignment (CCA). To be specific, ISS is presented to encapsulate discriminative information into the latent common space by elaborating a novel self-matching supervision mechanism. To alleviate the cross-domain discrepancy, CCA is proposed to align distinct domain-specific classifiers. Thanks to the ISS and CCA, our method could encode the discrimination into the domain-invariant embedding space for unsupervised cross-domain image retrieval. To verify the effectiveness of the proposed method, extensive experiments are conducted on four benchmark datasets compared with six state-of-the-art methods.

A triplet graph convolutional network with attention and similarity-driven dictionary learning for remote sensing image retrieval

A triplet graph convolutional network with attention and similarity-driven dictionary learning for remote sensing image retrieval

Group Ensemble Block: Subspace Diversity Improves Coarse-to-Fine Image Retrieval

The random subspace method is an ensemble learning technique of great potential. However, its popularity does not match its potential because of its prohibitive cost and the lack of plug-and-play reusable modules. To fill this gap, we propose the random-subspace-based group ensemble block. The proposed ensemble block increases the diversity among subspaces by masking each sampled subspace. It also utilizes the diversity among a large number of subspaces, e.g., 1024, which is facilitated by combining the advantages of both averaging and concatenating strategies for merging different subspaces' outcomes. In terms of cost, the proposed ensemble block contributes few parameters by parameter sharing, negligible computations by the linear transformation, and fast running speed by parallel processing for every subspace. In terms of reusability, it encapsulates every necessary step in one small block, and can be interchangeably used as a linear layer in neural networks. We show that our group ensemble block is complementary to existing methods for image retrieval such as class-level and instance-level discrimination, and can also be deployed for other tasks such as image classification. Experiments show that the proposed group ensemble block achieves the state-of-the-art accuracy on the CIFAR-100 and ImageNet datasets for the coarse-to-fine image retrieval problem, where the model is trained with coarse-level annotation (e.g., trees) and evaluated with fine-grained category (e.g., maple trees and oak trees).

Interpretable pap-smear image retrieval for cervical cancer detection with rotation invariance mask generation deep hashing

Interpretable pap-smear image retrieval for cervical cancer detection with rotation invariance mask generation deep hashing

Place Recognition with Memorable and Stable Cues for Loop Closure of Visual SLAM Systems

Visual Place Recognition (VPR) is a fundamental yet challenging task in Visual Simultaneous Localization and Mapping (V-SLAM) problems. The VPR works as a subsystem of the V-SLAM. VPR is the task of retrieving images upon revisiting the same place in different conditions. The problem is even more difficult for agricultural and all-terrain autonomous mobile robots that work in different scenarios and weather conditions. Over the last few years, many state-of-the-art methods have been proposed to solve the limitations of existing VPR techniques. VPR using bag-of-words obtained from local features works well for a large-scale image retrieval problem. However, the aggregation of local features arbitrarily produces a large bag-of-words vector database, limits the capability of efficient feature learning, and aggregation and querying of candidate images. Moreover, aggregating arbitrary features is inefficient as not all local features equally contribute to long-term place recognition tasks. Therefore, a novel VPR architecture is proposed suitable for efficient place recognition with semantically meaningful local features and their 3D geometrical verifications. The proposed end-to-end architecture is fueled by a deep neural network, a bag-of-words database, and 3D geometrical verification for place recognition. This method is aware of meaningful and informative features of images for better scene understanding. Later, 3D geometrical information from the corresponding meaningful features is computed and utilised for verifying correct place recognition. The proposed method is tested on four well-known public datasets, and Micro Aerial Vehicle (MAV) recorded dataset for experimental validation from Victoria Park, Adelaide, Australia. The extensive experimental results considering standard evaluation metrics for VPR show that the proposed method produces superior performance than the available state-of-the-art methods.

Research on the Application of Artificial Intelligence in Image and Text Database Retrieval

In the graphic database, the difficulty of query processing lies in how to query the contents of various data, that is, content-based retrieval, which is an effective means and an important technology to realize multimedia data retrieval. In graphic database, the difficulty of query processing lies in how to query the content based on unformatted data, that is, content-based retrieval, which is an effective means and an important technology to realize multimedia data retrieval. Intelligent information retrieval (IR) system is an intelligent computer IR system, which simulates the thinking process and intelligent activities of human beings about information processing, realizes the storage, retrieval and reasoning of information knowledge, and provides intelligent assistance to users. This paper analyzes the problems of image and text database retrieval driven by big data, explores the application effect of artificial intelligence (AI) in IR driven by big data, promotes the innovation and transformation of modern science and technology, and realizes the sustainable development of our society.

Text Spotting towards Perceptually Aliased Urban Place Recognition

Recognizing places of interest (POIs) can be challenging for humans, especially in foreign environments. In this study, we leverage smartphone sensors (i.e., camera, GPS) and deep learning algorithms to propose an intelligent solution to recognize POIs in an urban environment. Recent studies have approached landmark recognition as an image retrieval problem. However, visual similarity alone is not robust against challenging conditions such as extreme appearance variance and perceptual aliasing in urban environments. To this end, we propose to fuse visual, textual, and positioning information. Our contributions are as follows. Firstly, we propose VPR through text reading pipeline (VPRText) that uses off-the-shelf text spotting algorithms for word spotting followed by layout analysis and text similarity search modules. Secondly, we propose a hierarchical architecture that combines VPRText and image retrieval. Thirdly, we perform a comprehensive empirical study on the applicability of state-of-the-art text spotting methods for the VPR task. Additionally, we introduce a challenging purpose-built urban dataset for VPR evaluation. The proposed VPR architecture achieves a superior performance overall, especially in challenging conditions (i.e., perceptually aliased and illuminated environments).

Application of Digital Images and Corresponding Image Retrieval Paradigm

We live in a world where digital images are constantly generated during our daily activities, whether private or business. They play an important role in our private life, showing important moments, people, places, or events and keeping their memory. Images are unavoidable in business, especially in digital marketing, web sales, social networks, medicine, security, and education. In general, images contribute to a better understanding of the message, increase the attractiveness of textual content, provide a better user experience, and can convey emotion quickly. The key advantage of the image is that very often, even a cursory glance at the image is enough to convey a message and arouse emotion and interest. But with the increase in digital image numbers, storage, organization, and retrieval problems arise. The paper describes the importance of images in different areas of application and different image retrieval paradigms that include text-based, content-based, and combined approaches. Also, the most popular image search tools and cloud storage services are compared and discussed. The conclusion comments on the applicability of existing approaches to image searches in different application domains and highlights the advantages and disadvantages of each of the approaches.

Thermal to Visual Person Re-Identification Using Collaborative Metric Learning Based on Maximum Margin Matrix Factorization

Thermal to Visual Person Re-Identification Using Collaborative Metric Learning Based on Maximum Margin Matrix Factorization

Feature disentangling and reciprocal learning with label-guided similarity for multi-label image retrieval

Feature disentangling and reciprocal learning with label-guided similarity for multi-label image retrieval

Image Retrieval Technology of Economic Regulations Based on Semantic Segmentation

Content-based image retrieval (CBIR) is an important part of pattern recognition and artificial intelligence. It has broad application prospects in many important fields, such as digital library, medical image analysis, petroleum geological survey, and public security information retrieval. In this study, statistical modeling and discriminant learning methods are used to analyze and study some key problems in image retrieval, including image concept retrieval, image example retrieval, and relevance feedback. The main research results obtained are as follows: an image classification method based on the Gaussian mixture model (GMM) and max-min posterior pseudo-probability (MMP) discriminant learning is proposed, which is called GMM-MMP method for short; a concept retrieval method based on GMM-MMP is proposed. According to the image concept, the image is divided into two categories: the concept-related image and the concept-unrelated image. The Gaussian mixture model is used to establish the mapping from the image low-level features to the image concept, and the image is classified according to the posterior pseudo-probability classifier to realize the image concept retrieval; an example retrieval method based on GMM-MMP is proposed. According to the image similarity semantics, the image is divided into two categories: the related image and the uncorrelated image of the example image. The Gaussian mixture model is used to establish the mapping from the low-level features of the image to the image similarity semantics. The image is classified according to the posterior pseudo-probability classifier to realize the image case retrieval. Based on the above work, this study implements a content-based image retrieval system.