Problem Of Image Retrieval Research Articles

Location retrieval using qualitative place signatures of visible landmarks

Location retrieval based on visual information is to retrieve the location of an agent (e.g. human, robot) or the area they see by comparing their observations with a certain representation of the environment. Existing methods generally treat the problem as a content-based image retrieval problem and have demonstrated promising results in terms of localization accuracy. However, these methods are challenging to scale up due to the volume of reference data involved; and the image descriptions might not be easily understandable/communicable for humans to describe surroundings. Considering that humans often use less precise but easily produced qualitative spatial language and high-level semantic landmarks when describing an environment, a coarse-to-fine qualitative location retrieval method is proposed in this work to quickly narrow down the initial location of an agent by exploiting the available information in large-scale open data. This approach describes and indexes a location/place using the perceived qualitative spatial relations between ordered pairs of co-visible landmarks from the perspective of viewers, termed as ‘qualitative place signatures’ (QPS). The usability and effectiveness of the proposed method were evaluated using openly available datasets, together with simulated observations by considering different types perception errors.

Zero-Shot Sketch-Based Image Retrieval Using StyleGen and Stacked Siamese Neural Networks.

Sketch-based image retrieval (SBIR) refers to a sub-class of content-based image retrieval problems where the input queries are ambiguous sketches and the retrieval repository is a database of natural images. In the zero-shot setup of SBIR, the query sketches are drawn from classes that do not match any of those that were used in model building. The SBIR task is extremely challenging as it is a cross-domain retrieval problem, unlike content-based image retrieval problems because sketches and images have a huge domain gap. In this work, we propose an elegant retrieval methodology, StyleGen, for generating fake candidate images that match the domain of the repository images, thus reducing the domain gap for retrieval tasks. The retrieval methodology makes use of a two-stage neural network architecture known as the stacked Siamese network, which is known to provide outstanding retrieval performance without losing the generalizability of the approach. Experimental studies on the image sketch datasets TU-Berlin Extended and Sketchy Extended, evaluated using the mean average precision (mAP) metric, demonstrate a marked performance improvement compared to the current state-of-the-art approaches in the domain.

Semantic Image Retrieval Analysis Based on Deep Learning and Singular Value Decomposition

The exponential growth in the total quantity of digital images has necessitated the development of systems that are capable of retrieving these images. Content-based image retrieval is a technique used to get images from a database. The user provides a query image, and the system retrieves those photos from the database that are most similar to the query image. The image retrieval problem pertains to the task of locating digital photographs inside extensive datasets. Image retrieval researchers are transitioning from the use of keywords to the utilization of low-level characteristics and semantic features. The push for semantic features arises from the issue of subjective and time-consuming keywords, as well as the limitation of low-level characteristics in capturing high-level concepts that users have in mind. The main goal of this study is to examine how convolutional neural networks can be used to acquire advanced visual features. These high-level feature descriptors have the potential to be the most effective compared to the handcrafted feature descriptors in terms of image representation, which would result in improved image retrieval performance. The (CBIR-VGGSVD) model is an ideal solution for content-based image retrieval that is based on the VGG-16 algorithm and uses the Singular Value Decomposition (SVD) technique. The suggested model incorporates the VGG-16 model for the purpose of extracting features from both the query images and the images kept in the database. Afterwards, the dimensionality of the features retrieved from the VGG-16 model is reduced using SVD. Then, we compare the query photographs to the dataset images using the cosine metric to see how similar they are. When all is said and done, images that share a high degree of similarity will be successfully extracted from the dataset. A validation of the retrieval performance of the CBIR-VGGSVD model is performed using the Corel-1K dataset. When the VGG-16 standard model is the sole one used, the implementation will produce an average precision of 0.864. On the other hand, when the CBIR-VGGSVD model is utilized, this average precision is revealed to be (0.948). The findings of the retrieval ensured that the CBIR-VGGSVD model provided an improvement in performance on the test pictures that were utilized, surpassing the performance of the most recent approaches.

Multiple-environment Self-adaptive Network for aerial-view geo-localization

Aerial-view geo-localization tends to determine an unknown position through matching the drone-view image with the geo-tagged satellite-view image. This task is mostly regarded as an image retrieval problem. The key underpinning this task is to design a series of deep neural networks to learn discriminative image descriptors. However, existing methods meet large performance drops under realistic weather, such as rain and fog, since they do not take the domain shift between the training data and multiple test environments into consideration. To minor this domain gap, we propose a Multiple-environment Self-adaptive Network (MuSe-Net) to dynamically adjust the domain shift caused by environmental changing. In particular, MuSe-Net employs a two-branch neural network containing one multiple-environment style extraction network and one self-adaptive feature extraction network. As the name implies, the multiple-environment style extraction network is to extract the environment-related style information, while the self-adaptive feature extraction network utilizes an adaptive modulation module to dynamically minimize the environment-related style gap. Extensive experiments on three widely-used benchmarks, i.e., University-1652, SUES-200, and CVUSA, demonstrate that the proposed MuSe-Net achieves a competitive result for geo-localization in multiple environments. Furthermore, we observe that the proposed method also shows great potential to the unseen extreme weather, such as mixing the fog, rain and snow.

Study on the development strategy of information integration of packaging design and pattern design under visual communication course

Abstract In this paper, the importance of modern packaging design and pattern design and the application of pattern design in modern packaging design are the first two aspects of achieving the development of information integration of packaging design and pattern design, and the visual design based on Markov random field is proposed for the image retrieval and classification problems in traditional model packaging design and pattern design. Wavelet transforms, multi-scale analysis of image texture features and region merging conditions in visual design using GBIS algorithm, and image retrieval in pattern design are examined and analyzed. The results show that the model trained with both loss functions has the best retrieval top-k accuracy on both datasets, reaching 54.3% on the DARN dataset and 3.9% accuracy on Deep Fashion. This paper demonstrates the model’s effectiveness for modifying and retrieving pattern design image attributes.

CMLocate: A cross‐modal automatic visual geo‐localization framework for a natural environment without GNSS information

AbstractIn this paper, a new approach to visual geo‐localization for natural environments is proposed. The digital elevation model (DEM) data in virtual space is rendered and construct a panoramic skyline database is constructed. By combining the skyline database with real‐world image data (used as the “queries” to be localized), visual geo‐localization is treated as a cross‐modal image retrieval problem for panoramic skyline images, creating a unique new visual geo‐localization benchmark for the natural environment. Specifically, the semantic segmentation model named LineNet is proposed, for skyline extractions from query images, which has proven to be robust to a variety of complex natural environments. On the aforementioned benchmarks, the fully automatic method is elaborated for large‐scale cross‐modal localization using panoramic skyline images. Finally, the compound index is delicately designed to reduce the storage space of the positioning global descriptors and improve the retrieval efficiency. Moreover, the proposed method is proven to outperform most state‐of‐the‐art methods.

Collaborative Semantic Communication for Edge Inference

We study the collaborative image retrieval problem at the wireless edge, where multiple edge devices capture images of the same object from different angles and locations, which are then used jointly to retrieve similar images at the edge server over a shared multiple access channel (MAC). We propose two novel deep learning-based joint source and channel coding (JSCC) schemes for the task over both additive white Gaussian noise (AWGN) and Rayleigh slow fading channels, with the aim of maximizing the retrieval accuracy under a total bandwidth constraint. The proposed schemes are evaluated on a wide range of channel signal-to-noise ratios (SNRs), and shown to outperform the single-device JSCC and the separation-based multiple-access benchmarks. We also propose a channel state information-aware JSCC scheme with attention modules to enable our method to adapt to varying channel conditions.

Correspondence-Free Domain Alignment for Unsupervised Cross-Domain Image Retrieval

Cross-domain image retrieval aims at retrieving images across different domains to excavate cross-domain classificatory or correspondence relationships. This paper studies a less-touched problem of cross-domain image retrieval, i.e., unsupervised cross-domain image retrieval, considering the following practical assumptions: (i) no correspondence relationship, and (ii) no category annotations. It is challenging to align and bridge distinct domains without cross-domain correspondence. To tackle the challenge, we present a novel Correspondence-free Domain Alignment (CoDA) method to effectively eliminate the cross-domain gap through In-domain Self-matching Supervision (ISS) and Cross-domain Classifier Alignment (CCA). To be specific, ISS is presented to encapsulate discriminative information into the latent common space by elaborating a novel self-matching supervision mechanism. To alleviate the cross-domain discrepancy, CCA is proposed to align distinct domain-specific classifiers. Thanks to the ISS and CCA, our method could encode the discrimination into the domain-invariant embedding space for unsupervised cross-domain image retrieval. To verify the effectiveness of the proposed method, extensive experiments are conducted on four benchmark datasets compared with six state-of-the-art methods.

A triplet graph convolutional network with attention and similarity-driven dictionary learning for remote sensing image retrieval

With the explosion in the volume of collected high-resolution aerial image data, the development of effective image retrieval methods for remote sensing (RS) has become a popular area of research. The problem of content-based image retrieval (CBIR) for high-resolution remote sensing (HRRS) requires robust feature extraction and representation to extract high-level semantics followed by a similarity measurement method to compute the similarity between images based on these semantics. In this paper, we propose a novel sparse graph learning-based triplet network for deep metric learning-based HRRS CBIR. First, the graph representations of the aerial images are constructed via local feature descriptors and unsupervised segmentation that provide an efficient graph-structured representation of the images. Then, a novel attention graph convolutional network is developed to extract high-level semantics from the spatial patterns of the images’ segmented regions by aggregating region-node features. The attention mechanism and the pyramid pooling structure of the graph network help the proposed model extract powerful spatial features by preserving the structural information of the images while reducing the data’s dimensionality. Finally, a novel task-driven dictionary learning (TDDL) method based on triplet loss constructs a sparse metric space for the similarity measurement of the images in which the computed sparse codes for the images promote intra-class similarity between same-class images and inter-class dissimilarity between images of different classes. The proposed sparse features reduce the dimensionality of model parameters which mitigates overfitting and improves the generalization of the overall framework for deep metric learning. Our TDDL method adopts a bi-level optimization strategy so that the dictionary can be trained alongside the parameters of the graph representation network in an end-to-end fashion. Finally, we conduct extensive evaluations of the performance of our proposed architecture on real-world datasets compared to state-of-the-art models and obtain superior results.

Group Ensemble Block: Subspace Diversity Improves Coarse-to-Fine Image Retrieval

The random subspace method is an ensemble learning technique of great potential. However, its popularity does not match its potential because of its prohibitive cost and the lack of plug-and-play reusable modules. To fill this gap, we propose the random-subspace-based group ensemble block. The proposed ensemble block increases the diversity among subspaces by masking each sampled subspace. It also utilizes the diversity among a large number of subspaces, e.g., 1024, which is facilitated by combining the advantages of both averaging and concatenating strategies for merging different subspaces' outcomes. In terms of cost, the proposed ensemble block contributes few parameters by parameter sharing, negligible computations by the linear transformation, and fast running speed by parallel processing for every subspace. In terms of reusability, it encapsulates every necessary step in one small block, and can be interchangeably used as a linear layer in neural networks. We show that our group ensemble block is complementary to existing methods for image retrieval such as class-level and instance-level discrimination, and can also be deployed for other tasks such as image classification. Experiments show that the proposed group ensemble block achieves the state-of-the-art accuracy on the CIFAR-100 and ImageNet datasets for the coarse-to-fine image retrieval problem, where the model is trained with coarse-level annotation (e.g., trees) and evaluated with fine-grained category (e.g., maple trees and oak trees).

Interpretable pap-smear image retrieval for cervical cancer detection with rotation invariance mask generation deep hashing

Cervical cancer is a common disease in women, affecting their lives negatively and often resulting in death. Pap-smear tests are preferred by doctors as the primary tool in the early diagnosis and treatment of the disease. Physicians can be facilitated in the detection of five different categories of cervical cancer and similar cellular disease cases with the Pap-smear image retrieval technology. In this study, an algorithm for retrieval of cervical cancer images using hash coding with a Convolutional Neural Network (CNN) has been implemented. A sensitive deep hashing method combining interpretable mask generation and rotation invariance is proposed for cervical cancer detection. The distinctive features of cervical cancer cells with complex morphological features are focused on with the proposed hybrid dilated convolution spatial attention module and insignificant features are eliminated. Moreover, the loss function of Cauchy rotation invariance in terms of cervical cancer cell target is presented. In this way, the differences in the input samples are revealed, allowing the CNN to learn from different angles and achieve certain rotation invariance. The versatility and performance of the proposed method, as well as the efficiency of the loss function, have been tested on the SIPaKMeD and Mendeley LBC datasets consisting of cervical cancer images. In the experimental results obtained, it is shown that the proposed spatial attention module and rotational invariance deep hashing network generate high performance in cervical cancer image retrieval problems.

Place Recognition with Memorable and Stable Cues for Loop Closure of Visual SLAM Systems

Visual Place Recognition (VPR) is a fundamental yet challenging task in Visual Simultaneous Localization and Mapping (V-SLAM) problems. The VPR works as a subsystem of the V-SLAM. VPR is the task of retrieving images upon revisiting the same place in different conditions. The problem is even more difficult for agricultural and all-terrain autonomous mobile robots that work in different scenarios and weather conditions. Over the last few years, many state-of-the-art methods have been proposed to solve the limitations of existing VPR techniques. VPR using bag-of-words obtained from local features works well for a large-scale image retrieval problem. However, the aggregation of local features arbitrarily produces a large bag-of-words vector database, limits the capability of efficient feature learning, and aggregation and querying of candidate images. Moreover, aggregating arbitrary features is inefficient as not all local features equally contribute to long-term place recognition tasks. Therefore, a novel VPR architecture is proposed suitable for efficient place recognition with semantically meaningful local features and their 3D geometrical verifications. The proposed end-to-end architecture is fueled by a deep neural network, a bag-of-words database, and 3D geometrical verification for place recognition. This method is aware of meaningful and informative features of images for better scene understanding. Later, 3D geometrical information from the corresponding meaningful features is computed and utilised for verifying correct place recognition. The proposed method is tested on four well-known public datasets, and Micro Aerial Vehicle (MAV) recorded dataset for experimental validation from Victoria Park, Adelaide, Australia. The extensive experimental results considering standard evaluation metrics for VPR show that the proposed method produces superior performance than the available state-of-the-art methods.

Research on the Application of Artificial Intelligence in Image and Text Database Retrieval

In the graphic database, the difficulty of query processing lies in how to query the contents of various data, that is, content-based retrieval, which is an effective means and an important technology to realize multimedia data retrieval. In graphic database, the difficulty of query processing lies in how to query the content based on unformatted data, that is, content-based retrieval, which is an effective means and an important technology to realize multimedia data retrieval. Intelligent information retrieval (IR) system is an intelligent computer IR system, which simulates the thinking process and intelligent activities of human beings about information processing, realizes the storage, retrieval and reasoning of information knowledge, and provides intelligent assistance to users. This paper analyzes the problems of image and text database retrieval driven by big data, explores the application effect of artificial intelligence (AI) in IR driven by big data, promotes the innovation and transformation of modern science and technology, and realizes the sustainable development of our society.

Text Spotting towards Perceptually Aliased Urban Place Recognition

Recognizing places of interest (POIs) can be challenging for humans, especially in foreign environments. In this study, we leverage smartphone sensors (i.e., camera, GPS) and deep learning algorithms to propose an intelligent solution to recognize POIs in an urban environment. Recent studies have approached landmark recognition as an image retrieval problem. However, visual similarity alone is not robust against challenging conditions such as extreme appearance variance and perceptual aliasing in urban environments. To this end, we propose to fuse visual, textual, and positioning information. Our contributions are as follows. Firstly, we propose VPR through text reading pipeline (VPRText) that uses off-the-shelf text spotting algorithms for word spotting followed by layout analysis and text similarity search modules. Secondly, we propose a hierarchical architecture that combines VPRText and image retrieval. Thirdly, we perform a comprehensive empirical study on the applicability of state-of-the-art text spotting methods for the VPR task. Additionally, we introduce a challenging purpose-built urban dataset for VPR evaluation. The proposed VPR architecture achieves a superior performance overall, especially in challenging conditions (i.e., perceptually aliased and illuminated environments).

Application of Digital Images and Corresponding Image Retrieval Paradigm

We live in a world where digital images are constantly generated during our daily activities, whether private or business. They play an important role in our private life, showing important moments, people, places, or events and keeping their memory. Images are unavoidable in business, especially in digital marketing, web sales, social networks, medicine, security, and education. In general, images contribute to a better understanding of the message, increase the attractiveness of textual content, provide a better user experience, and can convey emotion quickly. The key advantage of the image is that very often, even a cursory glance at the image is enough to convey a message and arouse emotion and interest. But with the increase in digital image numbers, storage, organization, and retrieval problems arise. The paper describes the importance of images in different areas of application and different image retrieval paradigms that include text-based, content-based, and combined approaches. Also, the most popular image search tools and cloud storage services are compared and discussed. The conclusion comments on the applicability of existing approaches to image searches in different application domains and highlights the advantages and disadvantages of each of the approaches.

Thermal to Visual Person Re-Identification Using Collaborative Metric Learning Based on Maximum Margin Matrix Factorization

Thermal to visual person re-identification (T2V-ReID) is a cross-domain image retrieval problem. In this problem, the matching of a person’s image takes place, where the image is taken by different cameras (thermal and visual) at different times. This problem has numerous applications in night-time security surveillance. It is challenging due to the large intra-class variations and cross-domain discrepancies. Recently, deep metric learning methods are proposed for this problem. Still, there is a scope to improve the metric learning by generalizing the metric. In this paper, we have proposed the collaborative metric learning using Maximum Margin Matrix Factorization. It uses the group-wise similarities and collaboratively predicts the similarities. We can learn a more generalized metric by utilizing the maximized margin in this method. The proposed method is tested on the RegDB and RGB-D-T data sets, and the method outperforms the existing works in the few-shot learning settings.

Feature disentangling and reciprocal learning with label-guided similarity for multi-label image retrieval

Image retrieval usually faces scale-variance issues as the amount of image data is rapidly increasing, which calls for more accurate retrieval technology. Besides, existing methods usually treat pair-image similarity as a binary value which indicates whether two images share either at least one common label or none of shared labels. However, such similarity definition cannot truly describe the similarity ranking for different numbers of common labels when handling the multi-label image retrieval problem. In this paper, a Feature Disentangling and Reciprocal Learning (FDRL) method is introduced with label-guided similarity to solve the above multi-label image retrieval problem. Multi-scale features are first extracted by BNInception network and then disentangled to the corresponding high- and low-correlation features under the guidance of estimated global correlations. After that, the disentangled features are combined through a reciprocal learning approach to enhance the feature representation ability. Final hash codes are learned based on the global features derived from BNInception network and the combined features generated by reciprocal learning. The whole network is optimized by the proposed label-guided similarity loss function which aims to simultaneously preserve absolute similarity for hard image pairs and relative similarity for soft image pairs. Experimental results on three public benchmark datasets demonstrate that the proposed method outperforms current state-of-the-art techniques. The code is online here: ‘https://github.com/Yong-DAI/FDRL’.

Image Retrieval Technology of Economic Regulations Based on Semantic Segmentation

Content-based image retrieval (CBIR) is an important part of pattern recognition and artificial intelligence. It has broad application prospects in many important fields, such as digital library, medical image analysis, petroleum geological survey, and public security information retrieval. In this study, statistical modeling and discriminant learning methods are used to analyze and study some key problems in image retrieval, including image concept retrieval, image example retrieval, and relevance feedback. The main research results obtained are as follows: an image classification method based on the Gaussian mixture model (GMM) and max-min posterior pseudo-probability (MMP) discriminant learning is proposed, which is called GMM-MMP method for short; a concept retrieval method based on GMM-MMP is proposed. According to the image concept, the image is divided into two categories: the concept-related image and the concept-unrelated image. The Gaussian mixture model is used to establish the mapping from the image low-level features to the image concept, and the image is classified according to the posterior pseudo-probability classifier to realize the image concept retrieval; an example retrieval method based on GMM-MMP is proposed. According to the image similarity semantics, the image is divided into two categories: the related image and the uncorrelated image of the example image. The Gaussian mixture model is used to establish the mapping from the low-level features of the image to the image similarity semantics. The image is classified according to the posterior pseudo-probability classifier to realize the image case retrieval. Based on the above work, this study implements a content-based image retrieval system.

Individual Re-Identification in Blurred Image using SVM/PCA Technique

Individual re-identification (Re-ID) is the task of matching a target person across different cameras, which has drawn extensive attention in computer vision and has become an essential component in the video surveillance system. Pre-id can be considered as a problem of image retrieval. Existing person re-identification methods depend mostly on single-scale appearance information. In this work, to address issues, we demonstrate the benefits of a techniques using Support vector machine or Principal component analysis proposed for pre-id in this study. However, great challenges are being faced in the pre-id task. First, in different scenarios, appearance of the same pedestrian changes dramatically by reason of the body misalignment frequently, various background clutters, large variations of camera views and occlusion. Second, in a public space, different pedestrians wear the same or similar clothes. Therefore, the distinctions between different pedestrian images are subtle. These make the topic of pre-id a huge challenge. The proposed methods are only performed in the training phase and discarded in the testing phase, thus, enhancing the effectiveness of the model. Our model achieves the state-of-theart on the popular benchmark datasets including Market-1501, duke mtmc -re-id and CUHK03. Besides, we conduct a set of ablation experiments to verify the effectiveness of the proposed methods

Image retrieval based on weighted nearest neighbor tag prediction

Abstract With the development of communication and computer technology, the application of big data technology has become increasingly widespread. Reasonable, effective, and fast retrieval methods for querying information from massive data have become an important content of current research. This article provides an image retrieval method based on the weighted nearest neighbor label prediction for the problem of automatic image annotation and keyword image retrieval. In order to improve the performance of the test method, scientific experimental verification was implemented. The nearest neighbor weights are determined by maximizing the training image annotation, and experiments are carried out from multiple angles based on the Mahalanobis metric learning integration model. The experimental results show that the proposed tag correlation prediction propagation model has obvious improvements in accuracy, recall rate, break-even point, and overall average accuracy performance compared with other widely used algorithm models.