Content-based Image Retrieval Research Articles

Location retrieval using qualitative place signatures of visible landmarks

Location retrieval based on visual information is to retrieve the location of an agent (e.g. human, robot) or the area they see by comparing their observations with a certain representation of the environment. Existing methods generally treat the problem as a content-based image retrieval problem and have demonstrated promising results in terms of localization accuracy. However, these methods are challenging to scale up due to the volume of reference data involved; and the image descriptions might not be easily understandable/communicable for humans to describe surroundings. Considering that humans often use less precise but easily produced qualitative spatial language and high-level semantic landmarks when describing an environment, a coarse-to-fine qualitative location retrieval method is proposed in this work to quickly narrow down the initial location of an agent by exploiting the available information in large-scale open data. This approach describes and indexes a location/place using the perceived qualitative spatial relations between ordered pairs of co-visible landmarks from the perspective of viewers, termed as ‘qualitative place signatures’ (QPS). The usability and effectiveness of the proposed method were evaluated using openly available datasets, together with simulated observations by considering different types perception errors.

Content Based Image Retrieval Using Support Vector Machine

A significant issue in multimedia systems is image retrieval. It is identified as the procedure for looking through and obtaining pictures from a dataset. One important and difficult area of study in digital image processing is content-based image retrieval (CBIR). Retrieving pertinent data from a vast picture database after a query image with better system output is a fundamental requirement of the CBIR system. Regretfully, not all techniques can be applied to achieve high retrieval accuracy. Thus, the purpose of this research is to use Support Vector Machine (SVM) to classify query picture data with image database data in order to get similar images. Based on the classification, the accuracy of the image retrieval will be verified.

Neuromorphic computing spiking neural network edge detection model for content based image retrieval

ABSTRACT In contemporary times, content-based image retrieval (CBIR) techniques have gained widespread acceptance as a means for end-users to discern and extract specific image content from vast repositories. However, it is noteworthy that a substantial majority of CBIR studies continue to rely on linear methodologies such as gradient-based and derivative-based edge detection techniques. This research explores the integration of bioinspired Spiking Neural Network (SNN) based edge detection within CBIR. We introduce an innovative, computationally efficient SNN-based approach designed explicitly for CBIR applications, outperforming existing SNN models by reducing computational overhead by 2.5 times. The proposed SNN-based edge detection approach is seamlessly incorporated into three distinct CBIR techniques, each employing conventional edge detection methodologies including Sobel, Canny, and image derivatives. Rigorous experimentation and evaluations are carried out utilizing the Corel-10k dataset and crop weed dataset, a widely recognized and frequently adopted benchmark dataset in the realm of image analysis. Importantly, our findings underscore the enhanced performance of CBIR methodologies integrating the proposed SNN-based edge detection approach, with an average increase in mean precision values exceeding 3%. This study conclusively demonstrated the utility of our proposed methodology in optimizing feature extraction, thereby establishing its pivotal role in advancing edge centric CBIR approaches.

The use of pretrained neural networks for solving the problem of reverse searching of X-ray images of prohibited items and substances

The paper considers the application of pretrained neural networks to solve the problem of reverse searching of X-ray images of prohibited items and substances. The purpose of the work is to conduct an analysis and substantiate ways to improve the efficiency of baggage and passenger hand luggage X-ray image recognition systems. An analysis of existing domestic and foreign works in the field of baggage and passenger hand luggage X-ray image recognition is presented. It has been revealed that, despite the achieved results in the development of algorithms for recognizing prohibited items and substances, they do not fully cope with such a complexity factor as the overlay of objects. To solve this problem, the paper proposes to additionally analyze X-ray images with low confidence in object recognition. This stage includes the following steps: image segmentation, extraction of features of segmented image elements; search for similar images in the database; decision-making on the class of segmented image elements. This article discusses the last three steps. Variants of approaches to feature extraction from images are analyzed, particularly those based on the application of convolutional autoencoders and pretrained neural networks. The approach based on the application of pretrained neural networks is chosen. The ResNet-50 architecture neural network, pretrained on the ImageNet collection, is used during the work. In order to apply this model to extract image feature vectors, the last classification layer was preliminarily removed. All the previous layers of the model encode the image into a vector. ResNet-50 generates a 2048-dimensional feature vector of images. The principal component analysis is used to reduce the dimensionality of the image feature vectors. The decision of whether the segmented image element is a prohibited item or substance is considered as a reverse search problem using the k-nearest neighbor algorithm. In this case, the class of the X-ray image element is the class most frequently encountered among the K nearest neighbors. In order to test the proposed approach, a training dataset, including 4,635 images of individual items and substances that may be encountered in baggage and passenger hand luggage, was generated. A comparative analysis of image indexing and image search under different algorithms and feature number is presented. A comparative analysis of the model accuracy is provided. It is concluded that the most acceptable is the “Brute force” algorithm in combination with the principal component analysis.

A novel deep learning based CBIR model using Convolutional Siamese Neural Networks

Content-Based Image Retrieval (CBIR) is a technique that involves retrieving similar images from a large database by analysing the content features of the query image. The heavy usage of digital platforms and devices has in a way promoted CBIR and its allied technologies in computer vision and artificial intelligence. The process entails comparing the representative features of the query image with those of the images in the dataset to rank them for retrieval. Past research was centered around handcrafted feature descriptors based on traditional visual features. But with the advent of deep learning the traditional manual method of feature engineering gave way to automatic feature extraction. In this study, a cascaded network is utilised for CBIR. In the first stage, the model employs multi-modal features from variational autoencoders and super-pixelated image characteristics to narrow down the search space. In the subsequent stage, an end-to-end deep learning network known as a Convolutional Siamese Neural Network (CSNN) is used. The concept of pseudo-labeling is incorporated to categorise images according to their affinity and similarity with the query image. Using this pseudo-supervised learning approach, this network evaluates the similarity between a query image and available image samples. The Siamese network assigns a similarity score to each target image, and those that surpass a predefined threshold are ranked and retrieved. The suggested CBIR system undergoes testing on a widely recognized public dataset: the Oxford dataset and its performance is measured against cutting-edge image retrieval methods. The findings reveal substantial enhancements in retrieval performance in terms of several standard benchmarks such as average precision, average error rate, average false positive rate etc., providing strong support for utilising images from interconnected devices.

Content-based image retrieval of Indian traditional textile motifs using deep feature fusion

In the fast-paced fashion world, unique designs are like early birds, grabbing attention as online shopping surges. Fabric texture plays an immense role in selecting the perfect design. Indian Traditional textile motifs are pivotal, showing rich cultural origins and attracting worldwide art fanatics. Yet, technology-driven abstract forms are posing a challenge for them. The decline of handmade artistic ability due to computerization is concerning. Crafting new designs associated with the latest trends is time- consuming and requires diligence. In this work an interactive CBIR (content-based image retrieval) system is presented. It utilizes deep features from InceptionV3 and InceptionResNetV2 models to match query designs with a database of traditional Indian textiles. Its performance is tested with Caltech-101, Corel-1K state-of-the-art datasets, and Indian Textiles datasets and the results are shown to be finer than the existing approaches. The similarity-based fine-grained saliency maps (SBFGSM) approach is employed to visualize the importance of features. Our approach combines deep feature fusion with PCA dimensionality reduction and speeds up search using a clustering approach. Relevance feedback is employed to refine the retrievals. This tool is expected to benefit designers by accelerating the design cycles by bridging the gap between human creativity and A.I. assistance.

Advanced Image Processing Techniques for Medical Image Retrieval Using Visual Features and Distance Measures

Recently, there has been a rapid increase in the volume of medical image repositories because of the maximized utilization of digitized image data in hospitals. As a result, there is a complexity in querying and handling such large databases, which has led to the development of a novel Content-Based Medical Image Retrieval (CBMIR) technique. In this work, the CBMIR technique is developed using three types of visual features (i.e., color, texture and shape) along with 12 distance measures, and is optimized with the Crossover-Gravitational Search Algorithm (CRO-GSA). For ease, we named this technique Content-Based Medical Image Retrieval using Visual features with CRO-GSA (CBMIRVC). The initial step of our CBMIRVC technique is to extract three types of visual features from the images. Then, each type of feature is employed with a suitable distance measure, which is used for the computation of image similarities between the medical images in the database and a provided query image. The optimization of our proposed CBMIRVC technique is done by the CRO-GSA algorithm. This algorithm optimizes the CBMIRVC technique by identifying the optimal combinations of visual features and similarity measures. Additionally, optimal weights are computed in accordance with the three types of features for the three similarities. Experimental validations were done using retrieved cancer-related and endoscopic color images from databases holding numerous image categories. The experimental outcome shows that our proposed CBMIRVC technique outperforms other conventional image retrieval techniques by effectively retrieving similar images from large databases.

A Novel Semantic Content-Based Retrieval System for Hyperspectral Remote Sensing Imagery

With the growing use of hyperspectral remote sensing payloads, there has been a significant increase in the number of hyperspectral remote sensing image archives, leading to a massive amount of collected data. This highlights the need for an efficient content-based hyperspectral image retrieval (CBHIR) system to manage and enable better use of hyperspectral remote-sensing image archives. Conventional CBHIR systems characterize each image by a set of endmembers and then perform image retrieval based on pairwise distance measures. Such an approach significantly increases the computational complexity of the retrieval, mainly when the diversity of materials is high. Those systems also have difficulties in retrieving images containing particular materials with extremely low abundance compared to other materials, which leads to describing image content with inappropriate and/or insufficient spectral features. In this article, a novel CBHIR system to define global hyperspectral image representations based on a semantic approach to differentiate foreground and background image content for different retrieval scenarios is introduced to address these issues. The experiments conducted on a new benchmark archive of multi-label hyperspectral images, which is first introduced in this study, validate the retrieval accuracy and effectiveness of the proposed system. Comparative performance analysis with the state-of-the-art CBHIR systems demonstrates that modeling hyperspectral image content with foreground and background vocabularies has a positive effect on retrieval performance.

Content-based image retrieval through fusion of deep features extracted from segmented neutrosophic using depth map

Content-based image retrieval through fusion of deep features extracted from segmented neutrosophic using depth map

Efficient Image Retrieval Using Hierarchical K-Means Clustering.

The objective of content-based image retrieval (CBIR) is to locate samples from a database that are akin to a query, relying on the content embedded within the images. A contemporary strategy involves calculating the similarity between compact vectors by encoding both the query and the database images as global descriptors. In this work, we propose an image retrieval method by using hierarchical K-means clustering to efficiently organize the image descriptors within the database, which aims to optimize the subsequent retrieval process. Then, we compute the similarity between the descriptor set within the leaf nodes and the query descriptor to rank them accordingly. Three tree search algorithms are presented to enable a trade-off between search accuracy and speed that allows for substantial gains at the expense of a slightly reduced retrieval accuracy. Our proposed method demonstrates enhancement in image retrieval speed when applied to the CLIP-based model, UNICOM, designed for category-level retrieval, as well as the CNN-based R-GeM model, tailored for particular object retrieval by validating its effectiveness across various domains and backbones. We achieve an 18-times speed improvement while preserving over 99% accuracy when applied to the In-Shop dataset, the largest dataset in the experiments.

Image Processing Technique on Identification of Leaf Types and Detection of Diseases Portion on Tomato Leaves

This study presents novel techniques for leaf type identification, employing color histogram and edge histogram approaches. A color histogram serves as a model to represent color through intensity values, while each image is associated with a descriptive caption. Signatures, encompassing shape, color, and texture, provide a basis for comparing images. The edge histogram, delineating the distribution of five edge types in localized sub-images, further enhances the identification process. To address disease detection on tomato leaves, a color-based segmentation method utilizing the k-means clustering technique is proposed. This iterative approach partitions images into k clusters, facilitating the identification of diseases affecting tomato plants. Beyond environmental factors like rain and temperature, crop diseases emerge as primary influencers on production quality and crop yield. Early detection of diseases is crucial for effective control and mitigation. Leveraging technological advancements, the paper emphasizes the potential of using images of diseased leaves for accurate disease identification. This involves feature extraction from images, which can be subsequently employed in classification algorithms or content-based image retrieval systems. 

Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval

Large-scale digital whole slide image (WSI) datasets analysis have gained significant attention in computer-aided cancer diagnosis. Content-based histopathological image retrieval (CBHIR) is a technique that searches a large database for data samples matching input objects in both details and semantics, offering relevant diagnostic information to pathologists. However, the current methods are limited by the difficulty of gigapixels, the variable size of WSIs, and the dependence on manual annotations. In this work, we propose a novel histopathology language-image representation learning framework for fine-grained digital pathology cross-modal retrieval, which utilizes paired diagnosis reports to learn fine-grained semantics from the WSI. An anchor-based WSI encoder is built to extract hierarchical region features and a prompt-based text encoder is introduced to learn fine-grained semantics from the diagnosis reports. The proposed framework is trained with a multivariate cross-modal loss function to learn semantic information from the diagnosis report at both the instance level and region level. After training, it can perform four types of retrieval tasks based on the multi-modal database to support diagnostic requirements. We conducted experiments on an in-house dataset and a public dataset to evaluate the proposed method. Extensive experiments have demonstrated the effectiveness of the proposed method and its advantages to the present histopathology retrieval methods. The code is available at https://github.com/hudingyi/FGCR.

Deep Learning Model for Retrieving Color Logo Images in Content Based Image Retrieval

Content-Based Image retrieval (CBIR) has gained a magnificent deal of consideration because of the digital image data's epidemic growth. The advancement of deep learning has enabled Convolutional Neural Networks to become an influential technique for extraction of discriminative image features. In recent years, convolutional neural networks (CNNs) have proven extremely effective at extracting unique information from images. In contrast to text-based image retrieval, CBIR gathers comparable images based primarily on their visual content. The use of deep learning, especially CNNs, for feature extraction and image processing has been shown to perform better than other techniques. In the proposed study, we investigate CNNs for CBIR focusing on how well they extract discriminative visual features and facilitate accurate image retrieval.  Also Principal Component Analysis and Linear Discriminant Analysis are combined for optimization of features resulting in boosting the retrieval results.  Using hierarchical representations learned by CNNs, we aim to improve retrieval accuracy and efficiency. In comparison with conventional retrieval techniques, our proposed CBIR system shows superior performance on a benchmark dataset.

Zero-Shot Sketch-Based Image Retrieval Using StyleGen and Stacked Siamese Neural Networks.

Sketch-based image retrieval (SBIR) refers to a sub-class of content-based image retrieval problems where the input queries are ambiguous sketches and the retrieval repository is a database of natural images. In the zero-shot setup of SBIR, the query sketches are drawn from classes that do not match any of those that were used in model building. The SBIR task is extremely challenging as it is a cross-domain retrieval problem, unlike content-based image retrieval problems because sketches and images have a huge domain gap. In this work, we propose an elegant retrieval methodology, StyleGen, for generating fake candidate images that match the domain of the repository images, thus reducing the domain gap for retrieval tasks. The retrieval methodology makes use of a two-stage neural network architecture known as the stacked Siamese network, which is known to provide outstanding retrieval performance without losing the generalizability of the approach. Experimental studies on the image sketch datasets TU-Berlin Extended and Sketchy Extended, evaluated using the mean average precision (mAP) metric, demonstrate a marked performance improvement compared to the current state-of-the-art approaches in the domain.

VVS: Video-to-Video Retrieval with Irrelevant Frame Suppression

In content-based video retrieval (CBVR), dealing with large-scale collections, efficiency is as important as accuracy; thus, several video-level feature-based studies have actively been conducted. Nevertheless, owing to the severe difficulty of embedding a lengthy and untrimmed video into a single feature, these studies have been insufficient for accurate retrieval compared to frame-level feature-based studies. In this paper, we show that appropriate suppression of irrelevant frames can provide insight into the current obstacles of the video-level approaches. Furthermore, we propose a Video-to-Video Suppression network (VVS) as a solution. VVS is an end-to-end framework that consists of an easy distractor elimination stage to identify which frames to remove and a suppression weight generation stage to determine the extent to suppress the remaining frames. This structure is intended to effectively describe an untrimmed video with varying content and meaningless information. Its efficacy is proved via extensive experiments, and we show that our approach is not only state-of-the-art in video-level approaches but also has a fast inference time despite possessing retrieval capabilities close to those of frame-level approaches. Code is available at https://github.com/sejong-rcv/VVS

Block Chain-Assisted Privacy-Preserving Similarity Image Retrieval Over Cloud

With the advancement of digital technology, there has been an exponential increase in the storage of photographs across numerous servers. However, this proliferation of data raises concerns about integrity and reliability, particularly in multi-cloud environments. Existing research on content-based image retrieval has overlooked the challenges posed by untrustworthy cloud servers, hindering effective multi-cloud retrieval. Typically, centralized search servers are introduced in multicloud systems to facilitate image retrieval. Unfortunately, these centralized servers may produce inaccurate or incomplete results, compromising search precision and potentially impacting business interests. Furthermore, cloud servers themselves may exhibit dishonest behavior by tampering with or deleting images to conserve resources, leading to data loss and false image decryption.To address these issues, we propose a solution that ensures both the accuracy of search results and the integrity of images in multi-cloud environments. Our approach involves employing a single-image retrieval technique using blockchain technology to verify both the authenticity of search results and the integrity of stored images. We evaluate the effectiveness of our approach using real-world data, demonstrating its precision and efficiency in combating both over-retrieval and image manipulation. Keywords: Accuracy, Integrity, Privacy Protection, Block Chain, False/Incomplete Results, Multi-Cloud Retrieval, Central Search Server

Deep Representation-Based Fuzzy Graph Model for Content-Based Image Retrieval

Deep Representation-Based Fuzzy Graph Model for Content-Based Image Retrieval

Echoes of images: multi-loss network for image retrieval in vision transformers.

This paper introduces a novel approach to enhance content-based image retrieval, validated on two benchmark datasets: ISIC-2017 and ISIC-2018. These datasets comprise skin lesion images that are crucial for innovations in skin cancer diagnosis and treatment. We advocate the use of pre-trained Vision Transformer (ViT), a relatively uncharted concept in the realm of image retrieval, particularly in medical scenarios. In contrast to the traditionally employed Convolutional Neural Networks (CNNs), our findings suggest that ViT offers a more comprehensive understanding of the image context, essential in medical imaging. We further incorporate a weighted multi-loss function, delving into various losses such as triplet loss, distillation loss, contrastive loss, and cross-entropy loss. Our exploration investigates the most resilient combination of these losses to create a robust multi-loss function, thus enhancing the robustness of the learned feature space and ameliorating the precision and recall in the retrieval process. Instead of using all the loss functions, the proposed multi-loss function utilizes the combination of only cross-entropy loss, triplet loss, and distillation loss and gains improvement of 6.52% and 3.45% for mean average precision over ISIC-2017 and ISIC-2018. Another innovation in our methodology is a two-branch network strategy, which concurrently boosts image retrieval and classification. Through our experiments, we underscore the effectiveness and the pitfalls of diverse loss configurations in image retrieval. Furthermore, our approach underlines the advantages of retrieval-based classification through majority voting rather than relying solely on the classification head, leading to enhanced prediction for melanoma - the most lethal type of skin cancer. Our results surpass existing state-of-the-art techniques on the ISIC-2017 and ISIC-2018 datasets by improving mean average precision by 1.01% and 4.36% respectively, emphasizing the efficacy and promise of Vision Transformers paired with our tailor-made weighted loss function, especially in medical contexts. The proposed approach's effectiveness is substantiated through thorough ablation studies and an array of quantitative and qualitative outcomes. To promote reproducibility and support forthcoming research, our source code will be accessible on GitHub.

Advancing Content-Based Histopathological Image Retrieval Pre-Processing: A Comparative Analysis of the Effects of Color Normalization Techniques

Content-Based Histopathological Image Retrieval (CBHIR) is a search technique based on the visual content and histopathological features of whole-slide images (WSIs). CBHIR tools assist pathologists to obtain a faster and more accurate cancer diagnosis. Stain variation between hospitals hampers the performance of CBHIR tools. This paper explores the effects of color normalization (CN) in a recently proposed CBHIR approach to tackle this issue. In this paper, three different CN techniques were used on the CAMELYON17 (CAM17) data set, which is a breast cancer data set. CAM17 consists of images taken using different staining protocols and scanners in five hospitals. Our experiments reveal that a proper CN technique, which can transfer the color version into the most similar median values, has a positive impact on the retrieval performance of the proposed CBHIR framework. According to the obtained results, using CN as a pre-processing step can improve the accuracy of the proposed CBHIR framework to 97% (a 14% increase), compared to working with the original images.

Deep Learning based Smart Image Search Engine

Abstract: This paper introduces a new reverse search engine integration into content-based image retrieval (CBIR) systems that employs convolutional neural networks (CNNs) for feature extraction. It generates global descriptors using pre-trained CNN architectures such as ResNet50, InceptionV3, and InceptionResNetV2. It retrieves visually similar images without depending on linguistic annotations. Comparative analysis against existing methods, such as Gabor Wavelet, CNN-SVM, Metaheuristic Algorithm, etc., has been tested, and it proves the superiority of the proposed algorithm, the Cartoon Texture Algorithm, in CBIR. As the Internet sees an exponential growth of different data types, the importance of CBIR continues to grow. In order to efficiently retrieve images, solely relying on image features while ignoring metadata is exactly what we need. As such, this paper is a reminder of the need for CBIR in this changing world. They showed that CBIR continues to be quite effective in the age of the Internet. Their proposed model for CBIR, which integrates ResNet-50-based feature extraction, a neural network model trained on different image datasets, and clustering techniques to make retrieval fast, provides a significant improvement in accuracy and efficiency for content-dependent image retrieval. This methodology is likely to be very useful as we work with the increasingly huge data of vision and beyond on the Internet. It provides a good basis for an effective image search and retrieval system