Content-based Image Retrieval Research Articles

VVS: Video-to-Video Retrieval with Irrelevant Frame Suppression

In content-based video retrieval (CBVR), dealing with large-scale collections, efficiency is as important as accuracy; thus, several video-level feature-based studies have actively been conducted. Nevertheless, owing to the severe difficulty of embedding a lengthy and untrimmed video into a single feature, these studies have been insufficient for accurate retrieval compared to frame-level feature-based studies. In this paper, we show that appropriate suppression of irrelevant frames can provide insight into the current obstacles of the video-level approaches. Furthermore, we propose a Video-to-Video Suppression network (VVS) as a solution. VVS is an end-to-end framework that consists of an easy distractor elimination stage to identify which frames to remove and a suppression weight generation stage to determine the extent to suppress the remaining frames. This structure is intended to effectively describe an untrimmed video with varying content and meaningless information. Its efficacy is proved via extensive experiments, and we show that our approach is not only state-of-the-art in video-level approaches but also has a fast inference time despite possessing retrieval capabilities close to those of frame-level approaches. Code is available at https://github.com/sejong-rcv/VVS

Block Chain-Assisted Privacy-Preserving Similarity Image Retrieval Over Cloud

With the advancement of digital technology, there has been an exponential increase in the storage of photographs across numerous servers. However, this proliferation of data raises concerns about integrity and reliability, particularly in multi-cloud environments. Existing research on content-based image retrieval has overlooked the challenges posed by untrustworthy cloud servers, hindering effective multi-cloud retrieval. Typically, centralized search servers are introduced in multicloud systems to facilitate image retrieval. Unfortunately, these centralized servers may produce inaccurate or incomplete results, compromising search precision and potentially impacting business interests. Furthermore, cloud servers themselves may exhibit dishonest behavior by tampering with or deleting images to conserve resources, leading to data loss and false image decryption.To address these issues, we propose a solution that ensures both the accuracy of search results and the integrity of images in multi-cloud environments. Our approach involves employing a single-image retrieval technique using blockchain technology to verify both the authenticity of search results and the integrity of stored images. We evaluate the effectiveness of our approach using real-world data, demonstrating its precision and efficiency in combating both over-retrieval and image manipulation. Keywords: Accuracy, Integrity, Privacy Protection, Block Chain, False/Incomplete Results, Multi-Cloud Retrieval, Central Search Server

Deep Representation-Based Fuzzy Graph Model for Content-Based Image Retrieval

Deep Representation-Based Fuzzy Graph Model for Content-Based Image Retrieval

Echoes of images: multi-loss network for image retrieval in vision transformers.

This paper introduces a novel approach to enhance content-based image retrieval, validated on two benchmark datasets: ISIC-2017 and ISIC-2018. These datasets comprise skin lesion images that are crucial for innovations in skin cancer diagnosis and treatment. We advocate the use of pre-trained Vision Transformer (ViT), a relatively uncharted concept in the realm of image retrieval, particularly in medical scenarios. In contrast to the traditionally employed Convolutional Neural Networks (CNNs), our findings suggest that ViT offers a more comprehensive understanding of the image context, essential in medical imaging. We further incorporate a weighted multi-loss function, delving into various losses such as triplet loss, distillation loss, contrastive loss, and cross-entropy loss. Our exploration investigates the most resilient combination of these losses to create a robust multi-loss function, thus enhancing the robustness of the learned feature space and ameliorating the precision and recall in the retrieval process. Instead of using all the loss functions, the proposed multi-loss function utilizes the combination of only cross-entropy loss, triplet loss, and distillation loss and gains improvement of 6.52% and 3.45% for mean average precision over ISIC-2017 and ISIC-2018. Another innovation in our methodology is a two-branch network strategy, which concurrently boosts image retrieval and classification. Through our experiments, we underscore the effectiveness and the pitfalls of diverse loss configurations in image retrieval. Furthermore, our approach underlines the advantages of retrieval-based classification through majority voting rather than relying solely on the classification head, leading to enhanced prediction for melanoma - the most lethal type of skin cancer. Our results surpass existing state-of-the-art techniques on the ISIC-2017 and ISIC-2018 datasets by improving mean average precision by 1.01% and 4.36% respectively, emphasizing the efficacy and promise of Vision Transformers paired with our tailor-made weighted loss function, especially in medical contexts. The proposed approach's effectiveness is substantiated through thorough ablation studies and an array of quantitative and qualitative outcomes. To promote reproducibility and support forthcoming research, our source code will be accessible on GitHub.

Advancing Content-Based Histopathological Image Retrieval Pre-Processing: A Comparative Analysis of the Effects of Color Normalization Techniques

Content-Based Histopathological Image Retrieval (CBHIR) is a search technique based on the visual content and histopathological features of whole-slide images (WSIs). CBHIR tools assist pathologists to obtain a faster and more accurate cancer diagnosis. Stain variation between hospitals hampers the performance of CBHIR tools. This paper explores the effects of color normalization (CN) in a recently proposed CBHIR approach to tackle this issue. In this paper, three different CN techniques were used on the CAMELYON17 (CAM17) data set, which is a breast cancer data set. CAM17 consists of images taken using different staining protocols and scanners in five hospitals. Our experiments reveal that a proper CN technique, which can transfer the color version into the most similar median values, has a positive impact on the retrieval performance of the proposed CBHIR framework. According to the obtained results, using CN as a pre-processing step can improve the accuracy of the proposed CBHIR framework to 97% (a 14% increase), compared to working with the original images.

Deep Learning based Smart Image Search Engine

Abstract: This paper introduces a new reverse search engine integration into content-based image retrieval (CBIR) systems that employs convolutional neural networks (CNNs) for feature extraction. It generates global descriptors using pre-trained CNN architectures such as ResNet50, InceptionV3, and InceptionResNetV2. It retrieves visually similar images without depending on linguistic annotations. Comparative analysis against existing methods, such as Gabor Wavelet, CNN-SVM, Metaheuristic Algorithm, etc., has been tested, and it proves the superiority of the proposed algorithm, the Cartoon Texture Algorithm, in CBIR. As the Internet sees an exponential growth of different data types, the importance of CBIR continues to grow. In order to efficiently retrieve images, solely relying on image features while ignoring metadata is exactly what we need. As such, this paper is a reminder of the need for CBIR in this changing world. They showed that CBIR continues to be quite effective in the age of the Internet. Their proposed model for CBIR, which integrates ResNet-50-based feature extraction, a neural network model trained on different image datasets, and clustering techniques to make retrieval fast, provides a significant improvement in accuracy and efficiency for content-dependent image retrieval. This methodology is likely to be very useful as we work with the increasingly huge data of vision and beyond on the Internet. It provides a good basis for an effective image search and retrieval system

GPU-Based Parallel Processing Techniques for Enhanced Brain Magnetic Resonance Imaging Analysis: A Review of Recent Advances.

The approach of using more than one processor to compute in order to overcome the complexity of different medical imaging methods that make up an overall job is known as GPU (graphic processing unit)-based parallel processing. It is extremely important for several medical imaging techniques such as image classification, object detection, image segmentation, registration, and content-based image retrieval, since the GPU-based parallel processing approach allows for time-efficient computation by a software, allowing multiple computations to be completed at once. On the other hand, a non-invasive imaging technology that may depict the shape of an anatomy and the biological advancements of the human body is known as magnetic resonance imaging (MRI). Implementing GPU-based parallel processing approaches in brain MRI analysis with medical imaging techniques might be helpful in achieving immediate and timely image capture. Therefore, this extended review (the extension of the IWBBIO2023 conference paper) offers a thorough overview of the literature with an emphasis on the expanding use of GPU-based parallel processing methods for the medical analysis of brain MRIs with the imaging techniques mentioned above, given the need for quicker computation to acquire early and real-time feedback in medicine. Between 2019 and 2023, we examined the articles in the literature matrix that include the tasks, techniques, MRI sequences, and processing results. As a result, the methods discussed in this review demonstrate the advancements achieved until now in minimizing computing runtime as well as the obstacles and problems still to be solved in the future.

Interactive CBIR System for Various Similarity Metrics Based on Colour Content of Image

Interactive CBIR System for Various Similarity Metrics Based on Colour Content of Image

Evaluation of retrieval accuracy and visual similarity in content-based image retrieval of chest CT for obstructive lung disease

The aim of our study was to assess the performance of content-based image retrieval (CBIR) for similar chest computed tomography (CT) in obstructive lung disease. This retrospective study included patients with obstructive lung disease who underwent volumetric chest CT scans. The CBIR database included 600 chest CT scans from 541 patients. To assess the system performance, follow-up chest CT scans of 50 patients were evaluated as query cases, which showed the stability of the CT findings between baseline and follow-up chest CT, as confirmed by thoracic radiologists. The CBIR system retrieved the top five similar CT scans for each query case from the database by quantifying and comparing emphysema extent and size, airway wall thickness, and peripheral pulmonary vasculatures in descending order from the database. The rates of retrieval of the same pairs of query CT scans in the top 1–5 retrievals were assessed. Two expert chest radiologists evaluated the visual similarities between the query and retrieved CT scans using a five-point scale grading system. The rates of retrieving the same pairs of query CTs were 60.0% (30/50) and 68.0% (34/50) for top-three and top-five retrievals. Radiologists rated 64.8% (95% confidence interval 58.8–70.4) of the retrieved CT scans with a visual similarity score of four or five and at least one case scored five points in 74% (74/100) of all query cases. The proposed CBIR system for obstructive lung disease integrating quantitative CT measures demonstrated potential for retrieving chest CT scans with similar imaging phenotypes. Further refinement and validation in this field would be valuable.

3D Gaussian Geometric Moment Invariants

ABSTRACT 3D moment invariants are important tools for 3D image feature representation. In this paper, we introduced a novel approach for constructing 3D moment invariants using Gaussian geometric moments. Our proposed method demonstrated invariance under translation, rotation, and scale transformations. The numerical experiments validate the invariance and robustness of the proposed method, comparing it with traditional 3D geometric moments and revealing superior performance in the presence of noise and transformations. Additionally, the method is applied to content-based 3D image retrieval, exhibiting promising results through Minkowski distance-based retrieval on the Princeton Shape Benchmark (PSB) database.

CBIR-ACHS: compressed domain content-based image retrieval through auto-correloblock in HEVC standard

CBIR-ACHS: compressed domain content-based image retrieval through auto-correloblock in HEVC standard

Interactive content-based image retrieval with deep learning for CT abdominal organ recognition

Objective. Recognizing the most relevant seven organs in an abdominal computed tomography (CT) slice requires sophisticated knowledge. This study proposed automatically extracting relevant features and applying them in a content-based image retrieval (CBIR) system to provide similar evidence for clinical use. Approach. A total of 2827 abdominal CT slices, including 638 liver, 450 stomach, 229 pancreas, 442 spleen, 362 right kidney, 424 left kidney and 282 gallbladder tissues, were collected to evaluate the proposed CBIR in the present study. Upon fine-tuning, high-level features used to automatically interpret the differences among the seven organs were extracted via deep learning architectures, including DenseNet, Vision Transformer (ViT), and Swin Transformer v2 (SwinViT). Three images with different annotations were employed in the classification and query. Main results. The resulting performances included the classification accuracy (94%–99%) and retrieval result (0.98–0.99). Considering global features and multiple resolutions, SwinViT performed better than ViT. ViT also benefited from a better receptive field to outperform DenseNet. Additionally, the use of hole images can obtain almost perfect results regardless of which deep learning architectures are used. Significance. The experiment showed that using pretrained deep learning architectures and fine-tuning with enough data can achieve successful recognition of seven abdominal organs. The CBIR system can provide more convincing evidence for recognizing abdominal organs via similarity measurements, which could lead to additional possibilities in clinical practice.

Detecting microsatellite instability in colorectal cancer using Transformer-based colonoscopy image classification and retrieval.

Colorectal cancer (CRC) is a major global health concern, with microsatellite instability-high (MSI-H) being a defining characteristic of hereditary nonpolyposis colorectal cancer syndrome and affecting 15% of sporadic CRCs. Tumors with MSI-H have unique features and better prognosis compared to MSI-L and microsatellite stable (MSS) tumors. This study proposed establishing a MSI prediction model using more available and low-cost colonoscopy images instead of histopathology. The experiment utilized a database of 427 MSI-H and 1590 MSS colonoscopy images and vision Transformer (ViT) with different feature training approaches to establish the MSI prediction model. The accuracy of combining pre-trained ViT features was 84% with an area under the receiver operating characteristic curve of 0.86, which was better than that of DenseNet201 (80%, 0.80) in the experiment with support vector machine. The content-based image retrieval (CBIR) approach showed that ViT features can obtain a mean average precision of 0.81 compared to 0.79 of DenseNet201. ViT reduced the issues that occur in convolutional neural networks, including limited receptive field and gradient disappearance, and may be better at interpreting diagnostic information around tumors and surrounding tissues. By using CBIR, the presentation of similar images with the same MSI status would provide more convincing deep learning suggestions for clinical use.

Multimedia information retrieval using content-based image retrieval and context link for Chinese cultural artifacts

PurposeAn increasing number of images are generated daily, and images are gradually becoming a search target. Content-based image retrieval (CBIR) is helpful for users to express their requirements using an image query. Nevertheless, determining whether the retrieval system can provide convenient operation and relevant retrieval results is challenging. A CBIR system based on deep learning features was proposed in this study to effectively search and navigate images in digital articles.Design/methodology/approachConvolutional neural networks (CNNs) were used as the feature extractors in the author's experiments. Using pretrained parameters, the training time and retrieval time were reduced. Different CNN features were extracted from the constructed image databases consisting of images taken from the National Palace Museum Journals Archive and were compared in the CBIR system.FindingsDenseNet201 achieved the best performance, with a top-10 mAP of 89% and a query time of 0.14 s.Practical implicationsThe CBIR homepage displayed image categories showing the content of the database and provided the default query images. After retrieval, the result showed the metadata of the retrieved images and links back to the original pages.Originality/valueWith the interface and retrieval demonstration, a novel image-based reading mode can be established via the CBIR and links to the original images and contextual descriptions.

Self-attention-driven retrieval of chest CT images for COVID-19 assessment

Numerous methods have been developed for computer-aided diagnosis (CAD) of coronavirus disease-19 (COVID-19), based on chest computed tomography (CT) images. The majority of these methods are based on deep neural networks and often act as “black boxes” that cannot easily gain the trust of medical community, whereas their result is uniformly influenced by all image regions. This work introduces a novel, self-attention-driven method for content-based image retrieval (CBIR) of chest CT images. The proposed method analyzes a query CT image and returns a classification result, as well as a list of classified images, ranked according to similarity with the query. Each CT image is accompanied by a heatmap, which is derived by gradient-weighted class activation mapping (Grad-CAM) and represents the contribution of lung tissue and lesions to COVID-19 pathology. Beyond visualization, Grad-CAM weights are employed in a self-attention mechanism, in order to strengthen the influence of the most COVID-19-related image regions on the retrieval result. Experiments on two publicly available datasets demonstrate that the binary classification accuracy obtained by means of DenseNet-201 is 81.3% and 96.4%, for COVID-CT and SARS-CoV-2 datasets, respectively, with a false negative rate which is less than 3% in both datasets. In addition, the Grad-CAM-guided CBIR framework slightly outperforms the plain CBIR in most cases, with respect to nearest neighbour (NN) and first four (FF). The proposed method could serve as a computational tool for a more transparent decision-making process that could be trusted by the medical community. In addition, the employed self-attention mechanism increases the obtained retrieval performance.

Retracted: Content-Based Image Retrieval Using Gamma Distribution and Mixture Model

Retracted: Content-Based Image Retrieval Using Gamma Distribution and Mixture Model

Content-based image retrieval using multi-scale averaging local binary patterns

Feature extraction has a significant impact on the accuracy of Content-Based Image Retrieval (CBIR) algorithms since the content of images is encoded in feature vectors. In this paper, an effective method for texture feature extraction is proposed based on local patterns. In the proposed method, first the image is formed in different scales, then the texture features are extracted from the scales of the image. Finally, extracted features of different scales are concatenated to construct the final feature vector. To evaluate the proposed method, five datasets including Corel-1k, Brodatz, VisTex, Corel-10k, STex, Caltech256, and Oliva are used. In the evaluation process, the effect of different scales and different coding schemes on retrieval precision is investigated. The proposed method is compared with existing CBIR models based on local patterns. The proposed method achieves the best precision of 64.16%, 81.66%, 88.59%, 30.63%, and 69.63%, 13.59%, and 64.10% on the Corel-1k, Brodatz, VisTex, Corel-10k, STex, Caltech256, and Oliva datasets, respectively.

Exploring deep learning-based content-based video retrieval with Hierarchical Navigable Small World index and ResNet-50 features for anomaly detection

Surveillance video can capture a variety of realistic scenarios for providing security in both private and public spaces. The prediction of anomalies promptly is essential. However, existing methods are time-consuming due to marking conflicting clips in training videos. Therefore, this paper proposes a CBVR (Content-Based Video Retrieval) framework for detecting anomalies in surveillance videos. The proposed method leverages deep learning techniques and the HNSW (Hierarchical Navigable Small World) index to achieve efficient and accurate video retrieval. To preprocess the video data, frames are resized and normalized, and the dataset is augmented with data transformations such as rotations, translations, and flips to improve model generalization. The best results are offered while evaluating with the UCSD Ped2 dataset which produces the MAP score of 94.25%, and recall, precision, and F1-scores of 96.15%, 98.03%, and 92.63% respectively. The results proved that maximizing the number of layers in the HNSW index enhances the MAP score, suggesting that deeper layers are more effective at retrieving relevant results. Overall, the proposed CBVR framework provides an efficient and accurate method for anomaly detection in surveillance videos. It has potential applications in various domains, including video summarization, recommendation, and surveillance, among others.

Object Recognition to Content Based Image Retrieval: A Study of the Developments and Applications of Computer Vision

Natural Language Processing (NLP) and Computer Vision (CV) are interconnected fields within the domain of Artificial Intelligence (AI). CV is tasked with the process of engaging with computer systems to effectively interpret and recognize visual data, while NLP is responsible for comprehending and processing the human voice. The two fields have practical applicability in various tasks such as image description generation, object recognition, and question-based answering after a visual input. Deep learning algorithms such as word input are typically employed in enhancing the performance of Content-Based Image Processing (CBIR) techniques. Generally, NLP and CV play a vital role in enhancing computer comprehension and engagements with both visual and written information. This paper seeks to review various major elements of computer vision, such as CBIR, visual effects, image documentation, video documentation, visual learning, and inquiry to explore various databases, techniques, and methods employed in this field. The authors focus on the challenges and progress in each area and offer new strategies for improving the performance of CV systems.

Content Based Image Retrieval System Using CNN based Deep Learning Models

Digital information search archives on the internet and other platforms have been an essential part of our daily lives. With the use of huge amounts of digital images, it is paramount to develop efficient solutions to retrieve images seamlessly. Due to inadequate textual description, the task of automated image retrieval is quite complicated. Therefore, retrieval of images by analysing their visual content is an exciting task and research-worthy. Text based Image Search (TIBR) is time consuming as initially, every image has to be linked with a keyword or text to be extracted on query search. In the proposed system, an efficient algorithm for Content Based Image Retrieval (CBIR) using pre-trained CNN-based Deep Learning models to extract deep features of an image has been developed, which significantly increases the performance of the image retrieval process. Convolutional Neural Networks (CNN) has evolved as an efficient deep learning solution for the CBIR systems that work in image recognition applications. The results of retrieval are evaluated in terms of precision performance parameters. Emphasis has been given to the comparison of the efficiency of the system using two different pre-trained CNN-based Deep Learning models based on VGG16 and ResNet-50 architecture. Transfer learning enables us to develop robust and efficient working systems using pre-trained models. In the proposed system, pre-trained models have been used to derive features using a deep learning CNN network trained for a large image classification dataset ImageNet. This approach outperforms many contemporary CBIR systems. As most CBIR systems depend on query images and as most users of such systems are non-professional, we have developed a user interface for the built prototype systems. The efficiency of the CBIR systems with pre-trained models VGG16, and ResNet-50 are compared for better image retrieval and reliability. The proposed system has addressed a few existing problems in CBIR system design and developed a suitable solution with less semantic gap and better efficiency in image retrieval.