Content-based Image Retrieval Algorithms Research Articles

Content-based image retrieval using multi-scale averaging local binary patterns

Feature extraction has a significant impact on the accuracy of Content-Based Image Retrieval (CBIR) algorithms since the content of images is encoded in feature vectors. In this paper, an effective method for texture feature extraction is proposed based on local patterns. In the proposed method, first the image is formed in different scales, then the texture features are extracted from the scales of the image. Finally, extracted features of different scales are concatenated to construct the final feature vector. To evaluate the proposed method, five datasets including Corel-1k, Brodatz, VisTex, Corel-10k, STex, Caltech256, and Oliva are used. In the evaluation process, the effect of different scales and different coding schemes on retrieval precision is investigated. The proposed method is compared with existing CBIR models based on local patterns. The proposed method achieves the best precision of 64.16%, 81.66%, 88.59%, 30.63%, and 69.63%, 13.59%, and 64.10% on the Corel-1k, Brodatz, VisTex, Corel-10k, STex, Caltech256, and Oliva datasets, respectively.

Saliency-Enhanced Content-Based Image Retrieval for Diagnosis Support in Dermatology Consultation: Reader Study.

Previous research studies have demonstrated that medical content image retrieval can play an important role by assisting dermatologists in skin lesion diagnosis. However, current state-of-the-art approaches have not been adopted in routine consultation, partly due to the lack of interpretability limiting trust by clinical users. This study developed a new image retrieval architecture for polarized or dermoscopic imaging guided by interpretable saliency maps. This approach provides better feature extraction, leading to better quantitative retrieval performance as well as providing interpretability for an eventual real-world implementation. Content-based image retrieval (CBIR) algorithms rely on the comparison of image features embedded by convolutional neural network (CNN) against a labeled data set. Saliency maps are computer vision-interpretable methods that highlight the most relevant regions for the prediction made by a neural network. By introducing a fine-tuning stage that includes saliency maps to guide feature extraction, the accuracy of image retrieval is optimized. We refer to this approach as saliency-enhanced CBIR (SE-CBIR). A reader study was designed at the University Hospital Zurich Dermatology Clinic to evaluate SE-CBIR's retrieval accuracy as well as the impact of the participant's confidence on the diagnosis. SE-CBIR improved the retrieval accuracy by 7% (77% vs 84%) when doing single-lesion retrieval against traditional CBIR. The reader study showed an overall increase in classification accuracy of 22% (62% vs 84%) when the participant is provided with SE-CBIR retrieved images. In addition, the overall confidence in the lesion's diagnosis increased by 24%. Finally, the use of SE-CBIR as a support tool helped the participants reduce the number of nonmelanoma lesions previously diagnosed as melanoma (overdiagnosis) by 53%. SE-CBIR presents better retrieval accuracy compared to traditional CBIR CNN-based approaches. Furthermore, we have shown how these support tools can help dermatologists and residents improve diagnosis accuracy and confidence. Additionally, by introducing interpretable methods, we should expect increased acceptance and use of these tools in routine consultation.

Content-Based Image Retrieval using Hard Voting Ensemble Method of Inception, Xception, and Mobilenet Architectures

Advancements in internet accessibility and the affordability of digital picture sensors have led to the proliferation of extensive image databases utilized across a multitude of applications. Addressing the semantic gap between low-level attributes and human visual perception has become pivotal in refining Content-Based Image Retrieval (CBIR) methodologies, especially within this context. As this field is intensely researched, numerous efficient algorithms for CBIR systems have surfaced, precipitating significant progress in the artificial intelligence field. In this study, we propose employing a hard voting ensemble approach on features derived from three robust deep learning architectures: Inception, Exception, and Mobilenet. This is aimed at bridging the divide between low-level image features and human visual perception. The Euclidean method is adopted to determine the similarity metric between the query image and the features database. The outcome was a noticeable improvement in image retrieval accuracy. We applied our approach to a practical dataset named CBIR 50, which encompasses categories such as mobile phones, cars, cameras, and cats. The effectiveness of our method was thereby validated. Our approach outshone existing CBIR algorithms with superior accuracy (ACC), precision (PREC), recall (REC), and F1-score (F1-S), proving to be a noteworthy addition to the field of CBIR. Our proposed methodology could be potentially extended to various other sectors, including medical imaging and surveillance systems, where image retrieval accuracy is of paramount importance.

An Overview of Content-Based Image Retrieval Methods and Techniques

With the development of Internet technology and the popularity of digital devices, Content-Based Image Retrieval (CBIR) has been quickly developed and applied in various fields related to computer vision and artificial intelligence. Currently, it is possible to retrieve related images effectively and efficiently from a large-scale database with an input image. In the past ten years, great efforts have been made for new theories and models of CBIR, and many effective CBIR algorithms have been established. Content-based image retrieval helps to discover identical images in a big dataset that match a query image. The query image's representative feature similarities to the dataset images typically assist in ranking the images for retrieval. There are various past studies on different handicraft feature descriptors according to the visual features that describe the images: color, texture, and shape. However, deep learning has been the dominant alternative to manually planned feature engineering; it automatically takes the features from the data. The current work reviews recent advancements in content-based image retrieval. For a deeper understanding of the advancement, the explanation of current state-of-the-art approaches from various vantage points is also conducted. This review employs a taxonomy encompassing various retrieval networks, classification types, and descriptors and this study will help researchers make more progress in image retrieval

A Hybrid Filtering Technique of Digital Images in Multimedia Data Warehouses

The similarity search approach used for image Data Warehouse (DW) can provide better insights into discovering the most similar images compared to the input query. Due to the later innovation improvement, the mixed media complexity is discernibly expanded and modern inquires about regions are opened depending on comparable mixed media substance recovery. Content-Based Image Retrieval (CBIR) algorithms are utilized for the retrieval of images related to the inquiry image from gigantic databases or DW. The queries that are used for DW are complex, take a lot of time to process and many give less accurate results. For these reasons, this paper needs to have an effective technique to improve the similarity search query process that reflects a more positive result. In this paper, show how to extract features from a set of images (color, shape, and texture features) by using CBIR algorithm with Color Edge Detection (CED) method. Once these features are extracted, the proposed method will minimize the distance between these features vectors and the query image one using a Genetic Algorithm (GA). This paper illustrates the extraction of endless strong and imperative features from the database of the images, therefore, the capacity of these features in storing within the frame of features vectors. Accordingly, an imaginative closeness assessment with a metaheuristic algorithm (Genetic Algorithm (GA) with Simulating Annealing (SA)) has been attained between the query image features and those having a place in the database image. This paper introduces a new algorithm CEDF (Color Edge Detection with Gaussian Blur Filter) that applies the Gaussian Blur Filter after using CED method for feature detection of the image. Experimental results show that CEDF method gives better result than the other already-known methods.

Deep features optimization based on a transfer learning, genetic algorithm, and extreme learning machine for robust content-based image retrieval.

The recent era has witnessed exponential growth in the production of multimedia data which initiates exploration and expansion of certain domains that will have an overwhelming impact on human society in near future. One of the domains explored in this article is content-based image retrieval (CBIR), in which images are mostly encoded using hand-crafted approaches that employ different descriptors and their fusions. Although utilization of these approaches has yielded outstanding results, their performance in terms of a semantic gap, computational cost, and appropriate fusion based on problem domain is still debatable. In this article, a novel CBIR method is proposed which is based on the transfer learning-based visual geometry group (VGG-19) method, genetic algorithm (GA), and extreme learning machine (ELM) classifier. In the proposed method, instead of using hand-crafted features extraction approaches, features are extracted automatically using a transfer learning-based VGG-19 model to consider both local and global information of an image for robust image retrieval. As deep features are of high dimension, the proposed method reduces the computational expense by passing the extracted features through GA which returns a reduced set of optimal features. For image classification, an extreme learning machine classifier is incorporated which is much simpler in terms of parameter tuning and learning time as compared to other traditional classifiers. The performance of the proposed method is evaluated on five datasets which highlight the better performance in terms of evaluation metrics as compared with the state-of-the-art image retrieval methods. Its statistical analysis through a nonparametric Wilcoxon matched-pairs signed-rank test also exhibits significant performance.

Improve Content-based Image Retrieval using Deep learning model

The complexity of multimedia has expanded dramatically as a result of recent technology breakthroughs, and retrieval of similar multimedia material remains an ongoing research topic. Content-based image retrieval (CBIR) systems search huge databases for pictures that are related to the query image (QI). Existing CBIR algorithms extract just a subset of feature sets, limiting retrieval efficacy. The sorting of photos with a high degree of visual similarity is a necessary step in any image retrieval technique. Because a single feature is not resilient to image datasets modifications, feature combining, also known as feature fusion, is employed in CBIR to increase performance. This work describes a CBIR system in which combining DarkNet-19 and DarkNet-53 information to retrieve images. Experiments on the Wang (Corel 1K) database reveal a considerable improvement in precision over state-of-the-art classic techniques as well as Deep Convolutional Neural Network(DCNN).

Impact of Multi-Feature Extraction on Image Retrieval and classification Using Machine Learning Technique

Vast information generated due to the web requires proper mechanisms and tools for efficient management and retrieval of images. This led to the development of an efficient image retrieval system. Image retrieval is the process of retrieving relevant images from a large database. Text-Based Image Retrieval (TBIR) and Content-Based Image Retrieval (CBIR) are the two efficient techniques for image retrieval. It has become an active area of research for image retrieval. Most of the systems are designed with three basic phases in CBIR, i.e. feature extraction, feature selection and similarity matching. Feature extraction can be done based on the colour, shape or texture of the image. The studies conducted so far involve CBIR systems which utilize individual features or only two or more features combined for feature extraction by applying different feature extraction and CBIR techniques. The proposed system illustrates a comparison between the performance of the existing CBIR algorithm with multi-feature extraction using discrete wavelet transform, edge histogram descriptor, sobel operator, moment invariant, histogram-oriented gradient, local binary pattern and classification using Support Vector Machine (SVM) based on the accuracy measures—precision and recall. The experimental results show that multi-feature extraction techniques give better results in some of the individual categories but retrieval results using machine learning is better with respect to average accuracy.

Recent developments of content-based image retrieval (CBIR)

With the development of Internet technology and the popularity of digital devices, Content-Based Image Retrieval (CBIR) has been quickly developed and applied in various fields related to computer vision and artificial intelligence. Currently, it is possible to retrieve related images effectively and efficiently from a large scale database with an input image. In the past ten years, great efforts have been made for new theories and models of CBIR and many effective CBIR algorithms have been established. In this paper, we present a survey on the fast developments and applications of CBIR theories and algorithms during the period from 2009 to 2019. We mainly review the technological developments from the viewpoint of image representation and database search. We further summarize the practical applications of CBIR in the fields of fashion image retrieval, person re-identification, e-commerce product retrieval, remote sensing image retrieval and trademark image retrieval. Finally, we discuss the future research directions of CBIR with the challenge of big data and the utilization of deep learning techniques.

Content-based image retrieval algorithm for nuclei segmentation in histopathology images

In today’s world, the medical diagnostic system shows a high reliance on medical imagery and digital nosology. To facilitate the fast and precise screening of samples, technology is leading towards the computer-aided disease diagnosis and grading. Image segmentation possesses high worth in the computer-aided disease diagnosis and grading systems to extract the region of interest. This paper presents a content-based image retrieval algorithm for histopathology image segmentation for identification and extraction of nuclei. The proposed technique furnishes nuclei segmentation in three cascaded stages; pre-processing, nuclei points and region refining, and composite nuclei segmentation. The performance of nuclei segmentation is investigated on six hematoxylins and eosin (H&E) stained histopathology images datasets. Simulation outcomes of the segmentation schemes confirm the superiority of the proposed method for nuclei segmentation in histopathology images in qualitative and quantitative analysis.

Textural and color descriptor fusion for efficient content-based image retrieval algorithm

In communication, medium images are used for various applications such as social websites, education, bio-medical field and industrial applications. Indexing and retrieval of such large image database pose a big problem. The content-based image retrieval (CBIR) approaches are used to select information from the input images using feature descriptors. In this article, CBIR algorithm is proposed with fusion of color and texture descriptors such as binary Gabor pattern (BGP), segmentation based fractal texture analysis (SFTA), edge histogram descriptor (EHD), Gabor wavelet texture features and fuzzy histogram-based descriptors. Laplacian score has employed to reduce feature vector dimensions. Retrieval rate is computed using different distance metrics. Experimentation was performed with ten different classes of 1000 images using Wang database. The simulation results indicate improvement in terms of precision compared with the existing techniques.

Using content-based image retrieval of dermoscopic images for interpretation and education: A pilot study.

Dermoscopic content-based image retrieval (CBIR) systems provide a set of visually similar dermoscopic (magnified and illuminated) skinimages with a pathology-confirmed diagnosis for a given dermoscopic query image of a skin lesion. Although recent advances in machine learning have spurred novel CBIR algorithms, we have few insights into how end users interact with CBIRs and to what extent CBIRs can be useful for education and image interpretation. We developed an interactive user interface for a CBIR system with dermoscopic images as a decision support tool and investigated users' interactions and decisions with the system. We performed a pilot experiment with 14 non-medically trained users for a given set of annotated dermoscopic images. Our pilot showed that the number of correct classifications and users' confidence levels significantly increased with the CBIR interface compared with a non-CBIR interface, although the timing also increased significantly. The users found the CBIR interface of high educational value, engaging and easy to use. Overall, users became more accurate, found the CBIR approach provided a useful decision aid, and had educational value for learning about skin conditions.

Content-based image retrieval using local texture features in distributed environment

In this paper, we propose novel content-based image retrieval (CBIR) algorithms using Local Octa Patterns (LOtP), Local Hexadeca Patterns (LHdP) and Direction Encoded Local Binary Pattern (DELBP). LOtP and LHdP encode the relationship between center pixel and its neighbors based on the pixels’ direction obtained by considering the horizontal, vertical and diagonal pixels for derivative calculations. In DELBP, direction of a referenced pixel is determined by considering every neighboring pixel for derivative calculations which results in 256 directions. For this resultant direction encoded image, we have obtained LBP which is considered as feature vector. The proposed method’s performance is compared to that of Local Tetra Patterns (LTrP) using benchmark image databases viz., Corel 1000 (DB1) and Brodatz textures (DB2). Performance analysis shows that LOtP improves the average precision from 59.31% to 64.36% on DB1, and from 83.24% to 85.95% on DB2, LHdP improves it to 65.82% on DB1 and to 87.49% on DB2 and DELBP improves it to 60.35% on DB1 and to 86.12% on DB2 as compared to that of LTrP. Also, DELBP reduces the feature vector length by 66.62% as compared to that of LTrP. To reduce the retrieval time, the proposed algorithms are implemented on a Hadoop cluster consisting of 116 nodes and tested using Corel 10K (DB3), Mirflickr 100,000 (DB4) and ImageNet 511,380 (DB5) databases.

Multi-objective whale optimization algorithm for content-based image retrieval

In the recent years, there are massive digital images collections in many fields of our life, which led the technology to find methods to search and retrieve these images efficiently. The content-based is one of the popular methods used to retrieve images, which depends on the color, texture and shape descriptors to extract features from images. However, the performance of the content-based image retrieval methods depends on the size of features that are extracted from images and the classification accuracy. Therefore, this problem is considered as a multi-objective and there are several methods that used to manipulate it such as NSGA-II and NSMOPSO. However, these methods have drawbacks such as their time and space complexity are large since they used traditional non-dominated sorting methods. In this paper, a new non-dominated sorting based on multi-objective whale optimization algorithm is proposed for content-based image retrieval (NSMOWOA). The proposed method avoids the drawbacks in other non-dominated sorting multi-objective methods that have been used for content-based image retrieval through reducing the space and time complexity. The results of the NSMOWOA showed a good performance in content-based image retrieval problem in terms of recall and precision.

Multidimensional Markov Stationary Feature for Image Retrival Systems

Markov Stationary Features (MSF) not only considers the distribution of colors like histogram method does, also characterizes the spatial co-occurrence of histogram patterns. However, handling large scale database of images, simple MSF method is not sufficient to discriminate the images. In this paper, we have proposed a robust content based image retrieval algorithm that enhances the discriminating capability of the original MSF. The proposed Multidimensional MSF (MMSF) algorithm extends the MSF by generating multiple co-occurrence matrices with different quantization levels of an image. Publicly available WANG1000 and Corel10800 databases are used to evaluate the performance of the proposed algorithm. The experimental result justifies the effectiveness of the proposed method.

Scalable analysis of Big pathology image data cohorts using efficient methods and high-performance computing strategies.

BackgroundWe describe a suite of tools and methods that form a core set of capabilities for researchers and clinical investigators to evaluate multiple analytical pipelines and quantify sensitivity and variability of the results while conducting large-scale studies in investigative pathology and oncology. The overarching objective of the current investigation is to address the challenges of large data sizes and high computational demands.ResultsThe proposed tools and methods take advantage of state-of-the-art parallel machines and efficient content-based image searching strategies. The content based image retrieval (CBIR) algorithms can quickly detect and retrieve image patches similar to a query patch using a hierarchical analysis approach. The analysis component based on high performance computing can carry out consensus clustering on 500,000 data points using a large shared memory system.ConclusionsOur work demonstrates efficient CBIR algorithms and high performance computing can be leveraged for efficient analysis of large microscopy images to meet the challenges of clinically salient applications in pathology. These technologies enable researchers and clinical investigators to make more effective use of the rich informational content contained within digitized microscopy specimens.

Improved cuckoo search with particle swarm optimization for classification of compressed images

The need for a general purpose Content Based Image Retrieval (CBIR) system for huge image databases has attracted information-technology researchers and institutions for CBIR techniques development. These techniques include image feature extraction, segmentation, feature mapping, representation, semantics, indexing and storage, image similarity-distance measurement and retrieval making CBIR system development a challenge. Since medical images are large in size running to megabits of data they are compressed to reduce their size for storage and transmission. This paper investigates medical image retrieval problem for compressed images. An improved image classification algorithm for CBIR is proposed. In the proposed method, RAW images are compressed using Haar wavelet. Features are extracted using Gabor filter and Sobel edge detector. The extracted features are classified using Partial Recurrent Neural Network (PRNN). Since training parameters in Neural Network are NP hard, a hybrid Particle Swarm Optimization (PSO) – Cuckoo Search algorithm (CS) is proposed to optimize the learning rate of the neural network.

A comparative study using contours and skeletons as shape representations for binary image matching

Contours and skeletons are well-known shape representations that embody visual information by using a limited set of object points. Both representations have been applied in various pattern recognition applications, while studies in cognitive science have investigated their roles in human perception. Despite their importance has been shown in the above-mentioned fields, to our knowledge no existing studies have been conducted to compare their performances. Filling this gap, this paper is an empirical study of these two shape representations by comparing their performances over different binary image categories and variations. The image categories include thick, elongated, and nearly thin images. Image variations include addition of noise to the contours, blurring, and size reduction. The comparative evaluation is achieved by resorting to object classification (OC) and content-based image retrieval (CBIR) algorithms and evaluation metrics. The main findings highlight the superiority of contours but the improvements observed when skeletons are used for images with noisy contours.

Content-based Image Retrieval using Image Partitioning with Color Histogram and Wavelet-based Color Histogram of the Image

paper presents two content-based image retrieval algorithms that are based on image partitioning. The retrieval in the first algorithm is based only on the image color feature represented by the color histogram, while the retrieval in the second one is based on the image color and texture features represented by the color histogram and Haar wavelet transform, respectively. In these algorithms, each image in the database and the query image are divided into 4-equal sized blocks. Color and texture features are extracted for each block. Distances between the blocks of the query image and the blocks of a database image are calculated, then, the similarity between the query image and the database image is calculated by finding the minimum cost matching based on most similar highest priority (MSHP) principle. A CBIR system that implements the proposed algorithms has been developed. To evaluate the effectiveness of the proposed algorithms, experiments have been carried out using different color quantization schemes for three different color spaces (HSV, YIQ and YCbCr) with two similarity measures, namely the Histogram Euclidean Distance and Histogram Intersection Distance. The WANG image database, which contains 1000 general-purpose color images, has been used in the experiments.

A Visual Analytics Approach Using the Exploration of Multidimensional Feature Spaces for Content-Based Medical Image Retrieval.

Content-based image retrieval (CBIR) is a search technique based on the similarity of visual features and has demonstrated potential benefits for medical diagnosis, education, and research. However, clinical adoption of CBIR is partially hindered by the difference between the computed image similarity and the user's search intent, the semantic gap, with the end result that relevant images with outlier features may not be retrieved. Furthermore, most CBIR algorithms do not provide intuitive explanations as to why the retrieved images were considered similar to the query (e.g., which subset of features were similar), hence, it is difficult for users to verify if relevant images, with a small subset of outlier features, were missed. Users, therefore, resort to examining irrelevant images and there are limited opportunities to discover these "missed" images. In this paper, we propose a new approach to medical CBIR by enabling a guided visual exploration of the search space through a tool, called visual analytics for medical image retrieval (VAMIR). The visual analytics approach facilitates interactive exploration of the entire dataset using the query image as a point-of-reference. We conducted a user study and several case studies to demonstrate the capabilities of VAMIR in the retrieval of computed tomography images and multimodality positron emission tomography and computed tomography images.