Content-based Image Retrieval Approach Research Articles

Heritage Iconographic Content Structuring: from Automatic Linking to Visual Validation

This article presents a global framework dedicated to the structuring of iconographic heritage collections. To alleviate the poor interlinking both between collections and contents, a first step of automatic linking exploiting content-based image retrieval approaches is evaluated and adapted to the visual variability of such heritage contents. To ensure understanding and analysis of the contents in a structured fashion, a 3D immersive web platform is also introduced alongside visual-based analysis tools. Finally, by exploiting both automatic linking and manual interventions in the visualization platform, an iterative, semi-automatic structuring pipeline is proposed to solve difficult cases missed by automatic structuring, and then improve structuring optimally. Here, we demonstrate the potential of the proposal on the geographic iconographic heritage of Paris, with a dataset of 10k images belonging to several institutions, thus poorly connected nor organized globally.

Neuromorphic computing spiking neural network edge detection model for content based image retrieval

ABSTRACT In contemporary times, content-based image retrieval (CBIR) techniques have gained widespread acceptance as a means for end-users to discern and extract specific image content from vast repositories. However, it is noteworthy that a substantial majority of CBIR studies continue to rely on linear methodologies such as gradient-based and derivative-based edge detection techniques. This research explores the integration of bioinspired Spiking Neural Network (SNN) based edge detection within CBIR. We introduce an innovative, computationally efficient SNN-based approach designed explicitly for CBIR applications, outperforming existing SNN models by reducing computational overhead by 2.5 times. The proposed SNN-based edge detection approach is seamlessly incorporated into three distinct CBIR techniques, each employing conventional edge detection methodologies including Sobel, Canny, and image derivatives. Rigorous experimentation and evaluations are carried out utilizing the Corel-10k dataset and crop weed dataset, a widely recognized and frequently adopted benchmark dataset in the realm of image analysis. Importantly, our findings underscore the enhanced performance of CBIR methodologies integrating the proposed SNN-based edge detection approach, with an average increase in mean precision values exceeding 3%. This study conclusively demonstrated the utility of our proposed methodology in optimizing feature extraction, thereby establishing its pivotal role in advancing edge centric CBIR approaches.

Detecting microsatellite instability in colorectal cancer using Transformer-based colonoscopy image classification and retrieval.

Colorectal cancer (CRC) is a major global health concern, with microsatellite instability-high (MSI-H) being a defining characteristic of hereditary nonpolyposis colorectal cancer syndrome and affecting 15% of sporadic CRCs. Tumors with MSI-H have unique features and better prognosis compared to MSI-L and microsatellite stable (MSS) tumors. This study proposed establishing a MSI prediction model using more available and low-cost colonoscopy images instead of histopathology. The experiment utilized a database of 427 MSI-H and 1590 MSS colonoscopy images and vision Transformer (ViT) with different feature training approaches to establish the MSI prediction model. The accuracy of combining pre-trained ViT features was 84% with an area under the receiver operating characteristic curve of 0.86, which was better than that of DenseNet201 (80%, 0.80) in the experiment with support vector machine. The content-based image retrieval (CBIR) approach showed that ViT features can obtain a mean average precision of 0.81 compared to 0.79 of DenseNet201. ViT reduced the issues that occur in convolutional neural networks, including limited receptive field and gradient disappearance, and may be better at interpreting diagnostic information around tumors and surrounding tissues. By using CBIR, the presentation of similar images with the same MSI status would provide more convincing deep learning suggestions for clinical use.

SBHA: Sensitive Binary Hashing Autoencoder for Image Retrieval.

Binary hashing is an effective approach for content-based image retrieval, and learning binary codes with neural networks has attracted increasing attention in recent years. However, the training of hashing neural networks is difficult due to the binary constraint on hash codes. In addition, neural networks are easily affected by input data with small perturbations. Therefore, a sensitive binary hashing autoencoder (SBHA) is proposed to handle these challenges by introducing stochastic sensitivity for image retrieval. SBHA extracts meaningful features from original inputs and maps them onto a binary space to obtain binary hash codes directly. Different from ordinary autoencoders, SBHA is trained by minimizing the reconstruction error, the stochastic sensitive error, and the binary constraint error simultaneously. SBHA reduces output sensitivity to unseen samples with small perturbations from training samples by minimizing the stochastic sensitive error, which helps to learn more robust features. Moreover, SBHA is trained with a binary constraint and outputs binary codes directly. To tackle the difficulty of optimization with the binary constraint, we train the SBHA with alternating optimization. Experimental results on three benchmark datasets show that SBHA is competitive and significantly outperforms state-of-the-art methods for binary hashing.

Manta Ray Foraging Optimizer with Deep Learning-based Fundus Image Retrieval and Classification for Diabetic Retinopathy Grading

Diabetic Retinopathy (DR) is a major source of sightlessness and permanent visual damage. Manual Analysis of DR is a labor-intensive and costly task that requires skilled ophthalmologists to observe and evaluate DR utilizing digital fundus images. The images can be employed for analysis and disease screening. This laborious task can gain a great advantage in automated detection by exploiting Artificial Intelligence (AI) techniques. Content-Based Image Retrieval (CBIR) approaches are utilized to retrieve related images in massive databases and are helpful in many application regions and most healthcare systems. With this motivation, this article develops the new Manta Ray Foraging Optimizer with Deep Learning-based Fundus Image Retrieval and Classification (MRFODL-FIRC) approach for the grading of DR. The suggested MRFODL-FIRC model investigates the retinal fundus imaging effectively to retrieve the relevant images and identify class labels. To achieve this, the MRFODL-FIRC technique uses Median Filtering (MF) as a pre-processing step. The Capsule Network (CapsNet) model is used to produce feature vectors with the MRFO algorithm as a hyperparameter optimizer. For the image retrieval process, the Manhattan distance metric is used. Finally, the Variational Autoencoder (VAE) model is used for recognizing and classifying DR. The investigational assessment of the MRFODL-FIRC technique is accomplished on medical DR and the outputs highlighted the improved performance of the MRFODL-FIRC algorithm over the current approaches.

Trademark Similarity Evaluation Using a Combination of ViT and Local Features

The origin of the trademark similarity analysis problem lies within the legal area, specifically the protection of intellectual property. One of the possible technical solutions for this issue is the trademark similarity evaluation pipeline based on the content-based image retrieval approach. CNN-based off-the-shelf features have shown themselves as a good baseline for trademark retrieval. However, in recent years, the computer vision area has been transitioning from CNNs to a new architecture, namely, Vision Transformer. In this paper, we investigate the performance of off-the-shelf features extracted with vision transformers and explore the effects of pre-, post-processing, and pre-training on big datasets. We propose the enhancement of the trademark similarity evaluation pipeline by joint usage of global and local features, which leverages the best aspects of both approaches. Experimental results on the METU Trademark Dataset show that off-the-shelf features extracted with ViT-based models outperform off-the-shelf features from CNN-based models. The proposed method achieves a mAP value of 31.23, surpassing previous state-of-the-art results. We assume that the usage of an enhanced trademark similarity evaluation pipeline allows for the improvement of the protection of intellectual property with the help of artificial intelligence methods. Moreover, this approach enables one to identify cases of unfair use of such data and form an evidence base for litigation.

A clinically motivated self-supervised approach for content-based image retrieval of CT liver images.

Deep learning-based approaches for content-based image retrieval (CBIR) of computed tomography (CT) liver images is an active field of research, but suffer from some critical limitations. First, they are heavily reliant on labeled data, which can be challenging and costly to acquire. Second, they lack transparency and explainability, which limits the trustworthiness of deep CBIR systems. We address these limitations by: (1) Proposing a self-supervised learning framework that incorporates domain-knowledge into the training procedure, and, (2) by providing the first representation learning explainability analysis in the context of CBIR of CT liver images. Results demonstrate improved performance compared to the standard self-supervised approach across several metrics, as well as improved generalization across datasets. Further, we conduct the first representation learning explainability analysis in the context of CBIR, which reveals new insights into the feature extraction process. Lastly, we perform a case study with cross-examination CBIR that demonstrates the usability of our proposed framework. We believe that our proposed framework could play a vital role in creating trustworthy deep CBIR systems that can successfully take advantage of unlabeled data.

A deep learning content-based image retrieval approach using cloud computing

Due to the rapid growth in multimedia content and its visual complexity, contentbased image retrieval (CBIR) has become a very challenging task. Existing works achieve high precision values at first retrieval levels such as top 10 and top 20 images, but low precision values at subsequent levels such as top 40, 50, and 70, so the goal of this paper is to propose a new CBIR approach that achieves high precision values at all retrieval levels. The proposed method combines features extracted from the pre-trained AlexNet model and discrete cosine transform (DCT). Then principal components analysis (PCA) is performed on AlexNet’s features and feeding these combination to multiclass support vector machine (SVM). The euclidean distance is used to measure the similarity between query and stored images features within the predicted class by SVM. Finally top similar images are ranked and retrieved. All above techniques require huge computational power which may not be available on client machine thus, the processing of these tasks is processed on cloud. Experimental results on the benchmark Corel-1k show that the proposed method achieves high precision value 97% along all retrieval levels top 10, 20 and 70 images and requiring less memory compared to other methods.

An optimal approach for content-based image retrieval using deep learning on COVID-19 and pneumonia X-ray Images

Analyzing and comparing the multi-spectral images in the healthcare domain is a complex task as it involves features in the visible spectrum and beyond. Using Content-based image retrieval (CBIR) with deep learning in the healthcare domain provides an ease to identify similar images, which eventually helps in analysis. CBIR, with deep learning, removes the explicit definition of features and autonomously determines similarity on multiple attributes like shape, color, and texture. The proposed method uses multiple deep learning architectures fine-tuned by using max-pool overlapping pooling in all networks and placing the same number of decision layers with the same parameters to determine the result based on feature extraction layers. State-of-the-art Neural Network models VGG-16, VGG-19, Xception, InceptionResNetV2, DenseNet201, MobileNetV2, and NASNetLarge were considered for the experiment to identify the most optimal model. First, a classification process is applied to compare the performance of the state-of-the-art networks. Then, the CBIR process is applied using the feature extraction part. The experiment was conducted on the chest X-rays dataset, consisting of 21,165 images with COVID-19, pneumonia, and normal, with and without rotational invariant cases. The VGG-16 model proved to be the most optimal choice for image retrieval and achieved the highest precision of 99% and mAP of 94.34% compared to recent CBIR methods, which also used chest X-rays datasets. Rotational invariant cases were also tested and achieved mean precision of 86%.

A Smart Content-Based Image Retrieval Approach Based on Texture Feature and Slantlet Transform

With the advancement of digital storing and capturing technologies in recent years, an image retrieval system has been widely known for Internet usage. Several image retrieval methods have been proposed to find similar images from a collection of digital images to a specified query image. Content-based image retrieval (CBIR) is a subfield of image retrieval techniques that extracts features and descriptions content such as color, texture, and shapes from a huge database of images. This paper proposes a two-tier image retrieval approach, a coarse matching phase, and a fine-matching phase. The first phase is used to extract spatial features, and the second phase extracts texture features based on the Slantlet transform. The findings of this study revealed that texture features are reliable and capable of producing excellent results and unsusceptible to low resolution and proved that the SLT-based texture feature is the perfect mate. The proposed method's experimental results have outperformed the benchmark results with precision gaps of 28.0 % for the Caltech 101 dataset. The results demonstrate that the two-tier strategy performed well with the successive phase (fine-matching) and the preceding phase (coarse matching) working hand in hand harmoniously.

A deep learning fusion approach to retrieve images of People's unsafe behavior from construction sites

Retrieving unsafe behaviours from an existing digital database can provide managers and the like with the necessary information to put in place strategies to improve safety in construction. Prevailing studies have focused on developing content-based image retrieval (CBIR) approaches (e.g., color-based) to retrieve objects and materials obtained from construction sites. While CBIR approaches are effective in extracting low-level features from digital images they are unable to accurately retrieve unsafe behaviours those from existing databases. To address this limitation, we develop an improved CBIR approach to retrieve unsafe behaviour images more accurately and automatically, which combines features extracted from different models. We utilise a digital database developed by Huazhong University of Science and Technology to validate the feasibility of our proposed approach. Our research demonstrates that the fusion of ResNet-101 and VGG-19 can obtain higher levels of Top-K recall and outperform the one feature extraction method.

Secure Content-Based Image Retrieval with Copyright Protection within Cloud Computing Environment

Every day, a tremendous amount of image data is generated as a result of recent advances in imaging and computing technology. Several content-based image retrieval (CBIR) approaches have been introduced for searching image collections. These methods, however, involve greater computing and storage resources. Cloud servers can address this issue by offering a large amount of computational power at a low cost. However, cloud servers are not completely trustworthy, and data owners are concerned about the privacy of their personal information. In this research, we propose and implement a secure CBIR (SCBIR) strategy for searching and retrieving cipher text image databases. In the proposed scheme, the extract aggregated feature vectors to represent the related image collection and use a safe Asymmetric Scalar-Product-Preserving Encryption (ASPE) approach to encrypt these vectors while still allowing for similarity computation. To improve search time, all encrypted features are recursively clustered using the k-means method to create a tree index. The results reveal that SCBIR is faster at indexing and retrieving than earlier systems, with superior retrieval precision and scalability. In addition, our paper introduces the watermark to discover any illegal distributions of the images that are received by unlawful data users. Particularly, the cloud server integrates a unique watermark directly into the encrypted images before sending them to the data users. As a result, if an unapproved image copy is revealed, the watermark can be extracted and the unauthorized data users who spread the image can be identified. The performance of the proposed scheme is proved, while its performance is demonstrated through experimental results.

Satellite image search in AgoraEO

The growing operational capability of global Earth Observation (EO) creates new opportunities for data-driven approaches to understand and protect our planet. However, the current use of EO archives is very restricted due to the huge archive sizes and the limited exploration capabilities provided by EO platforms. To address this limitation, we have recently proposed MiLaN, a content-based image retrieval approach for fast similarity search in satellite image archives. MiLaN is a deep hashing network based on metric learning that encodes high-dimensional image features into compact binary hash codes. We use these codes as keys in a hash table to enable real-time nearest neighbor search and highly accurate retrieval. In this demonstration, we showcase the efficiency of MiLaN by integrating it with EarthQube, a browser and search engine within AgoraEO. EarthQube supports interactive visual exploration and Query-by-Example over satellite image repositories. Demo visitors will interact with EarthQube playing the role of different users that search images in a large-scale remote sensing archive by their semantic content and apply other filters.

A hybrid CBIR system using novel local tetra angle patterns and color moment features

The exponential growth of communal media platforms including Facebook and Twitter, and the accessibility of low-cost digital capturing devices have generated an enormous number of multimedia content including images. Effective handling of such massive image collection has boosted the development of content-based image retrieval (CBIR) approaches. Researchers have suggested both machine learning and non-learning-based techniques for CBIR. However, machine learning-based methods outperform the non-learning-based methods in the CBIR domain. The CBIR demands the development of reliable descriptors to attain the most appropriate images from the depository and better address the semantic gap problem. To counter these problems, we suggest a novel second-order Local Tetra Angle Patterns (LTAP) to better capture the texture features from the image. LTAPs are computed from adjacent pixels of 0°, 45°, 90°, and 135° using the second-order directional derivatives. Further, we propose a hybrid feature vector by concatenating LTAPs and RGB color features and using the genetic algorithm (GA) to select the finest appropriate features that enhance the image retrieval performance of our system. We employed our hybrid descriptor with the GA to optimize the support vector machine (SVM) for the image classification task and used the Chi-square quadratic distance measure to determine the resemblance between the query image and the images in the repository. Experimental results on three standard datasets including the Corel 1 k, Oxford flower, and CIFAR-10 indicate the effectiveness of the presented system over the contemporary CBIR methods.

Optimization of Content Based Image Retrieval Using Hybrid Approach

With the dawn of multimedia technology and the social web, retrieval from large image databases become possible. As a consequence of rising usage and incredible enthusiasm for inquiry about on Content-Based Image Retrieval (CBIR) systems, it needs improvement in the accuracy of CBIR systems in addition to the decline in monotonous results. Currently, the majority of research has focused on the representation and differentiation of images by an arrangement of low-level visual features. However, most of the retrieval systems produce repetitive or unnecessary retrieval results, termed a redundancy factor. In addition, content-based retrieval with reduced redundancy has a direct correlation with high-level semantics which is overlooked. Accurate content retrieval with reduced redundancy enables the user to focus on the actual problem in preference to nurture the retrieval results, augments the efficiency, and improves overall system performance. To enhance the retrieval accuracy and diminish the redundancy factor in image retrieval, we proposed an optimization-based technique that is blended with classification. In the proposed novel hybrid approach for CBIR, we used a two-tier architecture model. The first tier belongs to the feature extraction process via Particle Swarm Optimization with a Support Vector Machine as a classifier. On the way to reduce the redundancy factor, the K-Nearest Neighbor as a classifier is used with the Genetic Algorithm in the second tier. We noted a noteworthy increase in retrieval accuracy for images that is up to 25% (approx.). The proposed hybrid model is effective to enhance accuracy and reduce redundancy factors in CBIR systems. We used the WANG dataset for experimentation. Henceforward, improving the retrieval accuracy and reducing redundancy factor using unsupervised learning techniques is part of our future work.

Content-Based Image Retrieval Using Gamma Distribution and Mixture Model

Since the last decade, the complexity of multimedia data, specifically images, is emerging exponentially as millions of images are uploaded by users on daily basis. Searching for a relevant image from such a substantial amount of data is very hectic and resource-demanding. To cope with this issue, researchers are working on content-based image retrieval (CBIR) approaches. This article proposes an efficient and novel probabilistic technique as a solution for content-based image retrieval. The patterns formed by the glyph structure of an image are excavated to yield content representations. These representations are accumulatively used to form a distribution, whereas the characteristics of this distribution represent the semantic structure of the image. In the end, the mixture model for gamma distribution is applied and parameters are refined through maximum likelihood. Furthermore, a mechanism is devised to retrieve matching images having comparable distribution patterns. Experiments show not only that the proposed technique yields a comparable precision to other competitive techniques but it also demonstrates that it is sufficiently efficient with high performance compared as compared to the others and requires unsupervised training.

Bypassing Confines of Feature Extraction in Brain Tumor Retrieval via MR Images by CBIR

To check out the health of the patient, digital images are generated every single day and are used by the radiologist for extracting out the details and anomalies. The complicated part is to figure out the disease in those images. By the manual diagnosis of the images through the radiologists, the doctors can get to know exact scenario of the abnormalities in images, but is considerably more difficult with Content Based Image Retrieval (CBIR) to get those finer details from MR images. These CBIR approaches are now frequently employed in the automatic diagnosis of disease from MR images, mammograms, and other sources. Bridging this gap can be done with deep learning feature extraction algorithm and the canny edge detection technique we propose, and accuracy closer to the manual results of a human evaluator can be achieved to a significant extent as part of the goal of sustainable development through innovation.

Content-Based Image Retrieval Using Colour, Gray, Advanced Texture, Shape Features, and Random Forest Classifier with Optimized Particle Swarm Optimization.

In this paper, a new approach for Content-Based Image Retrieval (CBIR) has been addressed by extracting colour, gray, advanced texture, and shape features for input query images. Contour-based shape feature extraction methods and image moment extraction techniques are used to extract the shape features and shape invariant features. The informative features are selected from extracted features and combined colour, gray, texture, and shape features by using PSO. The target image has been retrieved for the given query image by training the random forest classifier. The proposed colour, gray, advanced texture, shape feature, and random forest classifier with optimized PSO (CGATSFRFOPSO) provide efficient retrieval of images in a large-scale database. The main objective of this research work is to improve the efficiency and effectiveness of the CBIR system by extracting the features like colour, gray, texture, and shape from database images and query images. These extracted features are processed in various levels like removing redundancy by optimal feature selection and fusion by optimal weighted linear combination. The Particle Swarm Optimization algorithm is used for selecting the informative features from gray and colour and texture features. The matching accuracy and the speed of image retrieval are improved by an ensemble of machine learning algorithms for the similarity search.

Semantic and Content-Based Medical Image Retrieval with Proven Pathology for Lung Cancer Diagnosis

In lung cancer computer-aided diagnosis (CAD) systems, having an accurate ground truth is critical and time consuming. Due to lack of ground truth and semantic information, lung CAD systems are not progressing in the manner these are supposed to. In this study, we have explored Lung Image Database Consortium (LIDC) database containing annotated pulmonary computed tomography (CT) scans, and we have used semantic and content-based image retrieval (CBIR) approach to exploit the limited amount of diagnostically labeled data in order to annotate unlabeled images with diagnoses. We evaluated the method by various combinations of lung nodule sets as queries and retrieves similar nodules from the diagnostically labeled dataset. In calculating the precision of this system Diagnosed dataset and computer-predicted malignancy data are used as ground truth for the undiagnosed query nodules. Our results indicate that CBIR expansion is an effective method for labeling undiagnosed images in order to improve the performance of CAD systems while tested on PGIMER data. Also a little knowledge of biopsy confirmed cases can also assist the physician’s as second opinion to mark the undiagnosed cases and avoid unnecessary biopsies

Remote Sensing Image Retrieval Using Convolutional Neural Network Features and Weighted Distance

Remote sensing image retrieval (RSIR) may be a fundamental task in remote sensing. Most content-based image retrieval (CBRSIR) approaches take an easy distance as similarity criteria. A retrieval method supported weighted distance and basic features of Convolutional Neural Network (CNN) is proposed during this letter. the strategy contains two stages. First, in offline stage, the pretrained CNN will be fine-tuned by some labelled images from our target data set, then accustomed extract CNN features, and labelled the pictures within the retrieval data set. Second, in online stage, we extract features of the query image by using fine-tuned CNN model and calculate the load of every image class and apply them to calculate the space between the query image and also the retrieved images. Experiments and methods are conducted on two Remote Sensing Image Retrieval data sets. Compared with the state-of the-art methods, the proposed method significantly improves retrieval performance.