Content-based Image Retrieval Research Articles

Image retrieval based on auto-encoder and clustering with centroid update

Abstract This paper suggests a revolutionary deep learning method using a dynamic deep auto-encoder for improving the performance of indexing the feature vectors of images by centroid updation. Feature vectors such as color, semantic, and spatial local binary pattern are extracted from the images for content-based image retrieval. The owner encrypts the images for protection using elliptic curve cryptography before uploading them to the cloud. A black hole entropic fuzzy clustering with Tversky indexing is used to retrieve similar information. When the new training image is matched with any of the centroid then the centroid gets updated by using dynamic deep auto-encoder. During the auto-encoder phase, the conflicted data points are dedicated to reconstruction and the reliable data points are helpful to centroid updation. The suggested BHE fuzzy clustering with dynamic deep auto-encoder approach fared better than the current methods, achieving the best accuracy of 97.605%, the highest $\boldsymbol{F_{1}}$ score of 90.210%, better precision of 90.001%, and the highest recall of 95.149%.

Secure and efficient content-based image retrieval using dominant local patterns and watermark encryption in cloud computing

Secure and efficient content-based image retrieval using dominant local patterns and watermark encryption in cloud computing

Multiple semantic X-ray medical image retrieval using efficient feature vector extracted by FPN.

Content-based medical image retrieval (CBMIR) has become an important part of computer-aided diagnostics (CAD) systems. The complex medical semantic information inherent in medical images is the most difficult part to improve the accuracy of image retrieval. Highly expressive feature vectors play a crucial role in the search process. In this paper, we propose an effective deep convolutional neural network (CNN) model to extract concise feature vectors for multiple semantic X-ray medical image retrieval. We build a feature pyramid based CNN model with ResNet50V2 backbone to extract multi-level semantic information. And we use the well-known public multiple semantic annotated X-ray medical image data set IRMA to train and test the proposed model. Our method achieves an IRMA error of 32.2, which is the best score compared to the existing literature on this dataset. The proposed CNN model can effectively extract multi-level semantic information from X-ray medical images. The concise feature vectors can improve the retrieval accuracy of multi-semantic and unevenly distributed X-ray medical images.

CBIF-M: A content-based image filtering and retrieval scheme using multi-classifier filtering framework

Recently, the image retrieval process appears to be a challenging task to filter out a huge volume of objectionable images from retrieval. It is very easy for people of all ages to obtain such obscene images with just a few clicks on the internet, which could harm the adolescent. To prevent such serious social problems, this paper introduces a content-based image filtering (CBIF) framework in an image retrieval system. The proposed CBIF approach is split into two components. In the first component, the skin region of exposed human body parts is extracted based on a color segmentation method, and an efficient feature vector generation process is introduced based on frequent skin color pixels and scale-invariant structural features. A novel texture feature representation is presented in the retrieval and filtering process obtained by block-paired local binary pattern (BP-LBP). The final component introduces a novel image filtering framework for the retrieved images found from the first component based on a majority of voting method on three different machine learning classifiers backed by SVM, MLP, and CNN. The retrieval efficiency is analyzed with standard performance indicators such as precision, recall, and F-score on two large datasets: NPDI and NSFW.

A Framework for Extensive Content-Based Image Retrieval System Incorporating Relevance Feedback and Query Suggestion

Over the last decade, there is a huge amount of images in which visual information has become increasingly important. If the images are to be utilized, they must be organized into databases, from there it can be searched based on different criteria. Content-based Image retrieval (CBIR) frameworks have turned into a mainstream subject of exploration; for their ability to retrieve images based on the real visual content rather than by manually connected textual descriptions. In the CBIR framework, analysis and interpretation of image information in large and diverse image databases is evidently complex because there is no prior information on the size or scale of individual structures within the images to be analyzed. In CBIR, retrieval is based on visual image features, which can be extracted automatically from the images with the help of human intervention, namely a technique called relevance feedback. Nonetheless, an efficient way of differentiating the visual content of images is complex to produce. Therefore, rather than a perfect solution CBIR systems must be able to exploit a partial solution to the problem of image understanding. In this paper, there is implementation of a CBIR framework is introduced that not only tries to efficiently capture the user intent based on the feedback but also provide query suggestions that can help its users to pose better queries in order to retrieve desired results in an efficient manner. The proposed technique is straightforward to implement and scopes efficiently to huge datasets. Extensive experiments on diverse real datasets with image similarity measures have revealed the dominance of the proposed method over original algorithms.

A privacy-preserving image retrieval scheme with access control based on searchable encryption in media cloud

With the popularity of the media cloud computing industry, individuals and organizations outsource image computation and storage to the media cloud server to reduce the storage burden. Media images usually contain a large amount of private information. To prevent disclosure of privacy of the image owners, media images are encrypted before uploading to the server. However, this operation will greatly limit the utilization of the image for the user, such as content-based image retrieval. We propose an efficient similarity query algorithm with access control based on Bkd-tree in this paper, in which a searchable encryption scheme is designed for similarity image retrieval, and the encrypted image is used to extract image features by a pre-trained CNN model. The Bkd-tree is utilized to generate an index tree for the image features to speed up retrieval and make it faster than linear indexing. Finally, the security performances of the proposed scheme is analyzed and the performance of this scheme is evaluated by experiments. The results show that the security of the image content and image features can be ensured, and it has a shorter retrieval time and higher retrieval efficiency.

DarwinGSE: Towards better image retrieval systems for intellectual property datasets.

A trademark's image is usually the first type of indirect contact between a consumer and a product or a service. Companies rely on graphical trademarks as a symbol of quality and instant recognition, seeking to protect them from copyright infringements. A popular defense mechanism is graphical searching, where an image is compared to a large database to find potential conflicts with similar trademarks. Despite not being a new subject, image retrieval state-of-the-art lacks reliable solutions in the Industrial Property (IP) sector, where datasets are practically unrestricted in content, with abstract images for which modeling human perception is a challenging task. Existing Content-based Image Retrieval (CBIR) systems still present several problems, particularly in terms of efficiency and reliability. In this paper, we propose a new CBIR system that overcomes these major limitations. It follows a modular methodology, composed of a set of individual components tasked with the retrieval, maintenance and gradual optimization of trademark image searching, working on large-scale, unlabeled datasets. Its generalization capacity is achieved using multiple feature descriptions, weighted separately, and combined to represent a single similarity score. Images are evaluated for general features, edge maps, and regions of interest, using a method based on Watershedding K-Means segments. We propose an image recovery process that relies on a new similarity measure between all feature descriptions. New trademark images are added every day to ensure up-to-date results. The proposed system showcases a timely retrieval speed, with 95% of searches having a 10 second presentation speed and a mean average precision of 93.7%, supporting its applicability to real-word IP protection scenarios.

A deep learning based interval type-2 fuzzy approach for image retrieval systems

Deep learning, that one of its key benefits is automated feature extraction, has become a principal solution for computer vision. This paper presents a Deep Type-2 Beta Fuzzy (DT2F) approach for Content-Based Image Retrieval (CBIR) systems. Firstly, the suggested architecture uses InceptionResNetv2 a pre-trained deep learning model on Image-Net data as a feature extractor. Secondly, the obtained feature space is fuzzified to handle the uncertainties associated with the extracted values of deep features. Thirdly, the reduction dimensionality step is efficiently applied using a Multi-Variational Auto-Encoder (MVAE) to reduce computational complexity and achieve better performance. Ultimately, we retrieve images using the nearest neighbors rule based on type-2 fuzzy similarity to having higher proximity sensitivity. Extensive experimentations were accomplished on various image-retrieving datasets of different scales the proposed system achieved an average precision of 97.15% exceeding other state-of-the-art methods over many systems on Corel datasets, which can open the door for several hybridization breakthroughs in the area of image retrieval.

SBHA: Sensitive Binary Hashing Autoencoder for Image Retrieval.

Binary hashing is an effective approach for content-based image retrieval, and learning binary codes with neural networks has attracted increasing attention in recent years. However, the training of hashing neural networks is difficult due to the binary constraint on hash codes. In addition, neural networks are easily affected by input data with small perturbations. Therefore, a sensitive binary hashing autoencoder (SBHA) is proposed to handle these challenges by introducing stochastic sensitivity for image retrieval. SBHA extracts meaningful features from original inputs and maps them onto a binary space to obtain binary hash codes directly. Different from ordinary autoencoders, SBHA is trained by minimizing the reconstruction error, the stochastic sensitive error, and the binary constraint error simultaneously. SBHA reduces output sensitivity to unseen samples with small perturbations from training samples by minimizing the stochastic sensitive error, which helps to learn more robust features. Moreover, SBHA is trained with a binary constraint and outputs binary codes directly. To tackle the difficulty of optimization with the binary constraint, we train the SBHA with alternating optimization. Experimental results on three benchmark datasets show that SBHA is competitive and significantly outperforms state-of-the-art methods for binary hashing.

Enhanced Plant Leaf Disease Identification by Integrating 2DCNN and Transfer Learning for a Content-Based Image Retrieval System

Enhanced Plant Leaf Disease Identification by Integrating 2DCNN and Transfer Learning for a Content-Based Image Retrieval System

Unveiling Visual Treasures: Harnessing Deep Learning for Content-Based Image Retrieval

Objective: An essential aspect of computer vision is content-based image retrieval (CBIR), which enables users to search for images based on their visual content instead of created annotations. Advances in technology have resulted in a significant rise in the complexity of multimedia content and the emergence of new research fields centered on similar multimedia material retrieval. The efficacy of retrieval is impacted by the limits of the present CBIR systems, which result from overlooked algorithms and computing restrictions. Methods: This research introduces a novel approach employing the Siamese Edge Attention Layered Convonet (SEAL Convonet) for Image Retrieval. We utilize the CBIR image dataset through Gaussian smoothing to enhance image quality for data preprocessing and the Canny Edge Detector (CED) for edge detection, following pre-processing. The Histogram of Oriented Gradients (HOG) is used for feature extraction to extract complex textures and patterns from the images. Findings: This approach is implemented and tested through simulations as well as the results indicate a substantial positive deviation in the performance and retrieval of the images compared to existing methods. The performance metrics are accuracy (97 %), precision (94 %), recall (91 %), F1-Score (97 %), False Positive Rate (FNR) (0.0013), Matthew's correlation coefficient (MCC) (0.85), and False Negative Rate (FPR) (0.0036) show the measurements of this proposed model. Application: The state of the art in this work is researching the influence of optimizers on the accuracy process, as indicated by the findings. Keywords: CBIR, Gaussian smoothing, Canny Edge Detector, Histogram of oriented gradients (HOG), Siamese Edge Attention Layered convonet (SEAL Convonet), Database

Novel Content Based Image Retrieval—Features of Correlated Visual Textons and MQLPP Descriptor

Novel Content Based Image Retrieval—Features of Correlated Visual Textons and MQLPP Descriptor

Collaborative group: Composed image retrieval via consensus learning from noisy annotations

Composed image retrieval extends content-based image retrieval systems by enabling users to search using reference images and captions that describe their intention. Despite great progress in developing image-text compositors to extract discriminative visual-linguistic features, we identify a hitherto overlooked issue, triplet ambiguity, which impedes robust feature extraction. Triplet ambiguity refers to a type of semantic ambiguity that arises between the reference image, the relative caption, and the target image. It is mainly due to the limited representation of the annotated text, resulting in many noisy triplets where multiple visually dissimilar candidate images can be matched to an identical reference pair (i.e., a reference image + a relative caption). To address this challenge, we propose the Consensus Network (Css-Net), inspired by the psychological concept that groups outperform individuals. Css-Net comprises two core components: (1) a consensus module with four diverse compositors, each generating distinct image-text embeddings, fostering complementary feature extraction and mitigating dependence on any single, potentially biased compositor; (2) a Kullback–Leibler divergence loss that encourages learning of inter-compositor interactions to promote consensual outputs. During evaluation, the decisions of the four compositors are combined through a weighting scheme, enhancing overall agreement. On benchmark datasets, particularly FashionIQ, Css-Net demonstrates marked improvements. Notably, it achieves significant recall gains, with a 2.77% increase in R@10 and 6.67% boost in R@50, underscoring its competitiveness in addressing the fundamental limitations of existing methods.

Benchmarking Deep Learning-Based Image Retrieval of Oral Tumor Histology.

Oral tumors necessitate a dependable computer-assisted pathological diagnosis system considering their rarity and diversity. A content-based image retrieval (CBIR) system using deep neural networks has been successfully devised for digital pathology. No CBIR system for oral pathology has been investigated because of the lack of an extensive image database and feature extractors tailored to oral pathology. This study uses a large CBIR database constructed from 30 categories of oral tumors to compare deep learning methods as feature extractors. The highest average area under the receiver operating characteristic curve (AUC) was achieved by models trained on database images using self-supervised learning (SSL) methods (0.900 with SimCLR and 0.897 with TiCo). The generalizability of the models was validated using query images from the same cases taken with smartphones. When smartphone images were tested as queries, both models yielded the highest mean AUC (0.871 with SimCLR and 0.857 with TiCo). We ensured the retrieved image result would be easily observed by evaluating the top 10 mean accuracies and checking for an exact diagnostic category and its differential diagnostic categories. Training deep learning models with SSL methods using image data specific to the target site is beneficial for CBIR tasks in oral tumor histology to obtain histologically meaningful results and high performance. This result provides insight into the effective development of a CBIR system to help improve the accuracy and speed of histopathology diagnosis and advance oral tumor research in the future.

Secureimagesec: A privacy-preserving framework for outsourced picture representation with content-based image retrieval

Content-Based Image Retrieval (CBIR) uses complicated algorithms to analyze visual attributes and retrieve relevant photos from large databases. CBIR is essential to a privacy-preserving feature extraction and protection method for outsourced picture representation. SecureImageSec combines essential methods with the system’s key entities to ensure secure, private and protected image feature processing during outsourcing. For a system to be implemented effectively, these techniques must be seamlessly integrated across critical entities, such as the client, the cloud server that is being outsourced, the component that protects secure features, the component that maintains privacy in communication, access control, and authorization, and the integration and system evaluation. The client entity initiates outsourcing using advanced encryption techniques to protect privacy. SecureImageSec protects outsourced data by using cutting-edge technologies like Fully Homomorphic Encryption (FHE) and Secure Multi-Party Computation (SMPC). Cloud servers hold secure feature protection entities and protect outsourced features’ privacy and security. SecureImageSec uses AES and FPE to protect data format. SecureImageSec’s cloud-outsourced privacy-preserving communication uses SSL/TLS and QKD to protect data transmission. Attribute-Based Encryption (ABE) and Functional Encryption (FE) in SecureImageSec limit access to outsourced features based on user attributes and allow fine-grained access control over decrypted data. SecureImageSec’s Information Leakage Rate (ILR) of 0.02 for a 1000-feature dataset shows its efficacy. SecureImageSec also achieves 4.5 bits of entropy, ensuring the encrypted feature set’s muscular cryptographic strength and randomness. Finally, SecureImageSec provides secure and private feature extraction and protection, including CBIR capabilities, for picture representation outsourcing.

Unsupervised Content Mining in CBIR: Harnessing Latent Diffusion for Complex Text-Based Query Interpretation.

The paper demonstrates a novel methodology for Content-Based Image Retrieval (CBIR), which shifts the focus from conventional domain-specific image queries to more complex text-based query processing. Latent diffusion models are employed to interpret complex textual prompts and address the requirements of effectively interpreting the complex textual query. Latent Diffusion models successfully transform complex textual queries into visually engaging representations, establishing a seamless connection between textual descriptions and visual content. Custom triplet network design is at the heart of our retrieval method. When trained well, a triplet network will represent the generated query image and the different images in the database. The cosine similarity metric is used to assess the similarity between the feature representations in order to find and retrieve the relevant images. Our experiments results show that latent diffusion models can successfully bridge the gap between complex textual prompts for image retrieval without relying on labels or metadata that are attached to database images. This advancement sets the stage for future explorations in image retrieval, leveraging the generative AI capabilities to cater to the ever-evolving demands of big data and complex query interpretations.

IMAGE CLASSIFIER FOR FAST SEARCH IN LARGE DATABASES

Relevance. The avalanche-like growth in the amount of information on the Internet necessitates the development of effective methods for quickly processing such information in information systems. Clustering of news information is carried out by taking into account both the morphological analysis of texts and graphic content. Thus, an urgent task is the clustering of images accompanying textual information on various web resources, including news portals. The subject of study is an image classifier that exhibits low sensitivity to increased information in databases. The purpose of the article is to enhance the efficiency of searching for identical images in databases experiencing a daily influx of 10-12 thousand images, by developing an image classifier. Methods used: mathematical modeling, content-based image retrieval, two-dimensional discrete cosine transform, image processing methods, decision-making methods. The following results were obtained. An image classifier has been developed with low sensitivity to increased database information. The properties of the developed classifier have been analyzed. The experiments demonstrated that clustering information based on images using the developed classifier proved to be sufficiently fast and cost-effective in terms of information volumes and computational power requirements.

Advancements in Non-Linear Content-Based Image Retrieval (CBIR) Systems for Image Analysis

Advancements in Non-Linear Content-Based Image Retrieval (CBIR) Systems for Image Analysis

Heritage Iconographic Content Structuring: from Automatic Linking to Visual Validation

This article presents a global framework dedicated to the structuring of iconographic heritage collections. To alleviate the poor interlinking both between collections and contents, a first step of automatic linking exploiting content-based image retrieval approaches is evaluated and adapted to the visual variability of such heritage contents. To ensure understanding and analysis of the contents in a structured fashion, a 3D immersive web platform is also introduced alongside visual-based analysis tools. Finally, by exploiting both automatic linking and manual interventions in the visualization platform, an iterative, semi-automatic structuring pipeline is proposed to solve difficult cases missed by automatic structuring, and then improve structuring optimally. Here, we demonstrate the potential of the proposal on the geographic iconographic heritage of Paris, with a dataset of 10k images belonging to several institutions, thus poorly connected nor organized globally.

Parallel Hierarchical Multi-View Feature Fusion Based on Canonical Correlation Analysis for Mammogram Retrieval

Due to the diversity of image sources, content-based multi-source image fusion and retrieval have shown promising capabilities in computer vision tasks, and especially when applied in Computer-Aided Diagnosis (CAD) to automate and improve the accuracy of medical image analysis. The combination of computer vision and CAD systems has the potential to revolutionize healthcare by augmenting the expertise of clinicians, improving overall diagnostic accuracy and helping experts in the clinical decision-making process by classifying and retrieving similar annotated clinical images to a given query. In the context of multi-view mammography interpretation, the concept of multi-view feature fusion has recently been studied to improve retrieval performance while effectively guaranteeing the complementarity of both MLO and CC views. However, conventional multi-view feature fusion makes descriptors long and lacks to take into consideration the relationship between descriptors. To deal with this issue, we propose two hierarchical multi-view feature fusion methods, for multi-view mammogram retrieval, based on the Canonical Correlation Analysis (CCA), which is the most commonly used multivariate parametric test. In fact, we have adapted CCA to determine the relationship between two descriptors by processing latent correlation factors. Moreover, after extracting descriptors for each view, a comparative study of texture and shape fusion descriptors is proposed in order to identify the more discriminative features for multi-view mammogram retrieval. Then, a query-dependent distance metric preserving both visual resemblance and semantic similarity is carried out to dynamically determine the more appropriate distance measure for each query image. Extensive experiments on the challenging Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) have demonstrated the effectiveness of the proposed hierarchical multi-view feature fusion for mammogram retrieval, which outperforms the performance achieved either by conventional fused information or by single view information. To improve the transparency of our paper, the source code of the proposed method and the related dataset (including readme files) are publicly accessible through the following GitHub link: https://github.com/ABDERRAHIMMAR/Multi-View-Feature-Fusion-for-Mammogram-Retrieval . This open-access resource empowers researchers and practitioners to delve deeper into our methodology, fostering collaboration and advancements in the field of computer-aided diagnosis and medical image analysis.