Content-based Image Retrieval Applications Research Articles

Neuromorphic computing spiking neural network edge detection model for content based image retrieval

ABSTRACT In contemporary times, content-based image retrieval (CBIR) techniques have gained widespread acceptance as a means for end-users to discern and extract specific image content from vast repositories. However, it is noteworthy that a substantial majority of CBIR studies continue to rely on linear methodologies such as gradient-based and derivative-based edge detection techniques. This research explores the integration of bioinspired Spiking Neural Network (SNN) based edge detection within CBIR. We introduce an innovative, computationally efficient SNN-based approach designed explicitly for CBIR applications, outperforming existing SNN models by reducing computational overhead by 2.5 times. The proposed SNN-based edge detection approach is seamlessly incorporated into three distinct CBIR techniques, each employing conventional edge detection methodologies including Sobel, Canny, and image derivatives. Rigorous experimentation and evaluations are carried out utilizing the Corel-10k dataset and crop weed dataset, a widely recognized and frequently adopted benchmark dataset in the realm of image analysis. Importantly, our findings underscore the enhanced performance of CBIR methodologies integrating the proposed SNN-based edge detection approach, with an average increase in mean precision values exceeding 3%. This study conclusively demonstrated the utility of our proposed methodology in optimizing feature extraction, thereby establishing its pivotal role in advancing edge centric CBIR approaches.

Jumping particle swarm optimization algorithm framework for content-based image retrieval system

Content-based image retrieval (CBIR) has been studied well in the last decades in numerous research fields such as medicine, journalism, and private life. Applications of CBIR have been widely employed in medical images due to their direct impact on human life. With continues growing of digital libraries, there is a need for an efficient method to retrieve images from large datasets. In this paper, a new method was developed for CBIR based on the jumping particle swarm optimization (JPSO) algorithm. The proposed algorithm represents a developed instant of particle swarm optimization (PSO). However, JPSO the approach does not consider the velocity components to guide particle movements in the problem space. Instead of relying on inertia and velocity, intermittently random jumps (moves) occur from one solution to another within the discrete search space. To test the performance of the proposed algorithm, three types of medical image databases were used in the experiment which are the endoscopy 100, dental 100, and 50 skull image databases. The results show that the proposed algorithm could achieve high accuracy in image extraction and retrieve the accurate image category compared with other research works.

Refining Content-Based Segmentation for Prediction of Coffee Bean Quality

Coffee has substantial economic value and is a key foreign exchange source for numerous nations, including Indonesia. Moreover, it is a primary livelihood for many of the country's farmers. Recently, there have been challenges in accurately predicting the quality of coffee beans, primarily due to time, inconsistency, and imprecision issues. Consequently, this study delves into the application of region-growing segmentation and content-based image retrieval (CBIR) techniques to enhance the prediction of coffee bean quality. The proposed hybrid approach, which combines region growing and CBIR methods, aims to improve the precision for forecasting cacao bean quality. Additionally, the research introduces an automated tool that employs these hybrid techniques for quality prediction. The study conducted experiments using a dataset of 400 premium and 400 low-quality coffee beans sourced from the University of Syiah Kuala in Indonesia. The results of the experiments demonstrate a commendable precision rate of 85.4%, showcasing significant improvement compared to certain previous studies.

FeatSet+: Visual Features Extracted from Public Image Datasets

Real-world applications generate large amounts of images every day. With the generalized use of social media, users frequently share images acquired by smartphones. Also, hospitals, clinics, exhibits, factories, and other facilities generate images with potential use for many applications. Processing the generated images usually requires feature extraction, which can be time-consuming and laborious. In this paper, we present FeatSet+, a compilation of color, texture and shape visual features extracted from 17 open image datasets reported in the literature. FeatSet+ provides a collection of 11 distinct visual features, extracted by well-known Feature Extraction Methods (FEMs) such as LBP, Haralick, and Color Layout. We organized the available features in a standard collection, including the metadata and labels, when available. Eleven of the datasets also contain classes, which aid the evaluation of supervised methods such as classifiers and clustering tasks. FeatSet+ is available for download in a public repository as sql scripts and csv files. Additionally, FeatSet+ provides a description of the domain of each dataset, including the reference to the original work and link. We show the potential applicability of FeatSet+ in four computational tasks: multi-attribute analysis and retrieval, visual analysis using Multidimensional Scaling (MDS) and Principal Components Analysis (PCA), global feature classification, and dimensionality reduction. FeatSet+ can be employed to evaluate supervised and non-supervised learning tasks, also widely supporting Content-Based Image Retrieval (CBIR) applications and complex data indexing using Metric Access Methods (MAMs).

Local neighborhood gradient pattern: A feature descriptor for content based image retrieval

Local feature descriptors are efficient encoders for capturing repeated local patterns in many of the computer vision applications. Majority of such descriptors consider only limited local neighborhood pixels to encode a pattern. One of the major issues while considering more number of neighborhood pixels is that it increases the dimensionality of the feature descriptor. The proposed descriptor addresses these issues by describing an effective encoding pattern with optimal feature vector length. In this paper, we have proposed Local Neighborhood Gradient Pattern (LNGP) for Content-Based Image Retrieval (CBIR) in which the relationship between a set of neighbours and the centre pixel is considered to obtain a compact 8-bit pattern in the respective pixel position. The relationship of the gradient information of immediate, next-immediate, and diagonal neighbours with the centre pixel is considered for pattern formation, and thus the local information based on pixels in three directions are captured. The experiments are conducted on benchmarked image retrieval datasets such as Wang’s 1K, Corel 5K, Corel 10K, Salzburg (Stex), MIT-Vistex, AT & T, and FEI datasets and it is observed that the proposed descriptor yields average precision of 71.88%, 54.57%, 40.66%, 71.85%, 86.12%, 82.54%, and 68.54% respectively in the mentioned datasets. The comparative analysis of the recent descriptors indicates that the proposed descriptor performs efficiently in CBIR applications.

Plasticity-Stability Preserving Multi-Task Learning for Remote Sensing Image Retrieval

Deep learning-based multi-task learning (MTL) methods have recently attracted attention for content-based image retrieval (CBIR) applications in remote sensing (RS). For a given set of tasks (e.g., scene classification, semantic segmentation, and image reconstruction), existing MTL methods employ a joint optimization algorithm on the direct aggregation of task-specific loss functions. Such an approach may provide limited CBIR performance when: 1) tasks compete or even distract each other; 2) one of the tasks dominates the whole learning procedure; or 3) characterization of each task is underperformed compared to single-task learning. This is mainly due to the lack of: 1) plasticity condition (which is associated with sensitivity to new information) or 2) stability condition (which is associated with protection from radical disruptions by new information) of the whole learning procedure. To avoid this issue, as a first time, we propose a novel plasticity-stability preserving MTL (PLASTA-MTL) approach to ensure the plasticity and the stability conditions of the whole learning procedure independently of the number and type of tasks. This is achieved by defining two novel loss functions. The first loss function is the plasticity preserving loss (PPL) function that aims to enforce the global image representation space to be sensitive to new information learned with each task. This is achieved by minimizing the difference of gradient magnitudes for the global representation and task-specific embedding spaces. The second loss function is the stability preserving loss (SPL) function that aims to protect the global representation space radically disrupted by a new task. This is achieved by minimizing the angular distances between the task gradients over global representation space. To effectively employ the proposed loss functions, we also introduce a novel sequential optimization algorithm. Experimental results show the effectiveness of the proposed approach compared to the state-of-the-art MTL methods in the context of CBIR.

Content Based Medical Image Retrieval for Accurate Disease Diagnosis

Introduction: Content Based Image Retrieval (CBIR) system is an innovative technology to retrieve images from various media types. One of the CBIR applications is Content Based Medical Image Retrieval (CBMIR). The image retrieval system retrieves the most similar images from the historical cases, and such systems can only support the physician's decision to diagnose a disease. To extract the useful features from the query image for linking similar types of images is the major challenge in the CBIR domain. The Convolution Neural Network (CNN) can overcome the drawbacks of traditional algorithms, dependent on the low-level feature extraction technique. Objective: The objective of the study is to develop a CNN model with a minimum number of convolution layers and to get the maximum possible accuracy for the CBMIR system. The minimum number of convolution layers reduces the number of mathematical operations and the time for the model's training. It also reduces the number of training parameters, like weights and bias. Thus, it reduces the memory requirement for the model storage. This work mainly focused on developing an optimized CNN model for the CBMIR system. Such systems can only support the physicians' decision to diagnose a disease from the images and retrieve the relevant cases to help the doctor decide the precise treatment. Methods: The deep learning-based model is proposed in this paper. The experiment is done with several convolution layers and various optimizers to get the maximum accuracy with a minimum number of convolution layers. Thus, the ten-layer CNN model is developed from scratch and used to derive the training and testing images' features and classify the test image. Once the image class is identified, the most relevant images are determined based on the Euclidean distance between the query features and database features of the identified class. Based on this distance, the most relevant images are displayed from the respective class of images. The general dataset CIFAR10, which has 60,000 images of 10 different classes, and the medical dataset IRMA, which has 2508 images of 9 various classes, have been used to analyze the proposed method. The proposed model is also applied for the medical x-ray image dataset of chest disease and compared with the other pre-trained models. Results: The accuracy and the average precision rate are the measurement parameters utilized to compare the proposed model with different machine learning techniques. The accuracy of the proposed model for the CIFAR10 dataset is 93.9%, which is better than the state-of-the-art methods. After the success for the general dataset, the model is also tested for the medical dataset. For the x-ray images of the IRMA dataset, it is 86.53%, which is better than the different pre-trained model results. The model is also tested for the other x-ray dataset, which is utilized to identify chest-related disease. The average precision rate for such a dataset is 97.25%. Also, the proposed model fulfills the major challenge of the semantic gap. The semantic gap of the proposed model for the chest disease dataset is 2.75%, and for the IRMA dataset, it is 13.47%. Also, only ten convolution layers are utilized in the proposed model, which is very small in number compared to the other pre-trained models. Conclusion: The proposed technique shows remarkable improvement in performance metrics over CNN-based state-of-the-art methods. It also offers a significant improvement in performance metrics over different pre-trained models for the two different medical x-ray image datasets.

Content Based Medical Image Retrieval for Accurate Disease Diagnosis

Introduction: Content Based Image Retrieval (CBIR) system is an innovative technology to retrieve images from various media types. One of the CBIR applications is Content Based Medical Image Retrieval (CBMIR). The image retrieval system retrieves the most similar images from the historical cases, and such systems can only support the physician's decision to diagnose a disease. To extract the useful features from the query image for linking similar types of images is the major challenge in the CBIR domain. The Convolution Neural Network (CNN) can overcome the drawbacks of traditional algorithms, dependent on the low-level feature extraction technique. Objective: The objective of the study is to develop a CNN model with a minimum number of convolution layers and to get the maximum possible accuracy for the CBMIR system. The minimum number of convolution layers reduces the number of mathematical operations and the time for the model's training. It also reduces the number of training parameters, like weights and bias. Thus, it reduces the memory requirement for the model storage. This work mainly focused on developing an optimized CNN model for the CBMIR system. Such systems can only support the physicians' decision to diagnose a disease from the images and retrieve the relevant cases to help the doctor decide the precise treatment. Methods: The deep learning-based model is proposed in this paper. The experiment is done with several convolution layers and various optimizers to get the maximum accuracy with a minimum number of convolution layers. Thus, the ten-layer CNN model is developed from scratch and used to derive the training and testing images' features and classify the test image. Once the image class is identified, the most relevant images are determined based on the Euclidean distance between the query features and database features of the identified class. Based on this distance, the most relevant images are displayed from the respective class of images. The general dataset CIFAR10, which has 60,000 images of 10 different classes, and the medical dataset IRMA, which has 2508 images of 9 various classes, have been used to analyze the proposed method. The proposed model is also applied for the medical x-ray image dataset of chest disease and compared with the other pre-trained models. Results: The accuracy and the average precision rate are the measurement parameters utilized to compare the proposed model with different machine learning techniques. The accuracy of the proposed model for the CIFAR10 dataset is 93.9%, which is better than the state-of-the-art methods. After the success for the general dataset, the model is also tested for the medical dataset. For the x-ray images of the IRMA dataset, it is 86.53%, which is better than the different pre-trained model results. The model is also tested for the other x-ray dataset, which is utilized to identify chest-related disease. The average precision rate for such a dataset is 97.25%. Also, the proposed model fulfills the major challenge of the semantic gap. The semantic gap of the proposed model for the chest disease dataset is 2.75%, and for the IRMA dataset, it is 13.47%. Also, only ten convolution layers are utilized in the proposed model, which is very small in number compared to the other pre-trained models. Conclusion: The proposed technique shows remarkable improvement in performance metrics over CNN-based state-of-the-art methods. It also offers a significant improvement in performance metrics over different pre-trained models for the two different medical x-ray image datasets.

Performance-efficient distributed transfer and transformation of big spatial histopathology datasets in the cloud

Whole Slide Image (WSI) datasets are giga-pixel resolution, unstructured histopathology datasets that consist of extremely big files (each can be as large as multiple GBs in compressed format). These datasets have utility in a wide range of diagnostic and investigative pathology applications. However, the datasets present unique challenges: The size of the files, propriety data formats, and lack of efficient parallel data access libraries limit the scalability of these applications. Commercial clouds provide dynamic, cost-effective, scalable infrastructure to process these datasets, however, we lack the tools and algorithms that will transfer/transform them onto the cloud seamlessly, providing faster speeds and scalable formats. In this study, we present novel algorithms that transfer these datasets onto the cloud while at the same time transforming them into symmetric scalable formats. Our algorithms use intelligent file size distribution, and pipelining transfer and transformation tasks without introducing extra overhead to the underlying system. The algorithms, tested in the Amazon Web Services (AWS) cloud, outperform the widely used transfer tools and algorithms, and also outperform our previous work. The data access to the transformed datasets provides better performance compared to the related work. The transformed symmetric datasets are fed into three different analytics applications: a distributed implementation of a content-based image retrieval (CBIR) application for prostate carcinoma datasets, a deep convolutional neural network application for classification of breast cancer datasets, and to show that the algorithms can work with any spatial dataset, a Canny Edge Detection application on satellite image datasets. Although different in nature, all of the applications can easily work with our new symmetric data format and performance results show near-linear speed-ups as the number of processors increases.

Gender Classification Using Proposed CNN-Based Model and Ant Colony Optimization

Pedestrian gender classification is one of the key assignments of pedestrian study, and it finds practical applications in content-based image retrieval, population statistics, human–computer interaction, health care, multimedia retrieval systems, demographic collection, and visual surveillance. In this research work, gender classification was carried out using a deep learning approach. A new 64-layer architecture named 4-BSMAB derived from deep AlexNet is proposed. The proposed model was trained on CIFAR-100 dataset utilizing SoftMax classifier. Then, features were obtained from applied datasets with this pre-trained model. The obtained feature set was optimized with ant colony system (ACS) optimization technique. Various classifiers of SVM and KNN were used to perform gender classification utilizing the optimized feature set. Comprehensive experimentation was performed on gender classification datasets, and proposed model produced better results than the existing methods. The suggested model attained highest accuracy, i.e., 85.4%, and 92% AUC on MIT dataset, and best classification results, i.e., 93% accuracy and 96% AUC, on PKU-Reid dataset. The outcomes of extensive experiments carried out on existing standard pedestrian datasets demonstrate that the proposed framework outperformed existing pedestrian gender classification methods, and acceptable results prove the proposed model as a robust model.

Adaptive tetrolet based color, texture and shape feature extraction for content based image retrieval application

Adaptive tetrolet based color, texture and shape feature extraction for content based image retrieval application

An Overview of Feature Extraction Techniques in Content-Based Image Retrieval Systems

The collection of images has grown rapidly and continues to increase in future due to widespread use of internet. Content based Image Retrieval (CBIR) system aims at retrieval of images of relevance to the query image input by the user from an enormous image database by low-level feature (such as color, texture and shape) extraction from the image. Some common applications of CBIR and existing commercial systems employing CBIR have been highlighted. General algorithm for explaining the working of CBIR is presented. This article presents a review on methods for low level feature extraction like extracting color, texture and edge features. CBIR still faces some challenges like judgement of human perception of visual content, less appropriate selection of similarity measure, semantic gap and other factors.

Recent developments of content-based image retrieval (CBIR)

With the development of Internet technology and the popularity of digital devices, Content-Based Image Retrieval (CBIR) has been quickly developed and applied in various fields related to computer vision and artificial intelligence. Currently, it is possible to retrieve related images effectively and efficiently from a large scale database with an input image. In the past ten years, great efforts have been made for new theories and models of CBIR and many effective CBIR algorithms have been established. In this paper, we present a survey on the fast developments and applications of CBIR theories and algorithms during the period from 2009 to 2019. We mainly review the technological developments from the viewpoint of image representation and database search. We further summarize the practical applications of CBIR in the fields of fashion image retrieval, person re-identification, e-commerce product retrieval, remote sensing image retrieval and trademark image retrieval. Finally, we discuss the future research directions of CBIR with the challenge of big data and the utilization of deep learning techniques.

A comprehensive survey on the reduction of the semantic gap in content-based image retrieval

In the last few decades, content-based image retrieval is considered as one of the most vivid research topics in the field of information retrieval. The limitation of current content-based image retrieval systems is that low-level features are highly ineffective to represent the semantic contents of the image. Most of the research work in content-based image retrieval is focused on bridging the semantic gap between the low-level features and high-level semantic concepts of image. This paper presents a thorough study of different techniques for the reduction of semantic gap. The existing techniques are broadly categorised as: 1) image annotation techniques to define the high-level concepts in image; 2) relevance feedback techniques to integrate user's perception; 3) machine learning and deep learning techniques to associate low-level features with high-level concepts. In addition, the general architecture of semantic-based image retrieval system has been discussed in this survey. This paper also highlights the current and future applications of content-based image retrieval. The paper concludes with promising future research directions.

Fusion of region based extracted features for instance- and class-based CBIR applications

One of the major challenges in content-based image retrieval (CBIR) is understanding the actual semantic meaning of images. Generally, natural images possess a rich collection of different visual features, each of which have different contributions to the image semantics. These vary drastically in their local sub-regions. However, most of the state-of-the-art CBIR techniques give equal importance to all the visual features, which are not always effective for generating relevant retrieval outcomes. To overcome these issues, a CBIR scheme has been proposed which considers the image semantic properties of the different regions with their positional significance in the overall image. In this paper, a novel regions-of-attention (ROA) based feature extraction and fusion scheme for efficient image retrieval is suggested. Initially, a salient key-points and opponent-color feature based scheme is used to locate ROA of the image. Multi-directional texture, and spatial correlation-based color features are extracted from this ROA of the image to obtain most of the image’s semantic meanings. Spatial features from salient non-ROA local regions are also obtained to capture vital background information. A dissimilarity factor-based fusion scheme then is utilized to fuse the ROA and non-ROA features effectively, considering the importance and uniqueness of each feature to support the positional invariance notion in their respective sub-regions. Finally, the fused feature vector is used to perform instance- and class-based image retrieval in five different image datasets using different classifiers. The outcomes of the retrieval experiment have shown competitive performances as compared to the current state-of-the-art architectures.

Deep Learning-based Self Organizing Map for Content-based Image Retrieval Applications

Deep Learning-based Self Organizing Map for Content-based Image Retrieval Applications

Application of CBIR in E-commerce

The rise of technology and the rapidly increasing inventions in Science have completely changed many aspects of the world today. Many sectors such as communication, banking, media, etc. have gained momentum because of the internet. Online shopping is one such sector that has flourished in recent times because of the internet. This paper presents a method which employs the system of Content Based Image Retrieval (CBIR) in online shopping. Using this system, the time required to shop online will be reduced. CBIR is the activity of fetching images from the database which have some similarity to the given query image. Traditionally customers would have to search from different categories and apply various filters to buy the product that they want. But in this system, they will be provided with an option to directly upload the image of the product that they wish to buy. If similar products are available, it will be displayed to the customer immediately. Thus, the time required for a customer to buy a product reduces considerably thereby making the shopping experience fun, easy and convenient. The system works in a way such that when an image is uploaded, the features of this image are extracted by using the deep learning method of Convolutional Neural Network (CNN). These extracted features are compared with the features of the available images stored in the database. Then, the similarity measure is calculated and images that are akin to the query image are found and are set out as result. This method significantly helps in reducing the time required to search for a particular product.

Effect of image partitioning on content-based image retrieval using colour and texture

The current application of content-based image retrieval (CBIR) was urgently needed to process image retrieval in large image databases. Previous research had succeeded in combining feature extraction methods using texture and colour. Combining the two methods of Gray Level Co-Occurrence matrix with Colour moment produces a higher level of precision than that produced when method is applied individually. This research examined more deeply about the effect of image partitioning using the grid partitioning method. The results of precision and recall on the CBIR method would be compared with no partitions, two partitions and three image partitions. The results obtained indicated that the addition of image partitions did not affect the precision value. Thus, it can be concluded that increasing the image partition increases the recall value of the overall CBIR performance.

CLASSIFICATION OF BIOMEDICAL IMAGES USING CONTENT BASED IMAGE RETRIEVAL SYSTEMS

Because of the numerous application of Content-based image retrieval (CBIR) system in various areas, it has always remained a topic of keen interest by the researchers. Fetching of the most similar image from the complete repository by comparing it to the input image in the minimum span of time is the main task of the CBIR. The purpose of the CBIR can vary from different types of requirements like a diagnosis of the illness by the physician, crime investigation, product recommendation by the e-commerce companies, etc. In the present work, CBIR is used for finding the similar patients having Breast cancer. Gray-Level Co-Occurrence Matrix along with histogram and correlation coefficient is used for creating CBIR system. Comparing the images of the area of interest of a present patient with the complete series of the image of a past patient can help in early diagnosis of the disease. CBIR is so much effective that even when the symptoms are not shown by the body the disease can be diagnosed from the sample images.

Local Ternary Cross Structure Pattern: A Color LBP Feature Extraction with Applications in CBIR

With the advent of medical endoscopes, earth observation satellites and personal phones, content-based image retrieval (CBIR) has attracted considerable attention, triggered by its wide applications, e.g., medical image analytics, remote sensing, and person re-identification. However, constructing effective feature extraction is still recognized as a challenging problem. To tackle this problem, we first propose the five-level color quantizer (FLCQ) to acquire a color quantization map (CQM). Secondly, according to the anatomical structure of the human visual system, the color quantization map (CQM) is amalgamated with a local binary pattern (LBP) map to construct a local ternary cross structure pattern (LTCSP). Third, the LTCSP is further converted into the uniform local ternary cross structure pattern (LTCSPuni) and the rotation-invariant local ternary cross structure pattern (LTCSPri) in order to cut down the computational cost and improve the robustness, respectively. Finally, through quantitative and qualitative evaluations on face, objects, landmark, textural and natural scene datasets, the experimental results illustrate that the proposed descriptors are effective, robust and practical in terms of CBIR application. In addition, the computational complexity is further evaluated to produce an in-depth analysis.