Image Retrieval Applications Research Articles

DIML: Deep Interpretable Metric Learning via Structural Matching.

In this paper, we present a new framework named DIML to achieve more interpretable deep metric learning. Unlike traditional deep metric learning method that simply produces a global similarity given two images, DIML computes the overall similarity through the weighted sum of multiple local part-wise similarities, making it easier for human to understand the mechanism of how the model distinguish two images. Specifically, we propose a structural matching strategy that explicitly aligns the spatial embeddings by computing an optimal matching flow between feature maps of the two images. We also devise a multi-scale matching strategy, which considers both global and local similarities and can significantly reduce the computational costs in the application of image retrieval. To handle the view variance in some complicated scenarios, we propose to use cross-correlation as the marginal distribution of the optimal transport to leverage semantic information to locate the important region in the images. Our framework is model-agnostic, which can be applied to off-the-shelf backbone networks and metric learning methods. To extend our DIML to more advanced architectures like vision Transformers (ViTs), we further propose truncated attention rollout and partial similarity to overcome the lack of locality in ViTs. We evaluate our method on three major benchmarks of deep metric learning including CUB200-2011, Cars196, and Stanford Online Products, and achieve substantial improvements over popular metric learning methods with better interpretability.

Photographer-guided attributes for underwater image aesthetics

Automated aesthetic assessment of photographs is an active research area with applications in image editing and retrieval. There are many suggestions on the various factors of importance in making an image ‘good’ or ‘aesthetically pleasing’. However, there is no consensus in the literature on a definitive set of attributes that contribute to image aesthetics for underwater images, which include features specific to an aquatic environment. In this research we interview underwater photographers and apply thematic analysis to their responses with the aim of determining which attributes are important for an aesthetically-pleasing underwater image. The results define a set of nine key attributes (i.e. Aesthetics, Aquatic features, Colour, Composition, Image precision, Lighting, Novelty, Subject(s), and Technical competence). These findings will guide future work in automated assessment of underwater image aesthetics.

Integration of statistical parameters-based colour-texture descriptors for radar remote sensing image retrieval applications

Integration of statistical parameters-based colour-texture descriptors for radar remote sensing image retrieval applications

Integration of statistical parameters-based colour-texture descriptors for radar remote sensing image retrieval applications

Integration of statistical parameters-based colour-texture descriptors for radar remote sensing image retrieval applications

Refining Content-Based Segmentation for Prediction of Coffee Bean Quality

Coffee has substantial economic value and is a key foreign exchange source for numerous nations, including Indonesia. Moreover, it is a primary livelihood for many of the country's farmers. Recently, there have been challenges in accurately predicting the quality of coffee beans, primarily due to time, inconsistency, and imprecision issues. Consequently, this study delves into the application of region-growing segmentation and content-based image retrieval (CBIR) techniques to enhance the prediction of coffee bean quality. The proposed hybrid approach, which combines region growing and CBIR methods, aims to improve the precision for forecasting cacao bean quality. Additionally, the research introduces an automated tool that employs these hybrid techniques for quality prediction. The study conducted experiments using a dataset of 400 premium and 400 low-quality coffee beans sourced from the University of Syiah Kuala in Indonesia. The results of the experiments demonstrate a commendable precision rate of 85.4%, showcasing significant improvement compared to certain previous studies.

Feature Extraction Methods by Various Concepts using SOM

Image retrieval systems gained traction with the increased use of visual and media data. It is critical to understand and manage big data, lot of analysis done in image retrieval applications. Given the considerable difficulty involved in handling big data using a traditional approach, there is a demand for its efficient management, particularly regarding accuracy and robustness. To solve these issues, we employ content-based image retrieval (CBIR) methods within both supervised , unsupervised pictures. Self-Organizing Maps (SOM), a competitive unsupervised learning aggregation technique, are applied in our innovative multilevel fusion methodology to extract features that are categorised. The proposed methodology beat state-of-the-art algorithms with 90.3% precision, approximate retrieval precision (ARP) of 0.91, and approximate retrieval recall (ARR) of 0.82 when tested on several benchmark datasets.

Deep global semantic structure-preserving hashing via corrective triplet loss for remote sensing image retrieval

With the explosive increase of remote sensing data, how to search for remote sensing data quickly and accurately in a vast dataset is an incredibly critical matter for research subjects. The deep hashing method has become the dominant method for remote sensing image retrieval because of its low-cost storage and high-speed retrieval. However, for the reason of the limitation of fixed convolutional kernels, deep hashing frameworks based on Convolutional Neural Networks (CNNs) fail to obtain the global semantic information well, which leads to the generation of suboptimal solutions. Furthermore, existing hashing methods commonly employ the random sampling strategy or hardest sample mining to build training batches, resulting in bad local minima. To remedy these problems, a novel Deep Global Semantic Structure-preserving Hashing framework via corrective triplet loss (DGSSH) is proposed for remote sensing image retrieval to learn a discriminative and stable embedding space, achieving intra-class confusion and inter-class diversity. Specifically speaking, the feature extraction module based on Swim Transformer architecture is developed to capture global semantic information and multiscale features from remote sensing images. Based on a distribution matching constraint, the corrective triplet loss for deep hashing schemes is designed to reduce the distribution shift caused by the random selection or hardest sample mining. Meanwhile, to reduce the time overhead of the model, the asymmetric learning strategy is employed to perform effective compact representation learning. Numerous experiments have been carried out on three publicly available benchmarks, which indicates that the proposed DGSSH framework could achieve optimal performance for remote sensing image retrieval applications. The source code of our DGSSH framework is hosted at https://github.com/QinLab-WFU/DGSSH.git.

Design of CSD based bi-orthogonal wavelet filter bank for medical image retrieval

Two-channel wavelet filter-banks (FBs) are universally used in many applications of signal processing. In this work, design of a bi-orthogonal wavelet FB with low complexity is suggested. The proposed design is based on canonical signed digit (CSD) implementation of the bi-orthogonal wavelet FB. CSD technique has been applied to the FB to eliminate the power consuming multipliers and to reduce the number of shifters and adders in the implementation of bi-orthogonal FB. The suggested FB design is suitable for the application of high speed signal processing applications. In addition, the proposed bi-orthogonal wavelet FB is implemented on hardware using Field Programmable Gate Array (FPGA) board Kintex-7 from Xilinx. It is found that the suggested bi-orthogonal wavelet FB attained low digital hardware requirement as compared to well-known rationalized existing bi-orthogonal wavelet FBs. The efficacy of the suggested wavelet FB is demonstrated on medical image retrieval application. NEMA, OASIS, and EXACT09, three publicly accessible medical image datasets, are utilized to measure the efficacy of image retrieval. It is shown that the suggested bi-orthogonal wavelet FB performs better than the bi-orthogonal wavelet FBs existing that are being used.

A DCT probability histogram-based ROI features for content-based natural and medical image retrieval applications

The effectiveness of a Content-based Image Retrieval (CBIR) system depends on selecting representative and salient visual features. Capturing the salient image features for forming a feature vector is an essential part of the CBIR process, as it can significantly affect retrieval efficiency. Most images contain a visually prominent region that perceives maximum image semantic information, known as Regions-of-Interest (ROIs). This paper suggests a new CBIR scheme that employs a Graph-based Visual Saliency (GBVS) map to extract the ROI of the image. Further, block-level Discrete Cosine Transforms (DCT) and subsequent histograms are computed for the DC and significant AC coefficients of the extracted ROI image. In addition, a low-dimensional feature vector is formed from the modified histograms. Subsequently, six statistical features from the background scene have also been computed for the final feature vector construction. As a result, the final feature vector contains ROI and background information in their proper proportion to improve retrieval performance. Moreover, the performance of the proposed CBIR scheme has been evaluated using one medical, three natural, two objects, one texture, and one natural heterogeneous image database. All these databases collectively contain 53732 different images from 200 image classes. The outcomes of the experiments demonstrate substantial improvement in the retrieval process with some existing related schemes. At the same time, it also shows that the performance of the proposed scheme is robust concerning all kinds of image databases.

Content-Based remote sensing image retrieval method using adaptive tetrolet transform based GLCM features

This paper proposed a novel texture feature extraction technique for radar remote sensing image retrieval application using adaptive tetrolet transform and Gray level co-occurrence matrix. Tetrolets have provided fine texture information in the radar image. Tetrominoes have been employed on each decomposed radar image and best pattern of tetrominoes has been chosen which represents the better radar image geometry at each decomposition level. All three high pass components of the decomposed radar image at each level and low pass component at the last level are considered as input values for Gray level co-occurrence matrix (GLCM), where GLCM provides the spatial relationship among the pixel values of decomposed components in different directions at certain distances. The GLCMs of decomposed components are computed in (1). (0, π/2, π, 3π/2), (2). (π/4, 3π/4, 5π/4, 7π/4) (3). (0, π/4, π/2, 3π/4, π, 3π/2, 5π/4, 7π/4) directions individually and subsequently a texture feature descriptor is constructed by computing statistical parameters from the corresponding GLCMs. The retrieval performance is validated on two standard radar remote sensing image databases: 20-class satellite remote sensing dataset and 21-class land-cover dataset. The average metrices i.e., precision, recall and F-score are 61.43%, 12.29% and 20.47% for 20-class satellite remote sensing dataset while 21-class land-cover dataset have achieved 67.75%, 9.03% and 15.94% average metrices. The retrieved results show the better accuracy as compared to the other related state of arts radar remote sensing image retrieval methods.

ENHANCED AI BASED FEATURE EXTRACTION TECHNIQUE IN MULTIMEDIA IMAGE RETRIEVAL

In the era of rapid technological advancements, the demand for efficient and accurate identification and retrieval of information from multimedia images has seen a substantial increase. To meet this growing demand, artificial intelligence (AI)-based technologies, particularly feature extraction techniques, have gained significant popularity. Feature extraction involves the extraction of salient features from multimedia images, such as edges, lines, curves, textures, and colors, with the aim of representing the data in a more suitable format for analysis. This paper presents an enhanced AI-based feature extraction technique for multimedia image retrieval. The proposed method introduces a novel approach that combines the power of deep learning and evolutionary algorithms in a neuro-symbolic computation framework. Specifically, the renowned VGG16 deep learning algorithm is employed as the initial feature extractor. VGG16 is a state-of-the-art deep convolutional neural network that has demonstrated exceptional performance in various computer vision tasks, including image classification and feature extraction. The primary idea behind this approach is to leverage the capabilities of AI to extract the most discriminative features from the source images using VGG16. These features are then further refined using evolutionary algorithms, which employ a search and optimization process inspired by natural evolution. By iteratively improving the extracted features through the evolutionary algorithms, the method aims to enhance the discriminative power and representational quality of the extracted features. To evaluate the performance of the proposed approach, extensive experiments were conducted. The results demonstrate that the method achieves superior performance in terms of precision, recall, and F-measure when compared to conventional feature extraction techniques. Furthermore, a comprehensive comparison with state-of-the-art AI-based feature extraction techniques further highlights the potential and effectiveness of the proposed approach in multimedia image retrieval applications.

Attention-Oriented Deep Multi-Task Hash Learning

Hashing has wide applications in image retrieval at large scales due to being an efficient approach to approximate nearest neighbor calculation. It can squeeze complex high-dimensional arrays via binarization while maintaining the semantic properties of the original samples. Currently, most existing hashing methods always predetermine the stable length of hash code before training the model. It is inevitable for these methods to increase the computing time, as the code length converts, caused by the task requirements changing. A single hash code fails to reflect the semantic relevance. Toward solving these issues, we put forward an attention-oriented deep multi-task hash learning (ADMTH) method, in which multiple hash codes of varying length can be simultaneously learned. Compared with the existing methods, ADMTH is one of the first attempts to apply multi-task learning theory to the deep hashing framework to generate and explore multi-length hash codes. Meanwhile, it embeds the attention mechanism in the backbone network to further extract discriminative information. We utilize two common available large-scale datasets, proving its effectiveness. The proposed method substantially improves retrieval efficiency and assures the image characterizing quality.

A self-attention multi-scale convolutional neural network method for SAR image despeckling

ABSTRACT The speckle noise found in synthetic aperture radar (SAR) images severely affects the efficiency of image interpretation, retrieval and other applications. Thus, effective methods for despeckling SAR image are required. The traditional methods for SAR image despeckling fail to balance in terms of the relationship between the intensity of speckle noise filtering and the retention of texture details. Deep learning based SAR image despeckling methods have been shown to have the potential to achieve this balance. Therefore, this study proposes a self-attention multi-scale convolution neural network (SAMSCNN) method for SAR image despeckling. The advantage of the SAMSCNN method is that it considers both multi-scale feature extraction and channel attention mechanisms for multi-scale fused features. In the SAMSCNN method, multi-scale features are extracted from SAR images through convolution layers with different depths. These are concatenated; then, and an attention mechanism is introduced to assign different weights to features of different scales, obtaining multi-scale fused features with weights. Finally, the despeckled SAR image is generated through global residual noise reduction and image structure fine-tuning. The despeckling experiments in this study involved a variety of scenes using simulated and real data. The performance of the proposed model was analysed using quantitative and qualitative evaluation methods and compared to probabilistic patch-based (PPB), SAR block-matching 3-D (SAR-BM3D) and SAR-CNN methods. The experimental results show that the method proposed in this paper improves the objective indexes and shows great advantages in visual effects compared to these classical methods. The method proposed in this study can provide key technical support for the practical application of SAR images.

Mathematical Morphology View of Topological Rough Sets and Its Applications

This article focuses on the relationship between mathematical morphology operations and rough sets, mainly based on the context of image retrieval and the basic image correspondence problem. Mathematical morphological procedures and set approximations in rough set theory have some clear parallels. Numerous initiatives have been made to connect rough sets with mathematical morphology. Numerous significant publications have been written in this field. Others attempt to show a direct connection between mathematical morphology and rough sets through relations, a pair of dual operations, and neighborhood systems. Rough sets are used to suggest a strategy to approximate mathematical morphology within the general paradigm of soft computing. A single framework is defined using a different technique that incorporates the key ideas of both rough sets and mathematical morphology. This paper examines rough set theory from the viewpoint of mathematical morphology to derive rough forms of the morphological structures of dilation, erosion, opening, and closing. These newly defined structures are applied to develop algorithm for the differential analysis of chest X-ray images from a COVID-19 patient with acute pneumonia and a health subject. The algorithm and rough morphological operations show promise for the delineation of lung occlusion in COVID-19 patients from chest X-rays. The foundations of mathematical morphology are covered in this article. After that, rough set theory ideas are taken into account, and their connections are examined. Finally, a suggested image retrieval application of the concepts from these two fields is provided.

Power-Efficient VLSI Architecture of a New Class of Dyadic Gabor Wavelets for Medical Image Retrieval

Gabor wavelet is widely used in the analysis of texture features. It is found that Gabor wavelet filter bank (FB) requires infinite precision (due to irrational coefficients) to extract accurate directional textural features for which it needs numerous computations and external memory access. Due to this, there is a requirement for huge digital hardware, increased dynamic power dissipation, and increased processing time. In order to solve these issues, this article presents the first dyadic Gabor wavelet FB (DGWFB) based on the slight alteration in orientation parameter without disturbing the remaining Gabor wavelet parameters. In addition, this article introduces a separable VLSI architecture for the proposed DGWFB. The proposed VLSI architecture is implemented in field-programmable gate array (FPGA) through Xilinx Kintex 7 platform. It is observed that the proposed DGWFB has reduced processing time, dynamic power dissipation and digital hardware requirement significantly. The effectiveness of the designed DGWFB is evaluated in medical image retrieval application using three different publicly available medical databases namely NEMA, OASIS, and EXACT09. The proposed DGWFB achieved better performance as compared to existing Gabor wavelet FBs with significantly reduced digital hardware and processing time.

Experimental evaluation of unsupervised image retrieval application using hybrid feature extraction by integrating deep learning and handcrafted techniques

Abstract Content Based Image Retrieval is ever growing technology for many applications such as medical, remote sensing, social media search engines and surveillance monitoring, etc,. Representing image content with appropriate features is tedious task using traditional low level feature extraction methods. Deep learning models achieved high precision in classification and object detection algorithms by extracting automated high level feature extraction process. This paper proposed a hybrid feature extraction technique by combining the high level features and low level features to improve the robustness of the feature vector. The proposed model used pre-trained Googlenet model as feature extractor and combined with Gabor multiscale texture features. The final feature vector will be used for retrieving the relevant image data from the large scale image dataset. It has achieved the precision of 91 percent which shows better than state of art methods.

Robust structural similarity index measure for images with non-Gaussian distortions

• Introduction of a new robust structure similarity index based on p th order statistics. • A simple and low cost special case: l1-norm based robust structure similarity index. • Extension of this measure to colour images. • Demonstration of the success of new index on various types of distortions. Structural Similarity Index Measure (SSIM) has been a very successful tool in image processing and computer vision . After almost two decades, it is still the most popular method for quantifying the similarity between original and distorted images with applications in image quality assessment , image retrieval , video coding, computer vision, image encryption and data-hiding. Despite its general success, there are some types of distortions, such as when the distortion is of non-Gaussian character, where SSIM does not sustain its success. We provide a generalized version of SSIM utilizing l p -norm ( 1 ≤ p ≤ 2 ) based sample moments as opposed to the classical SSIM which uses l 2 -norm based sample moments. In particular, we study the l 1 -norm special case and our simulations on well-known image-databases show superior performance compared to SSIM and related measures.

RCAE_BFV: Retrieve Encrypted Images using Convolution AutoEncoder and BFV

Content-Based Image Retrieval (CBIR) is an actual application in computer vision, which retrieves similar images from a database. Deep Learning (DL) is essential in many applications, including image retrieval applications. However, encryption techniques are used to protect data privacy because these data are vulnerable to being viewed by unauthorized parties while being transmitted over unsecured channels. This paper includes two parts for images retrieval. In the first part, features of all images of a Canadian Institute for Advanced Research CIFAR-10 dataset were extracted and stored on the Server-side. In the second part, the Brakerski/Fan-Vercauteren (BFV) homomorphic encryption scheme method for encrypting an image sent by the client-side. First, their decryption and image features are extracted depending on the trainer model when they arrive on the server-side. Then an extracted features are compared with stored features using the Cosine Distance method, and then the server encrypts the retrieved images and sends them to the client-side. Deep-learning results on plain images were 97% for classification and 96.7% for retriever images. At the same time, The National Institute of Standards and Technology (NIST ) test was used to check the security of BFV when applied to CIFAR-10 dataset. Index Terms— BFV, Convolution Autoencoder, Content-based image retrieval, Homomorphic encryption.

Weber Global Statistics Tri- Directional Pattern (WGSTriDP): A Texture Feature Descriptor for Image Retrieval

The texture is a high-flying feature in an image and has been extracted to represent the image for image retrieval applications. Many texture features are being offered for image retrieval. This paper proposes a local binary pattern-based texture feature called Weber Global Statistics Tri-Directional Pattern (WGSTriDP) to retrieve the images. This pattern combines the advantages of differential excitation components in the Weber Local Binary Pattern (WLBP), sign and magnitude components in the Local Tri-Directional Pattern (LTriDP), and global statistics. Differential Excitation (DE) and Global Statistics TriDirectional Pattern (GSTriDP) are two components of WGSTriDP. The WGSTriDP gains the benefit of discrimination concerning human perception from differential excitation as well as incorporates global statistics into sign and magnitude components in the pattern derived from the local neighborhoods. The effectiveness of the pattern in image retrieval is experimented with in two benchmark databases, such as ORL (face database) and UIUC (texture database). According to the results of the experiments, WGSTriDP outperforms other local patterns in retrieving similar images from the database.

FeatSet+: Visual Features Extracted from Public Image Datasets

Real-world applications generate large amounts of images every day. With the generalized use of social media, users frequently share images acquired by smartphones. Also, hospitals, clinics, exhibits, factories, and other facilities generate images with potential use for many applications. Processing the generated images usually requires feature extraction, which can be time-consuming and laborious. In this paper, we present FeatSet+, a compilation of color, texture and shape visual features extracted from 17 open image datasets reported in the literature. FeatSet+ provides a collection of 11 distinct visual features, extracted by well-known Feature Extraction Methods (FEMs) such as LBP, Haralick, and Color Layout. We organized the available features in a standard collection, including the metadata and labels, when available. Eleven of the datasets also contain classes, which aid the evaluation of supervised methods such as classifiers and clustering tasks. FeatSet+ is available for download in a public repository as sql scripts and csv files. Additionally, FeatSet+ provides a description of the domain of each dataset, including the reference to the original work and link. We show the potential applicability of FeatSet+ in four computational tasks: multi-attribute analysis and retrieval, visual analysis using Multidimensional Scaling (MDS) and Principal Components Analysis (PCA), global feature classification, and dimensionality reduction. FeatSet+ can be employed to evaluate supervised and non-supervised learning tasks, also widely supporting Content-Based Image Retrieval (CBIR) applications and complex data indexing using Metric Access Methods (MAMs).