Image Retrieval Using Convolutional Autoencoder, InfoGAN, and Vision Transformer Unsupervised Models

Abstract

Query by Image Content (QBIC), subsequently known as Content-Based Image Retrieval (CBIR) systems, may offer a more advantageous solution in a variety of applications, including medical, meteorological, search by image, and others. Such systems primarily use similarity matching algorithms to compare image content to get their relevance from databases. They are essentially measuring the spatial distance between extracted visual features from a query image and their correspondence in the dataset. One of the most challenging query retrieval problems is Facial Sketched-Real Image Retrieval (FSRIR), which is content similarity matching based. These facial retrieving systems are employed in a variety of contexts, including criminal justice. The difficulties of retrieving such sorts come from the composition of the human face and its distinctive parts. In addition, the comparison between these images is made from two different domains. Besides, to our knowledge, there is a rare existence of large-scale facial datasets that can be used to evolve the performance of the retrieving system. The success of the retrieval process is governed by the method used to calculate similarity and the efficient representation of compared images. However, by effectively representing visual features, the main challenge-posing component of such approaches might be resolved. Hence, this paper has several contributions that fill the research gap in content-based similarity matching and retrieving as follows: 1) The first contribution is extending the Chinese University Face Sketch (CUFS) dataset by including augmented images, introducing to the community a novel dataset named Extended Sketched-Real Image Retrieval (ESRIR). The CUFS dataset has been extended from 100 images to include 53,000 facial sketches and 53,000 real facial images. 2) The paper’s second contribution is proposing three new algorithms for sketched-real image retrieving based on convolutional autoencoder, Infogan, and Vision Transformer unsupervised models for large datasets. 3) Furthermore, to meet the subjective demands of the users because of the prevalence of multiple query formats. The third contribution of this paper is to train and assess the proposed algorithms across two additional facial datasets of various image sorts. 4) Recently, the majority of people have preferred searching for brand logo images, but it may be tricky to separate certain brand logo features from their alternatives and even from other features in an image. Thus, the fourth contribution is to compare logo image retrieval performance based on visual features derived from each of the three suggested retrieving systems. 5) The paper also proposes cloud-based energy-saving and computational complexity approaches in large-scale datasets. Due to the ubiquity of touchscreen devices, users often make drawings based on their fantasies for certain object image searches. Thus, the proposed algorithms were tested and assessed on a tough dataset of doodle-scratched human artworks. They are also studied for a multi-category dataset to cover practically all possible image types and situations. The results are compared with the most recent algorithms found in the literature. The results show that the proposed approaches are outperforming the recent algorithms.