Semantic-based Image Retrieval Research Articles

Ontology supported semantic based image retrieval

In this study, a two-stage approach for developing a Semantic-Based Image Retrieval system supported by Ontology is proposed. In the initial stage, the Object Detection process is employed to identify objects within the image. Subsequently, a predicate describing the relationship between these two objects is determined using the developed Bi-directional Recurrent Neural Network (Bi-RNN) model. In the second stage, relations defined in the form of <subject-predicate-object> are transformed into Ontologies and utilized to search for images that are semantically similar. In addressing the primary challenge of Semantic Gap within the Semantic-Based Image Retrieval approach, the proposed solution involves measuring the number of similar relationships between two images through the utilization of entropy. The Semantic Gap between two images was computed using the Joint Entropy method, leveraging the number of relationships (X) identified in the query image and the total number of relationships (Y) in the image with similar relationships obtained as a query result. The proposed approach exhibits characteristics of a novel method within this field, distinct from other similar methods employed in Semantic-Based Image Retrieval through the utilization of Ontologies. In the performance measurement of the developed model, 91% accuracy was obtained according to the Recall@100 (Top-5 accuracy) result.

A hierarchical approach based CBIR scheme using shape, texture, and color for accelerating retrieval process

In the traditional content-based image retrieval (CBIR) framework, images are retrieved based on the combined primitive features. However, such a kind of fusion is always not effective as one feature may overshadow other image feature. To overcome this issue, in this particular paper, we have suggested a hierarchical framework where features like color, texture, and shape are considered in a single hierarchy only. The main concern of the hierarchical system is the proper selection of the order of retrieving visual features. So, semantic-based image retrieval has been carried out. Here, at the first level, the salience map-based region of interest has been identified, and then edge histogram descriptor-based shape features are incorporated. In the second hierarchy, we have proposed a novel directional texture feature extraction based on the Tamura features’ directionality. Further, color is considered another primitive feature of an image, but human visual perception is not sensitive to each color. The image can be visualized by the salient colors, and in this work, we have developed a color image quantization-based approach. Now, to validate the system, extensive experimental results and its comparison with its contemporaries through Corel-1 K, GHIM-10 K, Olivia-2688, and Produce-1400 databases have been carried out.

A method for semantic-based image retrieval using hierarchical clustering tree and graph

Semantic extraction for images is an urgent problem and is applied in many different semantic retrieval systems. In this paper, a semantic-based image retrieval (SBIR) system is proposed based on the combination of growth partitioning tree (GP-Tree), which was built in the authors’ previous work, with a self-organizing map (SOM) network and neighbor graph (called SgGP-Tree) to improve accuracy. For each query image, a similar set of images is retrieved on the SgGP-Tree, and a set of visual words is extracted relying on the classes obtained from mask region-based convolutional neural networks (R-CNN), as the basis for querying semantic of input images on ontology by simple protocol and resource description framework query language (SPARQL) query. The experiment was performed on image datasets, such as ImageCLEF and MS-COCO, with precision values of 0.898453 and 0.875467, respectively. These results are compared with related works on the same image dataset, showing the effectiveness of the methods proposed.

Panorama-to-model registration through integration of image retrieval and semantic reprojection

Registering images to a building information model is an effective approach of associating as-built component status to as-designed information. However, registering a single still image to a digital twin is difficult due to pose ambiguities caused by the limited field of view in images. Several recent studies have started focusing on leveraging panoramic images to address problems, such as partial occlusions, repetitive facility components and textureless views, which could cause registration failures in existing image-based registration workflows. Even though having a bigger field of view helps in locating where the image was taken, registering a panoramic image to a building information model is still challenging due to inconsistent visual appearances, such as model vs. image visual differences caused by temporary objects in a scene, lighting condition changes, and different levels of details. In this paper, we present a novel method that registers panoramic images to a digital twin in a hierarchical way: the proposed method first performs rough registration through image retrieval using semantic segmentation, then localizes the image with a fine registration through minimizing semantic reprojection errors. Compared to existing methods, the developed method has the following contributions: 1) semantic-based image retrieval makes the rough registration process robust to lighting condition changes, texture differences, and temporary objects, 2) semantic-based image retrieval allows for bi-directional queries between a model and images, 3) reprojection-based fine registration further reduces the localization error due to the dimension reduction of features during image retrieval. Though the proposed method is developed for panoramic images, it can be generalized to monocular images at the cost of localization accuracy. The developed method was evaluated on a real-world academic building and a synthetic dataset, and the results showed that the proposed method can localize a panorama in less than a second and achieve sub-meter level localization error.

A Model of Semantic-Based Image Retrieval Using C-Tree and Neighbor Graph

The problems of image mining and semantic image retrieval play an important role in many areas of life. In this paper, a semantic-based image retrieval system is proposed that relies on the combination of C-Tree, which was built in our previous work, and a neighbor graph (called Graph-CTree) to improve accuracy. The k-Nearest Neighbor (k-NN) algorithm is used to classify a set of similar images that are retrieved on Graph-CTree to create a set of visual words. An ontology framework for images is created semi-automatically. SPARQL query is automatically generated from visual words and retrieve on ontology for semantics image. The experiment was performed on image datasets, such as COREL, WANG, ImageCLEF, and Stanford Dogs, with precision values of 0.888473, 0.766473, 0.839814, and 0.826416, respectively. These results are compared with related works on the same image dataset, showing the effectiveness of the methods proposed here.

Study and Analysis of User Desired Image Retrieval

Background: In the present digital world, Content Based Image Retrieval (CBIR) has gained significant importance. In this context, the image processing technology has become the most sought one, as a result its demand has increased to a large extend. The complex growth concerning computer technology offers a platform to apply the image processing application. Well-known image retrieval techniques suitable for application zone are 1.Text Based Image Retrieval (TBIR) 2. Content Based Image Retrieval (CBIR) and 3.Semantic Based Image Retrieval (SBIR) etc. In recent past, many researchers have conducted extensive research in the field of content-based image retrieval (CBIR). However, many related research studies on image retrieval and characterization have exemplified to be an immense issue and it should be progressively developed in its techniques. Hence, by putting altogether the research conducted in the recent years, this survey study makes a comprehensive attempt to review the state-of –the art in the field. Aims: This paper aims to retrieve similar images according to visual properties, which defined as Shape, color, Texture and edge detection. Objective: To investigate the CBIR to achieve the task because of the essential and fundamentals problems. The present and future trends are addressed to show come contributions and directions and it can inspire more research in the CBIR methods. Result: we present a deep analysis of the state of the art on CBIR methods; we explain the methods based on Color, Texture, and shape, and edge detection with performance evaluation metrics. In addition, we have discussed some significant future research directions reviewed. Methods: This paper has quickly anticipated the noteworthiness of CBIR and its related improvement, which incorporates Edge Detection Techniques, Various sorts of Distance Metric (DM), Performance measurements and various kinds of Datasets. This paper shows the conceivable outcomes to overcome the difficulties concerning re-positioning strategies with an exceptional spotlight on the improvement of accuracy and execution. Discussion: At last, we have proposed another technique for consolidating different highlights in a CBIR framework that can give preferred outcomes over the current strategies.

A comprehensive survey on the reduction of the semantic gap in content-based image retrieval

In the last few decades, content-based image retrieval is considered as one of the most vivid research topics in the field of information retrieval. The limitation of current content-based image retrieval systems is that low-level features are highly ineffective to represent the semantic contents of the image. Most of the research work in content-based image retrieval is focused on bridging the semantic gap between the low-level features and high-level semantic concepts of image. This paper presents a thorough study of different techniques for the reduction of semantic gap. The existing techniques are broadly categorised as: 1) image annotation techniques to define the high-level concepts in image; 2) relevance feedback techniques to integrate user's perception; 3) machine learning and deep learning techniques to associate low-level features with high-level concepts. In addition, the general architecture of semantic-based image retrieval system has been discussed in this survey. This paper also highlights the current and future applications of content-based image retrieval. The paper concludes with promising future research directions.

Performance Evaluation of Semantic Based Image Retrieval System from Cyberspace (SBIRSC) Framework

With the popularity and remarkable development of digital imaging technology and its usage in various domains, an extraordinary level of accuracy required in the existing image retrieval techniques. The retrieval of the requested images on the basis of the user query by convoluted features in cyberspace is a very big challenge. Various researchers were already proposed multiple techniques for image retrieval. But, after concluding the detailed research, the authors have analyzed that most of the search-engines does not produce an accurate result. In this context, a new generation of digital images retrieval from cyberspace applications must be developed. This paper presents the performance evaluation of the proposed semantic-based image retrieval system from the cyberspace (SBIRSC) framework. The proposed framework was tested on various datasets like Flickr, Shutterstock, Pixabay, Everypixel, Bing available on the cyberspace. The performance of the proposed framework is evaluated by using accuracy and response time.

Query Rewriting and Semantic Annotation in Semantic-Based Image Retrieval under Heterogeneous Ontologies of Big Data

Query Rewriting and Semantic Annotation in Semantic-Based Image Retrieval under Heterogeneous Ontologies of Big Data

A SELF-BALANCED CLUSTERING TREE FOR SEMANTIC-BASED IMAGE RETRIEVAL

The image retrieval and semantic extraction play an important role in the multimedia systems such as geographic information system, hospital information system, digital library system, etc. Therefore, the research and development of semantic-based image retrieval (SBIR) systems have become extremely important and urgent. Major recent publications are included covering different aspects of the research in this area, including building data models, low-level image feature extraction, and deriving high-level semantic features. However, there is still no general approach for semantic-based image retrieval (SBIR), due to the diversity and complexity of high-level semantics. In order to improve the retrieval accuracy of SBIR systems, our focus research is to build a data structure for finding similar images, from that retrieving its semantic. In this paper, we proposed a data structure which is a self-balanced clustering tree named C-Tree. Firstly, a method of visual semantic analysis relied on visual features and image content is proposed on C-Tree. The building of this structure is created based on a combination of methods including hierarchical clustering and partitional clustering. Secondly, we design ontology for the image dataset and create the SPARQL (SPARQL Protocol and RDF Query Language) query by extracting semantics of image. Finally, the semantic-based image retrieval on C-Tree (SBIR\_CT) model is created hinging on our proposal. The experimental evaluation 20,000 images of ImageCLEF dataset indicates the effectiveness of the proposed method. These results are compared with some of recently published methods on the same dataset and demonstrate that the proposed method improves the retrieval accuracy and efficiency.

A SELF-BALANCED CLUSTERING TREE FOR SEMANTIC-BASED IMAGE RETRIEVAL

The image retrieval and semantic extraction play an important role in the multimedia systems such as geographic information system, hospital information system, digital library system, etc. Therefore, the research and development of semantic-based image retrieval (SBIR) systems have become extremely important and urgent. Major recent publications are included covering different aspects of the research in this area, including building data models, low-level image feature extraction, and deriving high-level semantic features. However, there is still no general approach for semantic-based image retrieval (SBIR), due to the diversity and complexity of high-level semantics. In order to improve the retrieval accuracy of SBIR systems, our focus research is to build a data structure for finding similar images, from that retrieving its semantic. In this paper, we proposed a data structure which is a self-balanced clustering tree named C-Tree. Firstly, a method of visual semantic analysis relied on visual features and image content is proposed on C-Tree. The building of this structure is created based on a combination of methods including hierarchical clustering and partitional clustering. Secondly, we design ontology for the image dataset and create the SPARQL (SPARQL Protocol and RDF Query Language) query by extracting semantics of image. Finally, the semantic-based image retrieval on C-Tree (SBIR\_CT) model is created hinging on our proposal. The experimental evaluation 20,000 images of ImageCLEF dataset indicates the effectiveness of the proposed method. These results are compared with some of recently published methods on the same dataset and demonstrate that the proposed method improves the retrieval accuracy and efficiency.

A Novel Framework design for Semantic based Image retrieval as a Cyber Forensic Tool

Cyber forensics includes the areas of computer forensics, network forensics and internet forensics. Digital images were commonly used in cyber forensics to collect criminal images, fingerprints, images of crime events and so on. Since the current cyber forensic tools are not very much furnished with the course of action of huge image data, it becomes a big issue to obtain image evidence to prosecute the criminal, most of the evidence is available in the form of raw semantics. Cyber forensic investigators often face the challenge of manually examining an enormous quantity of digital image information to identify direct evidence with the assistance of this semantics. Semantic based image retrieval system (SBIR) is therefore the recent and best option to solve this drawback. The main purpose of this research is to design for cyber forensic tools a novel framework of semantic-based image retrieval system. We therefore present a deep learning framework based on the Convolution neural network(GoogLeNet) for recognizing distinct facial expressions from the Yale facial image database for cyber associated forensic tools, the presented framework is very effective for classification and detection based on semantics or verbal descriptions. The network has accomplished a decent accuracy of 86.25 % after training.

A two-stage hybrid probabilistic topic model for refining image annotation

Refining image annotation has become one of the core research topics in computer vision and pattern recognition due to its great potentials in image retrieval. However, it is still in its infancy and is not sophisticated enough to extract perfect semantic concepts just according to the image low-level features. In this paper, we propose a two-stage hybrid probabilistic topic model to improve the quality of automatic image annotation. To start with, a probabilistic latent semantic analysis model with asymmetric modalities is learned to estimate the posterior probabilities of each annotation keyword, during which the image-to-word relation can be well established. Next, a label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity of images associated with the corresponding labels. By this way, the information from image low-level visual features and high-level semantic concepts can be seamlessly integrated by fully taking into account the word-to-word and image-to-image relations. Finally, the rank-two relaxation heuristics is exploited to further mine the correlation of the candidate annotations so as to capture the refining results, which plays a critical role in semantic based image retrieval. Extensive experiments show that the proposed model achieves not only superior annotation accuracy but also better retrieval performance.

Multi-level diversification approach of semantic-based image retrieval results

With the increasing popularity of social photograph-sharing Web sites, a huge mass of digital images, associated with a set of tags voluntarily introduced by amateur photographers, is daily hosted and consequently, the Tag-based social Image Retrieval technique has been widely adopted. However, tag-based queries are often too ambiguous and abstract to be considered as an efficient solution for the retrieval of the most relevant images that meet the users’ needs. As an alternative, the Semantic-based social Image Retrieval technique has emerged for the purpose of retrieving the relevant images covering as much possible the topics that a given ambiguous query (q) may have. Actually, the diversification strategies are a great challenge for researchers. In this context, we jointly investigate two processes at the ambiguous query preprocessing and postprocessing levels. On the one hand, we propose a Tag-based Query Semantic Reformulation process, which aims at reformulating the tag-based users’ queries, according to multiple semantic facets of the different images’ views, by using a set of predefined ontological semantic rules. On the other hand, we propose a Multi-level Image Diversification process that can first perform a two-level-based image clustering offline, and second, filter and re-rank the image cluster retrieval results according to their pertinence versus the reformulated query online. The experimental results and statistical analysis performed on a collection of 25.000 socio-tagged images shared on Flickr demonstrate the effectiveness of the proposed technique, which is compared with the research technique based on one-level-based image clustering, tag-based image research technique and recent CBIR techniques.

SIR-DL: AN ARCHITECTURE OF SEMANTIC-BASED IMAGE RETRIEVAL USING DEEP LEARNING TECHNIQUE AND RDF TRIPLE LANGUAGE

The problem of finding and identifying semantics of images is applied in multimedia applications of many different fields such as Hospital Information System, Geographic Information System, Digital Library System, etc. In this paper, we propose the semantic-based image retrieval (SBIR) system based on the deep learning technique; this system is called as SIR-DL that generates visual semantics based on classifying image contents. At the same time we identify the semantics of similar images on Ontology, which describes semantics of visual features of images. Firstly, the color and spatial features of segmented images are we extracted and these visual feature vectors are trained on the deep neural network to obtain visual words vectors. The process of image retrieval is executed rely on semantic classification of SIR-DL according to the visual feature vector of the query image from which it produces a visual word vector. Then, we retrieve it on Ontology to provide the identities and the semantics of similar images corresponds to a similarity measure. In order to carry out SIR-DL, the algorithms and diagram of this image retrieval system are proposed after that we implement them on ImageCLEF@IAPR, which has 20,000 images. On the base of the experimental results, the effectiveness of our method is evaluated by the accuracy, precision, recall, and F-measure; these results are compared with some of works recently published on the same image dataset. It shows that SIR-DL effectively solves the problem of semantic-based image retrieval and can be used to build multimedia systems in many different fields.

A Comprehensive Survey on Semantic based Image Retrieval Systems for Cyber Forensics

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

Discriminative Deep Feature Learning for Semantic-Based Image Retrieval

Semantic-based image retrieval plays an important role in many practical applications, which aims to look for images with similar contents. Extracting discriminative representations of images is the real crux of this task. Directly utilizing the results of fully connected layers of convolutional neural networks (CNNs) is one of the best feature extraction methods. However, the fully connected layer is only supervised by the softmax loss, which only aims to maximize the accuracy of object classification and hardly pays attention to the spatial distribution of features, especially the intra-class and inter-class feature distances which are of great importance in semantic-based image retrieval. To compensate the performance degradation due to this reason, we try to address the spatial distribution of the features by two different loss functions. The first loss function jointly combines the softmax loss and the center loss in the training of the CNNs, which simultaneously ensures that the features of images with different contents are separable and the features of images in the same class are close. The second loss function is defined as an improved center loss which not only penalizes the distance between the obtained deep feature and the feature center of its own class, but also encourages a far distance between the feature and the feature centers of any other classes. We have conducted experiments on both ILSVRC data set and Caltech256 data set, demonstrated that the deep features got by our approaches can achieve better performance than other methods in semantic-based image retrieval, and the improved center loss can further outperform the joint supervision of the softmax loss and the center loss.

Semantic based Image Retrieval

Semantic based Image Retrieval

A Survey On Semantic Image Retrieval For Bigdata

From last few years, there is an enormous growth in size of digital image collections. Every day, millions of images are being generated, in that semantic Image retrieval is the most complex process in the real time scenario where the similarity finding would be more difficult in case of larger homogeneous image contents. In Semantic based image retrieval, the weight adjustment scheme is used to give the high priority for the contents which are semantically more related. However, the weight adjustment scheme doesn’t care about the user feedback which might reduce the user satisfaction level. In this survey, homogeneous image data is retrieved by the proposed fuzzy logic based feedback weight adjustment scheme which will increase the score of the class which is more preferred by the users.

Automatic image annotation based on Gaussian mixture model considering cross-modal correlations

Automatic image annotation has been an active topic of research in the field of computer vision and pattern recognition for decades. In this paper, we present a new method for automatic image annotation based on Gaussian mixture model (GMM) considering cross-modal correlations. To be specific, we first employ GMM fitted by the rival penalized expectation-maximization (RPEM) algorithm to estimate the posterior probabilities of each annotation keyword. Next, a label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity by seamlessly integrating the information from both image low level visual features and high level semantic concepts together, which can effectively avoid the phenomenon that different images with the same candidate annotations would obtain the same refinement results. Followed by the rank-two relaxation heuristics over the built label similarity graph is applied to further mine the correlation of the candidate annotations so as to capture the refining annotation results, which plays a crucial role in the semantic based image retrieval. The main contributions of this work can be summarized as follows: (1) Exploiting GMM that is trained by the RPEM algorithm to capture the initial semantic annotations of images. (2) The label similarity graph is constructed by a weighted linear combination of label similarity and visual similarity of images associated with the corresponding labels. (3) Refining the candidate set of annotations generated by the GMM through solving the max-bisection based on the rank-two relaxation algorithm over the weighted label graph. Compared to the current competitive model SGMM-RW, we can achieve significant improvements of 4% and 5% in precision, 6% and 9% in recall on the Corel5k and Mirflickr25k, respectively.