Content-based Retrieval System Research Articles

Multi-view expressive graph neural networks for 3D CAD model classification

The creation of effective content-based retrieval systems for the 3D models of engineering components created by Mechanical Computer Aided Design (MCAD) systems has been a subject of academic investigation since the 1990 s. Recently some of the most promising results have been reported by researchers using Deep Neural Nets (DNN) to classify industrial parts represented by collections of images generated from different viewpoints. Known as Multi-View Convolutional Neural Network (MVCNN) these systems have extended the architectures developed for 2D images analysis to handle 3D data by using a series of pictures rendered from different viewpoints. In 2016 Monti et al. used Graph Neural Networks (GNN) to classify the superpixel images for the first time and reported results which suggested that the approach could produce good classification accuracy (an observation confirmed by the application of GNNs to 2D image datasets of common objects such as MNIST and Cifar). To investigate the potential for GNNs to reason about spatial adjacency relationships this paper reports, for the first time, the application of GNNs to classifications of 3D MCAD models of mechanical components. Viewing GNN as a convolution operator, parallel GNN can be applied to multiple views of a 3D shape in the same way as CNN is structured in a MVCNN. After outlining the architecture implemented to support MVGNNs the impact of various hyperparameters on the expressive power are explored. When optimised these hyperparameters (located in both intra-layer and inter-layer structures) combined with the multi-view architecture produced a classification accuracy superior to previous methods. The results also suggest that GNNs have the potential to produce fast, accurate classification systems for 3D MCAD data using significantly smaller datasets than those used for transfer learning in MVCNNs.

Overview of the development of GIS in the education process in the Republic of North Macedonia

The paper presents an overview of the development of Geographic Information Systems in the educational system in the Republic of North Macedonia. It presents the period of development of geography and geographic education during the last century, while providing a more general review of the beginnings and the continuing course of development of GIS at the Institute of Geography at the Faculty of Natural Mathematics in Skopje, and across the public institutions in the Republic of North Macedonia overall. The goal is to understand the cultural overview of the events related to the development of GIS, the exchange of experiences and benefits from the organization of the educational processes of the undergraduate and postgraduate studies at the Institute of Geography at the Faculty of Natural Mathematics in Skopje and the contribution and guidance by the GIS-trained (graduate and Master degree) personnel in the development of GIS techniques and technologies in the application and design activities in the Republic of North Macedonia. The scientific contribution is multi-purpose in terms of advancing GIS techniques and technologies in various geographical and other disciplines, by recognizing the potentials that GIS has with regard to the work with massive databases and by obtaining more relevant analyses, automated process of creating various graphic and thematic cartographic database displays by using a content-based retrieval system.

Content-based image retrieval for categorized dataset by aggregating gradient and texture features

Content-based image retrieval is the process of retrieving images similar to the Query Image. The task of finding the dissimilarity among the different objects is the primary challenge in the content-based retrieval system. The challenge increases when the dissimilarity among similar objects is less. For instance, retrieving sunflowers from the dataset having different types of objects (i.e., the uncategorized dataset) like faces, rose, ball, mountains, etc. is a challenging task but what if the sunflowers are to be retrieved from the dataset having flowers only (i.e., the categorized dataset)? Such scenarios make content-based image retrieval more challenging because now the objects are very much similar to each other (categorized) and finding dissimilarity among them is a difficult task. In the state-of-the-art approaches, much work has been done on uncategorized data as compared to categorized one. The present paper proposes an image retrieval approach out of the objects of the same category as of Query Image. Gradient and texture features have been used for the retrieval of images out of the objects of the same category. The Proposed Approach has been applied to various categories of objects like flowers (roses, lily, sunflower), toys (cars, bears, frogs), dogs (Dalmatian, Pomeranian), etc. The Proposed Approach outperforms the state-of-the-art approaches by obtaining an average accuracy of 90 $$\%$$ .

Development of a system for efficient content-based retrieval to analyze large volumes of climate data

Analyses of large ensemble data on future climate are significantly useful for the probabilistic future projection of climate change in various interdisciplinary fields. However, the data volume of the Database for Policy Decision making for Future climate change or d4PDF, which is a mega-ensemble dataset, exceeds ∼ 3 PB, which is too large to download to local computers. To allow users for retrieve and downloading necessary data, we developed a user-friendly system called “System for Efficient content-based retrieval to Analyze Large volume climate data” (SEAL) under the Social Implementation Program on Climate Change Adaptation Technology (SI-CAT). Conventional web-based retrieval systems allow retrievals using metadata associated with a data file itself. In contrast, SEAL allows the users to retrieve the necessary data by using metadata associated with contents, such as physical values, of a data file. We confirmed that SEAL can reduce data sizes and total time required for obtaining necessary data to less than 0.5% and 1%, respectively, compared to conventional web-based retrieval systems.

Content-based retrieval system for surgery videos

There has been tremendous growth of laparoscopic surgical video archives in the past few years. Finding the surgical videos manually from the database is very troublesome and time consuming. As a consequence, efficient methods for retrieval of such videos from a database are needed. In this paper, a content-based video retrieval (CBVR) system is presented using average intensity, colour and shape feature. In the proposed retrieval system, average intensity, colour and shape feature are used. The ordinal measure introduced by Bhat and Nayar, which is based on ranks rather than of their values is used in this work. This is important in the laparoscopic videos because adjacent frames can be distinguished on the basis of ranking of their average intensities. Experimental results reveal that mean average precision of the proposed technique is better than the existing motion intensity and direction descriptor (MIDD) approach. This reflects the effectiveness of proposed method.

Using diverse set of features to design a content-based video retrieval system optimised by gravitational search algorithm

Using diverse set of features to design a content-based video retrieval system optimised by gravitational search algorithm

Content-based retrieval system for surgery videos

There has been tremendous growth of laparoscopic surgical video archives in the past few years. Finding the surgical videos manually from the database is very troublesome and time consuming. As a consequence, efficient methods for retrieval of such videos from a database are needed. In this paper, a content-based video retrieval (CBVR) system is presented using average intensity, colour and shape feature. In the proposed retrieval system, average intensity, colour and shape feature are used. The ordinal measure introduced by Bhat and Nayar, which is based on ranks rather than of their values is used in this work. This is important in the laparoscopic videos because adjacent frames can be distinguished on the basis of ranking of their average intensities. Experimental results reveal that mean average precision of the proposed technique is better than the existing motion intensity and direction descriptor (MIDD) approach. This reflects the effectiveness of proposed method.

Content Based Movie Scene Retrieval using Spatio Temporal Features

Thousands of movies along with TV shows, documentaries are being produced each year around the world with different genres and languages. Making a movie scene impactful as well as original is challenging task for the director. On the other hand, users demands to retrieve similar scenes from their queries is also challenging task as there is no proper maintenance of database of movie scene videos with proper semantic tags associated with it. So to fulfill the requirement of these two (but not the least) application areas there is a need of content based retrieval system for movie scenes. Content based video retrieval is a problem of retrieving most similar videos to a given query video by analyzing the visual contents of videos. Traditional video level features based on key frame level hand engineered features which does not exploit rich dynamics present in the video. In this paper we propose a Content based Movie Scene Retrieval (CB-MSR) framework using spatio-temporal features learned by deep learning. Specifically deep CNN along with LSTM is deploy to learn spatio-temporal representations of video. On the basis of these learned features similar movie scenes can be retrieve from the collection of movies. Hollywood2 dataset is used to test the proposed system. Two types of features: spatial and spatio-temporal features are used to evaluate the proposed framework.

Development of Retrieval System using Color from Multi Segments and Shape features from Images as Combination

The world of digital information has been generous to produce large data. The task of finding similar images in such a vast repository in real-time is very challenging. In this paper, Content Based Retrieval System is proposed to retrieve similar images with a novel approach of combining the image features like color and shape. The work includes extraction of color feature from images by dividing them into five significant portions, which emphasizes on center portion of an image where object may reside. This process is carried out and represented using color histograms. Next, shape features are extracted using Hu - moments for the entire image. Later, by combining these two features, similar image retrieval is done based on a query. Various trials of retrieval have been carried out with different categories and results are tabulated. These results are compared with existing works and shows high performance.

SketchHelper: Real-Time Stroke Guidance for Freehand Sketch Retrieval

Text-based retrieval systems have been popular, but content-based retrieval systems have gained widespread acceptance in recent years to directly retrieve diverse media based on their visual content, such as color, texture, and shape. Among many content-based retrieval systems, sketch-based media retrieval systems have attracted attention recently with the proliferation of tablet PCs and smart mobile devices. Sketch-based retrieval requires the user to draw a freehand sketch query, but freehand drawing can be challenging for those with limited drawing skills. This degrades retrieval performance, since successful retrieval depends on the quality of the sketch query image drawn by the user. To address this issue, we propose a real-time stroke guidance for freehand sketch retrieval that continuously displays next-stroke shadow sketches on the canvas based on the user's step-by-step partial strokes. We train a stroke guidance network that learns the mapping between the step-wise stroke relations to predict the user's next stroke. The proposed stroke guidance for freehand sketch retrieval system runs on a five step next-stroke prediction model that identifies candidate next-stroke sketches from a database of millions of sketches. The system retrieves variable number of sketch object classes at different drawing stages. During the initial sketching stage, diverse drawing possibilities are covered by retrieving multiple sketch classes; as the sketching progresses, the intended sketch class is narrowed down to one. Deep binary hashing is employed for efficient similarity matching of relevant next-stroke sketches. We extend the Google QuickDraw dataset to create a five step sketch stroke database. Qualitative and quantitative experiments are conducted to verify the effectiveness of the proposed system, which can be utilized for drawing guidance, tracing, and sketch retrieval. Tracing refers to the act of copying the shadowed line of a guiding image by drawing over its lines.

A Novel System for Content-Based Retrieval of Single and Multi-Label High-Dimensional Remote Sensing Images

This paper presents a novel content-based remote sensing (RS) image retrieval system that consists of the following. First, an image description method that characterizes both spatial and spectral information content of RS images. Second, a supervised retrieval method that efficiently models and exploits the sparsity of RS image descriptors. The proposed image description method characterizes the spectral content by three different novel spectral descriptors that are: raw pixel values, simple bag of spectral values and the extended bag of spectral values descriptors. To model the spatial content of RS images, we consider the well-known scale invariant feature transform-based bag of visual words approach. With the conjunction of the spatial and the spectral descriptors, RS image retrieval is achieved by a novel sparse reconstruction-based RS image retrieval method. The proposed method considers a novel measure of label likelihood in the framework of sparse reconstruction-based classifiers and generalizes the original sparse classifier to the case both single-label and multi-label RS image retrieval problems. Finally, to enhance retrieval performance, we introduce a strategy to exploit the sensitivity of the sparse reconstruction-based method to different dictionary words. Experimental results obtained on two benchmark archives show the effectiveness of the proposed system.

Using Diverse Set of Features to Design a Content-Based Video Retrieval System Optimized by Gravitational Search Algorithm

This paper explains about the content-based video retrieval approach (CBVR) using four varieties of features and 12 distance measurements, which is optimised by gravitational search algorithm (GSA). Initially, CBVR technique extracts five kinds of features such as colour, texture, shape, image and audio features that belong to each frame. Consequently, it emerges particular distance measurements for every sort of features to compute the similarity between query frame and remaining in the database frame. In this paper, we have used GSA to find the nearly optimal combination between the features and their respective similarity measurements. At last, from the video database, the query-based videos are recovered. For experimentation, here we used two types of databases such as sports video and UCF sports action datasets. The experimental results demonstrate that the proposed CBVR method shows better performance when contrasted with other existing methods.

Alike Scene Retrieval from Land-Cover Products Based on the Label Co-Occurrence Matrix (LCM) †

The management and application of remotely sensed data has become much more difficult due to the dramatically growing volume of remotely sensed imagery. To address this issue, content-based image retrieval (CBIR) has been applied to remote sensing image retrieval for information mining. As a consequence of the growing volume of remotely sensed imagery, the number of different types of image-derived products (such as land use/land cover (LULC) databases) is also increasing rapidly. Nevertheless, only a few studies have addressed the exploration and information mining of these products. In this letter, for the sake of making the most use of the LULC map, we propose an approach for the retrieval of alike scenes from it. Based on the proposed approach, we design a content-based map retrieval (CBMR) system for LULC. The main contributions of our work are listed below. Firstly, the proposed system can allow the user to select a region of interest as the reference scene with variable shape and size. In contrast, in the traditional CBIR/CBMR systems, the region of interest is usually of a fixed size, which is equal to the size of the analysis window for extracting features. In addition, the user can acquire various retrieval results by specifying the corresponding parameters. Finally, by combining the signatures in the base signature library, the user can acquire the retrieval result faster.

A Neutrosophic Image Retrieval Classifier

In this paper, we propose a two-phase Content-Based Retrieval System for images embedded in the Neutrosophic domain. In this first phase, we extract a set of features to represent the content of each image in the training database. In the second phase, a similarity measurement is used to determine the distance between the image under consideration (query image), and each image in the training database, using their feature vectors constructed in the first phase. Hence, the N most similar images are retrieved.

A Neutrosophic Image Retrieval Classifier

In this paper, we propose a two-phase Content-Based Retrieval System for images embedded in the Neutrosophic domain. In this first phase, we extract a set of features to represent the content of each image in the training database. In the second phase, a similarity measurement is used to determine the distance between the image under consideration (query image), and each image in the training database, using their feature vectors constructed in the first phase. Hence, the N most similar images are retrieved.

A learning automata framework based on relevance feedback for content-based image retrieval

The need for efficient image browsing and searching motivates the use of Content-Based Image Retrieval (CBIR) systems. However, they suffer from a big gap between high-level image semantics and low-level features. So, a learning process to reduce the gap seems quite useful. This paper presents a novel Learning Automata (LA)-based approach to improve the CBIR systems. Distributed Learning Automata (DLA) is used in this work to learn the relevant images from textual query feedbacks of the users. Subsequently, the retrieved images are ranked according to the learning outcome and similarity measure. In this study, the similarity between images is evaluated based on two color descriptors: the global color histogram and local color auto-correlogram. A thorough observation and comparison of these color descriptors performances are performed with different color spaces and also with various similarity measures. Experimental results on two publicly available databases demonstrate that the performance of the proposed CBIR system after each round is improved and the system could retrieve images compatible with the users’ perception.

In Defense of Locality-Sensitive Hashing.

Hashing-based semantic similarity search is becoming increasingly important for building large-scale content-based retrieval system. The state-of-the-art supervised hashing techniques use flexible two-step strategy to learn hash functions. The first step learns binary codes for training data by solving binary optimization problems with millions of variables, thus usually requiring intensive computations. Despite simplicity and efficiency, locality-sensitive hashing (LSH) has never been recognized as a good way to generate such codes due to its poor performance in traditional approximate neighbor search. We claim in this paper that the true merit of LSH lies in transforming the semantic labels to obtain the binary codes, resulting in an effective and efficient two-step hashing framework. Specifically, we developed the locality-sensitive two-step hashing (LS-TSH) that generates the binary codes through LSH rather than any complex optimization technique. Theoretically, with proper assumption, LS-TSH is actually a useful LSH scheme, so that it preserves the label-based semantic similarity and possesses sublinear query complexity for hash lookup. Experimentally, LS-TSH could obtain comparable retrieval accuracy with state of the arts with two to three orders of magnitudes faster training speed.

Content based Image Retrieval using Selective Region Matching with Region of Interest and SVM

Recently, very large collections of images and videos have grown rapidly. In parallel with this growth, content-based retrieval and querying the indexed collections are required to access visual information. Three main components of the visual information are color, texture and shape. In this paper, a selective region based content-based image retrieval system is presented that combines two visual descriptors of images and measures similarity of images by applying a SVM Classification. Paper. Here, the selective region matching with Region of Interest and SVM classification based CBIR retrieval system for imaging is presented in order to provide better image classification and fast image retrieval. In the proposed method the color and texture features like contrast, mean, standard deviation, energy and entropy are extracted from the image. Also it is shown through experimental results and analysis of retrieval effectiveness of querying that the content-based retrieval system is effective in terms of retrieval and scalability.

Automatic and online setting of similarity thresholds in content-based visual information retrieval problems

Several information recovery systems use functions to determine similarity among objects in a collection. Such functions require a similarity threshold, from which it becomes possible to decide on the similarity between two given objects. Thus, depending on its value, the results returned by systems in a search may be satisfactory or not. However, the definition of similarity thresholds is difficult because it depends on several factors. Typically, specialists fix a threshold value for a given system, which is used in all searches. However, an expert-defined value is quite costly and not always possible. Therefore, this study proposes an approach for automatic and online estimation of the similarity threshold value, to be specifically used by content-based visual information retrieval system (image and video) search engines. The experimental results obtained with the proposed approach prove rather promising. For example, for one of the case studies, the performance of the proposed approach achieved 99.5 % efficiency in comparison with that obtained by a specialist using an empirical similarity threshold. Moreover, such automated approach becomes more scalable and less costly.

A multimodal approach for extracting content descriptive metadata from lecture videos

The rapidly increasing availability of e-learning content and lecture videos over the internet, has brought forth an imperative need for developing effective content based retrieval systems. Comprehensive metadata extraction and support for topic-level search within videos are key factors in developing such systems. In this paper, we propose a multimodal metadata extraction system which extracts an optimal set of keyphrases and topic based segments that effectively summarize the content of a lecture video. The extraction process utilizes features from both audio transcripts and slide content in video streams. A hybrid approach combining a Naive Bayes classifier and a rule-based refiner is used for effective retrieval of the metadata in a lecture. The proposed content-descriptive metadata extraction technique has been evaluated using actual lecture videos from different sources, and our results show that our multimodal approach is effective in summarizing the lecture's content, potentially improving the user experience during retrieval and browsing.