Query Sketch Research Articles

Sketch-based 3D model retrieval utilizing adaptive view clustering and semantic information

Searching for relevant 3D models based on hand-drawn sketches is both intuitive and important for many applications, such as sketch-based 3D modeling and recognition, human computer interaction, 3D animation, game design, and etc. In this paper, our target is to significantly improve the current sketch-based 3D retrieval performance in terms of both accuracy and efficiency. We propose a new sketch-based 3D model retrieval framework by utilizing adaptive view clustering and semantic information. It first utilizes a proposed viewpoint entropy-based 3D information complexity measurement to guide adaptive view clustering of a 3D model to shortlist a set of representative sample views for 2D-3D comparison. To bridge the gap between the query sketches and the target models, we then incorporate a novel semantic sketch-based search approach to further improve the retrieval performance. Experimental results on several latest benchmarks have evidently demonstrated our significant improvement in retrieval performance.

Innovative Sketch Board Mining for Online image Retrieval

Developing an accurate and efficient Sketch-Based Image Retrieval (SBIR) method in determining the resemblances between the user's query and image stream has been a never-ending quest in digital data communication era. The main challenge is to overcome the asymmetry between a binary sketch and a full-color image. We introduce a unique sketch board mining method to recover the online web images. This image conceptual retrieval is performed by matching the sketch query with the relevant terminology of selected images. A systematic sequence is followed, including the sketch drawing by the user in interpreting its geometrical shape of the conceptual form based on annotation metadata matching technique achieved automatically from Google engines, indexing and clustering the selected images via data mining. The sketch mining board being built in dynamic drawing state used a set of features to generalize sketch board conceptualization in semantic level. Images from the global repository are retrieved via a semantic match of the user's sketch query with them. Excellent retrieval of hand-drawn sketches is found to achieve the recall rate within 0.1 to 0.8 and a precision rate is 0.7 to 0.98. The proposed technique solved many problems that stat-of-art suffered from SBIR (e.g. scaling, transport, imperfect) sketch. Furthermore, it is demonstrated that the proposed technique allowed us to exploit high-level features to search the web effectively and may constitute a basis for efficient and precise image recovery tool.

An Efficient Image Retrieval Scheme for Sketches using Fish Swarm Optimization with the Aid of Optimal Score Level Fusion

Objectives: Image retrieval is system software for browsing, examining and retrieving images from a large database of images. Images and sketches do not share numerous common modalities. Hence, Sketch to Image Retrieval is a tedious task in image processing. Sketch image retrieval focuses on the hand-drawn query and retrieves the similar images from a large database which is useful for further processing.Methods/Statistical Analysis:In the, most of the traditional/conventional image processing techniques considered edges and outlines for image retrieval. In this paper, a new methodology is developed by fusion of Edge Histogram Descriptors, Histogram of oriented gradients, Scale Invariant Feature Transform (SIFT) and Speeded up Robust Features (SURF). In the proposed model first feature Extraction is carried out and Euclidian distance is calculated amongst the query sketch and innovative image. Formerly the feature vectors are provided to the score level fusion stage, and then they obtained results are optimized. For optimization, Fish Swarm Optimization (FSO) is employed in the proposed method. Findings: The performance of the proposed method is evaluated through Benchmark sketch image database. Also, the attained results are compared with the existing evolutionary algorithm Genetic Algorithm (GA). Application/ Improvement: The experimental results showed that the projected method with FSO yields better results than GA.

Face sketch-photo recognition using local gradient checksum: LGCS

A new approach for matching of face sketch images with face photo images and vice versa has been presented here. For the extraction of local edge features from both the sketch and photo images, a new local feature called local gradient checksum (LGCS) has been developed. LGCS is a modality reduction local edge feature on gradient domain. It is calculated as the summation of four pairs of gradient differences between two local pixels that are at 180° with each other. The Euclidean distance between query sketch and gallery of photos are measured depending on extracted LGCS features. To further improve the result, a multi-scale LGCS is proposed. A rank-1 accuracy of 100 % is achieved in a gallery of 606 photos consisting of CUHK, AR, and XM2VTS face dataset. The proposed face sketch-photo recognition system requires neither learning procedures nor training data. Further, the experiment is extended to test the robustness of the proposed algorithm on blurred, noisy and disguised sketches, as well as photos. Under those situations also, LGCS has outperformed center-symmetric local binary pattern, directional local extrema pattern and weber local descriptor feature extraction techniques.

Learning Cross-Domain Neural Networks for Sketch-Based 3D Shape Retrieval

Sketch-based 3D shape retrieval, which returns a set of relevant 3D shapes based on users' input sketch queries, has been receiving increasing attentions in both graphics community and vision community. In this work, we address the sketch-based 3D shape retrieval problem with a novel Cross-Domain Neural Networks (CDNN) approach, which is further extended to Pyramid Cross-Domain Neural Networks (PCDNN) by cooperating with a hierarchical structure. In order to alleviate the discrepancies between sketch features and 3D shape features, a neural network pair that forces identical representations at the target layer for instances of the same class is trained for sketches and 3D shapes respectively. By constructing cross-domain neural networks at multiple pyramid levels, a many-to-one relationship is established between a 3D shape feature and sketch features extracted from different scales. We evaluate the effectiveness of both CDNN and PCDNN approach on the extended large-scale SHREC 2014 benchmark and compare with some other well established methods. Experimental results suggest that both CDNN and PCDNN can outperform state-of-the-art performance, where PCDNN can further improve CDNN when employing a hierarchical structure.

Survey paper on Sketch Based and Content Based Image Retrieval

This survey paper presents an overview of development of Sketch Based Image Retrieval (SBIR) and Content based image retrieval (CBIR) in the past few years. There is awful growth in bulk of images as well as the far-flung application in too many fields. The main attributes to represent as well index the images are color, shape, texture, spatial layout. These features of images are extracted to check similarity among the images. Generation of special query is the main problem of content based image retrieval. Features are extracted from the images and the similarity between them is check by using different algorithms. Sketch based image retrieval is effective and chief method which are not necessarily having a high skill to draw the query sketch. Over last twenty years different algorithms and models are explored for the retrieval of images. In concluding section we present the limitations of the current image retrieval algorithms.

Edge Detection Algorithms VS-active Contour for Sketch Matching: Comparative Study

Sketch based image retrieval method is still a new filed under development process in the research area. The main idea of image retrieval by sketch is to detect or bound the object in images which stored in the database and match it with a query which inserted as a sketch. This study displays the edge detection algorithms like (Sobel, Canny) and compare the results with contour active (Snakes) boundary methods. The results of those methods prepared to match with the query sketch. The experimental results show that Snakes algorithms registered as the best in matching process which fabricate a differentiated boundary detection of image objects. For more validity of this method used, the annotation of each matched image has been checked and approved that the contour (Snakes) boundary method is more precise in image retrieval.

Enhancing Sketch-Based Image Retrieval by Re-Ranking and Relevance Feedback.

A sketch-based image retrieval often needs to optimize the tradeoff between efficiency and precision. Index structures are typically applied to large-scale databases to realize efficient retrievals. However, the performance can be affected by quantization errors. Moreover, the ambiguousness of user-provided examples may also degrade the performance, when compared with traditional image retrieval methods. Sketch-based image retrieval systems that preserve the index structure are challenging. In this paper, we propose an effective sketch-based image retrieval approach with re-ranking and relevance feedback schemes. Our approach makes full use of the semantics in query sketches and the top ranked images of the initial results. We also apply relevance feedback to find more relevant images for the input query sketch. The integration of the two schemes results in mutual benefits and improves the performance of the sketch-based image retrieval.

Sketch4Image: a novel framework for sketch-based image retrieval based on product quantization with coding residuals

Sketch-based Image Retrieval (SBIR) is one important branch of Content-based Image Retrieval (CBIR). SBIR means dealing with retrieval using simple edge or contour images. However, SBIR is more difficult than CBIR due to the lack of visual information, this makes the Bag-of-Words (BoW) or codebook in SBIR hard to construct. In this paper, we propose a novel SBIR framework based on Product Quantization (PQ) with sparse coding (SC) to construct an optimized codebook. By using state-of-the-art local descriptors, we transform sketch images into features and then build the optimized codebook using PQ-based SC. In the retrieval stage, we can obtain a better representation of the query sketch and testing images by the optimized codebook with coding quantization residuals, by which the information loss during feature encoding process can be reduced; similarity computing is implemented by comparing the feature histograms between a query sketch and the testing data for the final results. We demonstrate the superiority and effectiveness of the proposed SBIR by comparing it with several state-of-the-art methods on three public sketch datasets.

A Methodology for Sketch based Image Retrieval based on Score level Fusion

sketches to match with a hand drawn sketch query is highly desired feature. This paper proposes a novel methodology for efficient retrieval of sketch based images. This system extracts features from the query sketch, HOG and GMM features are used and these features are combined using score level fusion which can match user drawn sketch with database sketches efficiently. The methodology is tested on bench mark images and the performance evaluation is carried out using metrics like Precession and Recall. The results derived are tested for efficiency against models based on HOG and GMM.

A novel sketch-based 3D model retrieval method by integrating skeleton graph and contour feature

With the rapid growth of 3D models on the Web, effective methods to retrieve appropriate 3D models are becoming crucial. In this paper, we propose a novel sketch-based 3D model retrieval approach which integrates the skeleton graph and contour feature. Skeleton graph represents the topology structure of the query sketch as well as representative 2D views of 3D models, and contour feature captures their detailed information. Moreover, A hierarchical coarse-to-fine searching strategy is employed in this method. Firstly, a skeleton graph based matching method is used to eliminate those 3D models which are dissimilar to the query sketch. Then, the selected candidate models are refined by measuring the similarity of their contour feature. In addition, we adopt a graph transduction method to improve the ranking order of the retrieved 3D models. The proposed method was tested on the public sketch benchmarks and compared with other leading sketch-based 3D model retrieval methods. The experiment results demonstrate that our approach can accurately retrieve desired 3D models for users and has a superior performance to other state-of-the-art approaches.

Retrieval of Flower Based on Sketches

In this paper, we propose a model for representation and indexing of flower images for the purpose of retrieving flowers of interest based a query sketch. In several applications, where the size of a database is supposed to be very huge, then the retrieval procedure typically takes unacceptably a long response time. A standard way to speed up the retrieval procedure is to design an indexing scheme prior to retrieval, which results in the reduction of number of flowers considered for searching by the retrieval algorithm. In this work, we swot the correctness of Kd-tree indexing scheme for flower retrieval system based on shape descriptors viz., Scale Invariant Feature Transform (SIFT), Histogram of Gradients (HOG) and Edge Orientation Histograms (EOH). To uphold the efficacy of the proposed method, an experiment has been conducted on our own flower data set of 127 classes of flowers, consists of 13169 flower images. For querying the database we collected about 100 flower sketches from 20 different users. Experimental results reveal the success of the proposed method.

BHoG: binary descriptor for sketch-based image retrieval

Due to the popularity of devices with touch screens, it is convenient to match images with a hand-drawn sketch query. However, existing methods usually care little about memory space and time efficiency thus is inadequate for the rapid growth of multimedia resources. In this paper, a BHoG descriptor is proposed for sketch-based image retrieval. Firstly, the boundary image is detected from natural image using Berkeley boundary detector, and then divided into many blocks. Secondly, we calculate the gradient feature of each block, and find the principal gradient orientation. Finally, the principal gradient orientation is encoded to binary codes, which is proved to be efficient and discriminative. We evaluated the performance of BHoG on a large-scale social media dataset. The experimental results have shown that BHoG not only has a better performance on flexibility and efficiency, but also occupies small memory.

Viewpoint-Aware Representation for Sketch-Based 3D Model Retrieval

We study the problem of sketch-based 3D model retrieval, and propose a solution powered by a new query-to-model distance metric and a powerful feature descriptor based on the bag-of-features framework. The main idea of the proposed query-to-model distance metric is to represent a query sketch using a compact set of sample views (called basic views) of each model, and to rank the models in ascending order of the representation errors. To better differentiate between relevant and irrelevant models, the representation is constrained to be essentially a combination of basic views with similar viewpoints. In another aspect, we propose a mid-level descriptor (called BOF-JESC) which robustly characterizes the edge information within junction-centered patches, to extract the salient shape features from sketches or model views. The combination of the query-to-model distance metric and the BOF-JESC descriptor achieves effective results on two latest benchmark datasets.

Similarity metric learning for sketch-based 3D object retrieval

Sketch-Based 3D Object Retrieval (SB3DOR) algorithms retrieve 3D models similar to hand-drawn sketch queries. It is one of the most effective modalities to query 3D models by their shape. However, comparison of a sketch, which is a 2D image, with a 3D model is not straightforward. Most of the SB3DOR algorithms compare a sketch image with images of 3D models rendered from multiple viewpoints. However, retrieval accuracies of state-of-the-art SB3DOR algorithms are still not satisfactory, due, in part, to stylistic variation, semantic influence, abstraction, and drawing error found in sketches. To improve retrieval accuracy of SB3DOR systems, we propose a manifold-based similarity metric learning algorithm that relates two kinds of features, that are, of sketches and 3D models. Features in a high dimensional space often lie on a lower dimensional subspace, or manifold. Feature similarity may be computed more accurately on the manifold than in the original high dimensional space. Our Cross-Domain Manifold Ranking (CDMR) algorithm tries to keep the two distinct feature manifolds, one for sketch features and the other for 3D model features, intact. These two manifolds are interlinked by using inter-feature similarity to form a Cross-Domain Manifold (CDM). If available, semantic labels may also be used in forming the CDM. Relevance diffusion is used to compute similarities between a sketch and 3D models in a database. Experimental evaluation showed that the CDMR algorithm produces higher retrieval accuracy than the algorithms we compared against.

Matching Facial Composite Sketches to Police Mug-Shot Images Based on Geometric Features

Composite sketches are frequently used to help track down suspects in a crime in which there is no photographic evidence. Automatic retrieval of photos of suspects from police mug-shot databases can help police narrow down potential suspects quickly. This paper describes a novel approach for matching facial composite sketches to police mug-shot images. Proposed method is based on Geometric Features. To recognize a facial composite we focussed on face geometric features like eyes, eyebrows, nose, lips, and face cutting along with their length and width ratios. This system has five components 1) Extract components from a face image/composite sketch 2) Compute Ratios of Length, Width and Area 3) Represent each image component as a vector 4) Mean of the vectors are computed and subtracted from each component vector to get a set of vectors called zero-mean vectors 5) Searching for mug shot images in the database based on a query sketch drawn by an artist. It has useful applications for both law enforcement and digital entertainment. We assume that mug shot images to be studied are in frontal pose, under normal lighting with neutral expression, and have no occlusion. Extensive experiment has been conducted with 50 male and female face/sketch database from CUHK face database.

A new sketch-based 3D model retrieval approach by using global and local features

With the rapid growth of available 3D models, fast retrieval of suitable 3D models has become a crucial task for industrial applications. This paper proposes a novel sketch-based 3D model retrieval approach which utilizes both global feature-based and local feature-based techniques. Unlike current approaches which use either global or local features, as well as do not take into account semantic relations between local features, we extract these two kinds of feature information from the representative 2D views of 3D models that can facilitate semantic description and retrieval for 3D models. Global features represent the gross exterior boundary shape information, and local features describe the interior details by compact visual words. Specifically, an improved bag-of-features method is provided to extract local features and their latent semantic relations. In addition, an efficient two-stage matching strategy is used to measure the distance between the query sketch and 3D models for selection and refinement. Experiment results demonstrate that our approach which combines these two kinds of complementary features significantly outperforms several state-of-the-art approaches.

Utilization of Textural Features in Video Retrieval System by Hand-writing Sketch

In this paper we propose a video retrieval system using texture features from a hand-drawn sketch. There is currently a lot of video content on the Internet due in part to the development of video-sharing Web sites, and sometimes users are unable to narrow the options down to the desired video because there is simply too much content to choose from. We should be able to solve this problem with a retrieval technique that uses internal movie features. Previously, our laboratory proposed a video retrieval system using three internal movie features and a sketch query. In the current study, our aim is to improve the retrieval precision by adding a new feature derived from texture.

Sketch-based shape retrieval

We develop a system for 3D object retrieval based on sketched feature lines as input. For objective evaluation, we collect a large number of query sketches from human users that are related to an e...

Sketch-based shape retrieval

We develop a system for 3D object retrieval based on sketched feature lines as input. For objective evaluation, we collect a large number of query sketches from human users that are related to an existing data base of objects. The sketches turn out to be generally quite abstract with large local and global deviations from the original shape. Based on this observation, we decide to use a bag-of-features approach over computer generated line drawings of the objects. We develop a targeted feature transform based on Gabor filters for this system. We can show objectively that this transform is better suited than other approaches from the literature developed for similar tasks. Moreover, we demonstrate how to optimize the parameters of our, as well as other approaches, based on the gathered sketches. In the resulting comparison, our approach is significantly better than any other system described so far.