Hand-drawn Sketches Research Articles

Forensic Face Sketch Recognition

In modern society, the overall crime rate is increasing day by day, and in order to overcome this, law enforcement department must also find ways to speed up the entire process and help bring justice. One such method is to use facial recognition technology to identify the criminal. The traditional approach here is to use a hand-drawn sketch of a face, drawn by forensic artist to identify the criminal. Upgrading it will allow us to identify the criminals using hand-drawn sketches and then matching it with law enforcement databases. This approach is time consuming as there are few forensic artists available compared to the increasing crime ratio which is subject to various limitations with the latest technology and increasing crime rates. Our project provides law enforcement department a standalone platform that allows users to create composite sketches of a suspect's face using the drag-and-drop feature without the aid of forensics, thereby reducing time gaps and speeding up the process. The application automatically matches complex face sketches with police database.

Retraction Note to: Automated Hand-drawn sketches retrieval and recognition using regularized Particle Swarm Optimization based deep convolutional neural network

Retraction Note to: Automated Hand-drawn sketches retrieval and recognition using regularized Particle Swarm Optimization based deep convolutional neural network

Generating Website Codes from Images

Brainstorm website layout ideas usually start with everyone giving out their mockups, and all the team members will discuss to finalize the layout of the user interface. Once a vision of that mockup is accepted, it is given to the designer to sketch it digitally on computer software (i.e., Photoshop, Figma, Sketch). When the designer completes, the developer based on the final design to code the UI/UX of the website. As we can see, the process requires three stages, which can be time-consuming. Therefore, if anyone has an idea for the professional website layout, they can visualize it by drawing on sketches. However, it can be impossible for them to make a usable website without designers and website developers. Due to that reason, our primary goal in this paper is to help individuals transform their hand-drawn sketch images into a website that can be deployed. To achieve that goal, we present two approaches: classical computer vision techniques and the other using a deep learning model to detect the sketch and execute the conversion. Furthermore, our evaluation shows that deep learning is the most promising direction. Still, classical techniques also improve the model’s input data by applying it in the pre-processing image.

Exploring the Spatial Image of Traditional Villages from the Tourists’ Hand-Drawn Sketches

As an important concept in cognitive psychology and behavioural geography, destination spatial image cognition has a significant impact on the quality of tourists’ experience, and on their behavioural intention. However, studies of spatial image cognition in small-scale traditional villages are limited. Therefore, the present study analyses the spatial image characteristics of four traditional villages of World Cultural Heritage sites in China through the use of tourists’ hand-drawn sketches, using a sample of 366 respondents to further explore the evolution process of cognitive map types and constituent elements with tourists’ stay days. Results indicate that the spatial cognitive map and landmarks are the main types and dominant elements of spatial image cognition, respectively. The tourists’ spatial cognitive process includes two sequences, as follows: the evolution sequence of dominant cognitive maps is “spatial + individual → spatial + individual + hybrid → spatial + individual”, while the evolution sequence of dominant cognition elements is “landmark + path + animal and plant → landmark + animal and plant + path”. This study extends the current destination spatial image cognition literature, and has substantial value for the destination in terms of developing traditional village sustainable tourism based on the tourists’ attitude, as obtained by the cognitive map method.

Sketch to Building: Architecture Image Translation Based on GAN

Image-to-image translation with Generative Adversarial Networks has been extensively studied recently. However, it remains still very challenging for sketch-to-image translation owing to large domain gap and information asymmetry problems. In this paper, we introduce a novel two-step method as well as designing an interactive system which can generate different styles of architecture images from a hand-drawn sketch. In our framework, either an architecture sketch can be drawn and then translated into a color image, or an architecture photo can be loaded, followed by sketch generation, modification, and translation. Style images can also be manually selected for different output styles. Experimental results show that our method has better performance on both image quality and diversity than the other competitive methods.

CHEHRA: An Application for Forensic Face Sketch Construction and Recognition

In this modern age, the overall crime rate is increasing day-by-day and to cope up with this the law enforcement departments too should find ways that would speed up the overall process and help them in bringing one to justice. One such way can be using face recognition technology for identifying and verifying the criminal. The traditional approach here is to use the hand-drawn face sketches drawn by forensic sketch artist to identify the criminal, modernizing this would mean using the hand drawn sketch and then matching them with the law enforcement departments database to identify the criminal. We believe, there is a need for creating an application which would not just provide a set of individual features like eyes, ears, mouth, etc. to be selected to create a face sketch but also would help in finding the criminal much faster and efficiently.

Exploring Local Detail Perception for Scene Sketch Semantic Segmentation.

In this paper, we aim to explore the fine-grained perception ability of deep models for the newly proposed scene sketch semantic segmentation task. Scene sketches are abstract drawings containing multiple related objects. It plays a vital role in daily communication and human-computer interaction. The study has only recently started due to a main obstacle of the absence of large-scale datasets. The currently available dataset SketchyScene is composed of clip art-style edge maps, which lacks abstractness and diversity. To drive further research, we contribute two new large-scale datasets based on real hand-drawn object sketches. A general automatic scene sketch synthesis process is developed to assist with new dataset composition. Furthermore, we propose to enhancing local detail perception in deep models to realize accurate stroke-oriented scene sketch segmentation. Due to the inherent differences between hand-drawn sketches and natural images, extreme low-level local features of strokes are incorporated to improve detail discrimination. Stroke masks are also integrated into model training to guide the learning attention. Extensive experiments are conducted on three large-scale scene sketch datasets. Our method achieves state-of-the-art performance under four evaluation metrics and yields meaningful interpretability via visual analytics.

Small facial image dataset augmentation using conditional GANs based on incomplete edge feature input.

Image data collection and labelling is costly or difficult in many real applications. Generating diverse and controllable images using conditional generative adversarial networks (GANs) for data augmentation from a small dataset is promising but challenging as deep convolutional neural networks need a large training dataset to achieve reasonable performance in general. However, unlabeled and incomplete features (e.g., unintegral edges, simplified lines, hand-drawn sketches, discontinuous geometry shapes, etc.) can be conveniently obtained through pre-processing the training images and can be used for image data augmentation. This paper proposes a conditional GAN framework for facial image augmentation using a very small training dataset and incomplete or modified edge features as conditional input for diversity. The proposed method defines a new domain or space for refining interim images to prevent overfitting caused by using a very small training dataset and enhance the tolerance of distortions caused by incomplete edge features, which effectively improves the quality of facial image augmentation with diversity. Experimental results have shown that the proposed method can generate high-quality images of good diversity when the GANs are trained using very sparse edges and a small number of training samples. Compared to the state-of-the-art edge-to-image translation methods that directly convert sparse edges to images, when using a small training dataset, the proposed conditional GAN framework can generate facial images with desirable diversity and acceptable distortions for dataset augmentation and significantly outperform the existing methods in terms of the quality of synthesised images, evaluated by Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) scores.

Remote Instruction for Data Visualization Design-A Report From the Trenches.

In this article, we report on our experiences of running visual design workshops within the context of a master's level data visualization course, in a remote setting. These workshops aim to teach students to explore visual design space for data by creating and discussing hand-drawn sketches. We describe the technical setup employed, the different parts of the workshop, how the actual sessions were run, and to what extent the remote version can substitute for in-person sessions. In general, the visual designs created by the students as well as the feedback provided by them indicate that the setup described here can be a feasible replacement for in-person visual design workshops.

Cooperation B.V. Klyuchevsky with S.F. Platonov on the distribution of his father's work

Much of S.F. Platonov's organizational work remains unknown. The revealed archival materials in his personal fund allow us to expand our understanding of his personal interest in providing the students of the Women's Pedagogical Institute, where he was the director, with the best scientific literature. The director's hand-drawn rough sketches provide an insight into the extent of his collaboration and informal relationships with his female students. The office materials introduced into scientific circulation for the first time give an idea of the unknown side of the activity of B.V. Klyuchevsky the son of the great historian. In historiography, the opinion has been established that he mainly performed the duties of secretary to his father. The researchers ' interest in the tragic fate of the only child of V.O. Klyuchevsky began to manifest itself recently. He was not spared the fate of a prisoner exiled to Kazakhstan. Previously unknown, his work in the warehouse of V.O. Klyuchevsky's printed products allows him to open a new page of his studies. Despite a certain dryness of office materials (receipts of postal orders, forms, invoices), they in this case allow us to analyze the functioning of the warehouse for the distribution of V.O. Klyuchevsky's works, to find out the scale of trade. Some inaccuracies established in historiography concerning the facts from the life of S.F. Platonov and B.V. Klyuchevsky are corrected

AE-Net: Fine-grained sketch-based image retrieval via attention-enhanced network

In this paper, we investigate the task of Fine-grained Sketch-based Image Retrieval (FG-SBIR), which uses hand-drawn sketches as input queries to retrieve the relevant images at the fine-grained instance level. The sketches and images come from different modalities, thus the similarity computation needs to consider both fine-grained and cross-modal characteristics. Existing solutions only focus on fine-grained details or spatial contexts, while ignoring the channel context and spatial sequence information. To mitigate such challenging problems, we propose a novel deep FG-SBIR model, which aims at inferring attention maps along channel dimension and spatial dimension, improving modules of channel attention and spatial attention, and exploring Transformer to enhance the model’s ability for constructing and understanding spatial sequence information. We focus not only on the correlation information between two modalities of sketch and image, but also on the discrimination information inside the single modality. Mutual Loss is especially proposed to enhance the traditional triplet loss, and promote the internal discrimination ability of the model on a single modality. Extensive experiments show that our AE-Net obtains promising results on Sketchy, which is the largest public dataset available for FG-SBIR at present.

Hand-drawn sketch recognition with a double-channel convolutional neural network

In hand-drawn sketch recognition, the traditional deep learning method has the problems of insufficient feature extraction and low recognition rate. To solve this problem, a new algorithm based on a dual-channel convolutional neural network is proposed. Firstly, the sketch is preprocessed to get a smooth sketch. The contour of the sketch is obtained by the contour extraction algorithm. Then, the sketch and contour are used as the input image of CNN. Finally, feature fusion is carried out in the full connection layer, and the classification results are obtained by using a softmax classifier. Experimental results show that this method can effectively improve the recognition rate of a hand-drawn sketch.

3D SKETCHES IN VIRTUAL REALITY AND THEIR EFFECT ON DEVELOPMENT TIMES

AbstractIn the product development process, digital support continues to advance. Some work steps during product development are still carried out without assistance. Sketch creation is one of these. Therefore, the content created here is rarely documented due to the effort required for digital transformation. An alternative can be sketching in virtual reality. This article explores whether 3D sketching in VR enables faster sketching and can offer the basic features of hand-drawn sketches. To verify this, a tool for 3D sketching was developed. 27 test subjects were asked to solve one out of two different design tasks using this tool. The experiments were evaluated using video coding to identify the subjects actions. The created solutions have been analyzed about quality. The study showed initial indications that sketching in VR generally enables faster processing while maintaining the same solution quality.

‘CADSketchNet’ - An Annotated Sketch dataset for 3D CAD Model Retrieval with Deep Neural Networks

Ongoing advancements in the fields of 3D modelling and digital archiving have led to an outburst in the amount of data stored digitally. Consequently, several retrieval systems have been developed depending on the type of data stored in these databases. However, unlike text data or images, performing a search for 3D models is non-trivial. Among 3D models, retrieving 3D Engineering/CAD models or mechanical components is even more challenging due to the presence of holes, volumetric features, presence of sharp edges etc., which make CAD a domain unto itself. The research work presented in this paper aims at developing a dataset suitable for building a retrieval system for 3D CAD models based on deep learning. 3D CAD models from the available CAD databases are collected, and a dataset of computer-generated sketch data, termed ‘CADSketchNet’, has been prepared. Additionally, hand-drawn sketches of the components are also added to CADSketchNet. Using the sketch images from this dataset, the paper also aims at evaluating the performance of various retrieval system or a search engine for 3D CAD models that accepts a sketch image as the input query. Many experimental models are constructed and tested on CADSketchNet. These experiments, along with the model architecture, choice of similarity metrics are reported along with the search results.

Survey on Hand Drawn Symbol Classification and Recognition

Due to this worldwide pandemic situation, there undergone a transformation in the education domain (schools, higher education, research, etc.,) into virtual throughout the world. It made several domains in education field challenging, where teaching-learning goes in an entire virtual manner. Subsequently, there is growing attention in making systems which can automatically recognize hand-drawn sketches. In contrast, many challenges persist with respect to accuracy of recognition, robustness to diverse drawing patterns, and capability to simplify amongst multiple domains. To tackle these issues, a different approach to hand drawn sketch recognition which emphasizes the diagrammatic appearance of the symbols is familiarized. This allows it to well handle the variety of visual variations observed in freehand drawings. A novel symbol classifier that is computationally efficient and invariant to change and local distortions is projected in few papers which reveals that this method surpasses performance on all domains evaluated in different works carried, including handwritten digits, and circuit symbols. Electric symbols are the basis of studies in electrical theory. The circuit diagram delivers many details regarding the system. Beneath any power-driven device there exists numerous electrical elements which do their individual functions, nowadays altogether the electrical package tools failed to efficiently transform the data spontaneously from the circuit diagram to a digital form. Henceforth, electrical engineers must by hand enter all data into PCs, and this procedure consumes time and gives errors with extreme possibility. Furthermore, when the diagram is drawn by hand, the problem gets still complicated for electrical analysis. Therefore, a unique method using Convolution Neural Network (CNN) plus other tactics are exploited in diverse types of literature to make a machine directly read the symbols from a hand-drawn image which is very much beneficial to solve this problem. This paper focuses on the detailed survey of methodologies introduced for automatic hand drawn sketch recognition. Particularly, the recognition procedure in CNN involves two steps: first is feature extraction using shape-based features, next the second step is a classification technique using CNN via a back-propagation algorithm which finally gave better recognition comparatively.

Héðinsdalsjökull, northern Iceland: geomorphology recording the recent complex evolution of a glacier

ABSTRACT The objective of this work is to conduct a detailed mapping of the Héðinsdalsjökull foreland, northern Iceland (65°39′N, 18°55′W). This cirque currently shows a variety of glacial and periglacial landforms derived from a complex deglaciation. Mapping was performed combining traditional hand-drawn and digital mapping. A hand-drawn sketch was georeferenced in ArcMap 10.7.1, supported on an aerial photograph (year 2000). Its vectorization, symbolization and final design were done in the computer-aided design (CAD) software MicroStation Connect. Complementary high-resolution Digital Surface Models were obtained from historical aerial photographs and ground-view field photographs through the application of Structure from Motion (SfM) photogrammetry. To improve the topographic expression of the geomorphological map, a photorealistic 3D view has been generated. The final map highlights the complexity of the foreland and the coexistence existence of a range of different units and landforms. The map will ease future studies on the transformation of receding glaciers.

Proposal of Image generation model using cGANs for sketching faces

The transition from sketches to realistic images of human faces has an important application in criminal investigation science to find criminals as depicted by witnesses. However, due to the difference between the sketch image and the real face image in terms of image detail and color, it is challenging and takes time to transform from hand-drawn sketches to actual faces. To solve this problem, we propose an image generation model using the conditional generative adversarial network with autoencoder (cGANs-AE) model to generate synthetic samples for variable length and multi-feature sequence datasets. The goal of the model is to learn how to encode a dataset that reduces its vector size. Using a vector with reducing the dimension, the autoencoder will have to recreate the image similar to the original image. The autoencoder aims to produce output as input and focus only on the essential features. Raw sketches over the cGANS create realistic images that quickly and easily make the sketch images raw images. The results show that the model achieves high accuracy of up to 75%, and PSNR is 25.5 dB that is potentially applicable for practice with only 606 face images. The performance of our proposed architecture is compared with other solutions, and the results show that our proposal obtains competitive performance in terms of output quality (25.5 dB) and efficiency (above 75%).

An Efficient 3D Human Pose Retrieval and Reconstruction from 2D Image-Based Landmarks.

We propose an efficient and novel architecture for 3D articulated human pose retrieval and reconstruction from 2D landmarks extracted from a 2D synthetic image, an annotated 2D image, an in-the-wild real RGB image or even a hand-drawn sketch. Given 2D joint positions in a single image, we devise a data-driven framework to infer the corresponding 3D human pose. To this end, we first normalize 3D human poses from Motion Capture (MoCap) dataset by eliminating translation, orientation, and the skeleton size discrepancies from the poses and then build a knowledge-base by projecting a subset of joints of the normalized 3D poses onto 2D image-planes by fully exploiting a variety of virtual cameras. With this approach, we not only transform 3D pose space to the normalized 2D pose space but also resolve the 2D-3D cross-domain retrieval task efficiently. The proposed architecture searches for poses from a MoCap dataset that are near to a given 2D query pose in a definite feature space made up of specific joint sets. These retrieved poses are then used to construct a weak perspective camera and a final 3D posture under the camera model that minimizes the reconstruction error. To estimate unknown camera parameters, we introduce a nonlinear, two-fold method. We exploit the retrieved similar poses and the viewing directions at which the MoCap dataset was sampled to minimize the projection error. Finally, we evaluate our approach thoroughly on a large number of heterogeneous 2D examples generated synthetically, 2D images with ground-truth, a variety of real in-the-wild internet images, and a proof of concept using 2D hand-drawn sketches of human poses. We conduct a pool of experiments to perform a quantitative study on PARSE dataset. We also show that the proposed system yields competitive, convincing results in comparison to other state-of-the-art methods.

Sketch-based 3D Shape Retrieval Using Similarity Weighting between Multi-View

3D shape retrieval has always been a hot research topic in the field of computer vision, and the research goal is to perform fast and efficient retrieval to obtain 3D shapes that meet user needs. With the rapid development and popularization of touch screen devices, hand-drawn sketches have undoubtedly become the most convenient and user-friendly input form. However, the huge difference between the 3D shape and the 2D sketch is the main challenge that affects retrieval performance. In this paper, we propose a method of adding a sketch and view feature similarity comparison module during the training process to obtain the scores for the final feature descriptors under the premise of feature extraction of the 3D shape based on multi-view. Specifically, we render the 3D shape into 2D views from multiple different perspectives to represent the shape. Perform feature extraction on two types of inputs through two different networks, and design a similarity weighting module to calculate the scores of each view, so as to obtain the final descriptors. Finally, a final descriptor similarity metric network is trained based on contrastive loss. The experimental results on SHREC’13 dataset demonstrate the superiority and robustness of our method.

The Wood Carvings of Two Traditional Timber Mosques in Kelantan and Terengganu

Wood carving has a long history in Malaysia. It is greatly influenced by the rich landscape of tropical forest in the living environment of the locals. The motifs of the carving are primarily derived from the images of flora and fauna. It mimics the intricate nature of leaves and flowers of the plants. The craftsmen initially use the bas-relief design consisting also of animal imageries on the boat’s sides or sometimes on the house components and even the dagger handle. However, with Islam’s arrival in the early 15th century, due to its teachings, contributed to the ban of animal images on art form. While most of the carvings are done on wood, there is no ample evidence of records available due to time and tides of the decaying process of the artwork. Likewise, there is no extensive research and documentation produced by academics to record the traditional wood carving of the Malays. Although few collections had documented the contemporary Malay houses produced by local scholars, sadly, there was no recount on the importance of housing elements and the technique of construction neither on wood carving. Hence, this paper attempts to examine the influence and pattern of wood carving on traditional mosques in Kelantan and Terengganu. The research conducted case studies, to seek the signs of the features and elements of wood carving on the traditional mosque in Kelantan and Terengganu, disclosing the influence of patterns and motifs on the selected images of its architecture. A measured drawing technique commonly used in conservation works guided the research operation which comprised of hand-drawn sketches to capture the details, measurement of building components, as well as transferring the images using tracing paper, while the other methods in gathering the information extended to direct interviews and photography. Then, the data collected were analyzed to determine the differences of each carving patterns and motifs in the two mosques in Kelantan and Terengganu. The result of the research established the future records on the variety of motifs and patterns of the Malay wood carving. Subsequently, it further elaborates the extensive techniques of wood carvings system and records the patterns found at Kelantan and Terengganu’s mosques for future inventory and recorded history.