User Sketch Research Articles

Freeform vector graphics with controlled thin-plate splines

Recent work defines vector graphics using diffusion between colored curves. We explore higher-order fairing to enable more natural interpolation and greater expressive control. Specifically, we build on thin-plate splines which provide smoothness everywhere except at user-specified tears and creases (discontinuities in value and derivative respectively). Our system lets a user sketch discontinuity curves without fixing their colors, and sprinkle color constraints at sparse interior points to obtain smooth interpolation subject to the outlines. We refine the representation with novel contour and slope curves, which anisotropically constrain interpolation derivatives. Compound curves further increase editing power by expanding a single curve into multiple offsets of various basic types (value, tear, crease, slope, and contour). The vector constraints are discretized over an image grid, and satisfied using a hierarchical solver. We demonstrate interactive authoring on a desktop CPU.

A mobile environment for sketching-based skeleton generation

Articulated character animation is typically performed by manually creating and rigging a skeleton into an unfolded 3D object. However, such tasks are not trivial, as they require a substantial amount of training and practices. Although automatic skeleton extraction methods have been proposed, they generally may not guarantee that the resulting skeleton can help produce desired animations according to user intention. In this paper, we present a sketching-based skeleton generation method suitable for use in the mobile environment. This method takes user sketching as an input, and based on the mesh segmentation result of a 3D object, it estimates a skeleton for articulated character animation. In addition, we are currently developing a Web-based mobile platform to support mesh editing by a group of collaborative users and we depict the system architecture of such a platform. Results show that our method can produce better skeletons in terms of joint positions and topological structure.

Sketch guided solid texturing

Compared to 2D textures, solid textures can represent not only the bounding surfaces, but also their interiors. Existing solid texture synthesis methods pay little attention to the generation of conforming textures that capture geometric structures or reflect the artists’ design intentions. In this paper, we propose a novel approach to synthesizing solid textures using 2D exemplars. The generated textures locally agree with a tensor field derived from user sketching curves. We use a deterministic approach and only a small portion of the voxels needs to be synthesized on demand. Correction is fundamental in deterministic texture synthesis. We propose a history windows representation, which is general enough to unifiedly represent various previous correction schemes, and a dual grid scheme based on it to significantly reduce the dependent voxels while still producing high quality results. Experiments demonstrate that our method produces significantly improved solid textures with a small amount of user interaction.

Sketching Clothoid Splines Using Shortest Paths

AbstractClothoid splines are gaining popularity as a curve representation due to their intrinsically pleasing curvature, which varies piecewise linearly over arc length. However, constructing them from hand‐drawn strokes remains difficult. Building on recent results, we describe a novel algorithm for approximating a sketched stroke with a fair (i.e., visually pleasing) clothoid spline. Fairness depends on proper segmentation of the stroke into curve primitives — lines, arcs, and clothoids. Our main idea is to cast the segmentation as a shortest path problem on a carefully constructed weighted graph. The nodes in our graph correspond to a vastly overcomplete set of curve primitives that are fit to every subsegment of the sketch, and edges correspond to transitions of a specified degree of continuity between curve primitives. The shortest path in the graph corresponds to a desirable segmentation of the input curve. Once the segmentation is found, the primitives are fit to the curve using non‐linear constrained optimization. We demonstrate that the curves produced by our method have good curvature profiles, while staying close to the user sketch.

Terrain Synthesis from Digital Elevation Models

In this paper, we present an example-based system for terrain synthesis. In our approach, patches from a sample terrain (represented by a height field) are used to generate a new terrain. The synthesis is guided by a user-sketched feature map that specifies where terrain features occur in the resulting synthetic terrain. Our system emphasizes large-scale curvilinear features (ridges and valleys) because such features are the dominant visual elements in most terrains. Both the example height field and user's sketch map are analyzed using a technique from the field of geomorphology. The system finds patches from the example data that match the features found in the user's sketch. Patches are joined together using graph cuts and Poisson editing. The order in which patches are placed in the synthesized terrain is determined by breadth-first traversal of a feature tree and this generates improved results over standard raster-scan placement orders. Our technique supports user-controlled terrain synthesis in a wide variety of styles, based upon the visual richness of real-world terrain data.

Tabletop Computed Lighting for Practical Digital Photography

We apply simplified image-based lighting methods to reduce the equipment, cost, time, and specialized skills required for high-quality photographic lighting of desktop-sized static objects such as museum artifacts. We place the object and a computer-steered moving-head spotlight inside a simple foam-core enclosure and use a camera to record photos as the light scans the box interior. Optimization, guided by interactive user sketching, selects a small set of these photos whose weighted sum best matches the user-defined target sketch. Unlike previous image-based relighting efforts, our method requires only a single area light source, yet it can achieve high-resolution light positioning to avoid multiple sharp shadows. A reduced version uses only a handheld light and may be suitable for battery-powered field photography equipment that fits into a backpack.

Classifier combination for sketch-based 3D part retrieval

In this paper, we present a search method with multi-class probability estimates for sketch-based 3D engineering part retrieval. The purpose of using probabilistic output from classification is to support high-quality part retrieval by motivating user relevance feedback from a ranked list of top categorical choices. Given a free-hand user sketch, we use an ensemble of classifiers to estimate the likelihood of the sketch belonging to each predefined category by exploring the strengths of various individual classifiers. Complementary shape descriptors are used to generate classifiers with probabilistic output using support vector machines (SVM). A weighted linear combination rule, called adapted minimum classification error (AMCE), is developed to concurrently minimize the classification errors and the log likelihood errors. Experiments are conducted using our Engineering Shape Benchmark database to evaluate the proposed combination rule. User studies show that users can easily identify the desired classes and then the parts under the proposed method and algorithms. Compared with the best individual classifier, the classification accuracy using AMCE increased by 7% for 3D models, and the average best rank improved by 11.6% for sketches.

Utilizing shape retrieval in sketch synthesis

This article describes SR-Sketch, a sketch creation tool that can act both as a front-end visual query module to visual information retrieval systems and as an aid tool for fast image composition. The system allows the user to draw shapes on the computer screen using the mouse cursor. At any time during this process the user can query a shape database for similar shapes and then select the ones s/he thinks are more relevant. The system can then automatically align and replace user-drawn objects with the chosen database shapes in the user sketch. For any match between a sketch shape and a database shape, the application can provide a visual explanation of how and why two shapes are considered similar. Evaluation results show that the system achieves favorable results with respect to noise tolerance, speed and reliability. SR-Sketch is freely available for download and experimentation.

SmoothSketch

We introduce SmoothSketch---a system for inferring plausible 3D free-form shapes from visible-contour sketches. In our system, a user's sketch need not be a simple closed curve as in Igarashi's Teddy [1999], but may have cusps and T-junctions, i.e., endpoints of hidden parts of the contour. We follow a process suggested by Williams [1994] for inferring a smooth solid shape from its visible contours: completion of hidden contours, topological shape reconstruction, and smoothly embedding the shape via relaxation. Our main contribution is a practical method to go from a contour drawing to a fairly smooth surface with that drawing as its visible contour. In doing so, we make several technical contributions: • extending Williams' and Mumford's work [Mumford 1994] on figural completion of hidden contours containing T-junctions to contours containing cusps as well, • characterizing a class of visible-contour drawings for which inflation can be proved possible, • finding a topological embedding of the combinatorial surface that Williams creates from the figural completion, and • creating a fairly smooth solid shape by smoothing the topological embedding using a mass-spring system.We handle many kinds of drawings (including objects with holes), and the generated shapes are plausible interpretations of the sketches. The method can be incorporated into any sketch-based free-form modeling interface like Teddy.

A 2D Sketch-Based User Interface for 3D CAD Model Retrieval

AbstractThis paper describes a sketch user interface enhanced by feedback for 3D CAD model retrieval. Users can express their intent by sketching 2D shape in the way as engineers draw three views of 3D models. It not only supports users’ free form sketches, but also accepts users’ further editing operations and feedbacks. The interaction paradigm proposed in this paper is supported by a 3D shape matching method, in which three problems are solved: determination of projecting planes and directions, 2D view generation, and similarity measuring between views. In addition, experiments are conducted to evaluate the performance of this sketch user interface by 3D model retrieval.

From on-line sketching to 2D and 3D geometry: a system based on fuzzy knowledge

The paper describes the development of a fuzzy knowledge-based prototype system for conceptual design. This real time system is designed to infer user's sketching intentions, to segment sketched input and generate corresponding geometric primitives: straight lines, circles; arcs, ellipses, elliptical arcs, and B-spline curves. Topology information (connectivity, unitary constraints and pairwise constraints) is received dynamically from 2D sketched input and primitives. From the 2D topology information, a more accurate 2D geometry can be built up by applying a 2D geometric constraint solver. Subsequently, 3D geometry can be received feature by feature incrementally. Each feature can be recognised by inference knowledge in terms of matching its 2D primitive configurations and connection relationships. The system accepts not only sketched input, working as an automatic design tool, but also accepts user interactive input of both 2D primitives and special positional 3D primitives. This makes it easy and friendly to use. The system has been tested with a number of sketched inputs of 2D and 3D geometry.