Sketch-based Interface Research Articles

Geometric reasoning in sketch-based volumetric decomposition framework for hexahedral meshing

This paper presents a sketch-based volumetric decomposition framework using geometric reasoning to assist in hex meshing. The sketch-based user interface makes the framework user-friendly and intuitive; and the geometric reasoning engine makes the framework smarter and improves the usability. The system first creates a data structure containing B-Rep and 3D medial to capture the exterior and interior of the input model, respectively. The four-step geometric reasoning process consists of (1) Determining sweeping direction and two types of sweepable regions, (2) Providing visual aids on sweeping direction and sweepable region for decomposition, (3) Understanding user’s intent by using prioritized B-Rep and medial entities, and (4) smart decomposition operation. Imprint and merge operations are then performed on the decomposed model before passing it to the sweeping algorithm to create hex meshes. The sketch-based framework has been tested on industrial models.

Sketch-based generation and editing of quad meshes

Coarse quad meshes are the preferred representation for animating characters in movies and video games. In these scenarios, artists want explicit control over the edge flows and the singularities of the quad mesh. Despite the significant advances in recent years, existing automatic quad remeshing algorithms are not yet able to achieve the quality of manually created remeshings. We present an interactive system for manual quad remeshing that provides the user with a high degree of control while avoiding the tediousness involved in existing manual tools. With our sketch-based interface the user constructs a quad mesh by defining patches consisting of individual quads. The desired edge flow is intuitively specified by the sketched patch boundaries, and the mesh topology can be adjusted by varying the number of edge subdivisions at patch boundaries. Our system automatically inserts singularities inside patches if necessary, while providing the user with direct control of their topological and geometrical locations. We developed a set of novel user interfaces that assist the user in constructing a curve network representing such patch boundaries. The effectiveness of our system is demonstrated through a user evaluation with professional artists. Our system is also useful for editing automatically generated quad meshes.

Analysis and Evaluation of Sketch Recognizers in the Creation of Physics Simulations

Sketch-based interfaces can provide a natural way for users to interact with applications. Since the core of a sketch-based interface is the gesture recognizer, there is a need to correctly evaluate various recognizers before choosing one. In this paper the authors present an evaluation of three gesture recognizers: Rubine's recognizer, CALI and the $1 Recognizer. The evaluation relied on a set of real gesture samples drawn by 32 subjects, with a gesture repertoire arranged for use in SketchyDynamics, a programming library that intends to facilitate the creation of applications by rapidly providing them with a sketch-based interface and physics simulation capabilities. The authors also discuss some improvements to the recognizers' implementation that helped achieving higher recognition rates. In the end, CALI had the best recognition rate with 94% accuracy, followed by $1 Recognizer with 87% and finally by Rubine's recognizer with 79%.

A sketch-based interface to modify and reproduce motion sequences

We present a sketch-based user interface, which was designed to help novices to create 3D character animations by multi-pass sketching, avoiding the ambiguities usually present in sketch input. Our system also contains sketch-based editing and reproducing tools, which allow paths and motions to be partially updated rather than wholly redrawn; and graphical block interface permits motion sequences to be organized and reconfigured easily. A user evaluation with participants of different skill levels suggest that novices using this sketch interface can produce animations almost as quickly as users who are experienced in 3D animation.

SketchyDynamics: A Library for the Development of Physics Simulation Applications with Sketch-Based Interfaces

Sketch-based interfaces provide a powerful, natural and intuitive way for users to interact with an application. By combining a sketch-based interface with a physically simulated environment, an application offers the means for users to rapidly sketch a set of objects, like if they are doing it on piece of paper, and see how these objects behave in a simulation. In this paper we present SketchyDynamics, a library that intends to facilitate the creation of applications by rapidly providing them a sketch-based interface and physics simulation capabilities. SketchyDynamics was designed to be versatile and customizable but also simple. In fact, a simple application where the user draws objects and they are immediately simulated, colliding with each other and reacting to the specified physical forces, can be created with only 3 lines of code. In order to validate SketchyDynamics design choices, we also present some details of the usability evaluation that was conducted with a proof-of-concept prototype.

Retrieval of motion composition in film

This article presents methods for the automatic retrieval of motion and motion compositions in movies. We introduce a user-friendly sketch-based query interface that enables the user to describe desired motion compositions. Based on this abstract description, a tolerant matching scheme extracts shots from a movie with a similar composition. We investigate and evaluate two application scenarios: the retrieval of motion compositions and the retrieval of matching motions (a technique in continuity editing). Experiments show that the developed methods accurately and promptly retrieve relevant shots. The presented methods enable new ways of searching and investigating movies.

원화의 음영 캡쳐 기반 카툰 캐릭터 렌더링

3차원 캐릭터 모델의 만화적 표현을 위하여 일반적으로 사용되고 있는 기존의 카툰렌더링 방법들은 아티스트가 직접 그린 원화의 표현을 그대로 살리는데 한계가 있다. 본 논문에서는 원화로부터 만화적인 음영모델을 캡쳐하는 기술을 제안하고 이를 이용하여 3차원 캐릭터 모델을 효과적으로 카툰 렌더링하는 새로운 시스템을 제공한다. 이 시스템의 특징은 음영을 캡쳐한 후에 캐릭터 모델의 만화적 특성을 부각시키기 위하여 음영을 이용하여 형태를 강조하는 알고리즘을 포함하며 아티스트가 포스트 프러덕션으로 음영을 편집할 수 있는 스케치 기반 인터페이스를 지원하는 것이다. 이를 위하여 선택된 영역의 색상 분포와 비율을 분석하는 RGB 색상 정렬 알고리즘을 이용하여 카툰 텍스처를 자동으로 생성하는 방법론을 제시한다. 또한 캐릭터의 형태적 특성을 강조하기 위하여 세일리언시 기반 카툰 렌더링 알고리즘을 제안하며, 음영을 지역적으로 편집하는 스케치 인터페이스를 제공한다. 마지막으로 사용자 평가를 통하여 제안하는 시스템의 유용성을 입증한다. Traditional rendering of cartoon character cannot revive the feeling of concept drawings properly. In this paper, we propose capture technology to get toon shading model from the concept drawings and with this technique, we provide a new novel system to render 3D cartoon character. Benefits of this system is to cartoonize the 3D character according to saliency to emphasize the form of 3D character and further support the sketch-based user interface for artists to edit shading by post-production. For this, we generate texture automatically by RGB color sorting algorithm to analyze color distribution and rates of selected region. In the cartoon rendering process, we use saliency as a measure to determine visual importance of each area of 3d mesh and we provide a novel cartoon rendering algorithm based on the saliency of 3D mesh. For the fine adjustments of shading style, we propose a user interface that allow the artists to freely add and delete shading to a 3D model. Finally, this paper shows the usefulness of the proposed system through user evaluation.

Sketch-based design for green geometry and image deformation

User interfaces have traditionally followed the WIMP (window, icon, menu, pointer) paradigm. Though functional and powerful, they are usually cumbersome for a novice user to design a complex model, requiring considerable expertise and effort. This paper presents a system for designing geometric models and image deformation with sketching curves, with the use of Green coordinates. In 3D modeling, the user first creates a 3D model by using a sketching interface, where a given 2D curve is interpreted as the projection of the 3D curve. The user can add, remove, and deform these control curves easily, as if working with a 2D line drawing. For a given set of curves, the system automatically identifies the topology and face embedding by applying graph rotation system. Green coordinates are then used to deform the generated models with detail-preserving property. Also, we have developed a sketch-based image-editing interface to deform image regions using Green coordinates. Hardware-assisted schemes are provided for both control shape deformation and the subsequent surface optimization, the experimental results demonstrate that 3D/2D deformations can be achieved in realtime.

A comparative evaluation of foreground/background sketch-based mesh segmentation algorithms

This paper presents an extensive comparative evaluation of five popular foreground/background sketch-based interactive mesh segmentation algorithms, addressing the quantitative assessment of the accuracy, efficiency, and stability of each algorithm. To facilitate the comparison, we have developed a complete framework with an intuitive and simple sketch-based interface to enable interactive mesh segmentation by marking strokes to specify the foreground and background with the mouse buttons, allowing us to quantify the algorithms in a unified manner. The evaluation has been performed via extensive user experiments in which each participant was assigned to segment models with the evaluated algorithms and the corresponding update of each segmentation was recorded as a new refinement when additional interactions were added. We then collected the segmentations from participants and evaluated them against the ground-truth corpus constructed from the Princeton segmentation database. To investigate how well the interactive segmentations match the ground-truth, five metrics were used to measure the boundary and region accuracy of segmentations. By studying the experimental results, we have analyzed the performance of the evaluated algorithms and provided valuable insights into their characteristics.

A sketching interface for feature curve recovery of free-form surfaces

In this paper, we present a semi-automatic approach to efficiently and robustly recover the characteristic feature curves of a given free-form surface where we do not have to assume that the input is a proper manifold. The technique supports a sketch-based interface where the user just has to roughly sketch the location of a feature by drawing a stroke directly on the input mesh. The system then snaps this initial curve to the correct position based on a graph-cut optimization scheme that takes various surface properties into account. Additional position constraints can be placed and modified manually which allows for an interactive feature curve editing functionality. We demonstrate the usefulness of our technique by applying it to two practical scenarios. At first, feature curves can be used as handles for surface deformation, since they describe the main characteristics of an object. Our system allows the user to manipulate a curve while the underlying non-manifold surface adopts itself to the deformed feature. Secondly, we apply our technique to a practical problem scenario in reverse engineering. Here, we consider the problem of generating a statistical (PCA) shape model for car bodies. The crucial step is to establish proper feature correspondences between a large number of input models. Due to the significant shape variation, fully automatic techniques are doomed to failure. With our simple and effective feature curve recovery tool, we can quickly sketch a set of characteristic features on each input model which establishes the correspondence to a pre-defined template mesh and thus allows us to generate the shape model. Finally, we can use the feature curves and the shape model to implement an intuitive modeling metaphor to explore the shape space spanned by the input models.

Volumetric modeling with diffusion surfaces

The modeling of volumetric objects is still a difficult problem. Solid texture synthesis methods enable the design of volumes with homogeneous textures, but global features such as smoothly varying colors seen in vegetables and fruits are difficult to model. In this paper, we propose a representation called diffusion surfaces (DSs) to enable modeling such objects. DSs consist of 3D surfaces with colors defined on both sides, such that the interior colors in the volume are obtained by diffusing colors from nearby surfaces. A straightforward way to compute color diffusion is to solve a volumetric Poisson equation with the colors of the DSs as boundary conditions, but it requires expensive volumetric meshing which is not appropriate for interactive modeling. We therefore propose to interpolate colors only locally at user-defined cross-sections using a modified version of the positive mean value coordinates algorithm to avoid volumetric meshing. DSs are generally applicable to model many different kinds of objects with internal structures. As a case study, we present a simple sketch-based interface for modeling objects with rotational symmetries that can also generate random variations of models. We demonstrate the effectiveness of our approach through various DSs models with simple non-photorealistic rendering techniques enabled by DSs.

A sketch interface to empower novices to create 3D animations

We introduce a multiple-pass sketch-based user interface, designed for novice users, which avoids spatial ambiguities and hence allows considerable freedom in creating 3D animations. The user first...

Sketch-based parameterization of L-systems using illustration-inspired construction lines and depth modulation

We present a sketch-based interface for parameter control of rule-based models. It allows intuitive specification and creation of plant structures with L-systems. Construction lines inspired by concept sketches are employed as a way to define and manipulate global-to-local characteristics of L-system models. The overall structure, posture and proportions of the plant are initially sketched by the user as 2D construction line arrangements. They are automatically encoded as a set of positional functions controlling internode lengths, branching angles, organ sizes, and stem shape. The depth in the stem is specified while sketching its construction line, by modulating input pen pressure as depth values. This is also inspired by line depth modulation techniques used in traditional illustration. The resulting positional functions are then used to parameterize pre-defined L-system templates representing phyllotactic patterns guiding the positioning of lateral organ surfaces such as leaves and petals. Results are presented for single monopodial plant structures, all generated from simple input construction line sketches.

Sketch-Based Solid Modeling for Conceptual Design

提出了一种简单、快速、高效的基于草绘设计的概念模型创建方法.该方法模仿传统的纸笔草绘设计方式,允许用户自由地在笔式用户界面上勾画特征笔划,然后根据笔划的形状和位置关系采用基于特征的实体建模方法构造不同的扫描特征体素.在此基础上,采用了基于意图捕捉的特征添加及切削机制创建复杂的实体模型,给出了一些关键算法的实现过程.实验结果表明,该方法能够较好地应用到联机三维概念模型快速创建过程中.;A simple, rapid and efficient approach to constructing 3D conceptual solid models on sketch-based user interface is presented. It emulates the traditional sketching design approach with paper and pencil to sketch strokes on the calligraphic interface, and then applies the feature-based modeling method to construct different sweep feature-based primitives according to the shape and location relationship of feature strokes. In order to construct complex solid models, the feature joining and cutting mechanism based on intention capturing is introduced. This paper illustrates the implementation of the algorithm, and experiment shows that the algorithm is efficient and well suitable for real time on-line 3D conceptual model creation.

A Sketch-Based Interface for Modeling Myocardial Fiber Orientation that Considers the Layered Structure of the Ventricles

We propose a sketch-based interface for modeling the myocardial fiber orientation required in the electrophysiological simulation of the heart, especially the ventricles. The user can create a volumetric vector field that represents the myocardial fiber orientation in two steps. First, a depth field over the three-dimensional (3D) ventricular model is defined to create layers of myocardium. The user can then peel these layers and draw strokes on them to specify the myocardial fiber orientation in each layer. We represent the 3D ventricular model as a tetrahedral mesh and perform Laplacian smoothing over the mesh vertices to interpolate the vector field defined by the user-drawn strokes. Our method also allows the user to perform deformations on volumetric models of myocardial fiber orientation, which is very important for studying heart disease associated with morphological abnormalities. We created several examples of myocardial fiber orientation and applied them to a simplified simulator to demonstrate the effectiveness of our method.

Improving the sketch-based interface

Improving the sketch-based interface

A Sketch-Based Interface for Modeling Myocardial Fiber Orientation

This article proposes a sketch-based interface for modeling muscle fiber orientation of a 3D virtual heart model. Our target was electrophysiological simulation of the heart and fiber orientation is one of the key elements to obtaining reliable simulation results. We designed the interface and algorithm based on the observation that fiber orientation is always parallel to the surface of the heart. The user specifies the fiber orientation by drawing a freeform stroke on the object surface. The system first builds a vector field on the surface by applying Laplacian smoothing to the mesh vertices and then builds a volumetric vector field by applying Laplacian smoothing to the voxels. We demonstrate the usefulness of the proposed method through a user study with a cardiologist.

Combining geometry and domain knowledge to interpret hand-drawn diagrams

One main challenge in building interpreters for hand-drawn sketches is the task of parsing a sketch to locate the individual symbols. Many existing pen-based systems avoid this problem by requiring the user to explicitly indicate the partitioning of the sketch with button clicks or pauses in drawing. We have created a parser that automatically locates symbols by looking for areas of high ink density, and for points at which the characteristics of the pen strokes change. To demonstrate our techniques, we have developed AC-SPARC, a sketch-based interface for the SPICE electric circuit analysis program. An evaluation of our interface has indicated that, even for novice users, our system can successfully locate and identify most of the circuit components in hand-drawn circuit diagrams.

A sketch-based interface for detail-preserving mesh editing

In this paper we present a method for the intuitive editing of surface meshes by means of view-dependent sketching. In most existing shape deformation work, editing is carried out by selecting and moving a handle , usually a set of vertices. Our system lets the user easily determine the handle, either by silhouette selection and cropping, or by sketching directly onto the surface. Subsequently, an edit is carried out by sketching a new, view-dependent handle position or by indirectly influencing differential properties along the sketch. Combined, these editing and handle metaphors greatly simplify otherwise complex shape modeling tasks.

LADDER, a sketching language for user interface developers

Sketch recognition systems are currently being developed for many domains, but can be time consuming to build if they are to handle the intricacies of each domain. In order to aid sketch-based user interface developers, we have developed tools to simplify the development of a new sketch recognition interface. We created LADDER, a language to describe how sketched diagrams in a domain are drawn, displayed, and edited. We then automatically transform LADDER structural descriptions into domain specific shape recognizers, editing recognizers, and shape exhibitors for use in conjunction with a domain independent sketch recognition system, creating a sketch recognition system for that domain. We have tested our framework by writing several domain descriptions and automatically generating a domain specific sketch recognition system from each description.