Transforming Hand-Drawn Sketches of Linkage Mechanisms Into Their Digital Representation

Abstract

Abstract This paper presents a deep neural network based approach for interactive digital transformation and simulation of n-bar planar linkages consisting of both revolute and prismatic joints from hand-drawn sketches. Instead of taking a pure computer vision approach, we combine the output of a convolutional deep neural network with the topological knowledge of linkage mechanisms to create a framework for recognition of hand-drawn sketches. To accomplish this, we first synthetically generate a dataset of images of linkage mechanism sketches similar to hand-drawn ones and then fine-tune a state of the art deep neural network capable of detecting discrete objects. While the network had previously been exposed to only a general class of images of every-day objects, it was for the first time trained with a set of building blocks of linkage mechanisms, viz. joints and links. Thereafter, we present a novel algorithm, which performs topological analysis on the set of detected objects to create a kinematic model of the sketched mechanisms. The results show that this algorithm performs well on hand-drawn sketches and could help with conversions of such sketches to their digital representation for effective communication, analysis, cataloging, and classification.