DCR: Disentangled component representation for sketch generation

Abstract

• We propose a novel end-to-end model to extract the disentangled representation of a sketch. • We discuss the performance of existing methods on the multi-class sketch generation task. • We investigate why disentangled representation can improve the performance of sketch generation. • We release a large online handwritten Chinese character dataset with component-level and image-level labels. Our model consists of three modules, a CNN autoencoder for extracting image-level representations f conv , an LSTM autoencoder for extracting component representations z t , and a feature disentangling module for disentangling the image-level representations to the component representations. Compared to directly mapping the pixel image into the whole sketch sequences, our model explores disentangling representations and gets a further understanding of components shared among sketches to make sketch generation more interpretable, flexible and diversifiable. We present a simple end-to-end model based on deep learning to automatically decompose sketched objects into components by disentangling the visual representation. The performance of visual representation learning based models degrades as categories increase. Rather than building a mapping from a static image to the whole sketch sequences, we propose an interpretable disentangled representation of sketch to understand component concepts and the relationship among such concepts. Our model takes the binary image of a sketched object and produces a component stroke sequence set corresponding to key components in the sketch. Experiments show that our method significantly outperforms all baselines quantitatively at the degree of disentanglement, and our method is more stable while training on tens of categories.