An automatic detection of a natural marker and augmentation of 3D models in AR with sketch-based object matching

Abstract

This paper introduces a sketch-based localisation approach to detect a desired natural marker from an input video image. The proposed method also retrieves a 3D virtual object to be augmented in augmented reality from a 3D database based on the object matching method. Sketch-based image matching has been used for content-based retrieval to compare the database images with a sketch-based image drawn by users and estimate the degree of similarity between the database images and the query image. In this paper, we adopt sketch-based object matching method to localise the natural marker of the video images to register a 3D virtual object in AR system. Most similar object in the input image is determined as a natural marker of the AR by comparing the user defined sketched image based on the basic features of the sketched object. Unlike other image matching methods, this matching technique is possible to produce query image without constraints by drawing the image intuitively. In addition, in the proposed sketched-based AR system, the 3D object augmented on the marker will be also determined by object matching between the detected marker and 3D database images.