Integrating Interactive Multimedia Objects in Mobile Augmented Reality for Sarawak Tourism

Abstract

Augmented Reality (AR) is a form of technology that superimpose virtual object or content in a real-world environment to create a mixture of reality. This research studies ARToolKitPlus tracking and developed an Android based AR system which integrate mobile graphics and interaction modules, such as audio, video and 3D objects. The development of multimedia based mobile AR system involved video capturing, image processing, mobile graphics and interactive multimedia features. Android camera Application Programming Interface (API) was implemented for video capturing purposes. In order to perform image processing tasks, an image library was built such as greyscale transformation and thresholding. The graphics module for this system was built using OpenGLES and PowerVR graphics API. Whereas, audio and video API for Android platform were used to build multimedia based audio and video module. As for interaction module, technique of color picking algorithm was applied. Henceforth, a prototype of mobile Sarawak Tourism system was developed to evaluate the multimedia based modules functionalities testing; text, animations, audio, video, 2D/3D object interactions. A preliminary survey was conducted to measure the interaction performance and usability of multimedia based modules AR on Sarawak Tourism application.