Augmented Reality Engine Research Articles

Augmented reality display of neurosurgery craniotomy lesions based on feature contour matching

Traditional neurosurgical craniotomy primarily uses two-dimensional cranial medical images to estimate the location of a patient’s intracranial lesions. Such work relies on the experience and skills of the doctor and may result in accidental injury to important intracranial physiological tissues. To help doctors more intuitively determine patient lesion information and improve the accuracy of surgical route formulation and craniotomy safety, an augmented reality method for displaying neurosurgery craniotomy lesions based on feature contour matching is proposed. This method uses threshold segmentation and region growing algorithms to reconstruct a 3-D Computed tomography image of the patient’s head. The augmented reality engine is used to adjust the reconstruction model’s relevant parameters to meet the doctor’s requirements and determine the augmented reality matching method for feature contour matching. By using the mobile terminal to align the real skull model, the virtual lesion model is displayed. Using the designed user interface, doctors can view the patient’s personal information and can zoom in, zoom out, and rotate the virtual model. Therefore, the patient’s lesions information can be visualized accurately, which provides a visual basis for preoperative preparation.

Flexible development of location-based mobile augmented reality applications with AREA

Mobile applications have garnered a lot of attention in the last years. The computational capabilities of mobile devices are the mainstay to develop completely new application types. The provision of augmented reality experiences on mobile devices paves one alley in this field. For example, in the automotive domain, augmented reality applications are used to experience, inter alia, the interior of a car by moving a mobile device around. The device’s camera then detects interior parts and shows additional information to the customer within the camera view. Another application type that is increasingly utilized is related to the combination of serious games with mobile augmented reality functions. Although the latter combination is promising for many scenarios, technically, it is a complex endeavor. In the AREA (Augmented Reality Engine Application) project, a kernel was implemented that enables location-based mobile augmented reality applications. Importantly, this kernel provides a flexible architecture that fosters the development of individual location-based mobile augmented reality applications. The work at hand shows the flexibility of AREA based on a developed serious game. Furthermore, the algorithm framework and major features of it are presented. As the conclusion of this paper, it is shown that mobile augmented reality applications require high development efforts. Therefore, flexible frameworks like AREA are crucial to develop respective applications in a reasonable time.

Augmented Reality as Learning Medium for Preservation of Traditional Musical Instruments in Bangka

Nowadays the use of technology is something that can be found anywhere. This condition has an impact on the loss of awareness of the Indonesian cultural treasures value in the form of traditional tools. No exception to traditional musical instruments on Bangka Island which began to lose its popularity. At present, most teenagers on Bangka Island cannot play traditional musical instruments. Likewise with the children who do not know and not even know their own regional musical instruments. With the continuation of this condition, it is feared that the existence of traditional Bangka musical instruments will disappear, as well as human resources that can play it. Augmented Reality (AR) is a visual technology that can display objects in 3D. The advantage of this technology is being able to give a display of real-time musical instrument in the form of dynamic 3D visualization of objects and it is in accordance with the movements of the user's smartphone camera. AR has also been applied in various cases as a solution to problem-solving. Therefore, to overcome this problem, an application to preserve the traditional Bangka musical instruments using Augmented Reality (AR) is forming. The 3D objects of musical instruments are made using Maya. Unity is also used as an engine for the application of 3D modeling on the Android system, and Vuforia SDK as it’s augmented reality engine. The results of performance testing obtained 100% running well. From the results of testing the user experience with the HARUS method, it is proven that the system has comprehensibility aspects of 75.98% and manipulability aspects of 80.74% so that the total value HARUS be 78.36%.

Implementation of an Open Source Based Augmented Reality Engine for Cloud Authoring Frameworks

In recent years the technology around Augmented Reality has grown considerably, in particular on cloud based. In this paper we present a pipeline model and sample implementation that shows how an Augmented Reality engine can be built by leveraging advances in open source algorithm implementations. We also show how such an engine can be effectively integrated with cloud authoring tools to take advantage of the network connectivity and its computing power.

The AREA Algorithm Framework Enabling Location-based Mobile Augmented Reality Applications

The dramatically increased computational capabilities of mobile devices have leveraged the opportunities for mobile application engineers. Respective scenarios, in which these opportunities can be exploited, emerge almost per day. In this context, mobile augmented reality applications play an important role in many business scenarios. In the automotive domain, they are mainly used to provide car customers with new experiences. For example, customers can use their own mobile device to experience the interior of a car by moving the mobile device around. The device’s camera then detects interior parts and shows additional information to the customer within the camera view. Although the computational capabilities have been increased, the realization of such mobile augmented reality applications is still a complex endeavor. In particular, the different mobile operating systems and their peculiarities must be carefully considered. In the AREA (Augmented Reality Engine Application) project, a powerful kernel was realized that enables location-based mobile augmented reality applications. This kernel, in turn, mainly focuses on robustness and performance. In addition, it provides a flexible architecture that fosters the development of individual location-based mobile augmented reality applications. As many aspects have to be considered to implement individual applications based on top of AREA, this paper provides the first comprehensive overview of the entire algorithm framework. Moreover, a recently realized algorithm and new features will be presented. To demonstrate the applicability of the kernel, its features are applied in the context of various mobile applications. As the major lesson learned, powerful mobile augmented reality applications can be efficiently run on present mobile operating systems and be effectively realized by engineers using AREA. We consider such mobile frameworks as being crucial to provide more generic concepts that are able to abstract from the peculiarities of the underlying mobile operating system and to support mobile application developers more properly.

Whistland: An Augmented Reality Crowd-Mapping System for Civil Protection and Emergency Management

The prevention and correct management of natural disaster event sequences play a key role in saving human lives. The availability of embedded and mobile smart computing systems opens new roads for the management of land and infrastructures by civil protection operators. To date, research has explored the use of social networks for the management of disasters connected to meteorological/hydrogeological events or earthquakes, but without emphasis on the importance of an integrated system. The main feature of the Whistland system proposed in this paper is to make synergistic use of augmented reality (AR), crowd-mapping (CM), social networks, the Internet of Things (IoT) and wireless sensor networks (WSN) by exploiting technologies and frameworks of Web 2.0 and GIS 2.0 to make informed decisions about the chain of events. The Whistland system is composed of a geo-server, a mobile application with AR and an analytics dashboard. The geo-server acts as the hub of the sensor and social networks. The abstracted concept in this sense is the transformation of the user domain into “intelligent sensors” for the whole scope of crisis management. The social network integration is made through an efficient pointer-like mechanism that keeps the storage requirement low through a mobile application based on an augmented reality engine and provides qualitative information that sensors are unable to capture. Real-time analyses, geo-searches and the capability to examine event histories with an augmented reality engine all help the stakeholders to understand better the state of the resources under observation/monitoring. The system has been extensively tested in the programmed maintenance of river basins, where it is necessary to log maintenance activities in order to keep the riverbank clean: a significant use-case in many countries affected by hydro-geological instability.

The AREA Framework for Location-Based Smart Mobile Augmented Reality Applications

During the last years, the computational capabilities of smart mobile devices have been continuously improved by hardware vendors, raising new opportunities for mobile application engineers. Mobile augmented reality can be considered as one demanding scenario demonstrating that smart mobile applications are becoming more and more mature. In the AREA (Augmented Reality Engine Application) project, we developed a powerful kernel that enables location-based, mobile augmented reality applications. On top of this kernel, mobile application developers can realize sophisticated individual applications. The AREA kernel, in turn, allows for both robustness and high performance. In addition, it provides a flexible architecture that fosters the development of individual location-based mobile augmented reality applications. As a particular feature, the kernel allows for the handling of points of interests (POI) clusters. Altogether, advanced concepts are required to realize a location-based mobile augmented reality kernel that are presented in this paper. Furthermore, results of an experiment are presented in which the AREA kernel was compared to other location-based mobile augmented reality applications. To demonstrate the applicability of the kernel, we apply it in the context of various mobile applications. As a lesson learned, sophisticated mobile augmented reality applications can be efficiently run on present mobile operating systems and be effectively realized by engineers using the AREA framework. We consider mobile augmented reality as a killer application for mobile computational capabilities as well as the proper support of mobile users in everyday life.

Enabling Tracks in Location-Based Smart Mobile Augmented Reality Applications

To assist users through contemporary mobile technology is demanded in a multitude of scenarios. Interestingly, more and more users crave for mobile assistance in their leisure time. Consequently, the number of mobile applications that support leisure activities increases significantly. Mobile augmented reality applications constitute an example for user assistance that is welcome in these scenarios. In the AREA (Augmented Reality Engine Application) project, we developed a kernel that enables sophisticated location-based mobile augmented reality applications. On top of this kernel, various projects were realized. In many of these projects, a feature to enable tracks was demanded. Tracks, for example, may assist users in the context of mountaineering. The development of an AREA algorithm that enables track handling requires new concepts that are presented in this paper. To demonstrate the performance of the developed algorithm, also results of an experiment are presented. As a lesson learned, mobile augmented reality applications that want to make use of the new algorithm can be efficiently run on present mobile operating systems and be effectively realized by engineers using the AREA framework. Altogether, the new track feature is another valuable step for AREA towards a comprehensive location-based mobile augmented reality framework.

Advanced Algorithms for Location-Based Smart Mobile Augmented Reality Applications

During the last years, the computational capabilities of smart mobile devices have been continuously improved by hardware vendors, raising new opportunities for mobile application engineers. Mobile augmented reality is one scenario demonstrating that smart mobile applications are becoming increasingly mature. In the AREA (Augmented Reality Engine Application) project, we developed a kernel that enables such location-based mobile augmented reality applications. On top of the kernel, mobile application developers can easily realize their individual applications. The kernel, in turn, focuses on robustness and high performance. In addition, it provides a flexible architecture that fosters the development of individual location-based mobile augmented reality applications. In the first stage of the project, the LocationView concept was developed as the core for realizing the kernel algorithms. This LocationView concept has proven its usefulness in the context of various applications, running on iOS, Android, or Windows Phone. Due to the further evolution of computational capabilities on one hand and emerging demands of location-based mobile applications on the other, we developed a new kernel concept. In particular, the new kernel allows for handling points of interests (POI) clusters or enables the use of tracks. These changes required new concepts presented in this paper. To demonstrate the applicability of our kernel, we apply it in the context of various mobile applications. As a result, mobile augmented reality applications could be run on present mobile operating systems and be effectively realized by engineers utilizing our approach. We regard such applications as a good example for using mobile computational capabilities efficiently in order to support mobile users in everyday life more properly.

An Augmented Reality Platform inside PURESAFE project

This paper presents the results of a three-year research project inside PURESAFE project. PURESAFE contains 15 research projects and aims to enhance the life-cycle management of facilities emitting ionizing radiation. The research project presented in this paper focuses in Augmented Reality solutions for maintenance purposes. The outcome of the research project is a new platform containing an Augmented Reality engine and an authoring tool for the creation of Augmented Reality applications for maintenance in large scientific facilities emitting ionizing radiation. The platform has been developed and tested in real conditions.

Dynamic learning, retrieval, and tracking to augment hundreds of photographs

Tracking is a major issue of virtual and augmented reality applications. Single object tracking on monocular video streams is fairly well understood. However, when it comes to multiple objects, existing methods lack scalability and can recognize only a limited number of objects. Thanks to recent progress in feature matching, state-of-the-art image retrieval techniques can deal with millions of images. However, these methods do not focus on real-time video processing and cannot track retrieved objects. In this paper, we present a method that combines the speed and accuracy of tracking with the scalability of image retrieval. At the heart of our approach is a bi-layer clustering process that allows our system to index and retrieve objects based on tracks of features, thereby effectively summarizing the information available on multiple video frames. Dynamic learning of new viewpoints as the camera moves naturally yields the kind of robustness and reliability expected from an augmented reality engine. As a result, our system is able to track in real-time multiple objects, recognized with low delay from a database of more than 300 entries. We released the source code of our system in a package called Polyora.

A Simple Framework to Develop Pedagogical Augmented Reality Programs: an Application Based on Plants Teaching

An augmented reality Engine for the Construction of Pedagogical Applications (ECPA) is presented with an ECPA menu composed of two (master and slave) sub-menus as the main interface. With simple LUA scripts that allow building the application features, ECPA can be used to easily program different educational applications in a short time. To illustrate the possibilities of ECPA engine, an application called Virtual Plant For Children (VPFC) is created. VPFC allows young people to interact with virtual plants thanks to an interactive L-system for plants growing simulation. A usability test is made to validate our interface and interaction method, in which twenty-four K4-K5 children are instructed to make virtual plants grow by giving them water, heat and light.

Interactive e-learning system using pattern recognition and augmented reality

This paper proposes an interactive e-learning system using pattern recognition and augmented reality. The goal of proposed system is to provide students with realistic audio-visual contents when they are leaning. The proposed e-learning system consists of image recognition, color and polka-dot pattern recognition, and augmented reality engine with audio-visual contents. When the Web camera on a PC captures the current page of textbook, the e-learning system first identifies the images on the page, and augments some audio-visual contents on the monitor. For interactive learning, the proposed e-learning system exploits the color-band or polka-dot markers which are stuck to the end of a finger. The color-band and polka-dot marker act like the mouse cursor to indicate the position in the textbook image. Appropriate interactive audio-visual contents are augmented as the marker is located on the predefined image objects in the textbook. The proposed e-learning system was applied to the educational courses in the elementary school, and we obtained satisfactory results for real applications. We expect that the proposed e-learning system is popular when the educational contents and scenarios are sufficiently provided.