Augmented Reality Agents Research Articles

Lightweight single pass numerical reading extraction for displays in the wild

Although considerable progress has been made in recognizing multi-character text from images, there are still cases where there is a lack of robust computationally-efficient methods that can execute on portable devices to read device displays in the wild. We specifically address the problem of parsing digits from 7 segment displays. Recognizing these displays is important for many tasks such as assisting users with tasks using augmented reality agents that need to verify actions or connecting legacy devices to the internet for process control using cheap cameras. Legacy techniques based on image processing operators and OCR are brittle whereas massive deep networks are too computationally expensive. We describe a computationally tractable VGG style backbone combined with a novel digit inference head that can be trained using a synthetic display generator with novel augmentations. We show the model trained on augmented synthetic data generalizes well to a corpus of real-world display images getting 97.8% single-frame accuracy and obtaining a throughput of 30 frames per second. We describe how the output can be further stabilized to improve accuracy through a kind of mode filtering.

A Framework for Self-Inspection Buildings Based on Augmented Reality Agents

A Framework for Self-Inspection Buildings Based on Augmented Reality Agents

Why, when and how to use augmented reality agents (AuRAs)

Over the last number of years, multiple research projects have begun to create augmented reality (AR) applications that use augmented reality agents, or AuRAs, as their principle interaction and development paradigm. This paper aims to address this new and distinct field of AuRAs by asking three questions: why should AuRAs be researched, when are they a useful paradigm, and how can they be developed? The first question explores the motivation behind applying AuRAs to AR. Specifically, it investigates whether AuRAs are purely an interaction paradigm, or whether they can also serve as a development paradigm, by outlining in which circumstances it is appropriate for a project to use AuRAs and where their addition would only add unnecessary complexity. A navigational experiment, performed in simulated AR, explores the second question of when AuRAs can be a useful concept in AR applications. Results from this experiment suggest that an embodied virtual character allows for faster navigation along a shorter route than directional arrows or marking the target with an AR "bubble". An exploration of the limitations of the simulated AR environment illuminates how faithfully the experiment recreated the environmental challenges that AuRAs can help to address. Finally, the question of how to develop such applications is addressed through the introduction of the agent factory augmented reality toolkit that allows the rapid prototyping of such applications. Results from a usability study on the toolkit are also presented.

Synthetic vision‐based perceptual attention for augmented reality agents

AbstractWe describe our model for synthetic vision‐based perceptual attention for autonomous agents in augmented reality (AR) environments. Since virtual and physical objects coexist in their environment, such agents must adaptively perceive and attend to objects relevant to their goals. To enable agents to perceive their surroundings, our approach allows the agents to determine currently visible objects from the scene description of what virtual and physical objects are configured in the camera's viewing area. In our model, a degree of attention is assigned to each perceived object based on its similarity to target objects related to an agent's goals. The agent can thus focus on a reduced set of perceived objects with respect to the estimated degree of attention. Moreover, by continuously and smartly updating the perceptual memory, it eliminates the processing loads associated to previously observed objects. To demonstrate the effectiveness of our approach, we implemented an animated character that was overlaid over a miniature version of campus in real‐time and that attended to building blocks relevant to given tasks. Experiments showed that our model could reduce a character's perceptual load at any time, even when surroundings change. Copyright © 2010 John Wiley & Sons, Ltd.

Augmented reality agents for user interface adaptation

AbstractMost augmented reality (AR) applications are primarily concerned with letting a user browse a 3D virtual world registered with the real world. More advanced AR interfaces let the user interact with the mixed environment, but the virtual part is typically rather finite and deterministic. In contrast, autonomous behavior is often desirable in ubiquitous computing (Ubicomp), which requires the computers embedded into the environment to adapt to context and situation without explicit user intervention. We present an AR framework that is enhanced by typical Ubicomp features by dynamically and proactively exploiting previously unknown applications and hardware devices, and adapting the appearance of the user interface to persistently stored and accumulated user preferences. Our framework explores proactive computing, multi‐user interface adaptation, and user interface migration. We employ mobile and autonomous agents embodied by real and virtual objects as an interface and interaction metaphor, where agent bodies are able to opportunistically migrate between multiple AR applications and computing platforms to best match the needs of the current application context. We present two pilot applications to illustrate design concepts. Copyright © 2007 John Wiley & Sons, Ltd.