MARIO: Mid-air Augmented Reality Interaction with Objects

Abstract

A seamless connection between a game space and the real world is a long-sought goal of the entertainment computing field. Recent games have intuitive interactions through gestures and motion-based control. However, their images are still confined to the displays. In this paper, we propose a novel interaction experience in which mid-air images interact with physical objects. Our “Mid-air Augmented Reality Interaction with Objects” (MARIO) system enables images to be displayed in 3D spaces beyond screens. By creating a spatial link between images appearing in mid-air and physical objects, we extend video games into the real world. For entertainment purposes, a game character appears in mid-air and jumps over real blocks that users have arranged with their hands. A user study was conducted at public exhibitions, and the results showed the popularity and effectiveness of the system as an entertainment application.