Abstract
Mixed reality applications can provide users with enhanced interaction experiences by integrating virtual and real world objects in a mixed environment. Through the mixed reality interface, a more realistic and immersive control style is achieved compared to the traditional keyboard and mouse input devices. The interface proposed in this paper consists of a stereo camera, which tracks the user's hands and fingers robustly and accurately in the 3D space. To enable a physically realistic experience in the interaction, a physics engine is adopted for the simulating the physics of virtual object manipulation. The objects can be picked up and tossed with physical characteristics, such as gravity and collisions which occur in the real world. Detection and interaction in our system is fully computer-vision based, without any markers or additional sensors. We demonstrate this gesture-based interface using two mixed reality game implementations: finger fishing, in which a player can simulate fishing for virtual objects with his/her fingers as in a real environment, and Jenga, which is a simulation of the well-known tower building game. A user study is conducted and reported to demonstrate the accuracy, effectiveness and comfort of using this interactive interface.
Highlights
With the development of highly advanced technologies, computerbased entertainment systems make extensive use of multimedia elements, such as graphics and sound, which help to create a stimulating atmosphere
We proposed a system for 3D finger interactions with virtual objects
The barehand interactions are implemented with the passive vision-based approach without attaching any extra intrusive devices to the users. This system can be used for mixed reality games, i.e. the game player is presented with virtual objects embedded in the physical world
Summary
Which lack the intuitive and natural interactions with real environments. Mixed reality is a possible solution to this problem, as it creates an environment that supports interactions with both virtual and real objects. More recent examples include mulTetris [1] which uses a graspable interface to manipulate bricks in the traditional Tetris game, and MRI [20] which is a novel input device that provides control of virtual targets through the operation on real-world objects. Examples include Magic-Book [2], a mixed reality game displaying virtual pictures on physical books, and the Harbour Game [11], an urban planning game for harbour areas on a physical board with markers These interfaces provide user-friendly interactions that people without computer knowledge can handle well. If the object is released by the user, it will fall under gravity and collide with other objects as what would happen in the real world To study these bare-hand interactions in the mixed reality space, we have applied our interaction techniques to two games: finger fishing and Jenga. We will discuss the future works and the conclusions in the last section
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
