Three-dimensional PC: toward novel forms of human-computer interaction

Abstract

The ongoing integration of IT systems is offering computer users a wide range of new (networked) services and the access to an explosively growing host of multimedia information. Consequently, apart from the currently established computer tasks, the access and exchange of information is generally considered as the driving computer application of the next decade. Major advances are required in the development of the human-computer interface, in order to enable end-users with the most varied backgrounds to master the added functionalities of their computers with ease, and to make the use of computers in general an enjoyable experience. Research efforts in computer science are concentrated on user interfaces that support the highly evolved human perception and interaction capabilities better than today's 2D graphic user interfaces with a mouse and keyboard. Thanks to the boosting computing power of general purpose computers, highly interactive interfaces are becoming feasible supporting a variety of input and output modalities. Multimodal interaction not only makes working with a computer more natural and intuitive but can substantially help to disambiguate the exchange of information in both directions between the user and the computer. Recent approaches aim to create a (virtual) three-dimensional interaction environment where users find a clear arrangement of the media objects, paths and links used, as well as highly responsive tools supporting continuous interaction. The article overviews the current concepts in general and describes an approach developed by the authors in greater detail as a case study of a 3D PC (the mUltimo3D project). This particular approach uses 3D displays which do not require stereo glasses to present a 3D graphic user interface. A newly developed 3D display makes it possible to seamlessly integrate the virtual interaction space into the real working space. Video trackers remotely sensing the user offer new modalities for unencumbered interaction with the displayed objects. Interface agents watch and interpret the user's actions with regard to the task context, fuse the input from different interaction modalities, and organize the visual interaction space accordingly. The concept aims at the vision of a command-free, continuous human-computer interaction. A test bed has been implemented and used for system evaluation with users. First results are reported.