Multimodality — The Future of the Wireless User Interface

Abstract

The wireless market is applying two conflicting requirements on mobile terminals — they must get smaller and lighter, yet at the same time they must also deliver more advanced multimedia applications. As the terminals shrink, screen size becomes limited and keypads must either occupy less space or vanish entirely. Meanwhile as the applications grow richer and more complex, the requirements to be able to control and interact with them only expand. How can these apparently contradictory goals be met? How can rich multimedia applications be controlled as the physical space for keypads shrinks or indeed vanishes? The answer lies in the use of speech technology. By using speech recognition, voice commands can supplement the use of a keypad/stylus, while audio output can deliver additional information that cannot fit on the screen. The multimodal interface, combining the best of voice-only and graphical user interfaces is the future of the wireless user interface. This paper looks at recent developments in multimodal interfaces and explores how they will affect the evolution of wireless applications.