Abstract
The popularity of the contemporary smartphone makes it an attractive platform for new applications. We are exploring the potential of such personal devices to control networked displays. In particular, we have developed a system that can sense mobile phone orientation to support two kinds of juggling-like play styles: padiddle and poi. Padiddling is spinning a flattish object (such as a tablet or board-mounted smartphone) on the tip of one’s finger. Poi involves whirling a weight (in this case the smartphone itself) at the end of a tether. Orientation of a twirled device can be metered, and with a communications infrastructure, this streamed azimuthal data can be used to modulate various distributed, synchronous, multimodal displays, including panoramic and photospherical imagery, diffusion of pantophonic and periphonic auditory soundscapes, and mixed virtuality scenes featuring avatars and props animated by real-world twirling. The unique nature of the twirling styles allows interestingly fluid perspective shifts, including orbiting “inspection gesture” virtual cameras with self-conscious ambidextrous avatars and “reality distortion” fields with perturbed affordance projection.
Highlights
Mobile position sensing We are exploring the expressive potential of personal devices for social, multimodal displays: “multimedia juggling.”
Billinghurst et al used smartphones to acquire hand gestures captured by a mobile camera (Billinghurst et al 2014), overlaying mixed reality graphics stabilized by internal position sensors
This interface is somewhat different from our system, which uses smartphone position data directly, but which effects are displayed across the network
Summary
Mobile position sensing We are exploring the expressive potential of personal devices for social, multimodal displays: “multimedia juggling.” We originally experimented with using laptop sudden motion sensor control (Cohen 2008), but eventually settled on tracking with mobile devices. Azimuthal tracking especially allows control of horizontally expressive displays, including panoramic imagery, spatial sound, rotary motion platforms, positions of objects in mixed virtuality environments, as well as rhythmic renderings such as musical sequencing. Embedding sensing devices into a spinnable affordance allows a “spinning plate”-style interface, as seen in Fig. 2 (Cohen et al 2011) Anyway, even with such deployment, padiddling skills are somewhat difficult to acquire. Billinghurst et al used smartphones to acquire hand gestures captured by a mobile camera (Billinghurst et al 2014), overlaying mixed reality graphics stabilized by internal position sensors This interface is somewhat different from our system, which uses smartphone position data directly, but which effects are displayed across the network.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
