Abstract
This article introduces the Off-The-Shelf Stylus (OTSS), a framework for 2D interaction (in 3D) as well as for handwriting and sketching with digital pen, ink, and paper on physically aligned virtual surfaces in Virtual, Augmented, and Mixed Reality (VR, AR, MR: XR for short). OTSS supports self-made XR styluses based on consumer-grade six-degrees-of-freedom XR controllers and commercially available styluses. The framework provides separate modules for three basic but vital features: 1) The stylus module provides stylus construction and calibration features. 2) The surface module provides surface calibration and visual feedback features for virtual-physical 2D surface alignment using our so-called 3ViSuAl procedure, and surface interaction features. 3) The evaluation suite provides a comprehensive test bed combining technical measurements for precision, accuracy, and latency with extensive usability evaluations including handwriting and sketching tasks based on established visuomotor, graphomotor, and handwriting research. The framework’s development is accompanied by an extensive open source reference implementation targeting the Unity game engine using an Oculus Rift S headset and Oculus Touch controllers. The development compares three low-cost and low-tech options to equip controllers with a tip and includes a web browser-based surface providing support for interacting, handwriting, and sketching. The evaluation of the reference implementation based on the OTSS framework identified an average stylus precision of 0.98 mm (SD= 0.54 mm) and an average surface accuracy of 0.60 mm (SD= 0.32 mm) in a seated VR environment. The time for displaying the stylus movement as digital ink on the web browser surface in VR was 79.40 ms on average (SD= 23.26 ms), including the physical controller’s motion-to-photon latency visualized by its virtual representation (M= 42.57 ms,SD= 15.70 ms). The usability evaluation (N= 10) revealed a low task load, high usability, and high user experience. Participants successfully reproduced given shapes and created legible handwriting, indicating that the OTSS and it’s reference implementation is ready for everyday use. We provide source code access to our implementation, including stylus and surface calibration and surface interaction features, making it easy to reuse, extend, adapt and/or replicate previous results (https://go.uniwue.de/hci-otss).
Highlights
Writing and sketching are important aspects of human communication
While the point capturing for the surface calibration is performed using the position and rotation of the controller, the calibrated surface is aligned in real-world coordinates, and the accuracy measurements of the Head-Mounted Display (HMD) are relevant
For the stylus calibration tasks, Stimulation was rated as neutral with some participants rating it as “boring” and “not interesting”, others as “valuable” and “motivating”
Summary
Writing and sketching are important aspects of human communication. Symbolically representing language and ideas allows to externalize information and share it across space and time. Several use cases in Virtual, Augmented, and Mixed Reality (VR, AR, MR, in short XR) are suited to embedding text and illustrations, e.g., in the areas of knowledge work and communication, social XR, or training and learning scenarios (Latoschik et al, 2019). Today, writing by hand is marginalized evermore in favor of typewriting. This preference starts during beginning writing instruction and carries through adult’s work and leisure activities with various implications related to the ease of writing with keyboards and the consistent output it produces (Mangen, 2018). Neuroscientific research shows that the motor process of writing longhand is linked to more brain activity than typewriting (Ose Askvik et al, 2020). Neuroscientific research shows that the motor process of writing longhand is linked to more brain activity than typewriting (Ose Askvik et al, 2020). van der Meer and van der Weel (2017) aptly summarized a similar finding in their title as “Only Three Fingers Write but the Whole Brain Works”
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
