Text Entry Performance Research Articles

Text Entry Performance and Situation Awareness of a Joint Optical See-Through Head-Mounted Display and Smartphone System.

Optical see-through head-mounted displays (OST HMDs) are a popular output medium for mobile Augmented Reality (AR) applications. To date, they lack efficient text entry techniques. Smartphones are a major text entry medium in mobile contexts but attentional demands can contribute to accidents while typing on the go. Mobile multi-display ecologies, such as combined OST HMD-smartphone systems, promise performance and situation awareness benefits over single-device use. We study the joint performance of text entry on mobile phones with text output on optical see-through head-mounted displays. A series of five experiments with a total of 86 participants indicate that, as of today, the challenges in such a joint interactive system outweigh the potential benefits.

Optimising virtual object position for efficient eye-gaze interaction in Hololens2

ABSTRACT Our study explored eye-tracking technology in the Hololens2 HMD. We assessed the effectiveness of eye-gaze interactions in a 3D environment, particularly in text entry applications. Existing recommendations for the placement and size of virtual objects are often followed without empirical validation. Therefore, we evaluated text entry target selection rates within the manufacturer’s specified optimal gaze interaction zone. We measured the spatial accuracy and precision of eye gaze data and optimised target positions for enhanced text entry performance. By establishing confidence ellipses covering 95% of gaze points per target, we derived an Area of Interest (AOI) recalibration function. Applying a receiver operating characteristic-based method, we quantified the recalibrated tracker’s performance at various AOIs. Our results indicate an optimal recalibrated AOI radius is 0.036 m, and the ideal object-plane distance from the eye-plane is 2.25 m. This recalibrated specification allows to efficiently interact with Hololens2 via eye movement, achieving target selection rates exceeding 90%.

Text Input for Non-Stationary XR Workspaces: Investigating Tap and Word-Gesture Keyboards in Virtual and Augmented Reality.

This article compares two state-of-the-art text input techniques between non-stationary virtual reality (VR) and video see-through augmented reality (VST AR) use-cases as XR display condition. The developed contact-based mid-air virtual tap and wordgesture (swipe) keyboard provide established support functions for text correction, word suggestions, capitalization, and punctuation. A user evaluation with 64 participants revealed that XR displays and input techniques strongly affect text entry performance, while subjective measures are only influenced by the input techniques. We found significantly higher usability and user experience ratings for tap keyboards compared to swipe keyboards in both VR and VST AR. Task load was also lower for tap keyboards. In terms of performance, both input techniques were significantly faster in VR than in VST AR. Further, the tap keyboard was significantly faster than the swipe keyboard in VR. Participants showed a significant learning effect with only ten sentences typed per condition. Our results are consistent with previous work in VR and optical see-through (OST) AR, but additionally provide novel insights into usability and performance of the selected text input techniques for VST AR. The significant differences in subjective and objective measures emphasize the importance of specific evaluations for each possible combination of input techniques and XR displays to provide reusable, reliable, and high-quality text input solutions. With our work, we form a foundation for future research and XR workspaces. Our reference implementation is publicly available to encourage replicability and reuse in future XR workspaces.

A Portable One-Hand Braille Input System Enabled by Dual-Interlocked Tactile Sensors for Communication

This article presents a novel portable one-hand Braille input system (BIS) to assist visually impaired users in entering text on electronic devices. The proposed BIS is designed based on a flexible Braille keyboard (FBK), it mainly consists of five dual-interlocked tactile sensors, a processing circuit, and a software modulus. The fully elastomeric graphene/carbon nanotube/ silicone rubber nanocomposites are synthesized and used as tactile sensing material and polydimethylsiloxane (PDMS) as the structural and encapsulation for the fabrication of sensors. Characterization tests showed the tactile sensor has relatively high force sensing sensitivity of 0.649 N <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^{-\text{1}}$</tex-math> </inline-formula> at sensing range of 0–5 N and a frequency sensing ability of 0.5–2.0 Hz, which can meet human finger pressing requirements. A specific Braille input method is proposed, and personalized parameters are set for users to fit different pressing habits. The average input speed of 3.37 words per minute (WPM) with an average accuracy of 92.2% for Braille characters input is demonstrated by evaluating text entry performance. The results indicated that our developed tactile sensor enabled portable one-hand BIS can assist visual impaired users with enhanced text entry on electronic devices effectively and privately.

Point and Select: Effects of Multimodal Feedback on Text Entry Performance in Virtual Reality

The current study examined the effects of visual, auditory, and haptic feedback on VR text entry performance and perceived mental workload across two experiments. In Experiment 1, the feedback was presented as users were pointing from one key to another, dubbed pointing feedback. Results from Experiment 1 indicated that providing haptic pointing feedback led to faster text entry. Results also showed that providing any modality of pointing feedback resulted in more accurate text entry compared to no feedback, but there was no substantial contribution of additional pointing feedback modality to text accuracy when one pointing feedback modality was present. Providing auditory pointing feedback led to greater perceived mental workload of the VR text entry task. In Experiment 2, the feedback was provided when users selected the keys by pressing the trigger on the controller, dubbed selection feedback. Results from Experiment 2 showed that providing auditory or haptic selection feedback led to faster text entry performance and that providing visual selection feedback improved accuracy. The presence of haptic selection feedback led to lower levels of perceived mental workload. These results indicate that the usability of VR text entry tasks changes as a function of when and how feedback is provided.

Investigating the Effects of Virtual Keyboard Position and Angle on Text Entry Performance and Subjective Evaluation

Investigating the Effects of Virtual Keyboard Position and Angle on Text Entry Performance and Subjective Evaluation

BackSwipe: Back-of-device Word-Gesture Interaction on Smartphones.

Back-of-device interaction is a promising approach to interacting on smartphones. In this paper, we create a back-of-device command and text input technique called BackSwipe, which allows a user to hold a smartphone with one hand, and use the index finger of the same hand to draw a word-gesture anywhere at the back of the smartphone to enter commands and text. To support BackSwipe, we propose a back-of-device word-gesture decoding algorithm which infers the keyboard location from back-of-device gestures, and adjusts the keyboard size to suit the gesture scales; the inferred keyboard is then fed back into the system for decoding. Our user study shows BackSwipe is feasible and a promising input method, especially for command input in the one-hand holding posture: users can enter commands at an average accuracy of 92% with a speed of 5.32 seconds/command. The text entry performance varies across users. The average speed is 9.58 WPM with some users at 18.83 WPM; the average word error rate is 11.04% with some users at 2.85%. Overall, BackSwipe complements the extant smartphone interaction by leveraging the back of the device as a gestural input surface.

Quadmetric Optimized Thumb-to-Finger Interaction for Force Assisted One-Handed Text Entry on Mobile Headsets

Augmented reality head-worn computers often feature small-sized touch interfaces that complicate interaction with content, provide insufficient space for comfortable text input, and can be awkward to use in social situations. This paper presents a novel one-handed thumb-to-finger text entry solution for augmented reality head-worn computers. We design a glove composed of 12 force-sensitive nodes featuring an ambiguous keyboard layout. We first explore the viability of force disambiguation to evaluate the force division within the force spectrum. We select a 3-level force division as it allows to considerably reduce the number of keys while featuring a high (83.9%) accuracy. Following this pilot study, we map the 26 English characters onto the 9 nodes located on the index, middle and ring fingers in a 3-3-3 configuration, and attribute the space, enter and backspace keys to the remaining three nodes. We consider text entry performance as a quadmetric optimization problem considering the following criteria: goodness of character pairs, layout similarity to the QWERTY keyboard, easiness of force interaction, and comfort level of thumb reach. The resulting layout strikes a balance between performance and usability. We finally evaluate the quadmetric optimized layout over 6 sessions with 12 participants. The participants achieve an average text entry rate of 6.47 WPM with 6.85% error rate in the final session, which is significantly faster than existing thumb-to-finger solutions. In addition, our one-handed text entry system enhances the user mobility compared to other state-of-the-art solutions by freeing one hand, while allowing the user to direct his visual attention to other activities.

Errata to "RingText: Dwell-Free and Hands-Free Text Entry for Mobile Head-Mounted Displays Using Head Motions".

In this paper, we present a case for text entry using a circular keyboard layout for mobile head-mounted displays (HMDs) that is dwell-free and does not require users to hold a dedicated input device for letter selection. To support the case, we have implemented RingText whose design is based on a circular layout with two concentric circles. The outer circle is subdivided into regions containing letters. Selection is made by using a virtual cursor controlled by the user's head movements—entering a letter region triggers a selection and moving back into the inner circle resets the selection. The design of RingText follows an iterative process, where we initially conduct one first study to investigate the optimal number of letters per region, inner circle size, and alphabet starting location. We then optimize its design by selecting the most suitable features from the first study: one letter per region, narrowing the trigger area to lower error rates, and creating candidate regions that incorporate two suggested words to appear next to the current letter region (close to the cursor) using a dynamic (rather than fixed) approach. Our second study compares text entry performance of RingText with four other hands-free techniques and the results show that RingText outperforms them. Finally, we run a third study lasting four consecutive days with 10 participants (5 novice users and 5 expert users) doing two daily sessions and the results show that RingText is quite efficient and yields a low error rate. At the end of the eighth session, the novice users can achieve a text entry speed of 11.30 WPM after 60 minutes of training while the expert (more experienced) users can reach an average text entry speed of 13.24 WPM after 90 minutes of training.

RingText: Dwell-free and hands-free Text Entry for Mobile Head-Mounted Displays using Head Motions.

In this paper, we present a case for text entry using a circular keyboard layout for mobile head-mounted displays (HMDs) that is dwell-free and does not require users to hold a dedicated input device for letter selection. To support the case, we have implemented RingText whose design is based on a circular layout with two concentric circles. The outer circle is subdivided into regions containing letters. Selection is made by using a virtual cursor controlled by the user's head movements-entering a letter region triggers a selection and moving back into the inner circle resets the selection. The design of RingText follows an iterative process, where we initially conduct one first study to investigate the optimal number of letters per region, inner circle size, and alphabet starting location. We then optimize its design by selecting the most suitable features from the first study: one letter per region, narrowing the trigger area to lower error rates, and creating candidate regions that incorporate two suggested words to appear next to the current letter region (close to the cursor) using a dynamic (rather than fixed) approach. Our second study compares text entry performance of RingText with four other hands-free techniques and the results show that RingText outperforms them. Finally, we run a third study lasting four consecutive days with 10 participants (5 novice users and 5 expert users) doing two daily sessions and the results show that RingText is quite efficient and yields a low error rate. At the end of the eighth session, the novice users can achieve a text entry speed of 11.30 WPM after 60 minutes of training while the expert (more experienced) users can reach an average text entry speed of 13.24 WPM after 90 minutes of training.

TOAST

Touch typing on flat surfaces (e.g. interactive tabletop) is challenging due to lack of tactile feedback and hand drifting. In this paper, we present TOAST, an eyes-free keyboard technique for enabling efficient touch typing on touch-sensitive surfaces. We first formalized the problem of keyboard parameter (e.g. location and size) estimation based on users' typing data. Through a user study, we then examined users' eyes-free touch typing behavior on an interactive tabletop with only asterisk feedback. We fitted the keyboard model to the typing data, results suggested that the model parameters (keyboard location and size) changed not only between different users, but also within the same user along with time. Based on the results, we proposed a Markov-Bayesian algorithm for input prediction, which considers the relative location between successive touch points within each hand respectively. Simulation results showed that based on the pooled data from all users, this model improved the top-1 accuracy of the classical statistical decoding algorithm from 86.2% to 92.1%. In a second user study, we further improved TOAST with dynamical model parameter adaptation, and evaluated users' text entry performance with TOAST using realistic text entry tasks. Participants reached a pick-up speed of 41.4 WPM with a character-level error rate of 0.6%. And with less than 10 minutes of practice, they reached 44.6 WPM without sacrificing accuracy. Participants' subjective feedback also indicated that TOAST offered a natural and efficient typing experience.

Text entry rate of access interfaces used by people with physical disabilities: A systematic review

ABSTRACTThis study systematically reviewed the research on assistive technology (AT) access interfaces used for text entry, and conducted a quantitative synthesis of text entry rates (TER) associated with common interfaces. We searched 10 databases and included studies in which: typing speed was reported in words per minute (WPM) or equivalent; the access interface was available for public use; and individuals with physical impairments were in the study population. For quantitative synthesis, we used only the TER reported for individuals with physical impairments. Studies also had to report the sample size, and the average and standard deviation for the TER data. Thirty-nine studies met the criteria for quantitative synthesis. Studies involved seven interface types: standard keyboard typing, on-screen keyboard (OSK) with cursor selection, OSK with scanning selection, automatic speech recognition (ASR), Morse code, brain–computer interface (BCI), and other. ASR, standard keyboard, cursor OSK, and scanning OSK had at least four studies and 30 subjects, with TER averaging 15.4, 12.5, 4.2, and 1.7 WPM, respectively. When combined with measurements of a particular client’s text entry performance, the TER from this review can be used within an evidence-based decision-making process for selecting control interfaces.

Visual and manual loadings with QWERTY-like ambiguous keyboards: Relevance of letter-key assignments on mobile phones

Mobile phone touchscreens have many ergonomic problems related to text entry. Previous studies, which attempted to use ambiguous keyboards to resolve problem of small keys, focused on the disambiguation process without consideration of the user loading on graphic user interface. This study investigates user loadings, which interfere with performance in the key selection phase when using an ambiguous keyboard. Hence, three QWERTY-like ambiguous keyboards and a standard QWERTY keyboard were compared via visual search and manual operation experiments. The visual search experiment shows that layouts with many letters per key were strongly related to long visual search times, and layouts with large keys were operated faster and more accurately in the manual operation experiment. Consequently, the trade-off between visual and manual loading differed among different letter-key assignments. This study is important in that it elucidates the impacts of visual and manual loadings on ambiguous keyboards, as well as in providing user interface designers with an enhanced understanding of how to design ambiguous keyboards based on user criteria. Relevance to industryAmbiguous keyboard designs in previous studies were generally far from the actual needs of users. This study examines the factors affecting text entry performance of users via the user-centered approach, improving the understanding of designers.

Investigation of Chinese text entry performance for mobile display interfaces

This study examined the effects of panel type, frequency of use and arrangement of phonetic symbols on operation time, usability, visual fatigue and workload in text entry performance. Three types of panel (solid, touch and mixed), three types of frequency of use (low, medium and high) and two types of the arrangement of phonetic symbols (vertical and horizontal) were investigated through 30 college students in the experiment. The results indicated that panel type, frequency of use, arrangement of phonetic symbols and the interaction between panel type and frequency of use were significant factors on operation time. Panel type was also a significant factor on usability, and a touch panel and a solid panel showed better usability than a mixed panel. Furthermore, a touch panel showed good usability and the lowest workload and therefore it is recommended to use a touch panel with vertical phonetic arrangement in sending Chinese text messages.Practitioner Summary: This study found, from ergonomics considerations, that a touch panel showed good usability and it is recommended to use a touch panel with vertical phonetic arrangement in sending Chinese text messages. Mobile display manufacturers can use the results of this study as a reference for future keyboard design.

Disambiguation of imprecise input with one-dimensional rotational text entry

We introduce a distinction between disambiguation supporting continuous versus discrete ambiguous text entry. With continuous ambiguous text entry methods, letter selections are treated as ambiguous due to expected imprecision rather than due to discretized letter groupings. We investigate the simple case of a one-dimensional character layout to demonstrate the potential of techniques designed for imprecise entry. Our rotation-based sight-free technique, Rotext, maps device orientation to a layout optimized for disambiguation, motor efficiency, and learnability. We also present an audio feedback system for efficient selection of disambiguated word candidates and explore the role that time spent acknowledging word-level feedback plays in text entry performance. Through a user study, we show that despite missing on average by 2.46--2.92 character positions, with the aid of a maximum a posteriori (MAP) disambiguation algorithm, users can average a sight-free entry speed of 12.6wpm with 98.9% accuracy within 13 sessions (4.3 hours). In a second study, expert users are found to reach 21wpm with 99.6% accuracy after session 20 (6.7 hours) and continue to grow in performance, with individual phrases entered at up to 37wpm. A final study revisits the learnability of the optimized layout. Our modeling of ultimate performance indicates maximum overall sight-free entry speeds of 29.0wpm with audio feedback, or 40.7wpm if an expert user could operate without relying on audio feedback.

Mobile text-entry and visual demands: reusing and optimizing current solutions

Mobile devices are increasingly used for text-entry in contexts where visual attention is fragmented and graphical information is inadequate, yet the current solutions to typing on virtual keyboards make it a visually demanding task. This work looks at assistive technologies and interface attributes as tools to ease the task. Two within-subject experiments were performed with 23 and 17 participants, respectively. The first experiment aimed to understand how walking affected text-entry performance and additionally to assess how effective assistive technologies can be in mobile contexts. In the second experiment, adaptive keyboards featuring character prediction and pre-attentive attributes to ease visual demands of text-entry interfaces were developed and evaluated. It has been found that both text-input speed and overall quality are affected in mobile situations. Contrary to the expectations, assistive technologies proved ineffective with visual feedback. The second experiment showed that pre-attentive attributes do not affect users' performance in task-entry tasks, even though a 3.3---4.3 % decrease in error rates was measured. It was found that users reduce walking speed to compensate for challenges placed by mobile text-entry. Caution should be exercised when transferring assistive technologies to mobile contexts, since they need adaptations to address mobile users' needs. Also, while pre-attentive attributes seemingly have no effect on experienced QWERTY typists' performance, they showed promise for both novice users and typists in attention-demanding contexts.

Establishing a baseline for text entry for a multi-touch virtual keyboard

Multi-touch, which has been heralded as a revolution in human–computer interaction, provides features such as gestural interaction, tangible interfaces, pen-based computing, and interface customization—features embraced by an increasingly tech-savvy public. However, multi-touch platforms have not been adopted as “everyday” computer interaction devices that support important text entry intensive applications such as word processing and spreadsheets. In this paper, we present two studies that begin to explore user performance and experience with entering text using a multi-touch input. The first study establishes a benchmark for text entry performance on a multi-touch platform across input modes that compare uppercase-only to mixed-case, single-touch to multi-touch and copy to memorization tasks. The second study includes mouse style interaction for formatting rich text to simulate a word processing task using multi-touch input. As expected, our results show that users do not perform as well in terms of text entry efficiency and speed using a multi-touch interface as with a traditional keyboard. Not as expected was the result that degradation in performance was significantly less for memorization versus copy tasks, and consequently willingness to use multi-touch was substantially higher (50% versus 26%) in the former case. Our results, which include preferred input styles of participants, also provide a baseline for further research to explore techniques for improving text entry performance on multi-touch systems.

ACT-R Predictive Model of Korean Text Entry on Touchscreen

Objective: The aim of this study is to predict Korean text entry on touchscreens using ACT-R cognitive architecture. Background: Touchscreen application in devices such as satellite navigation devices, PDAs, mobile phones, etc. has been increasing, and the market size is expanding. Accordingly, there is an increasing interest to develop and evaluate the interface to enhance the user experience and increase satisfaction in the touchscreen environment. Method: In this study, Korean text entry performance in the touchscreen environment was analyzed using ACT-R. The ACT-R model considering the characteristics of the Korean language which is composed of vowels and consonants was established. Further, this study analyzed if the prediction of Korean text entry is possible through the ACT-R cognitive model. Results: In the analysis results, no significant difference on performance time between model prediction and empirical data was found. Conclusion: The proposed model can predict the accurate physical movement time as well as cognitive processing time. Application: This study is useful in conducting model-based evaluation on the text entry interface of the touchscreen and enabled quantitative and effective evaluation on the diverse types of Korean text input interfaces through the cognitive models.

Text Entry via Character Stroke Disambiguation for an Adolescent With Severe Motor Impairment and Cortical Visual Impairment

ABSTRACT This study proposed a single-switch text entry system by hierarchical scanning of character strokes for an 11-year-old girl with severe physical disabilities and low vision. She could only perceive magnified straight line segments and chords presented against high-contrast, colored backgrounds. In a descriptive case study, the participant used the proposed system in the community for 8 months. Assessment included theoretical evaluation of text entry performance and empirical evaluation of the participant's proficiency. The proposed system had a lower error-free text entry rate but comparable proneness to user error as a real-world implementation of row-column virtual scanning keyboard with character frequency layout. The participant's proficiency, in terms of mean number of single-switch activations and time to type one character, showed statistically significant improvements as the case study progressed. The proposed system feasibly addressed the participant's typing needs, in a context where traditional row-column scanning and codeword-based text entry systems were not successful.

A Usability Evaluation of a Laser Projection Virtual Keyboard

This study compared typing performance using a laser projection virtual keyboard (VKB), and an electromechanical folding keyboard (FKB). Twenty undergraduate students (10 male, 10 female) participated as subjects in the study. Typing performance was measured for twelve 7.5-minute typing trials for each of the two keyboards. The results showed that typing was significantly slower with the VKB: by the twelfth typing trial the text entry performance with the VKB was 17.3 wpm compared to 40.4 wpm for the FKB, and the typing error rate was 8.6% for the VKB compared to 5.3% for the FKB. Text entry performance was 57.1% slower with the VKB than the FKB, but it was comparable to the speeds reported for other handheld device text entry methods (Multitap, T9, and handwriting recognition). A learning curve model showed that theoretically the typing speed for the VKB could reach 20 wpm after 20 hours of training.