Text Entry Rate Research Articles

Imperfect Surrogate Users: Understanding Performance Implications of Augmentative and Alternative Communication Systems through Bounded Rationality, Human Error, and Interruption Modeling

Nonspeaking individuals with motor disabilities frequently rely on augmentative and alternative communication (AAC) systems that allow users to communicate through a text entry interface coupled with a speech synthesizer. Such systems are notoriously difficult to evaluate with end-users. However, recent research has proposed envelope analysis as a method to estimate text entry rates and keystroke savings by simulating the interaction of an expert surrogate user entering sentences on a conceptual word-predictive text entry system. While only a part of the evaluation process of an AAC system, this method enables AAC designers to benefit from quantitative insights early on in the design process. This paper extends prior work by (1) demonstrating how to incorporate natural language generation, such as sentence generation, in such analyses; (2) presenting a model of an imperfect surrogate user that incorporates bounded rationality, human error, and interruptions to provide a more realistic simulation of text entry behavior; and (3) demonstrating how to estimate model parameters by observing users' actual typing behavior. We validate the model with data collected from eight participants using an AAC system on a touchscreen.

MyoKey: Inertial Motion Sensing and Gesture-Based QWERTY Keyboard for Extended Realities

Usability challenges and social acceptance of textual input in a context of extended realities (XR) motivate the research of novel input modalities. We investigate the fusion of inertial measurement unit (IMU) control and surface electromyography (sEMG) gesture recognition applied to text entry using a QWERTY-layout virtual keyboard. We design, implement and evaluate our proposed multi-modal solution, MyoBoard, which is an IMU-driven and EMG-based text entry system based on the QWERTY keyboard layout for XR headsets. The user can select characters with a combination of arm movements and hand gestures. MyoBoard employs a light-weight convolutional neural network classifier that can be deployed on a mobile device with insignificant inference time. We demonstrate the practicality of interruption-free text entry in mobile scenarios with MyoBoard, by recruiting 12 participants and by testing three sets of grasp micro-gestures in three scenarios - empty hand text input, input holding a pen, and a large object, such as an umbrella. With MyoBoard, users achieve an average text entry rate of 9.33 words per minute (WPM), 8.76 WPM, and 8.35 WPM for the freehand, pen, and umbrella conditions, respectively.

The effect of using entry-level keyboards in improving user-adaptability to new keyboards: Case of the Central-Bantu keyboards

Current trends in keyboard design show that QWERTY-similarity is a key factor for high user-adaptability keyboard design. This design approach has the challenge that the higher the similarity the lower the text-entry rate for the optimized keyboard. This article reports on the findings of an empirical study which we conducted on QWERTY-users to measure the effect of using an entry-level keyboard in improving user-ability to adapt to a new keyboard. The study used two Central-Bantu physical keyboards (entry-level: with high QWERTY-similarity, and advanced-level: with low similarity) which we had designed in an earlier study. The empirical study obtained learning-curves of the Advanced-level keyboard, of a 12-participant group which was first introduced to an entry-level keyboard, against a control-group of similar size, in a longitudinal study design. A two-sample t-test on the empirical results showed that the entry-level approach caused a marginally significant text-entry-rate improvement of 9.4% with p < .09. A two-sample U-test on word-error rates indicated a non-significant improvement of 8.4%. Our study has shown that the use of entry keyboards is an effective strategy in improving keyboard user-adaptability.

CrowbarLimbs: A Fatigue-Reducing Virtual Reality Text Entry Metaphor.

Text entry remains challenging in virtual environments, where users may quickly experience physical fatigue in some body parts using existing methods. In this paper, we propose "CrowbarLimbs," a novel virtual reality (VR) text entry metaphor with two deformable extended virtual limbs. By using a crowbar-like metaphor and placing the virtual keyboard at a user-preferred location based on the user's physical stature, our method can assist the user in placing their hands and arms in a comfortable posture, thus effectively reducing the physical fatigue in various body parts, such as hands, wrists, and elbows. In an initial user study, we found that CrowbarLimbs achieved text entry speed, accuracy, and system usability comparable to those of previous VR typing methods. To investigate the proposed metaphor in more depth, we further conducted two additional user studies to explore the ergonomically user-friendly shapes of CrowbarLimbs and virtual keyboard locations. The experimental results indicate that the shapes of CrowbarLimbs have significant effects on the fatigue ratings in various body parts and text entry speed. Furthermore, placing the virtual keyboard near the user and at half their height can lead to a satisfactory text entry rate of 28.37 words per minute.

C-PAK: Correcting and Completing Variable-Length Prefix-Based Abbreviated Keystrokes

Improving keystroke savings is a long-term goal of text input research. We present a study into the design space of an abbreviated style of text input called C-PAK (Correcting and completing variable-length Prefix-based Abbreviated Keystrokes) for text entry on mobile devices. Given a variable length and potentially inaccurate input string (e.g., “li g t m”), C-PAK aims to expand it into a complete phrase (e.g., “looks good to me”). We develop a C-PAK prototype keyboard, PhraseWriter , based on a current state-of-the-art mobile keyboard consisting of 1.3 million n -grams and 164,000 words. Using computational simulations on a large dataset of realistic input text, we found that, in comparison to conventional single-word suggestions, PhraseWriter improves the maximum keystroke savings rate by 6.7% (from 46.3% to 49.4,), reduces the word error rate by 14.7%, and is particularly advantageous for common phrases. We conducted a lab study of novice user behavior and performance which found that users could quickly utilize the C-PAK style abbreviations implemented in PhraseWriter, achieving a higher keystroke savings rate than forward suggestions (25% vs. 16%). Furthermore, they intuitively and successfully abbreviated more with common phrases. However, users had a lower overall text entry rate due to their limited experience with the system (28.5 words per minute vs. 37.7). We outline future technical directions to improve C-PAK over the PhraseWriter baseline, and further opportunities to study the perceptual, cognitive, and physical action trade-offs that underlie the learning curve of C-PAK systems.

Designing Mobile MR Workspaces

Virtual Reality (VR) is increasingly used in everyday contexts for a variety of tasks. We particularly look at the confined space for passengers inside cars, where head-mounted displays (HMDs) could complement the prevalent use of mobile devices for work. In a field study (N=19), we tested three mobile workspace setups along the reality-virtuality continuum (Mounted Tablet, Augmented Focus Bubble, and Virtual Office) and let users re-position the virtual keyboard and display while typing on a physical keyboard in a parked car. The results revealed that using HMDs lowered users' awareness of their real surroundings but increased their perceived workload with a performance impairment of text entry rate compared to just using a tablet. Letting users customize their workspace layout improved their perceived performance and decreased pitch-axis head movements for switching between the virtual display and keyboard. This paper discusses challenges and strategies for future work regarding dynamic incorporation of productivity tools, adaptive mixed reality (MR) work environment designs, and optimizing upper thresholds of physical discomfort in mobile MR workspaces.

A Usability Study of the Central-Bantu Multilingual Keyboards

ABSTRACT Central-Bantu speakers usually use the predominant QWERTY keyboard on a combination of some local and European languages, such as Chichewa and English. There has been no research that looks into usability issues of Central-Bantu speakers typing text in English and in local languages on QWERTY. This article presents our experiments that assess the usability of multilingual keyboards for Central-Bantu language speakers. We first obtained a Central-Bantu layout that is partially optimized from QWERTY, based on the German-QWERTZ keyboard, and a highly optimized version based on the French-AZERTY keyboard. The two keyboard layouts (Entry and advanced-level) were optimized for both English and Chichewa, for a two-finger text-entry mode. Simulation tests measured text-entry rates for Chichewa and English. The usability experiments show that the text-entry rate of English is about higher than that of Chichewa, which is close to the prediction of the simulation test. The Bantu keyboards reduced this imbalance to about . While the simulation tests predicted an increase in text-entry rate, the experimental tests show a decrease in performance. Typing-error rates almost double from QWERTY to the Advanced-Bantu keyboard. In this study, we show that the Central-Bantu keyboards have high practical-performance potential for both Chichewa and English, at balanced usability levels.

VectorEntry

Mobile phones are now touch-enabled, which allows the use of on-screen keyboards for text entry. Text entry tasks are among the most frequently occurring tasks performed by mobile phone users. However, people with visual impairments find it difficult to use on-screen keyboards, and this affects their digital literacy. In this article, a text entry mechanism is proposed to solve this problem using directional movement gestures suitable for people with visual impairments. Two forms of directional movement gestures, guided and unguided movements, are first studied, and the analysis reveals that unguided directional movement gestures are more suitable for a text entry mechanism for individuals who are visually impaired. Based on this insight, a text entry mechanism called VectorEntry is developed. It uses eight unguided directional movement gestures to select characters on the keyboard of a touch-enabled mobile phone. The keyboard is designed in accordance with the traditional 4×3 telephone keypad. The results of experiments show that the average text entry rate of VectorEntry was 3.3 wpm, 83.3% higher than the state-of-the-art No-Look Notes used for similar tasks. Its average rate of error in text entry was only 0.19% per character.

Integration of eye tracking and lip motion for hands-free computer access

Healthy people use a keyboard and mouse as standard input devices for controlling a computer. However, these input devices are usually not suitable for people with severe physical disabilities. This study aims to design and implement a suitable and reliable human–computer interactive (HCI) interface for disabled users by integrating both eye tracking technology and lip motion recognition. Eye movements control the cursor position on the computer screen. An eye gaze followed by a lip motion of the mouth opening served as a mouse click action. Seven lip motion features were extracted to discriminate mouth openings and mouth closures with the cumulative sum control chart algorithm. A novel smoothing technique called the threshold-based Savitzky–Golay smoothing filter was proposed to stabilize the cursor movement due to the inherent jittery motions of eyes and to reduce eye tracking latencies. In this study, a fixation experiment with nine dots was carried out to evaluate the efficacy of eye gaze data smoothing. A Chinese text entry experiment based on the on-screen keyboard with four keypad sizes was designed to evaluate the influence of the keypad size on the Chinese text entry rate. The results of the fixation experiment indicated that the threshold-based Savitzky–Golay smoothing filter with a threshold value of two standard deviations, a polynomial order of 3, and a window length of 61 can significantly improve the stability of eye cursor movements by 44.86% on average. The averaged Chinese text entry rate achieved 4.41 wpm with the dynamical enlargeable keypads. The current results encouraged us to utilize the proposed HCI interface for disabled users in the future.

From seen to unseen: Designing keyboard-less interfaces for text entry on the constrained screen real estate of Augmented Reality headsets

Text input is a very challenging task in the constrained screen real-estate of Augmented Reality headsets. Typical keyboards spread over multiple lines and occupy a significant portion of the screen. In this article, we explore the feasibility of single-line text entry systems for smartglasses. We first design FITE, a dynamic keyboard where the characters are positioned depending on their probability within the current input. However, the dynamic layout leads to mediocre text input and low accuracy. We then introduce HIBEY, a fixed 1-line solution that further decreases the screen real-estate usage by hiding the layout. Despite its hidden layout, HIBEY surprisingly performs much better than FITE, and achieves a mean text entry rate of 9.95 words per minute (WPM) with 96.06% accuracy, which is comparable to other state-of-the-art approaches. After 8 days, participants achieve an average of 13.19 WPM. In addition, HIBEY only occupies 13.14% of the screen real estate at the edge region, which is 62.80% smaller than the default keyboard layout on Microsoft Hololens.

Controller-based Text-input Techniques for Virtual Reality: An Empirical Comparison

Existing consumer VR systems support text input using handheld controllers in combination with virtual keyboards and many designers have attempted to build on these widely used techniques. However, information on current and well-established VR text-input techniques is lacking. In this work, we conduct a comparative empirical evaluation of four controller-based VR text-input techniques, namely, raycasting, drum-like keyboard, head-directed input, and split keyboard. We focus on their text-entry rate and accuracy, usability, and user experience. Twenty-two participants evaluated the techniques by completing a typing session, answering usability and user-experience questionnaires, and participating in a semi-structured interview. The drum-like keyboard and the raycasting techniques stood out, achieving good usability scores, positive experiential feedback, satisfactory text-entry rates, and moderate error rates that can be reduced in future studies. The specific documented usability and experiential characteristics of the techniques are presented and discussed herein.

Quadmetric Optimized Thumb-to-Finger Interaction for Force Assisted One-Handed Text Entry on Mobile Headsets

Augmented reality head-worn computers often feature small-sized touch interfaces that complicate interaction with content, provide insufficient space for comfortable text input, and can be awkward to use in social situations. This paper presents a novel one-handed thumb-to-finger text entry solution for augmented reality head-worn computers. We design a glove composed of 12 force-sensitive nodes featuring an ambiguous keyboard layout. We first explore the viability of force disambiguation to evaluate the force division within the force spectrum. We select a 3-level force division as it allows to considerably reduce the number of keys while featuring a high (83.9%) accuracy. Following this pilot study, we map the 26 English characters onto the 9 nodes located on the index, middle and ring fingers in a 3-3-3 configuration, and attribute the space, enter and backspace keys to the remaining three nodes. We consider text entry performance as a quadmetric optimization problem considering the following criteria: goodness of character pairs, layout similarity to the QWERTY keyboard, easiness of force interaction, and comfort level of thumb reach. The resulting layout strikes a balance between performance and usability. We finally evaluate the quadmetric optimized layout over 6 sessions with 12 participants. The participants achieve an average text entry rate of 6.47 WPM with 6.85% error rate in the final session, which is significantly faster than existing thumb-to-finger solutions. In addition, our one-handed text entry system enhances the user mobility compared to other state-of-the-art solutions by freeing one hand, while allowing the user to direct his visual attention to other activities.

The dynamic grouping keyboard: a general keyboard optimization approach for users with motor impairment

An important feature of software keyboards is its ability to flexibly adapt the interface. In this paper, we present the Dynamic Grouping Keyboard, as a general approach to optimize text entry for users with motor impairments. Each time after a key is selected, the Dynamic Grouping Keyboard adapts the grouping of keys to optimize the interaction cost, such as the number of keystrokes and the distance of cursor movement, while maintaining the key locations to preserve keyboard layout. We identify the technical challenge, and propose a two-step algorithm for optimization: The first step estimates the amount of information that the current step should provide, while the second step minimizes the interaction cost by applying dynamic programming. Two lab studies with twelve able-bodied users were conducted, in which three keyboard cases for motor impairment were tested. Results showed that compared to an optimal static grouping (ambiguous) keyboard, the Dynamic Grouping Keyboard reduced interaction cost by 23–75%, and improved text entry rate by 16–71% across all tested cases. Subjective user feedback showed consistent preference for the Dynamic Grouping Keyboard due to its faster input speed and less fatigue.

Text Input in Virtual Reality: A Preliminary Evaluation of the Drum-Like VR Keyboard

The drum-like virtual reality (VR) keyboard is a contemporary, controller-based interface for text input in VR that uses a drum set metaphor. The controllers are used as sticks which, through downward movements, “press” the keys of the virtual keyboard. In this work, a preliminary feasibility study of the drum-like VR keyboard is described, focusing on the text entry rate and accuracy as well as its usability and the user experience it offers. Seventeen participants evaluated the drum-like VR keyboard by having a typing session and completing a usability and a user experience questionnaire. The interface achieved a good usability score, positive experiential feedback around its entertaining and immersive qualities, a satisfying text entry rate (24.61 words-per-minute), as well as moderate-to-high total error rate (7.2%) that can probably be further improved in future studies. The work provides strong indications that the drum-like VR keyboard can be an effective and entertaining way to type in VR.

Design and evaluation of a time adaptive multimodal virtual keyboard

The usability of virtual keyboard based eye-typing systems is currently limited due to the lack of adaptive and user-centered approaches leading to low text entry rate and the need for frequent recalibration. In this work, we propose a set of methods for the dwell time adaptation in asynchronous mode and trial period in synchronous mode for gaze based virtual keyboards. The rules take into account commands that allow corrections in the application, and it has been tested on a newly developed virtual keyboard for a structurally complex language by using a two-stage tree-based character selection arrangement. We propose several dwell-based and dwell-free mechanisms with the multimodal access facility wherein the search of a target item is achieved through gaze detection and the selection can happen via the use of a dwell time, soft-switch, or gesture detection using surface electromyography in asynchronous mode; while in the synchronous mode, both the search and selection may be performed with just the eye-tracker. The system performance is evaluated in terms of text entry rate and information transfer rate with 20 different experimental conditions. The proposed strategy for adapting the parameters over time has shown a significant improvement (more than 40%) over non-adaptive approaches for new users. The multimodal dwell-free mechanism using a combination of eye-tracking and soft-switch provides better performance than adaptive methods with eye-tracking only. The overall system receives an excellent grade on adjective rating scale using the system usability scale and a low weighted rating on the NASA task load index, demonstrating the user-centered focus of the system.

Optimization of Split Keyboard Design for Touchscreen Devices

ABSTRACTWhile many large touchscreen devices have a split keyboard option to enable two-thumb typing, there are significant differences in the design which affect users’ text entry rate. A mixed-integer programming model is developed to optimize key-to-thumb assignment with the objective of minimizing the expected time to type a character. Computer simulations are conducted to determine the optimal key dimension under different values of the Fitts’ law’s slope coefficient, typing error rate, and alternate-hand advantage phenomenon. The results show that text entry rate and the optimal key-to-thumb assignment depend on key dimensions, user’s speed-accuracy profile, and the level of alternate-hand advantage. The optimal keyboard is proposed. To validate the analytical findings, an empirical study is conducted with eighteen users and six different keyboards in terms of key dimensions and typing zone. Empirical results report between 7% and 18% improvement in the text entry rate over the other split keyboards tested.

Effectiveness and usability of Scanning Wizard software: a tool for enhancing switch scanning

Scanning Wizard software helps scanning users improve the setup of their switch and scanning system. This study evaluated Scanning Wizard’s effectiveness and usability. Ten people who use switch scanning and ten practitioners used Scanning Wizard in the initial session. Usability was high, based on survey responses averaging over 4.5 out of 5, and qualitative feedback was very positive. Five switch users were able to complete the multi-week protocol, using settings on their own scanning system that were recommended from the Scanning Wizard session. Using these revised settings, text entry rates improved by an average of 71%, ranging from 29% to 172% improvement. Results suggest that Scanning Wizard is a useful tool for improving the configuration of scanning systems for people who use switch scanning to communicate.Implications for RehabilitationSome individuals with severe physical impairments use switch scanning for spoken and written communication.Scanning Wizard software helps scanning users improve the setup of their switch and scanning system.This study demonstrated high usability of Scanning Wizard (with 10 switch userpractitioner teams) and increased text entry rate by an average of 71% (for five switch users).Results suggest that Scanning Wizard is a useful tool for improving the configuration of scanning systems for people who use switch scanning to communicate.

BlindType

Eyes-free input is desirable for ubiquitous computing, since interacting with mobile and wearable devices often competes for visual attention with other devices and tasks. In this paper, we explore eyes-free typing on a touchpad using one thumb, wherein a user taps on an imaginary QWERTY keyboard while receiving text feedback on a separate screen. Our hypothesis is that users can transfer their typing ability obtained from visible keyboards to eyes-free use. We propose two statistical decoding algorithms to infer users’ eyes-free input: the absolute algorithm and the relative algorithm. The absolute algorithm infers user input based on the absolute position of touch endpoints, while the relative algorithm infers based on the vectors between successive touch endpoints. Evaluation results showed users could achieve satisfying performance with both algorithms. Text entry rate was 17-23 WPM (words per minute) depending on the algorithm used. In comparison, a baseline cursor-based text entry method yielded only 7.66 WPM. In conclusion, our research demonstrates for the first time the feasibility of thumb-based eyes-free typing, which provides a new possibility for text entry on ubiquitous computing platforms such as smart TVs and HMDs.

Text entry rate of access interfaces used by people with physical disabilities: A systematic review

ABSTRACTThis study systematically reviewed the research on assistive technology (AT) access interfaces used for text entry, and conducted a quantitative synthesis of text entry rates (TER) associated with common interfaces. We searched 10 databases and included studies in which: typing speed was reported in words per minute (WPM) or equivalent; the access interface was available for public use; and individuals with physical impairments were in the study population. For quantitative synthesis, we used only the TER reported for individuals with physical impairments. Studies also had to report the sample size, and the average and standard deviation for the TER data. Thirty-nine studies met the criteria for quantitative synthesis. Studies involved seven interface types: standard keyboard typing, on-screen keyboard (OSK) with cursor selection, OSK with scanning selection, automatic speech recognition (ASR), Morse code, brain–computer interface (BCI), and other. ASR, standard keyboard, cursor OSK, and scanning OSK had at least four studies and 30 subjects, with TER averaging 15.4, 12.5, 4.2, and 1.7 WPM, respectively. When combined with measurements of a particular client’s text entry performance, the TER from this review can be used within an evidence-based decision-making process for selecting control interfaces.

Novice and Expert Performance of KeyScretch: A Gesture-Based Text Entry Method for Touch-Screens

KeyScretch is a text entry method for devices equipped with touch-screens, based on a menu-augmented soft keyboard. In these keyboards, a menu containing a small number of frequent characters is shown, while a key is pressed, allowing further character entry by menu selection. KeyScretch improves the previously studied menu-based methods by enabling the interpretation of compound strokes, which allow the input of text chunks longer than two characters. The performance of the method is analyzed on different kinds of touch-screens: First, we present a 25-session user study on a stylus-based device, showing that an instance of the method optimized for Italian can be learned in a reasonable time by the users and significantly outperforms the traditional method based on the tapping interaction. Then, we define and validate a model for predicting expert text entry rates on finger-based devices. The predicted rates for instances of KeyScretch optimized for different Western languages vary from about 44-50 words/min on the Qwerty layout, enabling improvements in the range of 30-49% as compared with the traditional method.