Virtual Keyboard Research Articles

Designing Hand Pose Aware Virtual Keyboard With Hand Drift Tolerance

An unintentional hand drift adversely affects the typing performance of conventional virtual keyboards. To overcome this, we proposed to utilize the typing patterns of skilled typists. First, as most typists enter the keys in the same column with a predetermined finger only, we restricted these keys to be typed by their corresponding fingers. Second, our investigation of skilled typists demonstrated that hand poses vary when the typists touch different keys. Thus, rather than locating the touch point as in the case of existing virtual keyboards, we attempted to use unique hand poses to infer the target key. Based on these two techniques, we implemented a novel hand poses aware virtual keyboard that is tolerant of hand drift. Our experimental studies yielded the following results: 1) most of the QWERTY-familiar typists who have varying typing habits were easily adaptable to the proposed keyboard design and 2) the proposed keyboard outperformed existing virtual keyboards in terms of typing speed and several error rates, and eventually achieved a typing speed of approximately 56 WPM.

A Single-Channel SSVEP-Based BCI Speller Using Deep Learning

This paper aims to develop a speller system based on a bipolar single-channel electroencephalogram with sufficient accuracy. The proposed system consists of a custom-designed headset, a new virtual keyboard with 58 characters, special symbols, and digits, and a five-target steady-state visual-evoked potential (SSVEP)-based brain–computer interface (BCI) utilizing one-dimensional convolutional neural network (1-D CNN) for SSVEP frequency detection. The deep learning model is implemented and trained under the training mode before being applied in the operation mode of the system. To validate the proposed model, we acquire the training dataset with numerous testing conditions, including different frequency resolutions of the feature and different time-window lengths of analysis. Two types of features based on the frequency domain are investigated to compare their performances in terms of classification accuracy of the model. The experimental results from eight subjects shows that on average, the proposed model can classify five-class SSVEP data with a high accuracy of 99.2%. The proposed BCI is then employed in an online experiment of spelling the word “SPELLER” using 2-s time window. Consequently, the system achieves an average accuracy of 97.4% and an information transfer rate of 49 ± 7.7 bpm, showing the practicality and feasibility of implementing a reliable single-channel SSVEP-based speller utilizing 1-D CNN.

VIKAS: A new virtual keyboard based simple and efficient text CAPTCHA verification scheme

Nowadays online transactions are becoming ubiquitous that must be protected from bots using different techniques, and CAPTCHA is one of them. Text-CAPTCHA preferred due to its simplicity amongst different types of CAPTCHAs. Text-CAPTCHA can be strengthened by adding some distortion to prevent bot-attacks but cause usability issues for humans. This results in multiple attempts by a user to gain access to the required service and may give frustration to the user. Hence, there is a need to design CAPTCHA which is easy for humans to recognise but difficult for bots. This paper proposes virtual keyboard-based simple and efficient text-CAPTCHA verification scheme (VIKAS) that makes CAPTCHA verification easy for humans but difficult for bots. VIKAS uses simple text-CAPTCHA and verifies the same using positions of the keys pressed by the user using an image-based virtual keyboard. VIKAS is sustainable against segmentation scheme, replay attacks and possible attacks with keyloggers.

Wireless IoT Motion-Recognition Rings and a Paper Keyboard

In this paper, we present a new scheme for implementing virtual keyboards, which uses only two to four motion-recognition rings per hand and a two-dimensional keyboard template (e.g., an A4 size paper with printed key positions). It has the benefit of portability, customizability, and low-cost when compared with existing approaches. Essentially, we have shown that wearing two wireless IoT rings on the middle phalanges of two fingers of each hand, users can input the alphabetic characters into a computing device by typing on a flat paper on a desk, and potentially in mid-air. We have demonstrated that two rings are sufficient in capturing the gestures and motions of all fingers in a typing hand for keystrokes recognition. A single wireless IoT ring, which weighs 7.8 grams, consists of a Bluetooth low energy (BLE) unit, a micro inertial measurement unit (mIMU), and a cell battery. The 3-axes attitude angles and the Z-axis acceleration of each ring are adopted as the features for keystroke recognition. The overall keystroke recognition accuracy rate can reach as high as 94.8% when two IoT rings are worn by a user on each hand; this accuracy rate increases to 98.6%, when four rings are worn on each typing hand.

Information Technology for Entering Text Based on Tools of the Special Virtual Keyboard Mobile and Auxiliary Devices

Information Technology for Entering Text Based on Tools of the Special Virtual Keyboard Mobile and Auxiliary Devices

Electronic Diachronic Corpus and Dictionaries of Old Bulgarian

The electronic system histdict is designed as a tool for research, adequate presentation and popularization of a part of Bulgaria’s cultural and historical heritage: the Bulgarian language and its medieval literature. The article describes the various steps in the development of histdict. Attention is paid to each component of the resource: specialized Unicode fonts, electronic diachronic corpus, dictionary of Old Bulgarian, historical dictionary equipped with tools for writing and editing dictionary entries, grammatical dictionary, prototypical search engine, and virtual keyboard. The article also lays out the principles followed in the development of the diachronic grammatical dictionary of the Bulgarian language.

Fast and Precise Touch-Based Text Entry for Head-Mounted Augmented Reality with Variable Occlusion

We present the VISAR keyboard: An augmented reality (AR) head-mounted display (HMD) system that supports text entry via a virtualised input surface. Users select keys on the virtual keyboard by imitating the process of single-hand typing on a physical touchscreen display. Our system uses a statistical decoder to infer users’ intended text and to provide error-tolerant predictions. There is also a high-precision fall-back mechanism to support users in indicating which keys should be unmodified by the auto-correction process. A unique advantage of leveraging the well-established touch input paradigm is that our system enables text entry with minimal visual clutter on the see-through display, thus preserving the user’s field-of-view. We iteratively designed and evaluated our system and show that the final iteration of the system supports a mean entry rate of 17.75wpm with a mean character error rate less than 1%. This performance represents a 19.6% improvement relative to the state-of-the-art baseline investigated: A gaze-then-gesture text entry technique derived from the system keyboard on the Microsoft HoloLens. Finally, we validate that the system is effective in supporting text entry in a fully mobile usage scenario likely to be encountered in industrial applications of AR HMDs.

가상키보드의 명암대비 디자인이 문자입력 시의 터치행태 및 사용성에 미치는 효과

가상키보드의 명암대비 디자인이 문자입력 시의 터치행태 및 사용성에 미치는 효과

Exploring touch feedback display of virtual keyboards for reduced eye movements

When typing on smartphones or palm tablets, users generally make an effort to type correctly while simultaneously checking the small keyboard and the text display. Unlike physical keyboards that allow users to perform typing based on long-term muscle memory, virtual keyboards typically require more frequent eye movements between the keyboard and the text display areas.This study proposes a new way of designing a virtual keyboard display to reduce the effort associated with frequent eye-movements. For this study, we developed virtual keyboard display systems featuring both static and dynamic word-by-word (WBW) feedback displays. The two display systems were examined in comparison with a more conventional method known as character-by-character (CBC) feedback display. We investigated user satisfaction, typing performance and the user’s eye gaze shifts. Eye gaze shifts were measured between the keyboard and the text display areas across the three conditions using self-report, log, and eye-tracking measures. In the static WBW condition, the words being typed displayed in a fixed area at the top of the virtual keyboard; in the dynamic WBW display, the words displayed in a small popup window at the tip of the selected key.Using a repeated measure experiment for the three display conditions, participants were asked to type fifteen phrases using a palm tablet while wearing eye-tracking glasses for each condition. We conducted a mixed-model design ANOVA with group (SLOW vs. FAST typing; men vs. women) as between-subject factors and display condition (CBC vs. WBW). We found a significant (11%) improvement in typing speed with the dynamic WBW over the CBC display for less experienced keyboard users. In addition, participants reported higher satisfaction with the two WBW conditions than the CBC condition. Eye fixations, dwell times, and heat map data also supported that WBW displays are advantageous for less experienced, slower typists by helping them stay focused more on the keyboard, thus reducing eye transitions to the text display. Our study systematically demonstrates how and to what extent the virtual keyboard display strategy influences typing performance and subjective experience based on self-reports and eye-tracking measures. The approach and findings of this study should provide useful information and practical guidance to mobile application developers and designers who are interested in improving virtual keyboard functionalities and user satisfaction.

The human-computer connection: An overview of brain-computer interfaces

This article introduces the field of brain-computer interfaces (BCI), which allows the control of devices without the generation of any active motor output but directly from the decoding of the user’s brain signals. Here we review the current state of the art in the BCI field, discussing the main components of such an interface and illustrating ongoing research questions and prototypes for controlling a large variety of devices, from virtual keyboards for communication to robotics systems to replace lost motor functions and even clinical interventions for motor rehabilitation after a stroke. The article concludes with some insights into the future of BCI.

ВИКОРИСТАННЯ ІНФОРМАЦІЙНО-КОМУНІКАЦІЙНИХ ТЕХНОЛОГІЙ ДЛЯ ЗАБЕЗПЕЧЕННЯ ДОСТУПНОСТІ І РОЗВИТКУ ІНКЛЮЗИВНОЇ ОСВІТИ

Information and communication technologies as support for the inclusive education are one of the effective tools for the development, education and training of students with special educational needs. The article deals with assistive hardware and software tools that compensate some of the natural functions of the child’s body. The possibilities of modern Microsoft, Google, Facebook and Youtube platforms have been described that make it possible to improve the fulfilment of educational tasks via the settings of keyboard shortcuts, virtual keyboard, word prediction functions, sound signals or screen indicators. The article also highlights the potential of messengers, social networks and e-mails as the most modern and adapted communicators for addressing teaching and educational needs of schoolchildren. The Viber and Facebook instant messaging programs have also been described and were found to be the most widespread means of communication in Ukraine having functions of voice and video calls with the interface adapted to maximize screen readability, allowing to create constructive dialogue between teachers, parents or schoolchildren with special educational needs. The use of ICT in inclusive education for didactic purposes have been presented practically by the LearningApps.org service. Examples of setting objectives for children with mild and moderate mental retardation have been presented, in particular: for pupils who suffer from autistic disorders, for schoolchildren who have problems with auditory or visual perception of information. Providing education through the suggested resource allows to carry out remedial work with children with special educational needs that intensifies their perception and attention, improves their memory and stimulates their motivation for educational activities. Introduction of information and communication technologies encompassing their three aspects - compensatory, communication and didactic purposes - makes it possible to adjust educational programmes and contribute effectively to the development of pupils according to their individuality.

EEG-EOG based Virtual Keyboard: Toward Hybrid Brain Computer Interface.

The past twenty years have ignited a new spark in the research of Electroencephalogram (EEG), which was pursued to develop innovative Brain Computer Interfaces (BCIs) in order to help severely disabled people live a better life with a high degree of independence. Current BCIs are more theoretical than practical and are suffering from numerous challenges. New trends of research propose combining EEG to other simple and efficient bioelectric inputs such as Electro-oculography (EOG) resulting from eye movements, to produce more practical and robust Hybrid Brain Computer Interface systems (hBCI) or Brain/Neuronal Computer Interface (BNCI). Working towards this purpose, existing research in EOG based Human Computer Interaction (HCI) applications, must be organized and surveyed in order to develop a vision on the potential benefits of combining both input modalities and give rise to new designs that maximize these benefits. Our aim is to support and inspire the design of new hBCI systems based on both EEG and EOG signals, in doing so; first the current EOG based HCI systems were surveyed with a particular focus on EOG based systems for communication using virtual keyboard. Then, a survey of the current EEG-EOG virtual keyboard was performed highlighting the design protocols employed. We concluded with a discussion of the potential advantages of combining both systems with recommendations to give deep insight for future design issues for all EEG-EOG hBCI systems. Finally, a general architecture was proposed for a new EEG-EOG hBCI system. The proposed hybrid system completely alters the traditional view of the eye movement features present in EEG signal as artifacts that should be removed; instead EOG traces are extracted from EEG in our proposed hybrid architecture and are considered as an additional input modality sharing control according to the chosen design protocol.

User-Adaptive Key Click Vibration on Virtual Keyboard

This study focuses on design of user-adaptive tactile keyboard on mobile device. We are particularly interested in its feasibility of user-adaptive keyboard in mobile environment. Study 1 investigates how tactile feedback intensity of the virtual keyboard in mobile devices affects typing speed and user preference. We report how different levels of feedback intensity affect user preferences in terms of typing speed and accuracy in different user groups with different typing performance. Study 2 investigates different tactile feedback modes (i.e., whether feedback intensity is linearly increased, linearly decreased, or constant from the centroid of the key, and whether tactile feedback is delivered when a key is pressed, released, or both pressed and released). We finally design and implement user-adaptive tactile keyboards on mobile device to explore the design space of our keyboards. We close by discussing the benefits of our design along with its future work.

EOG-based eye movement detection and gaze estimation for an asynchronous virtual keyboard

Abstract This work aims to develop a novel electrooculography (EOG)-based virtual keyboard with a standard QWERTY layout which, unlike similar state-of-the-art systems, allows users to reach any icon from any location directly and asynchronously. The saccadic EOG potential displacement is mapped to angular gaze displacement using a novel two-channel input linear regression model, which considers features extracted from both the horizontal and vertical EOG signal components jointly. Using this technique, a gaze displacement estimation error of 1.32 ± 0.26° and 1.67 ± 0.26° in the horizontal and vertical directions respectively was achieved, a performance which was also found to be generally statistically significantly better than the performance obtained using one model for each EOG component to model the relationship in the horizontal and vertical directions separately, as typically used in the literature. Furthermore, this work also proposes a threshold-based method to detect eye movements from EOG signals in real-time, which are then classified as saccades or blinks using a novel cascade of a parametric and a signal-morphological classifier based on the EOG peak and gradient features. This resulted in an average saccade and blink labelling accuracy of 99.92% and 100.00% respectively, demonstrating that these two eye movements could be reliably detected and discriminated in real-time using the proposed algorithms. When these techniques were used to interface with the proposed asynchronous EOG-based virtual keyboard, an average writing speed across subjects of 11.89 ± 4.42 characters per minute was achieved, a performance which has been shown to improve substantially with user experience.

A camera based human computer interaction through virtual keyboard assistant

Human Computer Interaction has evolved with time and exploited almost all the possible relationships that human can have with computers. The present scenario of using the computer is often inaccessible to people with severe motor impairment, which cannot inhabit the traditional keyboard and mouse, and require another specialized input device. A Camera mouse Human-Computer Interface (HCI) based virtual assistant is designed and planned to be evaluated with 15 subjects and hoped to work for individuals with advanced Amyotrophic Lateral Sclerosis (ALS), all of whom were given tasks to execute through the virtual assistant. In the proposed system, this includes primarily, a virtual keyboard with an embedded Internet browser. The virtual keyboard allows users with movement impairments, basically hand movement issues, to directly interact with the computer. The keyboard has varied functions, apart from just typing; namely, searching anything on google, login to social media and checking emails, text to speech facilities and has a database that holds all the basic helper commands for the person. The camera mouse is software that allows people to interact with the computer by registering any part of their body and then using that part as the pointing device to work on the computer system. The participants of the evaluation will be both, subjects that have knowledge about the HCI interface and those completely new to it. We define additional criteria for unrestricted internet access for evaluation of the presented and future internet browsers, Camera based browser provides unrestricted access and enables free web surfing for individuals with motor impairments. This system improves the subject’s quality of life and provides uninterrupted access for the subject to use the computer.

A Gaze-Based Virtual Keyboard Using a Mouth Switch for Command Selection.

Portable eye-trackers provide an efficient way to access the point of gaze from a user on a computer screen. Thanks to eyetracking, gaze-based virtual keyboard can be developed by taking into account constraints related to the gaze detection accuracy. In this paper, we propose a new gaze-based virtual keyboard where all the letters can be accessed directly through a single command. In addition, we propose a USB mouth switch that is directly connected through a computer mouse, with the mouse switch replacing the left click button. This approach is considered to tackle the Midas touch problem with eye-tracking for people who are severely disabled. The performance is evaluated on 10 participants by comparing the following three conditions: gaze detection with mouth switch, gaze detection with dwell time by considering the distance to the closest command, and the gaze detection within the surface of the command box. Finally, a workload using NASA-TLX test was conducted on the different conditions. The results revealed that the proposed approach with the mouth switch provides a better performance in terms of typing speed (36.6 ± 8.4 letters/minute) compared to the other conditions, and a high acceptance as an input device.

A multimodal virtual keyboard using eye-tracking and hand gesture detection.

A large number of people with disabilities rely on assistive technologies to communicate with their families, to use social media, and have a social life. Despite a significant increase of novel assitive technologies, robust, non-invasive, and inexpensive solutions should be proposed and optimized in relation to the physical abilities of the users. A reliable and robust identification of intentional visual commands is an important issue in the development of eye-movements based user interfaces. The detection of a command with an eyetracking system can be achieved with a dwell time. Yet, a large number of people can use simple hand gestures as a switch to select a command. We propose a new virtual keyboard based on the detection of ten commands. The keyboard includes all the letters of the Latin script (upper and lower case), punctuation marks, digits, and a delete button. To select a command in the keyboard, the user points the desired item with the gaze, and select it with hand gesture. The system has been evaluated across eight healthy subjects with five predefined hand gestures, and a button for the selection. The results support the conclusion that the performance of a subject, in terms of speed and information transfer rate (ITR), depends on the choice of the hand gesture. The best gesture for each subject provides a mean performance of $8 . 77 \pm 2 .90$ letters per minute, which corresponds to an ITR of $57 . 04 \pm 14 .55$ bits per minute. The results highlight that the hand gesture assigned for the selection of an item is inter-subject dependent.

A virtual keyboard security system for automated teller machine

The Automated Teller Machines (ATM) are used for cash withdrawals. Meanwhile as ATM lacks security, thefts occurring in the ATMs are also high. Our proposed system minimizes those break-ins occurring in the ATM’s by instigating a low cost standalone embedded sys-tem using Arduino Microcontroller. This system is proposed for ATM security, comprising of the components namely RGB tag, Global System for Mobile communication (GSM), Global Positioning System (GPS), Virtual Keyboard and camera. Authentication by virtual keyboard thus helps the public to protect their credentials from being captured by malicious bots, key loggers and spyware. Whenever theft occurs, Buzzer makes an alarm, DC Motor turns ON and the door of ATM is closed. The nearby police station and corresponding bank authorities is alerted by the GSM Modem that sends messages along with the location. This will prevent the robberies and the person in-volving in the robbery can be easily caught. In short, it discusses the methods that are used to detect the venture, initiate preemptive measures and forewarn the officials through GSM network. This system thus heightens the safety of ATM users as well as the ATM’s against forthcoming attacks effectively.

Internal scrambled virtual keyboard for password protection

Due to the Wide Growth of Internet the Electronic Commerce and Transactions is also Increasing. Numerous Companies have built up their web based exchanging stages Such as Banking Portals, Online Shopping, Ticket Booking and so forth. Many of these platforms are Providing Virtual Keyboard for making the Authentication of user more Secure. But the Virtual Keyboard is also having some draw-backs which the attackers make take as an advantage. A few of these consolidate Click based Screenshot Capturing and shoulder Surfing. To Maintain a strategic distance from these disadvantages, we require to outline Virtual Console which will change the Position of Keys on console for each on click activity and each key will have a fallacy value through a Rotation Pattern i.e the key which is pressed is not the actual key value, the user only knows the Actual Position of the key. So, the Screen Captures or Screen Recordings will not have the capacity to discover the Actual key. Our Proposed Framework makes the utilization of virtual console more Secured for clients through Key Rotation Technique.

Toward Optimization of Gaze-Controlled Human-Computer Interaction: Application to Hindi Virtual Keyboard for Stroke Patients.

Virtual keyboard applications and alternative communication devices provide new means of communication to assist disabled people. To date, virtual keyboard optimization schemes based on script-specific information, along with multimodal input access facility, are limited. In this paper, we propose a novel method for optimizing the position of the displayed items for gaze-controlled tree-based menu selection systems by considering a combination of letter frequency and command selection time. The optimized graphical user interface layout has been designed for a Hindi language virtual keyboard based on a menu wherein 10 commands provide access to type 88 different characters, along with additional text editing commands. The system can be controlled in two different modes: eye-tracking alone and eye-tracking with an access soft-switch. Five different keyboard layouts have been presented and evaluated with ten healthy participants. Furthermore, the two best performing keyboard layouts have been evaluated with eye-tracking alone on ten stroke patients. The overall performance analysis demonstrated significantly superior typing performance, high usability (87% SUS score), and low workload (NASA TLX with 17 scores) for the letter frequency and time-based organization with script specific arrangement design. This paper represents the first optimized gaze-controlled Hindi virtual keyboard, which can be extended to other languages.