MmKey: Universal Virtual Keyboard Using A Single Millimeter-Wave Radio

Abstract

Keyboard acts as one of the most commonly used mediums for human–computer interaction. Today, massive Internet-of-Things (IoT) devices are designed without a physical keyboard as they go tiny, but are almost all equipped with a wireless module for networks. In this work, we aim to enable a universal virtual keyboard using wireless signals, which would allow a typing interface for tiny IoT devices or serve as a portable alternative to the unwieldy physical keyboards. To this end, we present <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mmKey</i> , the first universal virtual keyboard system using a single millimeter-wave (mmWave) radio. By leveraging the unique advantages of mmWave signals, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mmKey</i> converts any flat surface, with a printed paper keyboard, into an effective typing medium. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mmKey</i> enables concurrent keystrokes and supports multiple keyboard layouts (e.g., computer keyboard, piano keyboard, or phone keypad). We design a novel signal processing pipeline to detect, segment and separate, and finally, recognize keystrokes. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mmKey</i> does not need any training except for a minimal one-time effort of only three key-presses for keyboard calibration upon the initial setup. We prototype <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">mmKey</i> using a commodity 802.11ad/ay chipset, customized to support radar-like operations, and evaluate it with different keyboard layouts under various settings. Experimental results with ten participants demonstrate a keystroke recognition accuracy of >95% for single-key case and >90% for multikey scenario, which leads to a word recognition accuracy of >97%.