On-Device User-Adaptive Next Word Prediction System

Abstract

Recent advancements in natural language processing have contributed to the success of language modeling in the task of next word prediction (NWP). The NWP systems are widely used on mobile devices, where deploying huge language models for offline prediction is a major bottleneck due to the limitation of storage and computing capacity. Moreover, due to the diversity of users, different users could have varying typing styles within the same language. Hence a system that does not adapt itself to the user’s style has an adverse effect on the user experience. This work proposes a low latency 3-gram NWP system using the Markov model with a ranking mechanism. The ranking mechanism ranks the next word prediction words in a way that the users preferred words gradually get a higher rank. With a 50K vocabulary size, our NWP system has an average latency of 6.41ms and a size of 3. 2MB.