Recognizing the Operating Hand and the Hand-Changing Process for User Interface Adjustment on Smartphones.

Hansong Guo,Liusheng Huang,He Huang,Yu-E Sun

doi:10.3390/s16081314

Abstract

As the size of smartphone touchscreens has become larger and larger in recent years, operability with a single hand is getting worse, especially for female users. We envision that user experience can be significantly improved if smartphones are able to recognize the current operating hand, detect the hand-changing process and then adjust the user interfaces subsequently. In this paper, we proposed, implemented and evaluated two novel systems. The first one leverages the user-generated touchscreen traces to recognize the current operating hand, and the second one utilizes the accelerometer and gyroscope data of all kinds of activities in the user’s daily life to detect the hand-changing process. These two systems are based on two supervised classifiers constructed from a series of refined touchscreen trace, accelerometer and gyroscope features. As opposed to existing solutions that all require users to select the current operating hand or confirm the hand-changing process manually, our systems follow much more convenient and practical methods and allow users to change the operating hand frequently without any harm to the user experience. We conduct extensive experiments on Samsung Galaxy S4 smartphones, and the evaluation results demonstrate that our proposed systems can recognize the current operating hand and detect the hand-changing process with 94.1% and 93.9% precision and 94.1% and 93.7% True Positive Rates (TPR) respectively, when deciding with a single touchscreen trace or accelerometer-gyroscope data segment, and the False Positive Rates (FPR) are as low as 2.6% and 0.7% accordingly. These two systems can either work completely independently and achieve pretty high accuracies or work jointly to further improve the recognition accuracy.

Highlights

As technology advances, smartphones with abundant built-in sensors are becoming more and more ubiquitous in our daily lives, which stimulates the blooming of smartphone sensing research, such as healthcare, localization and human computer interaction and makes our lives more efficient, more intelligent and more enjoyable
We study the performance for recognizing the hand-changing process of our second system, when utilizing pattern recognition algorithms and Dynamic Time Warping (DTW), respectively
We can observe that the combination of Random Forest (RF) and LDA achieves better recognition performance, namely 93.9% precision, 93.7% True Positive Rates (TPR) of recognizing the hand-changing process, 0.7% False Positive Rates (FPR) of recognizing the activities of daily life and 68 misrecognition segments in total, which reduces the features to two dimensions

Summary

Introduction

Smartphones with abundant built-in sensors are becoming more and more ubiquitous in our daily lives, which stimulates the blooming of smartphone sensing research, such as healthcare, localization and human computer interaction and makes our lives more efficient, more intelligent and more enjoyable. The result of our investigation about the dominant hand when operating smartphones by 500 randomly-selected students from the University of Science and Technology of China shows that 34% of them usually operate the smartphones with the left hand, 50% usually with the right hand and almost 16% operate the smartphones utilizing the right or left hand with the same frequency. This problem was not that severe previously since the sizes of smartphone screens were small. The screen size of iPhone 6 has already reached 4.7 inches [1], while the screen size of iPhone 4 is only 3.5 inches [1], and the screen sizes of the

Results

Discussion

Conclusion