Control Your Home With a Smartwatch

Yubing Li,Xinyi Cao,Meichen Duan,Kun Zhao,Yang Liu,Wei Shi,Jizhong Zhao,Liangliang Lin

doi:10.1109/access.2020.3007328

Abstract

With the wide applications of smart devices and mobile computing, smart home becomes a hot issue in the household appliance industry. The controlling and interaction approach plays a key role in users' experience and turns into one of the most important selling points for profit growth. Considering the robustness and privacy protection, wearable devices equipped with MEMS, e.g., smartphones, smartwatches, or smart wristbands, are thought of one of the most feasible commercial solutions for interaction. However, the low-cost built-in MEMS sensors do not perform well in capturing finely grained human activity directly. In this paper, we propose a method that leverages the arm constraint and historical information recorded by MEMS sensors to estimate the maximum likelihood action in a two-phases model. First, in the arm posture estimation phase, we leverage the kinematics model to analyze the maximum likelihood position of users' arms. Second, in the trajectory recognition phase, we leverage the gesture estimation model to identify the key actions and output the instructions to devices by SVM. Our substantial experiments show that the proposed solution can recognize eight kinds of postures defined for man-machine interaction in the smart home application scene, and the solution implements efficient and effective interaction using low-cost smartwatches, and the interaction accuracy is >87%. The experiments also show that the algorithm proposed in this paper can be well applied to the perceptual control of smart household appliances, and has high practical value for the application design of the perceptual interaction function of household appliances.

Highlights

With the rapid development of smart hardware and mobile computing, Internet of Things (IoT) is wildly applied in multiple domains, e.g., Vehicular ad hoc networks (VANETs) [1], edge computing [2]–[5], quality of service (QoS) prediction [6], intention detection, and smart home or intelligent appliances interactions and controlling [7]–[12]
Smart watch is a common wearable device, which is often equipped with Micro-electromechanical System (MEMS) sensors such as gyroscope, accelerometer and magnetometer
OVERALL DESIGN As shown in Figure 1, by mathematical modeling of the skeleton of human arm, the data collected by the built-in MEMS sensors of smart watch is converted into the data of degree of freedom (DOF) which is closely related to the movement of arm

Summary

INTRODUCTION

With the rapid development of smart hardware and mobile computing, Internet of Things (IoT) is wildly applied in multiple domains, e.g., Vehicular ad hoc networks (VANETs) [1], edge computing [2]–[5], quality of service (QoS) prediction [6], intention detection, and smart home or intelligent appliances interactions and controlling [7]–[12]. Smart watch is a common wearable device, which is often equipped with MEMS sensors such as gyroscope, accelerometer and magnetometer These sensors can collect a multitude of data related to the movement of wrist and arm. This kind of interaction has the advantages of low cost, convenient to use and wide application range. Low precision and high noise are the common characteristics of the built-in MEMS sensors of smart watch This is very disadvantageous to the accuracy of action recognition. The main idea of our solution is to record the latest interaction between users and device through smart watch, and guess the position and trajectory of the targets This proposed method mainly consists of two steps: the arm posture estimation and trajectory recognition. Our extreme experiments show that the proposed solution can be successfully applied to the interactive control of smart household appliances, e.g., smart air conditioners, smart TV, smart curtain and other smart household goods

RELATED WORKS

9: Add new values to

EXPERIMENTS SETUP

Findings

CONCLUSION