Built-in Motion Sensors Research Articles

MotionID: Towards practical behavioral biometrics-based implicit user authentication on smartphones

Traditional one-time authentication mechanisms cannot authenticate smartphone users’ identities throughout the session — the concept of using behavioral-based biometrics captured by the built-in motion sensors and touch data is a candidate to solve this issue. Many studies proposed solutions for behavioral-based continuous authentication; however, they are still far from practicality and generality for real-world usage. To date, no commercially deployed implicit user authentication scheme exists because most of those solutions were designed to improve detection accuracy without addressing real-world deployment requirements. To bridge this gap, we tackle the limitations of existing schemes and reach towards developing a more practical implicit authentication scheme, dubbed MotionID, based on a one-class detector using behavioral data from motion sensors when users touch their smartphones. Compared with previous studies, our work addresses the following challenges: ① Global mobile average to dynamically adjust the sampling rate for sensors on any device and mitigate the impact of using sensors’ fixed sampling rate; ② Over-all-apps to authenticate a user across all the mobile applications, not only on-specific application; ③ Single-device-evaluation to measure the performance with multiple users’ (i.e., genuine users and imposters) data collected from the same device; ④ Rapid authentication to quickly identify users’ identities using a few samples collected within short durations of touching (1–5 s) the device; ⑤ Unconditional settings to collect sensor data from real-world smartphone usage rather than a laboratory study. To show the feasibility of MotionID for those challenges, we evaluated the performance of MotionID with ten users’ motion sensor data on five different smartphones under various settings. Our results show the impracticality of using a fixed sampling rate across devices that most previous studies have adopted. MotionID is able to authenticate users with an F1-score up to 98.5% for some devices under practical requirements and an F1-score up to roughly 90% when considering the drift concept and rapid authentication settings. Finally, we investigate time efficiency, power consumption, and memory usage considerations to examine the practicality of MotionID.

Multi-motion sensor behavior based continuous authentication on smartphones using gated two-tower transformer fusion networks

With continuous authentication, smartphones can constantly calculate authentication scores to verify a user's identity while interacting with the smartphones after logging in. Among various behavioral biometrics based continuous authentication methods, motion dynamics biometrics based methods draw a lot of attention, which relies on potential differences in signals from smartphone built-in motion sensors to provide an implicit and secure authentication. However, existing motion dynamics based methods not only fail to model correlations between motion sensors with multiple channels but also ignore temporal patterns contained in each channel. In this paper, we develop a two-tower neural networks based continuous authentication system TNNAuth based on the built-in multi-motion sensors on smartphones. Specifically, TNNAuth divides the raw multi-motion sensor data into channel and temporal stream data, and proposes a gated two-tower transformer fusion network to capture both time-over-channel and channel-over-time motion patterns. To evaluate TNNAuth, we collect a large-scale multi-motion sensor dataset in the unconstrained real life. Extensive experiments demonstrate that TNNAuth can provide the lowest authentication error rate, with a false-acceptance rate of 8.3% and a false-rejection rate of 9.4%.

MAUTH: Continuous User Authentication Based on Subtle Intrinsic Muscular Tremors

Continuous authentication is viewed to be increasingly important for mobile devices, which store a wide range of private data and sensitive information of users. Traditional continuous authentication methods need user inputs (e.g. typing, sliding). In this work, we present MAUTH, a zero-effect continuous authentication scheme for mobile devices. With the built-in motion sensors on commercial off-the-shelf (COTS) devices, MAUTH can continuously extract, classify and verify the unique tremor features of users on how their body intrinsically shakes during the normal use of such devices. As a result, it is extremely difficult if not impossible to reproduce the same set of tremors as individuals differ in their muscle development. We implement MAUTH as a software on Android-based smartphones, which demonstrates that MAUTH is light-weight and unobtrusive to its users. We conduct extensive real-world experiments and trace-driven simulations in controlled and uncontrolled environments on 21 volunteers. The results show that MAUTH is difficult to counterfeit and achieves a low average false positive rate (FPR) of 6.73% under real-world spoofing attacks. Moreover, MAUTH is comfortable to use and can achieve a low average false negative rate (FNR) of 2.2% during uncontrolled and continuous usage of devices, leveraging isolation-forest-based classifiers trained with only 40 training samples.

Human Lower Limb Motion Capture and Recognition Based on Smartphones.

Human motion recognition based on wearable devices plays a vital role in pervasive computing. Smartphones have built-in motion sensors that measure the motion of the device with high precision. In this paper, we propose a human lower limb motion capture and recognition approach based on a Smartphone. We design a motion logger to record five categories of limb activities (standing up, sitting down, walking, going upstairs, and going downstairs) using two motion sensors (tri-axial accelerometer, tri-axial gyroscope). We extract the motion features and select a subset of features as a feature vector from the frequency domain of the sensing data using Fast Fourier Transform (FFT). We classify and predict human lower limb motion using three supervised learning algorithms: Naïve Bayes (NB), K-Nearest Neighbor (KNN), and Artificial Neural Networks (ANNs). We use 670 lower limb motion samples to train and verify these classifiers using the 10-folder cross-validation technique. Finally, we design and implement a live detection system to validate our motion detection approach. The experimental results show that our low-cost approach can recognize human lower limb activities with acceptable accuracy. On average, the recognition rate of NB, KNN, and ANNs are 97.01%, 96.12%, and 98.21%, respectively.

Augmented PIN Authentication through Behavioral Biometrics.

Personal Identification Numbers (PINs) are widely used today for user authentication on mobile devices. However, this authentication method can be subject to several attacks such as phishing, smudge, and side-channel. In this paper, we increase the security of PIN-based authentication by considering behavioral biometrics, specifically the smartphone movements typical of each user. To this end, we propose a method based on anomaly detection that is capable of recognizing whether the PIN is inserted by the smartphone owner or by an attacker. This decision is taken according to the smartphone movements, which are recorded during the PIN insertion through the built-in motion sensors. For each digit in the PIN, an anomaly score is computed using Machine Learning (ML) techniques. Subsequently, these scores are combined to obtain the final decision metric. Numerical results show that our authentication method can achieve an Equal Error Rate (EER) as low as 5% in the case of 4-digit PINs, and 4% in the case of 6-digit PINs. Considering a reduced training set, composed of solely 50 samples, the EER only slightly worsens, reaching 6%. The practicality of our approach is further confirmed by the low processing time required, on the order of fractions of milliseconds.

Improving the Authentication with Built-In Camera Protocol Using Built-In Motion Sensors: A Deep Learning Solution

In this paper, we propose an enhanced version of the Authentication with Built-in Camera (ABC) protocol by employing a deep learning solution based on built-in motion sensors. The standard ABC protocol identifies mobile devices based on the photo-response non-uniformity (PRNU) of the camera sensor, while also considering QR-code-based meta-information. During registration, users are required to capture photos using their smartphone camera. The photos are sent to a server that computes the camera fingerprint, storing it as an authentication trait. During authentication, the user is required to take two photos that contain two QR codes presented on a screen. The presented QR code images also contain a unique probe signal, similar to a camera fingerprint, generated by the protocol. During verification, the server computes the fingerprint of the received photos and authenticates the user if (i) the probe signal is present, (ii) the metadata embedded in the QR codes is correct and (iii) the camera fingerprint is identified correctly. However, the protocol is vulnerable to forgery attacks when the attacker can compute the camera fingerprint from external photos, as shown in our preliminary work. Hence, attackers can easily remove their PRNU from the authentication photos without completely altering the probe signal, resulting in attacks that bypass the defense systems of the ABC protocol. In this context, we propose an enhancement to the ABC protocol, using motion sensor data as an additional and passive authentication layer. Smartphones can be identified through their motion sensor data, which, unlike photos, is never posted by users on social media platforms, thus being more secure than using photographs alone. To this end, we transform motion signals into embedding vectors produced by deep neural networks, applying Support Vector Machines for the smartphone identification task. Our change to the ABC protocol results in a multi-modal protocol that lowers the false acceptance rate for the attack proposed in our previous work to a percentage as low as 0.07%. In this paper, we present the attack that makes ABC vulnerable, as well as our multi-modal ABC protocol along with relevant experiments and results.

A Robust Heading Estimation Solution for Smartphone Multisensor-Integrated Indoor Positioning

As a part of Internet-of-Things applications, various smartphone-based indoor location services have considerable commercial value. It is largely agreed that the integration of multiple sensors is the preferable solution for improving the performance of smartphone indoor positioning, thanks to the diversity of built-in sensors. However, heading error remains a challenge for smartphone indoor positioning, especially in complex indoor scenes. This article, therefore, proposes a heading estimation solution to enhance the accuracy and reliability of smartphone indoor positioning. The extended Kalman filter (EKF)-based solution fuses smartphone built-in motion sensors, magnetometers, building map knowledge, and fingerprinting coarse positions from Wi-Fi or Bluetooth. First, the context of pedestrian mobility and scene knowledge is inferred by combining these data. Then, a scene augmentation strategy and magnetic interference online detection method are applied to calibrate the gyro accumulation error and improve the heading estimation accuracy. Additionally, quasistatic and low-dynamic judgments and online calibration are used to mitigate gyro drift. The proposed solution is implemented on a smartphone device and validated in several experiments under natural pedestrian mobility and complex indoor scenarios. Experiments show that the accuracy of heading estimation is improved from 13.1° to 2.0°. The improved heading estimation enhances the accuracy of smartphone indoor positioning from 3.57 to 0.90 m. The proposed solution is applicable to real location-based service scenarios.

Validation of a smartphone application and wearable sensor for measurements of wrist motions.

We developed a smartphone application to measure wrist motion using the mobile device’s built-in motion sensors or connecting it via Bluetooth to a wearable sensor. Measurement of wrist motion with this method was assessed in 33 participants on two occasions and compared with those obtained with a standard goniometer. The test–retest reproducibility in healthy individuals ranged from good to excellent (intraclass correlation (ICC) 0.76–0.95) for all motions, both with and without the wearable sensor. These results improved to excellent (ICC 0.90–0.96) on the second test day, suggesting a learning effect. The day-to-day reproducibility was overall better with the wearable sensor (mean ICC 0.87) compared with the application without using sensor or goniometer (mean ICC 0.82 and 0.60, respectively). This study suggests that smartphone-based measurements of wrist range of motion are feasible and highly accurate, making it a powerful tool for outcome studies after wrist surgery.

Motion-to-BMI: Using Motion Sensors to Predict the Body Mass Index of Smartphone Users.

Obesity has become a widespread health problem worldwide. The body mass index (BMI) is a simple and reliable index based on weight and height that is commonly used to identify and classify adults as underweight, normal, overweight (pre-obesity), or obese. In this paper, we propose a hybrid deep neural network for predicting the BMI of smartphone users, based only on the characteristics of body movement captured by the smartphone’s built-in motion sensors without any other sensitive data. The proposed deep learning model consists of four major modules: a transformation module for data preprocessing, a convolution module for extracting spatial features, a long short-term memory (LSTM) module for exploring temporal dependency, and a fully connected module for regression. We define motion entropy (MEn), which is a measure of the regularity and complexity of the motion sensor, and propose a novel MEn-based filtering strategy to select parts of sensor data that met certain thresholds for training the model. We evaluate this model using two public datasets in comparison with baseline conventional feature-based methods using leave-one-subject-out (LOSO) cross-validation. Experimental results show that the proposed model with the MEn-based filtering strategy outperforms the baseline approaches significantly. The results also show that jogging may be a more suitable activity of daily living (ADL) for BMI prediction than walking and walking upstairs. We believe that the conclusions of this study will help to develop a long-term remote health monitoring system.

One-Handed Input Through Rotational Motion for Smartwatches

ABSTRACTOne-handed input for smartwatches is crucial when users’ hands are occupied. A rotational motion is leveraged, which can be detected by built-in motion sensors in most smartwatches, as input to provide item selection. Users control a cursor by rotating the smartwatch into different orientations using the proposed item selection gestures. To understand human wrist dexterity and the ability to perform rotational motion input along each degree of freedom on smartwatches, a human-factor study was performed. Except dexterity, users’ consensus and agreement of user-defined rotational motion input gestures for selection and commitment on smartwatches in an exploratory user study were also observed. Based on both the studies, gestures roll and tilt are proposed as selection gestures, and shake and nod are used as commitment gestures. The item selection performance using the proposed gestures in 2D layouts in a user study was further evaluated. The study results showed that tilt-nod and tilt-shake are suitable for item numbers (4 × 3) and (5 × 5), respectively. Furthermore, due to high error rate for tilt-shake in (2 × 2) caused by false positive in shake detection, roll-shake with better social acceptance can be used as an alternative for smartwatches without a built-in front camera in (2 × 2).

Sensor fusion‐based precise obstacle localisation for automatic parking systems

A precise obstacle localisation method for automatic parking systems is proposed. The proposed method improves the obstacle localisation accuracy by fusing only the sensors already installed on mass-produced vehicles. This method finds the initial obstacle location using ultrasonic sensors. On the basis of the initial obstacle location, it detects the obstacle's vertical outlines using images provided by an around view monitor system while the ego-vehicle is moving. The sequentially detected vertical outlines are combined based on the ego-vehicle positions estimated by built-in motion sensors. Finally, the obstacle location is refined by calculating the intersection point of the combined vertical outlines. In experiments, the proposed method was quantitatively evaluated and outperformed the previous method.

Understanding Keystroke Dynamics for Smartphone Users Authentication and Keystroke Dynamics on Smartphones Built-In Motion Sensors

Personal Identification Numbers (PINs) and pattern drawing have been used as common authentication methods especially on smartphones. Such methods, however, are very vulnerable to the shoulder surfing attack. Thus, keystroke dynamics that authenticate legitimate users based on their typing manner have been studied for years. However, many of the studies have focused on PC keyboard keystrokes. More studies on mobile and smartphones keystroke dynamics are warranted; as smartphones make progress in both hardware and software, features from smartphones have been diversified. In this paper, using various features including keystroke data such as time interval and motion data such as accelerometers and rotation values, we evaluate features with motion data and without motion data. We also compare 5 formulas for motion data, respectively. We also demonstrate that opposite gender match between a legitimate user and impostors has influence on authenticating by our experiment results.

Standalone Wearable Driver Drowsiness Detection System in a Smartwatch

Drowsiness while driving is one of the main causes of fatal accidents, especially on monotonous routes such as highways. The goal of this paper is to design a completely standalone, distraction-free, and wearable system for driver drowsiness detection by incorporating the system in a smartwatch. The main objective is to detect the driver’s drowsiness level based on the driver behavior derived from the motion data collected from the built-in motion sensors in the smartwatch, such as the accelerometer and the gyroscope. For this purpose, the magnitudes of hand movements are extracted from the motion data and are used to calculate the time, spectral, and phase domain features. The features are selected based on the feature correlation method. Eight features serve as an input to a support vector machine (SVM) classifier. After the SVM training and testing, the highest obtained accuracy was 98.15% (Karolinska sleepiness scale). This user-predefined system can be used by both left-handed and right-handed users, because different SVM models are used for different hands. This is an effective, safe, and distraction-free system for the detection of driver drowsiness.

The Quick Radio Fingerprint Collection Method for a WiFi-Based Indoor Positioning System

In the WiFi-based indoor positioning system (IPS), by using WiFi access points (APs) on hand and without requiring the APs' positions, the scene analysis method has better positioning accuracy and attracts many researchers to devote their efforts to it. However, this method needs to perform a sampling process in order to collect the received signal strength indicators (RSSIs) of the APs from the places of interest to build the building's WiFi radio fingerprint database in advance. It also needs to resample frequently in order to keep the fingerprint database updated and accurate. Both tasks are time-consuming, therefore we propose a quick radio fingerprint collection (QRFC) algorithm for collecting the sampling information. The Android smartphone and its built-in motion sensors were used to help the collection process. QRFC makes the sampling process simpler, and the time needed for the sampling is close to the time of walking slowly through the path. We also propose a heuristic AP RSSI shaping algorithm to compensate for the signal attenuation caused by multi-path, shadowing, and mask effects. Several field tests were performed to compare the QRFC method to the traditional method. Experiments show that there is no significant difference in their accuracy, but QRFC takes much less time to complete.

The rise of keyloggers on smartphones: A survey and insight into motion-based tap inference attacks

Smartphone sensing capabilities have opened new opportunities for innovative User Interface (UI) and context-aware applications. They have also opened new possibilities for potential risks to user privacy and security infiltration. Researchers have recently explored a new attack vector that exploits the built-in motion sensors to infer user taps on smartphone touchscreens. This new side channel has introduced the threat of keylogging to smartphones despite the lack of physical keyboards. In this paper, we review this type of attack and survey the leading works in the literature to highlight the underpinning motivations and threat model. We also discuss the main issues in the design and implementation of the new attack, in order to provide insights into the practicality, prospects, and limitations of the different approaches. Different countermeasures that can mitigate the rising threat are investigated, and recommendations for further research on this emerging trend are discussed. A comparative summary of the surveyed works is also presented.

Development of a Smartphone Application to Measure Physical Activity Using Sensor-Assisted Self-Report

Introduction: Despite the known advantages of objective physical activity monitors (e.g., accelerometers), these devices have high rates of non-wear, which leads to missing data. Objective activity monitors are also unable to capture valuable contextual information about behavior. Adolescents recruited into physical activity surveillance and intervention studies will increasingly have smartphones, which are miniature computers with built-in motion sensors.Methods: This paper describes the design and development of a smartphone application (“app”) called Mobile Teen that combines objective and self-report assessment strategies through (1) sensor-informed context-sensitive ecological momentary assessment (CS-EMA) and (2) sensor-assisted end-of-day recall.Results: The Mobile Teen app uses the mobile phone’s built-in motion sensor to automatically detect likely bouts of phone non-wear, sedentary behavior, and physical activity. The app then uses transitions between these inferred states to trigger CS-EMA self-report surveys measuring the type, purpose, and context of activity in real-time. The end of the day recall component of the Mobile Teen app allows users to interactively review and label their own physical activity data each evening using visual cues from automatically detected major activity transitions from the phone’s built-in motion sensors. Major activity transitions are identified by the app, which cues the user to label that “chunk,” or period, of time using activity categories.Conclusion: Sensor-driven CS-EMA and end-of-day recall smartphone apps can be used to augment physical activity data collected by objective activity monitors, filling in gaps during non-wear bouts and providing additional real-time data on environmental, social, and emotional correlates of behavior. Smartphone apps such as these have potential for affordable deployment in large-scale epidemiological and intervention studies.