Smartphone Sensors Research Articles

Automated step detection with 6-minute walk test smartphone sensors signals for fall risk classification in lower limb amputees.

Predictive models for fall risk classification are valuable for early identification and intervention. However, lower limb amputees are often neglected in fall risk research despite having increased fall risk compared to age-matched able-bodied individuals. A random forest model was previously shown to be effective for fall risk classification of lower limb amputees, however manual labelling of foot strikes was required. In this paper, fall risk classification is evaluated using the random forest model, using a recently developed automated foot strike detection approach. 80 participants (27 fallers, 53 non-fallers) with lower limb amputations completed a six-minute walk test (6MWT) with a smartphone at the posterior pelvis. Smartphone signals were collected with The Ottawa Hospital Rehabilitation Centre (TOHRC) Walk Test app. Automated foot strike detection was completed using a novel Long Short-Term Memory (LSTM) approach. Step-based features were calculated using manually labelled or automated foot strikes. Manually labelled foot strikes correctly classified fall risk for 64 of 80 participants (accuracy 80%, sensitivity 55.6%, specificity 92.5%). Automated foot strikes correctly classified 58 of 80 participants (accuracy 72.5%, sensitivity 55.6%, specificity 81.1%). Both approaches had equivalent fall risk classification results, but automated foot strikes had 6 more false positives. This research demonstrates that automated foot strikes from a 6MWT can be used to calculate step-based features for fall risk classification in lower limb amputees. Automated foot strike detection and fall risk classification could be integrated into a smartphone app to provide clinical assessment immediately after a 6MWT.

Falling and Drowning Detection Framework Using Smartphone Sensors.

Advancements in health monitoring using smartphone sensor technologies have made it possible to quantify the functional performance and deviations in an individual's routine. Falling and drowning are significant unnatural causes of silent accidental deaths, which require an ambient approach to be detected. This paper presents the novel ambient assistive framework Falling and Drowning Detection (FaDD) for falling and drowning detection. FaDD perceives input from smartphone sensors, such as accelerometer, gyroscope, magnetometer, and GPS, that provide accurate readings of the movement of an individual's body. FaDD hierarchically recognizes the falling and drowning actions by applying the machine learning model. The approach activates embedding, in a smartphone application, to notify emergency alerts to various stakeholders (i.e., guardian, rescue, and close circle community) about drowning of an individual. FaDD detects falling, drowning, and routine actions with good accuracy of 98%. Furthermore, the FaDD framework enhances coordination to provide more efficient and reliable healthcare services to people.

Analysis of the Spectral Sensitivity of Luxmeters and Light Sensors of Smartphones in Terms of Their Influence on the Results of Illuminance Measurements—Example Cases

The article presents spectral sensitivity curves of 22 typical luxmeters and 3 smartphones as a result of laboratory measurements. Based on the measurement data, errors in the spectral correction of the considered instruments and devices were calculated. Due to the fact that the value of the measured illuminance is influenced by the spectral characteristics of the radiation, in order to determine the spectral corrections coefficients, 10 light sources with different spectral distributions of radiation were taken into account. Spectral corrections factors were determined using two methods for all luxmeters and smartphones. This paper presents the relationship between the shape of the spectral sensitivity curves of a given luxmeter and the measured values of f′1 errors. Moreover, the limitations related to the use of spectral corrections in environmental measurements with the use of a luxmeter were demonstrated.

A low-cost pavement-rating system, based on machine learning, utilising smartphone sensors

One of the key innovations of this study is the development of a cost-efficient approach for pavement monitoring. This work aims to develop a system that evaluates roadway pavement surface conditions with improved frequency by utilising unsupervised machine learning algorithms and smartphone sensors. The evaluation of roadways utilising complex contemporary data sets is currently conducted periodically because of the high cost of collection methods. For this purpose, the study presents a data-driven framework on the use of a vehicle, a smartphone, an on-board diagnostic device and machine learning for the rating of pavement surfaces, while statistical features are considered in both time- and frequency-domain forms. The selection of features is performed utilising unsupervised classification algorithms. Moreover, the proposed system architecture has been field-tested for the detection of pavement anomalies and the classification of five rating categories. Furthermore, the proposed system may provide daily information on roadway pavement surface conditions, which can be used by agencies for automating the planning of pavement maintenance operations and for improving driving safety.

Deriving stair-climbing performance outcome measures using the smartphone barometer: Results of an algorithm development study

As we seek to gain richer insights to understand intervention effects, and increasingly decentralise aspects of clinical trials to simplify participation, there is a growing interest in leveraging wearables and sensors to generate novel and informative clinical outcome measures for at-home assessment. The sensors embedded within smartphone technology provide one approach to capture of this data, and may be particularly useful when patients are already using mobile devices for at-home capture of other clinical trials data, such as patient-reported outcomes.We describe the results of an initial algorithm development study to determine whether the atmospheric pressure data provided by an onboard smartphone sensor is sufficiently informative to enable detection of a small height gain, such as that achieved during a short stair climb performance test. We were able to sufficiently distinguish height changes of 0.6 m in indoor conditions, representing around 4 stairs on an average staircase. This suggests that the smartphone barometer may indeed be suitable for inclusion within future work developing a stair-climbing performance outcome test instrumented using a mobile application.

Predicting states of elevated negative affect in adolescents from smartphone sensors: a novel personalized machine learning approach.

Adolescence is characterized by profound change, including increases in negative emotions. Approximately 84% of American adolescents own a smartphone, which can continuously and unobtrusively track variables potentially predictive of heightened negative emotions (e.g. activity levels, location, pattern of phone usage). The extent to which built-in smartphone sensors can reliably predict states of elevated negative affect in adolescents is an open question. Adolescent participants (n = 22; ages 13-18) with low to high levels of depressive symptoms were followed for 15 weeks using a combination of ecological momentary assessments (EMAs) and continuously collected passive smartphone sensor data. EMAs probed negative emotional states (i.e. anger, sadness and anxiety) 2-3 times per day every other week throughout the study (total: 1145 EMA measurements). Smartphone accelerometer, location and device state data were collected to derive 14 discrete estimates of behavior, including activity level, percentage of time spent at home, sleep onset and duration, and phone usage. A personalized ensemble machine learning model derived from smartphone sensor data outperformed other statistical approaches (e.g. linear mixed model) and predicted states of elevated anger and anxiety with acceptable discrimination ability (area under the curve (AUC) = 74% and 71%, respectively), but demonstrated more modest discrimination ability for predicting states of high sadness (AUC = 66%). To the extent that smartphone data could provide reasonably accurate real-time predictions of states of high negative affect in teens, brief 'just-in-time' interventions could be immediately deployed via smartphone notifications or mental health apps to alleviate these states.

Build-in sensors and analysis algorithms aided smartphone-based sensors for point-of-care tests

Over the last few years, smartphone-based sensors present a rapid development in the field of point-of-care (POC) testing owing to the properties of ubiquity and convenience. Considering the test accuracy and sensitivity, kinds of external devices were equipped on smartphones for a better on-site medical diagnosis. However, some devices were not easily fabricated or obtained. The compatibility and practicality of the equipped devices may often be overlooked in the process of application. This review pays attention to the application of smartphone independent detections for the POC testing, especially in the cases that were performed only by build-in sensors and analysis algorithms of smartphones, as well as some universal and compatible tools. Four types of build-in sensor systems, including optical sensors, sound sensors, electrochemical sensors, and other sensors, were mainly introduced to improve the public cognition and understanding of smartphone sensors. At last, a brief perspective on scientific challenges and prospects was discussed, aiming to open new ways for exploiting smartphone-based POC or home healthcare.

A robust machine learning structure for driving events recognition using smartphone motion sensors

Driving behavior monitoring by smartphone sensors is one of the most investigated approaches to ameliorate road safety. Various methods are adopted in the literature; however, to the best of our knowledge, their robustness to the prediction of new unseen data from different drivers and road conditions is not explored. In this paper, a two-phase Machine Learning (ML) method with taking advantage of high-pass, low-pass, and wavelet filters is developed to detect driving brakes and turns. In the first phase, accelerometer and gyroscope filtered time series are fed into Random Forest and Artificial Neural Network classifiers, and the suspicious intervals are extracted by a high recall. Following that, in the next phase, statistical features calculated based on the obtained intervals are used to determine the false and true positive events. To compare the predicted and real labels of the recorded events and calculate the accuracy, a method that covers the limitations of previous sliding windows is also employed. Real-world experimental result shows that the proposed method can predict new unseen datasets with average F1-scores of 71% in brake detection and 82% in turn detection which is comparable with previous works. Moreover, by sensitivity analysis of our proposed model, it is proven that implementing high-pass and low-pass filters can affect the accuracy for turn detection up to 30%.

Native Smartphone Single- and Dual-Frequency GNSS-PPP/IMU Solution in Real-World Driving Scenarios

The Global Navigation Satellite System (GNSS) capability in smartphones has seen significant upgrades over the years. The latest ultra-low-cost GNSS receivers are capable of carrier-phase tracking and multi-constellation, dual-frequency signal reception. However, due to the limitations of these ultra-low-cost receivers and antennas, smartphone GNSS position solutions suffer significantly from urban multipath, poor signal reception, and signal blockage. This paper presents a novel sensor fusion technique using Precise Point Positioning (PPP) and the inertial sensors in smartphones, combined with a single- and dual-frequency (SFDF) optimisation scheme for smartphones. The smartphone is field-tested while attached to a vehicle’s dashboard and is driven in multiple real-world situations. A total of five vehicle experiments were conducted and the solutions show that SFDF-PPP outperforms single-frequency PPP (SF-PPP) and dual-frequency PPP (DF-PPP). Solutions can be further improved by integrating with native smartphone IMU measurements and provide consistent horizontal positioning accuracy of <2 m rms through a variety obstructions. These results show a significant improvement from the existing literature using similar hardware in challenging environments. Future work will improve optimising inertial sensor calibration and integrate additional sensors.

Model Of Physics Experiments Using Smartphone Sensors In Senior High School 1 Pangkalpinang

The aim of this research is to develop a model physics experiment manual in high school by utilizing sensors on smartphones as a source of observing data on physics experiments to increase students' level of understanding. The research method used is research and development. The teacher arranges physics experiments manual so that students can use it. Retrieval of data in the experiments uses the Research Based Learning (RBL) method so that students get the freedom to be creative using smartphone sensors.The results of the study are expected to help students understand the concepts of physics through experiments data using smartphone sensors

Security and Privacy Analysis of Smartphone-Based Driver Monitoring Systems from the Developer's Point of View.

Nowadays, the whole driver monitoring system can be placed inside the vehicle driver’s smartphone, which introduces new security and privacy risks to the system. Because of the nature of the modern transportation systems, the consequences of the security issues in such systems can be crucial, leading to threat to human life and health. Moreover, despite the large number of security and privacy issues discovered in smartphone applications on a daily basis, there is no general approach for their automated analysis that can work in conditions that lack data and take into account specifics of the application area. Thus, this paper describes an original approach for a security and privacy analysis of driver monitoring systems based on smartphone sensors. This analysis uses white-box testing principles and aims to help developers evaluate and improve their products. The novelty of the proposed approach lies in combining various security and privacy analysis algorithms into a single automated approach for a specific area of application. Moreover, the suggested approach is modular and extensible, takes into account specific features of smartphone-based driver monitoring systems and works in conditions of lack or inaccessibility of data. The practical significance of the approach lies in the suggestions that are provided based on the conducted analysis. Those suggestions contain detected security and privacy issues and ways of their mitigation, together with limitations of the analysis due to the absence of data. It is assumed that such an approach would help developers take into account important aspects of security and privacy, thus reducing related issues in the developed products. An experimental evaluation of the approach is conducted on a car driver monitoring use case. In addition, the advantages and disadvantages of the proposed approach as well as future work directions are indicated.

StudentSADD

The growing prevalence of depression and suicidal ideation among college students further exacerbated by the Coronavirus pandemic is alarming, highlighting the need for universal mental illness screening technology. With traditional screening questionnaires too burdensome to achieve universal screening in this population, data collected through mobile applications has the potential to rapidly identify at-risk students. While prior research has mostly focused on collecting passive smartphone modalities from students, smartphone sensors are also capable of capturing active modalities. The general public has demonstrated more willingness to share active than passive modalities through an app, yet no such dataset of active mobile modalities for mental illness screening exists for students. Knowing which active modalities hold strong screening capabilities for student populations is critical for developing targeted mental illness screening technology. Thus, we deployed a mobile application to over 300 students during the COVID-19 pandemic to collect the Student Suicidal Ideation and Depression Detection (StudentSADD) dataset. We report on a rich variety of machine learning models including cutting-edge multimodal pretrained deep learning classifiers on active text and voice replies to screen for depression and suicidal ideation. This unique StudentSADD dataset is a valuable resource for the community for developing mobile mental illness screening tools.

Digital Phenotyping for Stress, Anxiety, and Mild Depression: Systematic Literature Review.

Unaddressed early-stage mental health issues, including stress, anxiety, and mild depression, can become a burden for individuals in the long term. Digital phenotyping involves capturing continuous behavioral data via digital smartphone devices to monitor human behavior and can potentially identify milder symptoms before they become serious. This systematic literature review aimed to answer the following questions: (1) what is the evidence of the effectiveness of digital phenotyping using smartphones in identifying behavioral patterns related to stress, anxiety, and mild depression? and (2) in particular, which smartphone sensors are found to be effective, and what are the associated challenges? We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) process to identify 36 papers (reporting on 40 studies) to assess the key smartphone sensors related to stress, anxiety, and mild depression. We excluded studies conducted with nonadult participants (eg, teenagers and children) and clinical populations, as well as personality measurement and phobia studies. As we focused on the effectiveness of digital phenotyping using smartphones, results related to wearable devices were excluded. We categorized the studies into 3 major groups based on the recruited participants: studies with students enrolled in universities, studies with adults who were unaffiliated to any particular organization, and studies with employees employed in an organization. The study length varied from 10 days to 3 years. A range of passive sensors were used in the studies, including GPS, Bluetooth, accelerometer, microphone, illuminance, gyroscope, and Wi-Fi. These were used to assess locations visited; mobility; speech patterns; phone use, such as screen checking; time spent in bed; physical activity; sleep; and aspects of social interactions, such as the number of interactions and response time. Of the 40 included studies, 31 (78%) used machine learning models for prediction; most others (n=8, 20%) used descriptive statistics. Students and adults who experienced stress, anxiety, or depression visited fewer locations, were more sedentary, had irregular sleep, and accrued increased phone use. In contrast to students and adults, less mobility was seen as positive for employees because less mobility in workplaces was associated with higher performance. Overall, travel, physical activity, sleep, social interaction, and phone use were related to stress, anxiety, and mild depression. This study focused on understanding whether smartphone sensors can be effectively used to detect behavioral patterns associated with stress, anxiety, and mild depression in nonclinical participants. The reviewed studies provided evidence that smartphone sensors are effective in identifying behavioral patterns associated with stress, anxiety, and mild depression.

Pavement Distress Recognition via Wavelet-Based Clustering of Smartphone Accelerometer Data

The ubiquity of smartphones has led to a variety of studies on how phone accelerometers can be used to assess road pavement quality. The majority of prior studies have emphasized supervised machine learning, assuming the ability to collect labeled data for model development. However, variances in vehicle dynamics and roadway quality, as well as the reliance on data labeling, limit the generalizability, scalability, and reproducibility of these approaches. Here, we propose an unsupervised learning framework that combines Pareto-optimized wavelet featurization and clustering. We first demonstrate the applicability of wavelets as features for dimensionally reduced accelerometer data. Wavelet featurization typically requires significant empirical tuning for optimal featurization. The presented framework automates tuning and selection of wavelet features based on subsequent clustering metrics. These metrics are related to inherent cluster characteristics such as intercluster variance and between-cluster variance. Rather than select a clustering method a priori, the proposed approach optimizes across a variety of clustering algorithms and hyperparameter configurations, again based on a set of clustering metrics. Experimental evaluation shows that the framework is able to detect road pavement distress and distinguish between classes of pavement defects, but that low-cost smartphone sensor data may not be as reliable in discriminating the more nuanced characteristics of pavement distress. This was most notable in the case of cracking, where the type and range of cracking severity caused undesirable cluster separation. The presented framework is general and cost-efficient and opens the way to further research on automatic pavement distress recognition from crowdsourced, low-cost data.

Collaborative smartphone experiments for large audiences with phyphox

We present methods to implement collaborative experimentation with smartphone sensors for larger audiences ranging from typical university undergraduate courses of hundreds of students in a lecture hall to world-wide outreaches on the Internet. These methods are based on the app ‘phyphox’, which is being developed by the authors, and encompass simple data collection via web forms as well as a new network interface for ‘phyphox’, allowing to collect real-time experiment data from an audience on-site or easy data submission for remote participants. Examples are given with practical considerations derived from first implementations of this method in a lecture hall with 350 undergraduate students as well as a global experiment to determine the Earth’s axial tilt with smartphones.

User Authentication by Gait Data from Smartphone Sensors Using Hybrid Deep Learning Network

User authentication and verification by gait data based on smartphones’ inertial sensors has gradually attracted increasing attention due to their compact size, portability and affordability. However, the existing approaches often require users to walk on a specific road at a normal walking speed to improve recognition accuracy. In order to recognize gaits under unconstrained conditions on where and how users walk, we proposed a Hybrid Deep Learning Network (HDLN), which combined the advantages of a long short-term memory (LSTM) network and a convolutional neural network (CNN) to reliably extract discriminative features from complex smartphone inertial data. The convergence layer of HDLN was optimized through a spatial pyramid pooling and attention mechanism. The former ensured that the gait features were extracted from more dimensions, and the latter ensured that only important gait information was processed while ignoring unimportant data. Furthermore, we developed an APP that can achieve real-time gait recognition. The experimental results showed that HDLN achieved better performance improvements than CNN, LSTM, DeepConvLSTM and CNN+LSTM by 1.9%, 2.8%, 2.0% and 1.3%, respectively. Furthermore, the experimental results indicated our model’s high scalability and strong suitability in real application scenes.

Convolutional Neural Network-Based Travel Mode Recognition Based on Multiple Smartphone Sensors

Nowadays, large-scale human mobility has led to increasingly severe traffic congestion in cities, how to accurately identify people’s travel mode has become particularly important for urban traffic planning and management. However, traditional methods are based on telephone interviews or questionnaires, which makes it difficult to obtain accurate and effective data. Nowadays, numbers of smartphones are equipped with various sensors, including accelerometers, gyroscopes, and GPS, providing a novel social sensing data source to detect people’s travel modes. The fusion of multiple sensor data is a promising way for travel mode detection. However, how to use these sensor data to accurately detect travel mode is still a challenging task. In this paper, we presented a light-weight method for travel mode detection based on four types of smartphone sensor data collected from an accelerometer, gyroscope, magnetometer, and barometer, and a prototype application was developed. Then, a novel convolutional neural network (CNN) was designed to identify five representative travel modes (walk, bicycle, bus, car, and metro). We compared the overall performance of the proposed method via different hyperparameters, and the experimental results show that the F value of the proposed method reaches 97%, which verified the effectiveness of the proposed method for travel mode classification.

Predicting Depression in Adolescents Using Mobile and Wearable Sensors: Multimodal Machine Learning-Based Exploratory Study.

BackgroundDepression levels in adolescents have trended upward over the past several years. According to a 2020 survey by the National Survey on Drug Use and Health, 4.1 million US adolescents have experienced at least one major depressive episode. This number constitutes approximately 16% of adolescents aged 12 to 17 years. However, only 32.3% of adolescents received some form of specialized or nonspecialized treatment. Identifying worsening symptoms earlier using mobile and wearable sensors may lead to earlier intervention. Most studies on predicting depression using sensor-based data are geared toward the adult population. Very few studies look into predicting depression in adolescents.ObjectiveThe aim of our work was to study passively sensed data from adolescents with depression and investigate the predictive capabilities of 2 machine learning approaches to predict depression scores and change in depression levels in adolescents. This work also provided an in-depth analysis of sensor features that serve as key indicators of change in depressive symptoms and the effect of variation of data samples on model accuracy levels.MethodsThis study included 55 adolescents with symptoms of depression aged 12 to 17 years. Each participant was passively monitored through smartphone sensors and Fitbit wearable devices for 24 weeks. Passive sensors collected call, conversation, location, and heart rate information daily. Following data preprocessing, 67% (37/55) of the participants in the aggregated data set were analyzed. Weekly Patient Health Questionnaire-9 surveys answered by participants served as the ground truth. We applied regression-based approaches to predict the Patient Health Questionnaire-9 depression score and change in depression severity. These approaches were consolidated using universal and personalized modeling strategies. The universal strategies consisted of Leave One Participant Out and Leave Week X Out. The personalized strategy models were based on Accumulated Weeks and Leave One Week One User Instance Out. Linear and nonlinear machine learning algorithms were trained to model the data.ResultsWe observed that personalized approaches performed better on adolescent depression prediction compared with universal approaches. The best models were able to predict depression score and weekly change in depression level with root mean squared errors of 2.83 and 3.21, respectively, following the Accumulated Weeks personalized modeling strategy. Our feature importance investigation showed that the contribution of screen-, call-, and location-based features influenced optimal models and were predictive of adolescent depression.ConclusionsThis study provides insight into the feasibility of using passively sensed data for predicting adolescent depression. We demonstrated prediction capabilities in terms of depression score and change in depression level. The prediction results revealed that personalized models performed better on adolescents than universal approaches. Feature importance provided a better understanding of depression and sensor data. Our findings can help in the development of advanced adolescent depression predictions.

Depressive Symptoms Feature-Based Machine Learning Approach to Predicting Depression Using Smartphone.

With the impact of the COVID-19 pandemic, the number of patients suffering from depression is rising around the world. It is important to diagnose depression early so that it may be treated as soon as possible. The self-response questionnaire, which has been used to diagnose depression in hospitals, is impractical since it requires active patient engagement. Therefore, it is vital to have a system that predicts depression automatically and recommends treatment. In this paper, we propose a smartphone-based depression prediction system. In addition, we propose depressive features based on multimodal sensor data for predicting depressive mood. The multimodal depressive features were designed based on depression symptoms defined in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). The proposed system comprises a “Mental Health Protector” application that collects data from smartphones and a big data-based cloud platform that processes large amounts of data. We recruited 106 mental patients and collected smartphone sensor data and self-reported questionnaires from their smartphones using the proposed system. Finally, we evaluated the performance of the proposed system’s prediction of depression. As the test dataset, 27 out of 106 participants were selected randomly. The proposed system showed 76.92% on an f1-score for 16 patients with depression disease, and in particular, 15 patients, 93.75%, were successfully predicted. Unlike previous studies, the proposed method has high adaptability in that it uses only smartphones and has a distinction of evaluating prediction accuracy based on the diagnosis.

Activities tracking by smartphone and smartwatch biometric sensors using fuzzy set theory

Activities tracking by smartphone and smartwatch biometric sensors using fuzzy set theory