Mobile Sensing System Research Articles

FacePsy: An Open-Source Affective Mobile Sensing System - Analyzing Facial Behavior and Head Gesture for Depression Detection in Naturalistic Settings

Depression, a prevalent and complex mental health issue affecting millions worldwide, presents significant challenges for detection and monitoring. While facial expressions have shown promise in laboratory settings for identifying depression, their potential in real-world applications remains largely unexplored due to the difficulties in developing efficient mobile systems. In this study, we aim to introduce FacePsy, an open-source mobile sensing system designed to capture affective inferences by analyzing sophisticated features and generating real-time data on facial behavior landmarks, eye movements, and head gestures - all within the naturalistic context of smartphone usage with 25 participants. Through rigorous development, testing, and optimization, we identified eye-open states, head gestures, smile expressions, and specific Action Units (2, 6, 7, 12, 15, and 17) as significant indicators of depressive episodes (AUROC=81%). Our regression model predicting PHQ-9 scores achieved moderate accuracy, with a Mean Absolute Error of 3.08. Our findings offer valuable insights and implications for enhancing deployable and usable mobile affective sensing systems, ultimately improving mental health monitoring, prediction, and just-in-time adaptive interventions for researchers and developers in healthcare.

Initial Deployment of a Mobile Sensing System for Water Quality in Urban Canals

Although water quality has extensively improved over the last decade, recreational uses of the canal network in Amsterdam are limited by variations in water quality associated with stormwater runoff and episodic harmful algal blooms. The current systems for monitoring water quality are based on a stationary network of sampling points, offline testing methods, and online measurements of conventional water quality parameters on board a boat that continuously navigates the urban canal network. Here we describe the development and deployment of online algal sensors on board the boat, including a prototype LED-induced fluorescence instrument for algal identification and quantification. We demonstrate that by using only a single patrol vessel, we are able to achieve enough sampling coverage to observe spatiotemporal heterogeneity of algal and chemical water quality within the canal network. The data provide encouraging evidence that opportunistic measurements from a small number of mobile platforms can enable high-resolution mapping and can be used to improve the monitoring of water quality across the city compared to the current network of fixed sampling locations. We also discuss the challenges of operating water quality sensors for long-term autonomous monitoring.

A Mobile Sensing System for Future Gas Mapping in Confined Space Using an Olfactory Quadruped Robot

A Mobile Sensing System for Future Gas Mapping in Confined Space Using an Olfactory Quadruped Robot

Extending Coverage of Stationary Sensing Systems with Mobile Sensing Systems for Human Mobility Modeling

Human mobility modeling has many applications in location-based services, mobile networking, city management, and epidemiology. Previous sensing approaches for human mobility are mainly categorized into two types: stationary sensing systems (e.g., surveillance cameras and toll booths) and mobile sensing systems (e.g., smartphone apps and vehicle tracking devices). However, stationary sensing systems only provide mobility information of human in limited coverage (e.g., camera-equipped roads) and mobile sensing systems only capture a limited number of people (e.g., people using a particular smartphone app). In this work, we design a novel system Mohen to model human mobility with a heterogeneous sensing system. The key novelty of Mohen is to fundamentally extend the sensing coverage of a large-scale stationary sensing system with a small-scale sensing system. Based on the evaluation on data from real-world urban sensing systems, our system outperforms them by 35% and achieves a competitive result to an Oracle method.

Memory-Aware Active Learning in Mobile Sensing Systems.

We propose a novel active learning framework for activity recognition using wearable sensors. Our work is unique in that it takes limitations of the oracle into account when selecting sensor data for annotation by the oracle. Our approach is inspired by human-beings' limited capacity to respond to prompts on their mobile device. This capacity constraint is manifested not only in the number of queries that a person can respond to in a given time-frame but also in the time lag between the query issuance and the oracle response. We introduce the notion of mindful active learning and propose a computational framework, called EMMA, to maximize the active learning performance taking informativeness of sensor data, query budget, and human memory into account. We formulate this optimization problem, propose an approach to model memory retention, discuss the complexity of the problem, and propose a greedy heuristic to solve the optimization problem. Additionally, we design an approach to perform mindful active learning in batch where multiple sensor observations are selected simultaneously for querying the oracle. We demonstrate the effectiveness of our approach using three publicly available activity datasets and by simulating oracles with various memory strengths. We show that the activity recognition accuracy ranges from 21% to 97% depending on memory strength, query budget, and difficulty of the machine learning task. Our results also indicate that EMMA achieves an accuracy level that is, on average, 13.5% higher than the case when only informativeness of the sensor data is considered for active learning. Moreover, we show that the performance of our approach is at most 20% less than the experimental upper-bound and up to 80% higher than the experimental lower-bound. To evaluate the performance of EMMA for batch active learning, we design two instantiations of EMMA to perform active learning in batch mode. We show that these algorithms improve the algorithm training time at the cost of a reduced accuracy in performance. Another finding in our work is that integrating clustering into the process of selecting sensor observations for batch active learning improves the activity learning performance by 11.1% on average, mainly due to reducing the redundancy among the selected sensor observations. We observe that mindful active learning is most beneficial when the query budget is small and/or the oracle's memory is weak. This observation emphasizes advantages of utilizing mindful active learning strategies in mobile health settings that involve interaction with older adults and other populations with cognitive impairments.

Detecting On-Street Parking Spaces in Smart Cities: Performance Evaluation of Fixed and Mobile Sensing Systems

As the number of vehicles continues to grow, parking spaces are at a premium in city streets. In addition, due to the lack of knowledge about street parking spaces, heuristic circling in the streets not only costs drivers’ time and fuel, but also increases city congestion. In the wake of the recent trend to build convenient, green, and energy-efficient smart cities, common techniques adopted by high-profile smart parking systems are reviewed, and the performance of the various approaches are compared. A mobile sensing unit has been developed as an alternative to the fixed sensing approach. It is mounted on the passenger side of a car to measure the distance from the vehicle to the nearest roadside obstacle. By extracting parked vehicles’ features from the collected trace, a supervised learning algorithm has been developed to estimate roadside parking occupancy. Multiple road tests were conducted around Wheatley (Oxfordshire) and Guildford (Surrey) in the U.K. In the case of accurate GPS readings, enhanced by a map matching technique, the accuracy of the system is above 90%. A quantity estimation model is derived to calculate the density of sensing units required to cover urban streets. The estimation is quantitatively compared with a fixed sensing solution. The results show that the mobile sensing approach can perform at the same level as fixed sensing solutions when accurate location information is available but substantially fewer sensors are needed compared with the fixed sensing system.

SADHealth: A Personal Mobile Sensing System for Seasonal Health Monitoring

People’s health, mood, and activities are closely related to their environment and the seasons. Countries at extreme latitudes (e.g., Sweden, U.K., and Norway) experience huge variations in their light levels, impacting the population’s mental state, well-being and energy levels. Advanced sensing technologies on smartphones enable nonintrusive and longitudinal monitoring of user states. The collected data make it possible for healthcare professionals and individuals to diagnose and rectify problems caused by seasonality. In this paper, we present a personal mobile sensing system that exploits technologies on smartphones to efficiently and accurately detect the light exposure, mood, and activity levels of individuals. We conducted a 2-year experiment with many users to test the functionality and performance of our system. The results show that we can obtain accurate light exposure estimation by opportunistically measuring light data on smartphones, tracking both personal light exposure and the general seasonal trends. An optional questionnaire also provides insight into the correlation between a user’s mood and energy level. Our system is able to inform users how little light they are experiencing in the winter time. It can also correlate light exposure data with reduced mood and energy, and provide quantitative measurements for lifestyle changes.

Passive and In-situ Assessment of Mental and Physical Well-being using Mobile Sensors.

The idea of continuously monitoring well-being using mobile-sensing systems is gaining popularity. In-situ measurement of human behavior has the potential to overcome the short comings of gold-standard surveys that have been used for decades by the medical community. However, current sensing systems have mainly focused on tracking physical health; some have approximated aspects of mental health based on proximity measurements but have not been compared against medically accepted screening instruments. In this paper, we show the feasibility of a multi-modal mobile sensing system to simultaneously assess mental and physical health. By continuously capturing fine grained motion and privacy-sensitive audio data, we are able to derive different metrics that reflect the results of commonly used surveys for assessing well-being by the medical community. In addition, we present a case study that highlights how errors in assessment due to the subjective nature of the responses could potentially be avoided by continuous sensing and inference of social interactions and physical activities.

A Mobile Sensing and Imaging System for Real-Time Monitoring of Spine Health

Poor posture or extra stress on the spine has been shown to lead to a variety of spinal disorders including chronic back pain, and to incur numerous health costs to society. For this reason, workplace ergonomics is rapidly becoming indispensable in all major corporations. Making the individual continuously aware of poor posture may reduce out-of-posture tendencies and encourage healthy spinal habits. Spine stress can also worsen existing structural deformities in the spine such as adolescent idiopathic scoliosis (AIS). In this work we developed a system to monitor spine health through both dynamic monitoring and structural imaging. The dynamic sensing method monitors spine stress in realtime by detecting poor back posture and strain on the back due to prolonged sitting or standing, and provides real-time user feedback when poor posture is sustained. The imaging method extracts the structural curvature of the spine and is used for the diagnosis of AIS in a non-invasive and inexpensive manner. Namely, the image is obtained using a photograph where the spinous processes have been marked to trace the shape of the spine. The spine curvature is then extracted automatically and modeled by a curve-fitting polynomial. The approach is simple and practical and allows scoliosis patients to monitor their curvature progress from home while minimizing the use of x-rays. The theme of our work is spine health, which we monitor through the wireless sensing system and the orthopedic imaging system. The two are complementary: the mobile wireless system assesses spine health during daily activity while the imaging system can assess the progression of a patient’s structural spine curvature. We demonstrate effectiveness of our sensing system in simultaneously monitoring posture and position by testing in numerous situations. Furthermore, experiments show that our imaging method is accurate and robust under different brightness conditions. X-ray data used for this study was obtained from the international, electronic database of surgical cases of AIS, Scolisoft®.

Autonomous hazardous waste drum inspection vehicle

Hundreds of thousands of hazardous, radioactive, and mixed waste drums are being stored throughout the world, and the anticipated decommissioning of facilities will generate many more drums. Currently, in compliance with federal regulations, waste storage facilities at U.S. Department of Energy (DOE) sites are inspected manually for degradation and to verify inventories. An Intelligent Mobile Sensing System (IMSS) has been developed for the automated inspection of radioactive and hazardous waste storage containers in warehouse facilities at DOE sites. The IMSS will reduce the risk of exposure to personnel and create accurate, high-quality inspection reports to ensure regulatory compliance. The IMSS includes an autonomous robotic device with enhanced intelligence and maneuverability, capable of conducting routine multisensor inspection of stored waste drums.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>