Types Of Sensor Data Research Articles

Robust Fetal Heart Rate Tracking through Fetal Electrocardiography (ECG) and Photoplethysmography (PPG) Fusion.

Fetal electrocardiogram (fECG) or photoplethysmogram (fPPG) devices are being developed for fetal heart rate (FHR) monitoring. However, deep tissue sensing is challenged by low fetal signal-to-noise ratio (SNR). Data quality is easily degraded by motion, or interference from maternal tissues and data losses can happen due to communication faults. In this paper, we propose to combine fECG and fPPG measurements in order to increase robustness against such dynamic challenges and increase FHR estimation accuracy. To the author's knowledge the fusion of two sensory data types (fECG, fPPG) has not been investigated for FHR tracking purposes in the literature. The proposed methods are evaluated on real-world data captured from gold-standard large pregnant animal experiments. A particle filtering algorithm with sensor fusion in the measurement likelihood, called KUBAI, is used to estimate FHR. Fusion of PPG&ECG data resulted in 36.6% improvement in root-mean-square-error (RMSE) and 20.3% improvement in R2 correlation between estimated and reference FHR values compared to single sensor-type (PPG-only or ECG-only) data. We demonstrate that using different types of sensory data improves the robustness and accuracy of FHR tracking.

Lost in data: recognizing type of time series sensor data using signal pattern classification

AbstractWith the increase in number and size of Internet of Things systems, there is an ever-growing risk of (meta)data loss, as well as the maintenance overhead to mitigate such risks. The experts recognize three main challenges in this area that need to be tackled, namely (1) downsizing the manual work required for configuring sensor networks, (2) recovering metadata, such as sensor type, in case of connection issues, malfunctions or malicious actions in sensor networks, (3) rebuilding metadata lost due to unexpected problems within a data storage. Fortunately, all three challenges can be tackled with a uniform solution, namely the signal type classification approach, which is able to match raw signal to an appropriate data type. In this research, we evaluate and compare different approaches for signal type classification that can be used to recognize a signal type being read from an IoT sensor. This is done by using machine learning methods for modelling a signal represented as raw time series data. Three machine learning classification approaches are taken into a consideration, namely one class, two class and multi-class. According to the results of the evaluation, the most accurate multi-class random forest algorithm can correctly classify unknown signals in $$\sim {75}\%$$ ∼ 75 % of the cases based on only 20 consecutive sensor readings. Moreover, multi-class random forest can detect two most probable classes of monitored signal with the accuracy of 95%.

Trusted and privacy-preserving sensor data onloading

To personalize their services (e.g., advertisement, navigation, healthcare), mobile apps collect sensor data. Typically, they upload the collected sensor data to the cloud, which returns the inferred user profiles required to personalize mobile services. However, privacy concerns and network connectivity/congestion issues can render cloud-based processing inapplicable. If different apps collect the same type of sensor data, app providers can collaborate by combining their data collections to infer on-device the user profiles required for personalization. Although major mobile platforms provide on-device data sharing mechanisms, these direct data exchanges provide no privacy protection. As an alternative to direct data sharing, we present differentially privatized sensor data onloading for app providers’ collaboration. With our approach, app providers can safely collaborate by using shared sensor data to personalize their mobile services. We realize our approach as a middleware that acts as a trusted intermediary. The middleware aggregates the sensor data contributed by individual apps, which execute statistical queries against the combined datasets. Furthermore, the middleware’s adaptive privacy-preserving scheme (1) computes and adds the required amount of noise to the query results so as to balance utility and privacy; (2) introduces a Trust-Data Theory so as to detect and remove spurious data from the combined collections; (3) rewards active contributing app providers so as to incentivize data contribution; (4) integrates a Trusted Execution Environment (TEE) so as to secure all data processing. Our evaluation shows that it is feasible and useful to personalize mobile services while protecting data privacy: queries’ execution time is within 10 ms; participants’ dissimilar privacy/utility requirements are satisfied; untrustworthy data are effectively detected; mobile services are personalized, and data privacy of both app providers and users are preserved.11This article is a revised and extended version of our prior paper, published in the 12th EAI International Conference on Mobile Computing, Applications and Services (MobiCASE 2021) (Liu et al., 2022).

A review of high-definition map creation methods for autonomous driving

Autonomous driving has been among the most popular and challenging topics in the past few years. Among all modules in autonomous driving, High-definition (HD) map has drawn lots of attention in recent years due to its high precision and informative level in localization. Since localization is a significant module for automated vehicles to navigate an unknown environment, it has immediately become one of the most critical components of autonomous driving. Big organizations like HERE, NVIDIA, and TomTom have created HD maps for different scenes and purposes for autonomous driving. However, such HD maps are not open-source and are only available for internal research or automotive companies. Even though researchers have proposed various methods to create HD maps using different types of sensor data, there are few papers that review and summarize those methods. New researchers do not have a clear insight into the current state of HD map creation methods to work on their HD map research. Due to the reason above, reviewing, classifying, comparing, and summarizing the state-of-the-art techniques for HD map creation is necessary. This paper reviews recent HD map creation methods that leverage both 2D and 3D map generation. This review introduces the concept of HD maps and their usefulness in autonomous driving and gives a detailed overview of HD map creation methods. We will also discuss the limitations of the current HD map creation methods to motivate future research. Additionally, a chronological overview is created with the most recent HD map creation methods in this paper.

ForETaxi: Data-Driven Fleet-Oriented Charging Resource Allocation in Large-Scale Electric Taxi Networks

Charging processes are the key to promoting electric taxis and improving their operational efficiency due to frequent charging activities and long charging time. Nevertheless, optimizing charging resource allocation in real time is extremely challenging because of uneven charging demand/supply distributions, heuristic-based charging behaviors of drivers, and city-scale of the fleets. The existing solutions have utilized real-time contextual information for charging recommendation, but they do not consider the much-richer fleet information, leading to the suboptimal individual-based charging recommendation. In this paper, we design a data-driven fleet-oriented charging recommendation system for charging resource allocation called ForETaxi for electric taxis , which aims to minimize the overall charging overhead for the entire fleet, instead of individual vehicles. ForETaxi considers not only current charging requests but also possible charging requests of other nearby electric taxis in the near future by inferring their status in real time. More importantly, we implement ForETaxi with multiple types of sensor data from the Chinese Shenzhen city including GPS data, and taxi transaction data from more than 13,000 electric taxis, combined with road network data and charging station data. The data-driven evaluation results show that compared to the state-of-the-art individual-based recommendation methods, our fleet-oriented ForETaxi outperforms them by 16% in the total charging time reduction and 82% in the queuing time reduction.

Cost–benefit analysis of remote sensing data types for mapping mosquito breeding sites

Cost–benefit analysis of remote sensing data types for mapping mosquito breeding sites

Anomaly detection using a model-agnostic meta-learning-based variational auto-encoder for facility management

In a smart building, various types of sensor data are generated and employed to control devices within the building. Detecting anomalies in these sensor data is critical for effective facility management because it can prevent device malfunction or failure. Although deep learning-based methods have been used to detect anomalies, effective model construction is challenging because abnormal data tend to be rare under real-world conditions. In image classification and load forecasting, model-agnostic meta-learning (MAML) has recently been proposed to alleviate this problem using common knowledge learned from various tasks. This paper proposes an MAML-based unsupervised anomaly detection method called MAVAE for time-series sensor data. The proposed method uses a variational auto-encoder (VAE) as an anomaly detection model and adapts the model to a new target task with few unlabeled anomaly data via MAML. To our knowledge, this is the first study to train a VAE using MAML with time-series data. Extensive experiments on public data reveals that the proposed method outperforms existing anomaly detection methods, achieving an average improvement in prediction performance of 45% compared to state-of-the-art methods. The robustness of the proposed method is also demonstrated by evaluating its performance with a different number of samples for MAML. Our code is available at github.com/17011813/MAVAE.

Dynamic Super Round-Based Distributed Task Scheduling for UAV Networks

Networks of Unmanned Aerial Vehicles (UAVs) are emerging in many application domains, e.g., military surveillance. To perform collaborative tasks, the involved UAVs exchange several types of information, e.g., sensor data and commands. The major question here is how to schedule the tasks under dynamic traffic flows to provide network services. Existing solutions use the Round-Robin Strategy (RRS), where the tasks are scheduled statistically by dividing the time into fixed-length rounds. However, the RRS wastes significant network and device resources due to task scheduling in each round. This paper proposes DROVE– a novel clustering approach that allows the UAVs for dynamic task scheduling. However, determining the task scheduling is crucial, as it significantly affects several network parameters, e.g., throughput. Therefore, we devise the problem of distributed task scheduling under dynamic traffic flow scenarios to optimize the throughput. We propose a clustering task scheduling algorithm to serve dynamic traffic flows. Particularly, we integrate the dynamic traffic flows into the Lyapunov drift analysis framework, and determine the throughput optimality of our proposed scheduling algorithm. We perform extensive simulations to validate the effectiveness of DROVE. The results show that DROVE outperforms the state-of-the-art solutions in terms of energy consumption, clustering overhead, throughput, end-to-end delay, flow success rate and packet drop rate.

Comparison of Point-of-Care and Laboratory Glycated Hemoglobin A1c and Its Relationship to Time-in-Range and Glucose Variability: A Real-World Study.

Introduction The main objective of the current study was to perform a comparison of point-of-care testing for hemoglobin A1c (POCT-HbA1c) versus the standard laboratory method (Lab HbA1c) and their relationship to time-in-range (TIR) and glucose variability (GV) among patients with diabetes mellitus (DM) presented to the outpatient diabetes clinics. Methods This single-center cross-sectional study was carried out on diabetic patients (aged ≥14 years of both genders) who undergo routine follow-up at our institution and whose physicians ordered HbA1c analysis for routine care. The included patients were those using the intermittently scanned continuous glucose monitoring (isCGM) Abbott's FreeStyle Libre system for at least three months and regular CGM users with at least 70% use. Results We included 97 diabetic patients (41 female and 56 male), with a median age of 25 years (Interquartile range= 18) and a mean DM duration of 10.33±5.48 years. The mean values of Lab-HbA1c and POCT HbA1c were 8.82%±0.85% and 8.53%±0.89%, respectively. The TIR, time below range, and time above range were 33.47±14.38 minutes (47.78%±14.32%), 5.44±2.58 minutes (8.41%±4.42%), and 28.8±8.27 minutes (43.81%±13.22%), respectively. According to the Bland-Altman plot analysis, the POCT-HbA1c values are consistent with the standard Lab-HbA1c values (SD of bias= 0.55, and 95% CI= -0.78 to 1.4). The univariate linear regression analysis showed a statistically significant relationship between laboratory HbA1c and POCT HbA1c (R2= 0.637,p<0.001), TIR (R2= 0.406,p<0.001), and GV (R2= 0.048,p= 0.032). After adjusting for age, gender, disease duration, diabetes type, and percentage of sensor data in a multivariable linear regression model, the linear associations remained significant (allp< 0.05). Conclusion The current findings show that TIR and GV can be used as endpoints and valuable parameters for the therapy of DM.

A Systematic Review of Gait Analysis in the Context of Multimodal Sensing Fusion and AI.

Neurological diseases are a leading cause of disability and mortality. Gait, or human walking, is a significant predictor of quality of life, morbidity, and mortality. Gait patterns and other kinematic, kinetic, and balance gait features are accurate and powerful diagnostic and prognostic tools. This review article focuses on the applicability of gait analysis using fusion techniques and artificial intelligence (AI) models. The aim is to examine the significance of mixing several types of wearable and non-wearable sensor data and the impact of this combination on the performance of AI models. In this systematic review, 66 studies using more than two modalities to record and analyze gait were identified. 40 studies incorporated multiple gait analysis modalities without the use of artificial intelligence to extract gait features such as kinematic, kinetic, margin of stability, temporal, and spatial gait parameters, as well as cerebral activity. Similarly, 26 studies analyzed gait data using multimodal fusion sensors and AI algorithms. The research summarized here demonstrates that the quality of gait analysis and the effectiveness of AI models can both benefit from the integration of data from many sensors. Meanwhile, the utilization of EMG signals in fusion data is especially advantageous. The findings of this review suggest that a smart, portable, wearable-based gait and balance assessment system can be developed using multimodal sensing of the most cutting-edge, clinically relevant tools and technology available. The information presented in this article may serve as a vital springboard for such development.

A Deep-Learning-Based Multi-Modal Sensor Fusion Approach for Detection of Equipment Faults

Condition monitoring is a part of the predictive maintenance approach applied to detect and prevent unexpected equipment failures by monitoring machine conditions. Early detection of equipment failures in industrial systems can greatly reduce scrap and financial losses. Developed sensor data acquisition technologies allow for digitally generating and storing many types of sensor data. Data-driven computational models allow the extraction of information about the machine’s state from acquired sensor data. The outstanding generalization capabilities of deep learning models have enabled them to play a significant role as a data-driven computational fault model in equipment condition monitoring. A challenge of fault detection applications is that single-sensor data can be insufficient in performance to detect equipment anomalies. Furthermore, data in different domains can reveal more prominent features depending on the fault type, but may not always be obvious. To address this issue, this paper proposes a multi-modal sensor fusion-based deep learning model to detect equipment faults by fusing information not only from different sensors but also from different signal domains. The effectiveness of the model’s fault detection capability is shown by utilizing the most commonly encountered equipment types in the industry, such as electric motors. Two different sensor types’ raw time domain and frequency domain data are utilized. The raw data from the vibration and current sensors are transformed into time-frequency images using short-time Fourier transform (STFT). Then, time-frequency images and raw time series data were supplied to the designed deep learning model to detect failures. The results showed that the fusion of multi-modal sensor data using the proposed model can be advantageous in equipment fault detection.

SensorFormer: Efficient Many-to-Many Sensor Calibration With Learnable Input Subsampling

Accurate calibration of low-cost environmental sensors is a prerequisite for their successful use in many monitoring applications. State-of-the-art calibration methods vary from simple linear regression to sophisticated deep models based on LSTMs and GRUs. The latter take past measurements to improve calibration accuracy. In this article, we argue that both recent past and close future measurements help to achieve accurate calibration, whereas accuracy improvements beyond the past come with a delay introduced by the occurrence of the future. We propose a generalized many-to-many calibration scheme called SensorFormer based on the successful Transformer model which takes both past and future raw measurements into account. We show that the proposed approach: 1) outperforms other methods by improving calibration accuracy by 16.5%–20.4% on public data sets and own field data and 2) can efficiently run on low-power microcontrollers with very limited computational and storage capabilities. The latter is achieved by a novel optimization technique based on learnable input subsampling taking advantage of the properties of typical sensor data. We manage to reduce the model size by 20%–33% and minimize the overall floating point operations per second (FLOPs) by 65% while maintaining superior accuracy than state-of-the-art methods.

IDVS: interactive 2D and 3D visualizations of proximal sensor data for rapid characterization of soil profiles

Knowledge of the soil’s physical and chemical properties in field-scale geographical areas is crucial for farmers and policymakers for agronomic productivity and environmental quality assessment. Proximal sensors can successfully model soil properties for these purposes and offer a way to rapidly acquire data from soil profiles. However, existing data analysis approaches are ill-suited to explore this type of multivariate proximal sensor data over large land areas and in a sizeable three-dimensional volume. Therefore, this work proposes a multifaceted approach with seamless integration of a star pattern for soil sample collection, data acquisition using proximal sensor devices, and an interactive data visualization solution for processing, analyzing, and reporting analysis results. This solution is the result of an interdis- ciplinary project in which data visualizers worked closely with soil scientists and agronomists to develop a tool called iDVS for rapid characterizations of soil profiles over larger geographical areas using proximal sensor technologies.

Design of Multimodal Sensor Module for Outdoor Robot Surveillance System

Recent studies on surveillance systems have employed various sensors to recognize and understand outdoor environments. In a complex outdoor environment, useful sensor data obtained under all weather conditions, during the night and day, can be utilized for application to robots in a real environment. Autonomous surveillance systems require a sensor system that can acquire various types of sensor data and can be easily mounted on fixed and mobile agents. In this study, we propose a method for modularizing multiple vision and sound sensors into one system, extracting data synchronized with 3D LiDAR sensors, and matching them to obtain data from various outdoor environments. The proposed multimodal sensor module can acquire six types of images: RGB, thermal, night vision, depth, fast RGB, and IR. Using the proposed module with a 3D LiDAR sensor, multimodal sensor data were obtained from fixed and mobile agents and tested for more than four years. To further prove its usefulness, this module was used as a monitoring system for six months to monitor anomalies occurring at a given site. In the future, we expect that the data obtained from multimodal sensor systems can be used for various applications in outdoor environments.

Convolutional Neural Network-Based Travel Mode Recognition Based on Multiple Smartphone Sensors

Nowadays, large-scale human mobility has led to increasingly severe traffic congestion in cities, how to accurately identify people’s travel mode has become particularly important for urban traffic planning and management. However, traditional methods are based on telephone interviews or questionnaires, which makes it difficult to obtain accurate and effective data. Nowadays, numbers of smartphones are equipped with various sensors, including accelerometers, gyroscopes, and GPS, providing a novel social sensing data source to detect people’s travel modes. The fusion of multiple sensor data is a promising way for travel mode detection. However, how to use these sensor data to accurately detect travel mode is still a challenging task. In this paper, we presented a light-weight method for travel mode detection based on four types of smartphone sensor data collected from an accelerometer, gyroscope, magnetometer, and barometer, and a prototype application was developed. Then, a novel convolutional neural network (CNN) was designed to identify five representative travel modes (walk, bicycle, bus, car, and metro). We compared the overall performance of the proposed method via different hyperparameters, and the experimental results show that the F value of the proposed method reaches 97%, which verified the effectiveness of the proposed method for travel mode classification.

Detecting Mental Health Behaviors Using Mobile Interactions: Exploratory Study Focusing on Binge Eating.

BackgroundBinge eating is a subjective loss of control while eating, which leads to the consumption of large amounts of food. It can cause significant emotional distress and is often accompanied by purging behaviors (eg, meal skipping, overexercising, or vomiting).ObjectiveThe aim of this study was to explore the potential of mobile sensing to detect indicators of binge-eating episodes, with a view toward informing the design of future context-aware mobile interventions.MethodsThis study was conducted in 2 stages. The first involved the development of the DeMMI (Detecting Mental health behaviors using Mobile Interactions) app. As part of this, we conducted a consultation session to explore whether the types of sensor data we were proposing to capture were useful and appropriate, as well as to gather feedback on some specific app features relating to self-reporting. The second stage involved conducting a 6-week period of data collection with 10 participants experiencing binge eating (logging both their mood and episodes of binge eating) and 10 comparison participants (logging only mood). An optional interview was conducted after the study, which discussed their experience using the app, and 8 participants (n=3, 38% binge eating and n=5, 63% comparisons) consented.ResultsThe findings showed unique differences in the types of sensor data that were triangulated with the individuals’ episodes (with nearby Bluetooth devices, screen and app use features, mobility features, and mood scores showing relevance). Participants had a largely positive opinion about the app, its unobtrusive role, and its ease of use. Interacting with the app increased participants’ awareness of and reflection on their mood and phone usage patterns. Moreover, they expressed no privacy concerns as these were alleviated by the study information sheet.ConclusionsThis study contributes a series of recommendations for future studies wishing to scale our approach and for the design of bespoke mobile interventions to support this population.

Digital soil mapping of soil total nitrogen based on Landsat 8, Sentinel 2, and WorldView-2 images in smallholder farms in Yellow River Basin, China.

Predicting spatial explicit information of soil nutrients is critical for sustainable soil management and food security under climate change and human disturbance in agricultural land. Digital soil mapping (DSM) techniques can utilize soil-landscape information from remote sensing data to predict the spatial pattern of soil nutrients, and it is important to explore the effects of remote sensing data types on DSM. This research utilized Landsat 8 (LT), Sentinel 2 (ST), and WorldView-2 (WV) remote sensing data and employed partial least squares regression (PLSR), random forest (RF), and support vector machine (SVM) algorithms to characterize the spatial pattern of soil total nitrogen (TN) in smallholder farm settings in Yellow River Basin, China. The overall relationships between TN and spectral indices from LT and ST were stronger than those from WV. Multiple red edge band-based spectral indices from ST and WV were relevant variables for TN, while there were no red band-based spectral indices from ST and WV identified as relevant variables for TN. Soil moisture and vegetation were major driving forces of soil TN spatial distribution in this area. The research also concluded that farmlands of crop rotation had relatively higher TN concentration compared with farmlands of monoculture. The soil prediction models based on WV achieved relatively lower model performance compared with those based on ST and LT. The effects of remote sensing data spectral resolution and spectral range on enhancing soil prediction model performance are higher than the effects of remote sensing data spatial resolution. Soil prediction models based on ST can provide location-specific soil maps, achieve fair model performance, and have low cost. This research suggests DSM research utilizing ST has relatively high prediction accuracy, and can produce soil maps that are fit for the spatial explicit soil management for smallholder farms.

Deep learning model inspired by lateral line system for underwater object detection

Inspired by the lateral line systems of various aquatic organisms that are capable of hydrodynamic imaging using ambient flow information, this study develops a deep learning-based object localization model that can detect the location of objects using flow information measured from a moving sensor array. In numerical simulations with the assumption of a potential flow, a two-dimensional hydrofoil navigates around four stationary cylinders in a uniform flow and obtains two types of sensory data during a simulation, namely flow velocity and pressure, from an array of sensors located on the surface of the hydrofoil. Several neural network models are constructed using the flow velocity and pressure data, and these are used to detect the positions of the hydrofoil and surrounding objects. The model based on a long short-term memory network, which is capable of learning order dependence in sequence prediction problems, outperforms the other models. The number of sensors is then optimized using feature selection techniques. This sensor optimization leads to a new object localization model that achieves impressive accuracy in predicting the locations of the hydrofoil and objects with only 40% of the sensors used in the original model.

Detecting individuals' spatial familiarity with urban environments using eye movement data

The spatial familiarity of environments is an important high-level user context for location-based services (LBS). Knowing users' familiarity level of environments is helpful for enabling context-aware LBS that can automatically adapt information services according to users' familiarity with the environment. Unlike state-of-the-art studies that used questionnaires, sketch maps, mobile phone positioning (GPS) data, and social media data to measure spatial familiarity, this study explored the potential of a new type of sensory data - eye movement data - to infer users' spatial familiarity of environments using a machine learning approach. We collected 38 participants' eye movement data when they were performing map-based navigation tasks in familiar and unfamiliar urban environments. We trained and cross-validated a random forest classifier to infer whether the users were familiar or unfamiliar with the environments (i.e., binary classification). By combining basic statistical features and fixation semantic features, we achieved a best accuracy of 81% in a 10-fold classification and 70% in the leave-one-task-out (LOTO) classification. We found that the pupil diameter, fixation dispersion, saccade duration, fixation count and duration on the map were the most important features for detecting users' spatial familiarity. Our results indicate that detecting users' spatial familiarity from eye tracking data is feasible in map-based navigation and only a few seconds (e.g., 5 s) of eye movement data is sufficient for such detection. These results could be used to develop context-aware LBS that adapt their services to users' familiarity with the environments.

Dementia Scale Score Classification Based on Daily Activities Using Multiple Sensors

Early detection of age-related disease symptoms in older people by the use of daily activity data is one of the central challenges of home sensor systems. This paper focuses on dementia scale classification from daily activity data collected using sensors that can be deployed in actual residential environments. Activity data collected by four sensors (a door sensor, human motion sensor, location sensor, and sleep sensor) were obtained by recording 56 older adults living in common residences. We analyzed the effects of different types of sensor data, such as time spent in an individual room according to human motion sensors, location in a facility, and sleep patterns, on dementia detection. We then developed a feature extraction method related to daily activity patterns based on a clustering algorithm and analyzed its effectiveness. In the experimental evaluation, we trained binary classification models to classify dementia scale scores based on the Mini-Mental State Examination (MMSE) from these datasets. The experimental results show that a maximum accuracy of 0.871 was obtained with a linear support vector machine (SVM) model by fusing the door, location, and sleep features and by clustering activity patterns using the X-means algorithm.