Most existing multi-user Augmented Reality (AR) systems only support multiple co-located users to view a common set of virtual objects but lack the ability to enable each user to directly interact with other users appearing in his/her view. Such multi-user AR systems should be able to detect the human keypoints and estimate device poses (for identifying different users) in the meanwhile. However, due to the stringent low latency requirements and the intensive computation of the above two capabilities, previous research only enables either of the two capabilities for mobile devices even with the aid of the edge server. Integrating the above two capabilities is promising but non-trivial in terms of latency, accuracy, and matching. To fill this gap, we propose DiTing to achieve real-time ID-aware multi-device visual interaction for multi-user AR applications, which contains three key innovations: Shared On-device Tracking to merge the similar computation for optimized latency, Tightly Coupled Dual Pipeline to enhance the accuracy of each task through mutual assistance, Body Affinity Particle Filter to precisely match device poses with human bodies. We implement DiTing on four types of mobile AR devices and develop a multi-user AR game as a case study. Extensive experiments show that DiTing can provide high-quality human keypoint detection and pose estimation in real-time (30fps) for ID-aware multi-device interaction and outperform the SOTA baseline approaches.

Non-invasive human vital sign detection has gained significant attention in recent years, with its potential for contactless, long-term monitoring. Advances in radar systems have enabled non-contact detection of human vital signs, emerging as a crucial area of research. The movements of key human organs influence radar signal propagation, offering researchers the opportunity to detect vital signs by analyzing received electromagnetic (EM) signals. In this review, we provide a comprehensive overview of the current state-of-the-art in millimeter-wave (mmWave) sensing for vital sign detection. We explore human anatomy and various measurement methods, including contact and non-contact approaches, and summarize the principles of mmWave radar sensing. To demonstrate how EM signals can be harnessed for vital sign detection, we discuss four mmWave-based vital sign sensing (MVSS) signal models and elaborate on the signal processing chain for MVSS. Additionally, we present an extensive review of deep learning-based MVSS and compare existing studies. Finally, we offer insights into specific applications of MVSS ( e.g. , biometric authentication) and highlight future research trends in this domain.

Data imputation is prevalent in crowdsensing, especially for Internet of Things (IoT) devices. On the one hand, data collected from sensors will inevitably be affected or damaged by unpredictability. On the other hand, extending the active time of sensor networks has urgently aspired environmental monitoring. Using neural networks to design a data imputation algorithm can take advantage of the prior information stored in the models. This paper proposes a preprocessing algorithm to extract a subset for training a neural network on an IoT dataset, including time window determination, sensor aggregation, sensor exclusion and data frame shape selection. Moreover, we propose a data imputation algorithm using deep compressed sensing with generative models. It explores novel representation matrices and can impute data in the case of a high missing ratio situation. Finally, we test our subset extraction algorithm and data imputation algorithm on the EPFL SensorScope dataset, respectively, and they effectively improve the accuracy and robustness even with extreme data loss.

Gait is regarded as a unique feature for identifying people, and gait recognition is the basis of various customized services of the IoT. Unlike traditional techniques for identifying people, the Wi-Fi-based technique is unconstrained by illumination conditions and such that it eliminates the need for dense, specialized sensors and wearable devices. Although deep learning-based sensing models are conducive to the development of Wi-Fi-based identification, the latter technique relies on a large amount of data and requires a long training time, where this limits the scope of its use for identifying people. In this study, we propose a Wi-Fi sensing model called Wave-CapNet for human identification. We use data processing to eliminate errors in the raw data so that the model can extract the characteristics in channel state information (CSI). We also design a dedicated adaptive wavelet neural network to extract representative features from Wi-Fi signals with only a few epochs of training and a small number of parameters. Experiments show that it can identify human gait with an average accuracy of 99%. Moreover, it can achieve an average accuracy of 95% by using only 10% of the data and fewer than five epochs, and outperforms state-of-the-art (SOTA) methods.

Cardiac patterns are being used to provide hard-to-forge biometric signatures in identification applications. However, this performance is obtained under controlled scenarios where cardiac signals maintain a relatively uniform pattern, facilitating the identification process. In this work, we analyze cardiac signals collected in more realistic (uncontrolled) scenarios and show that their high signal variability makes them harder to obtain stable and distinct features. When faced with these irregular signals, the state-of-the-art (SOTA) reduces its performance significantly. To solve these problems, we propose the CardioID framework 1 with two novel properties. First, we design an adaptive method that achieves stable and distinct features by tailoring the filtering process according to each user’s heart rate. Second, we show that users can have multiple cardiac morphologies, offering us a bigger pool of cardiac signals compared to the SOTA. Considering three uncontrolled datasets, our evaluation shows two main insights. First, while using a PPG sensor with healthy individuals, the SOTA’s balanced accuracy (BAC) reduces from 90-95% to 75-80%, while our method maintains a BAC above 90%. Second, under more challenging conditions (using smartphone cameras or monitoring unhealthy individuals), the SOTA’s BAC reduces to values between 65-75%, and our method increases the BAC to values between 75-85%.

Local differential privacy (LDP) is a promising privacy model for distributed data collection. It has been widely deployed in real-world systems (e.g. Chrome, iOS, macOS). In LDP-based mechanisms, an aggregator collects private values perturbed by each user and then analyses these values to estimate their statistics, such as frequency and mean. Most existing works focus on simple scalar value types, such as boolean and categorical values. However, with the emergence of smart sensors and Internet of Things, high-dimensional data are gaining increasing popularity. In many cases where more than one type of sensor data are collected simultaneously, correlations exist between various attributes of such data, e.g. temperature and luminance. To ensure LDP for high-dimensional data, existing solutions either partition the privacy budget ϵ among these correlated attributes or adopt sampling, both of which dilute the density of useful information and thus result in poor data utility. In this paper, we propose a relaxed LDP model, namely, univariate dominance local differential privacy (UDLDP), for high-dimensional data. We quantify the correlations between attributes and present a correlation-bounded perturbation (CBP) mechanism that optimizes the partitioning of privacy budget on each correlated attribute. Furthermore, we extend CBP to support sampling, which is a common bandwidth reduction technique in sensor networks and Internet of Things. We derive the best allocation strategy of sampling probabilities among attributes in terms of data utility, which leads to the correlation-bounded perturbation mechanism with sampling (CBPS). Finally, we discuss how to collect and leverage the correlation from real-time data stream with a by-round algorithm to enhance the utility. The performance of the proposed mechanisms is evaluated and compared with state-of-the-art LDP mechanisms on real-world and synthetic datasets.

Cooperative edge caching enables edge servers to jointly utilize their cache to store popular contents, thus drastically reducing the latency of content acquisition. One fundamental problem of cooperative caching is how to coordinate the cache replacement decisions at edge servers to meet users’ dynamic requirements and avoid caching redundant contents. Online deep reinforcement learning (DRL) is a promising way to solve this problem by learning a cooperative cache replacement policy using continuous interactions (trial and error) with the environment. However, the sampling process of the interactions is usually expensive and time-consuming, thus hindering the practical deployment of online DRL-based methods. To bridge this gap, we propose a novel Delay-awarE Cooperative cache replacement method based on Offline deep Reinforcement learning (DECOR), which can exploit the existing data at the mobile edge to train an effective policy while avoiding expensive data sampling in the environment. A specific convolutional neural network is also developed to improve the training efficiency and cache performance. Experimental results show that DECOR can learn a superior offline policy from a static dataset compared to an advanced online DRL-based method. Moreover, the learned offline policy outperforms the behavior policy used to collect the dataset by up to 35.9%.

This paper proposes a sensor data anonymization model that is trained on decentralized data and strikes a desirable trade-off between data utility and privacy, even in heterogeneous settings where the sensor data have different underlying distributions. Our anonymization model, dubbed Blinder, is based on a variational autoencoder and one or multiple discriminator networks trained in an adversarial fashion. We use the model-agnostic meta-learning framework to adapt the anonymization model trained via federated learning to each user’s data distribution. We evaluate Blinder under different settings and show that it provides end-to-end privacy protection on two IMU datasets at the cost of increasing privacy loss by up to \(4.00\% \) and decreasing data utility by up to \(4.24\% \) , compared to the state-of-the-art anonymization model trained on centralized data. We also showcase Blinder’s ability to anonymize the radio frequency sensing modality. Our experiments confirm that Blinder can obscure multiple private attributes at once, and has sufficiently low power consumption and computational overhead for it to be deployed on edge devices and smartphones to perform real-time anonymization of sensor data.

Enabled by the increasingly available embedded hardware accelerators, the capability of executing advanced machine learning models at the edge of the Internet of Things (IoT) triggers interest of applying Artificial Intelligence of Things (AIoT) systems for industrial applications. The in situ inference and decision made based on the sensor data allow the industrial system to address a variety of heterogeneous, local-area non-trivial problems in the last hop of the IoT networks. Such a scheme avoids the wireless bandwidth bottleneck and unreliability issues, as well as the cumbersome cloud. However, the literature still lacks presentations of industrial AIoT system developments that provide insights into the challenges and offer lessons for the relevant research and industry communities. In light of this, we present the design, deployment, and evaluation of an industrial AIoT system for improving the quality control of HP Inc.’s ink cartridge manufacturing lines. While our development has obtained promising results, we also discuss the lessons learned from the whole course of the work, which could be useful to the developments of other industrial AIoT systems for quality control in manufacturing.

The growing popularity of contact-free smart sensing has contributed to the development of the Artificial Intelligence of Things (AIoT). The contact-free sensory data has great potential to mine and analyze the hidden information for AIoT-enabled applications. However, due to the limited storage resource of contact-free smart sensing devices, data is naturally stored in the cloud, which is at risk of privacy leakage. Cloud storage is generally considered insecure. On the one hand, the openness of the cloud environment makes the data easy to be attacked, and the complex AIoT environment also makes the data transmission process vulnerable to the third party. On the other hand, the Cloud Service Provider (CSP) is untrusted. In this paper, to ensure the security of data from contact-free smart sensing devices, a Cloud-Edge-End cooperative storage scheme is proposed, which takes full advantage of the differences in the cloud, edge, and end. Firstly, the processed sensory data is stored separately in the three layers by utilizing well-designed data partitioning strategy. This scheme can increase the difficulty of privacy leakage in the transmission process and avoid internal and external attacks. Besides, the contact-free sensory data is highly time-dependent. Therefore, combined with the Cloud-Edge-End cooperation model, this paper proposes a delta-based data update method and extends it into a hybrid update mode to improve the synchronization efficiency. Theoretical analysis and experimental results show that the proposed cooperative storage method can resist various security threats in bad situations and outperform other update methods in synchronization efficiency, significantly reducing the synchronization overhead in AIoT.