Abstract
We study the problem of handling timeliness and criticality trade-off when gathering data from multiple resources in complex environments. In IoT environments, where several sensors transmitting data packets - with various criticality and timeliness, the rate of data collection could be limited due to associated costs (e.g., bandwidth limitations and energy considerations). Besides, environment complexity regarding data generation could impose additional challenges to balance criticality and timeliness when gathering data. For instance, when data packets (either regarding criticality or timeliness) of two or more sensors are correlated, or there exists temporal dependency among sensors, incorporating such patterns can expose challenges to trivial policies for data gathering. Motivated by the success of the Asynchronous Advantage Actor-Critic (A3C) approach, we first mapped vanilla A3C into our problem to compare its performance in terms of criticality-weighted deadline miss ratio to the considered baselines in multiple scenarios. We observed degradation of the A3C performance in complex scenarios. Therefore, we modified the A3C network by embedding long short term memory (LSTM) to improve performance in cases that vanilla A3C could not capture repeating patterns in data streams. Simulation results show that the modified A3C reduces the criticality-weighted deadline miss ratio from 0.3 to 0.19.
Highlights
Connected devices are part of Internet of Things (IoT) environments in which every device talks to other related devices - by gathering and transmitting data - to timely communicate important/critical sensor data to interested parties for further usage
OVERVIEW OF DEEP REINFORCEMENT LEARNING and before discussing the A3C network, we first introduce the basic concepts of RL and Deep Learning (DL), based on which Deep Reinforcement Learning (DRL) is defined
QUANTITATIVE EVALUATION Having described the proposed approach based on deep reinforcement learning, we discuss other heuristics that we can use to compare with our proposal
Summary
Connected devices are part of IoT environments in which every device talks to other related devices - by gathering and transmitting data - to timely communicate important/critical sensor data to interested parties for further usage. We propose an approach based on the Asynchronous Advantage Actor-Critic (A3C) [13] by improving the network structure such that it captures the likely case of having temporal dependency and correlation within data streams We achieved this improvement by adding an LSTM layer to consider some previous states (rather than only one state) to learn recurring patterns within data. CONTEXT The Internet of Things (IoT) refers to the vast number of things (i.e., electronics-infused devices) connected to the internet, which acts as sensors in their hosting environment, generating massive volumes of data In such settings, everything from data acquisition to processing and analysis can leverage Machine Learning (ML) techniques to preserve efficiency and performance. Such observation confirms the advantage of the proposed solution over the vanilla A3C in all studied scenarios
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
