Average Average Age Of Information Research Articles

Deep Reinforcement Learning-Driven UAV Data Collection Path Planning: A Study on Minimizing AoI

As a highly efficient and flexible data collection device, Unmanned Aerial Vehicles (UAVs) have gained widespread application because of the continuous proliferation of Internet of Things (IoT). Addressing the high demands for timeliness in practical communication scenarios, this paper investigates multi-UAV collaborative path planning, focusing on the minimization of weighted average Age of Information (AoI) for IoT devices. To address this challenge, the multi-agent twin delayed deep deterministic policy gradient with dual experience pools and particle swarm optimization (DP-MATD3) algorithm is presented. The objective is to train multiple UAVs to autonomously search for optimal paths, minimizing the AoI. Firstly, considering the relatively slow learning speed and susceptibility to local minima of neural network algorithms, an improved particle swarm optimization (PSO) algorithm is utilized for parameter optimization of the multi-agent twin delayed deep deterministic policy gradient (MATD3) neural network. Secondly, with the introduction of the dual experience pools mechanism, the efficiency of network training is significantly improved. Experimental results show DP-MATD3 outperforms MATD3 in average weighted AoI. The weighted average AoI is reduced by 33.3% and 27.5% for UAV flight speeds of v = 5 m/s and v = 10 m/s, respectively.

Resource allocation and passive beamforming for IRS‐assisted short packet systems

AbstractThis paper investigates an intelligent reflecting surface (IRS) assisted downlink short packet transmission system, where an access point sends short packets to multiple devices with the help of an IRS. Specifically, a performance comparison between the frequency division multiple access and time division multiple access is conducted for the considered system, from the perspective of average age of information (AoI). To minimize the maximum average AoI among all devices, the resource allocation and passive beamforming are jointly optimized. However, the formulated problem is difficult to solve due to the non‐convex objective function and coupled variables. Thus, an alternating optimization based algorithm is proposed by exploiting the semidefinite relaxation and bisection search techniques. Simulation results show that time division multiple access can achieve lower AoI by exploiting the time‐selective passive beamforming of IRS for maximizing the signal to noise ratio of each device consecutively. Moreover, it also shows that as the length of information bits becomes sufficiently large as compared to the available bandwidth, the proposed frequency division multiple access transmission scheme becomes more favourable due to more flexible power allocation.

Enhancing information freshness in multi-class mobile edge computing systems using a hybrid discipline

Timely status updating in mobile edge computing (MEC) systems has recently gained the utmost interest in internet of things (IoT) networks, where status updates may need higher computations to be interpreted. Moreover, in real-life situations, the status update streams may also be of different priority classes according to their importance and timeliness constraints. The classical disciplines used for priority service differentiation, preemptive and non-preemptive disciplines, pose a dilemma of information freshness dissatisfaction for the whole priority network. This work proposes a hybrid preemptive/non-preemptive discipline under an M/M/1/2 priority queueing model to regulate the priority-based contention of the status update streams in MEC systems. For this hybrid discipline, a probabilistic discretionary rule for preemption is deployed to govern the server and buffer access independently, introducing distinct probability parameters to control the system performance. The stochastic hybrid system approach is utilized to analyze the average age of information (AoI) along with its higher moments for any number of classes. Then, a numerical study on a three-class network is conducted by evaluating the average AoI performance and the corresponding dispersion. The numerical observations underpin the significance of the hybrid-discipline parameters in ensuring the reliability of the whole priority network. Hence, four different approaches are introduced to demonstrate the setting of these parameters. Under these approaches, some outstanding features are manifested: exploiting the buffering resources efficiently, conserving the aggregate sensing power, and optimizing the whole network satisfaction. For this last feature, a near-optimal low-complex heuristic method is proposed.

On the Adaptive Buffer-Aided TDMA Uplink System with AoI-Aware Status-Update Services and Timely Throughput Traffics.

In this paper, we study a buffer-aided TDMA uplink network, where multiple status-update devices and throughput-demand devices are supposed to upload their data to one information access point (AP), and all devices are assumed to be provisioned with a data buffer to temporarily store the randomly generated data from either the installed sensor or upper-layer applications. To fulfill the communication requirements using two types of devices, the average Age of Information (AoI) is utilized to characterize the data freshness of the status-update devices, while the average sum rate is employed to capture the average transmission performance of the throughput-demand devices. On this basis, a joint-optimization problem was formulated to minimize the average AoI for status-update devices and to maximize the average sum rate for the throughput-demand devices. Lyapunov optimization framework was used to solve the problem of obtaining an AoI-aware adaptive TDMA uplink scheme. Numerical results are presented to show that an AoI-aware adaptive TDMA uplink scheme can effectively fulfill the heterogeneous service requirements using status-update devices and throughput-demand devices.

Analysis of Age of Information in Dual Updating Systems

We study the average Age of Information (AoI) and peak AoI (PAoI) of a dual-queue status update system that monitors a common stochastic process through two independent channels. Although the double queue parallel transmission is instrumental in reducing AoI, the out of order of data arrivals also imposes a significant challenge to the performance analysis. We consider two settings: the M-M system where the service time of two servers is exponentially distributed; the M-D system in which the service time of one server is exponentially distributed and that of the other is deterministic. For the two dual-queue systems, closed-form expressions of average AoI and PAoI are derived by resorting to the graphic method and state flow graph analysis method. Our analysis reveals that when the two servers have the same service rate, compared with the single-queue system with an exponentially distributed service time, the average PAoI and the average AoI of the M-M system decrease by 33.3% and 37.5%, respectively, and those of the M-D system decrease by 27.7% and 39.7%, respectively. Numerical results show that the two dual-queue systems also outperform the M/M/2 single queue dual-server system with optimized arrival rate in terms of average AoI and PAoI.

Hierarchical Deep Reinforcement Learning for Age-of-Information Minimization in IRS-Aided and Wireless-Powered Wireless Networks

In this paper, we focus on a wireless-powered sensor network coordinated by a multi-antenna access point (AP). Each node can generate sensing information and report the latest information to the AP using the energy harvested from the AP’s signal beamforming. We aim to minimize the average age-of-information (AoI) by adapting the nodes’ scheduling and the transmission control strategies jointly. To reduce the transmission delay, an intelligent reflecting surface (IRS) is used to enhance the channel conditions by controlling the AP’s beamforming strategy and the IRS’s phase shifting matrix. Considering dynamic data arrivals at different sensing nodes, we propose a hierarchical deep reinforcement learning (DRL) framework to for AoI minimization in two steps. The users’ transmission scheduling is firstly determined by the outer-loop DRL approach, e.g. the DQN or PPO algorithm, and then the inner-loop optimization is used to adapt either the uplink information transmission or downlink energy transfer to all nodes. A simple and efficient approximation is also proposed to reduce the inner-loop rum time overhead. Numerical results verify that the hierarchical learning framework outperforms typical baselines in terms of the average AoI and proportional fairness among different nodes.

Adaptive Packet Length Adjustment for Minimizing Age of Information Over Fading Channels

Motivated by the high information freshness requirement in the Internet of Things (IoT), this paper investigates adaptive packet length adjustment schemes to minimize the average age of information (AoI) for status update systems, where a machine type communication device adaptively adjusts the packet length in real time by exploiting the channel state information and AoI. Since the status packets in the IoT are often short, a significant packet error rate is introduced. Due to the instability of channel fading and the high packet error rate, optimizing the AoI performance is tricky. Under a power consumption constraint, the AoI minimization problem is modeled as a constrained Markov decision process (CMDP), and the structure of the optimal scheme is revealed. Then, under the CMDP framework, this paper transforms the AoI minimization problem into a linear programming problem and proposes a probabilistic packet length adjustment scheme, which can lead to the optimal solution. When the power consumption constraint is loose, a low-complexity suboptimal scheme is further proposed, where the expected average AoI of one period length is minimized. Simulation results verify the superiority of the proposed optimal scheme and show that the proposed low-complexity scheme can reach near-optimal performance.

Average Age-of-Information Minimization in Aerial IRS-Assisted Data Delivery

Aerial intelligent reconfigurable surface (IRS) is a promising technology to enhance channel quality in data delivery. In this paper, we study an aerial IRS deployment problem to enable timely and reliable data delivery in a remote Internet of Things (IoT) scenario, in which an IRS mounted on an unmanned aerial vehicle (UAV) is adopted as a mobile relay to assist devices in uploading data to the base station (BS). The objective is to minimize the average age of information (AoI) of the data received by the BS over time by jointly determining the aerial IRS deployment position and phase shift, transmit power of devices, and data uploading time. Under the requirements of peak AoI (PAoI) and communication reliability, we formulate an average AoI minimization problem. Since the non-linear relations among optimization variables make the formulated problem non-convex and intractable to solve, we propose a block coordinate descent (BCD) based iterative algorithm which decomposes the formulated problem into several sub-problems. The variables are optimized in each sub-problem individually in an alternately iterative manner to attain a near-optimal solution. Simulation results demonstrate the superiority of the proposed algorithm in improving the information freshness compared with the benchmark schemes.

Average AoI Minimization for Energy Harvesting Relay-Aided Status Update Network Using Deep Reinforcement Learning

A dual-hop status update system aided by energy-harvesting (EH) relays with finite data and energy buffers is studied in this work. To achieve timely status updates, the best relays should be selected to minimize the average age of information (AoI), which is a recently proposed metric to evaluate information freshness. The average AoI minimization can be formulated as a Markov decision process (MDP), but the state space for capturing channel and buffer evolution grows exponentially with the number of relays, leading to high solution complexity. We propose a relay selection (RS) scheme based on deep reinforcement learning (DRL) according to the instantaneous channel packet freshness and buffer information of each relay. Simulation results show a significant improvement of the proposed DRL-based RS scheme over state-of-art approaches.

Cooperative Data Collection With Multiple UAVs for Information Freshness in the Internet of Things

Maintaining the freshness of information in the Internet of Things (IoT) is a critical yet challenging problem. In this paper, we study cooperative data collection using multiple Unmanned Aerial Vehicles (UAVs) with the objective of minimizing the total average Age of Information (AoI). We consider various constraints of the UAVs, including kinematic, energy, trajectory, and collision avoidance, in order to optimize the data collection process. Specifically, each UAV, which has limited on-board energy, takes off from its initial location and flies over sensor nodes to collect update packets in cooperation with the other UAVs. The UAVs must land at their final destinations with non-negative residual energy after the specified time duration to ensure they have enough energy to complete their missions. It is crucial to design the trajectories of the UAVs and the transmission scheduling of the sensor nodes to enhance information freshness. We model the multi-UAV data collection problem as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), as each UAV is unaware of the dynamics of the environment and can only observe a part of the sensors. To address the challenges of this problem, we propose a multi-agent Deep Reinforcement Learning (DRL)-based algorithm with centralized learning and decentralized execution. In addition to the reward shaping, we use action masks to filter out invalid actions and ensure that the constraints are met. Simulation results demonstrate that the proposed algorithms can significantly reduce the total average AoI compared to the baseline algorithms, and the use of the action mask method can improve the convergence speed of the proposed algorithm.

Average AoI minimization for data collection in UAV-enabled IoT backscatter communication systems with the finite blocklength regime

Thanks to the autonomy of unmanned aerial vehicle (UAV), UAV-enabled data collection in Internet of Things (IoT) networks has become a key application for the next generation communication network. In this paper, we consider a scenario where an UAV is responsible for collecting data from sensor equipments (SEs) one by one and finally carrying the collected data to the computation center for processing. Different from the commonly used assumption that SEs always generate data at the beginning of each time slot, it is assumed that SEs can generate data at any instant during one time slot, which is more practical. Since SEs are energy limited, they upload data to the UAV by adopting backscatter communication technology to reduce energy consumption. Meanwhile, the updated information usually contains a small number of information bits but requires low latency and high reliability, thus the finite blocklength regime in ultra-reliable and low-latency communication is adopted for SEs’ data transmission to the UAV. To keep the freshness of the updated information, a joint resource allocation problem including data collection time allocation, transmission power and trajectory design of the UAV is formulated as an optimization problem to minimize the average age of information (AoI) of all SEs. The formulated problem mixes discrete and continuous variables, which makes it difficult to solve. Thereby, we decompose the optimization problem into data collection time minimization subproblem and UAV trajectory design subproblem, which are solved by the successive convex approximation method, and the backtracking algorithm and the genetic algorithm, respectively. Numerical results show that the backtracking-based algorithm that can obtain the optimal trajectory of the UAV gains the minimal average AoI, and the genetic-based algorithm achieves sub-optimal average AoI with much lower computational complexity. The results also demonstrate that the average AoI of the backtracking-based algorithm and the genetic-based algorithm is reduced by up to 54% and 46% compared with the greedy-based benchmark algorithm, respectively.

Impact of Channel Memory on the Data Freshness

In this letter, we investigate the impact of channel memory on the average age of information (AoI) for networks with various packet arrival models under the first-come-first-served (FCFS) and the preemptive last-generated-first-served (pLGFS) policies over the Gilbert-Elliott (GE) erasure channel. For networks with Bernoulli and generate-at-will arrival models, the AoI performances under the FCFS and pLGFS policies are derived explicitly as functions of the channel state transition probabilities. For networks with periodic arrival model, we derive the closed-form expression for the average AoI under pLGFS and propose a numerical algorithm for efficient evaluation of the AoI under FCFS. It is revealed that for pLGFS policy, the average AoI increases monotonically with channel memory <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\eta $ </tex-math></inline-formula> at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\frac {\eta }{1-\eta }$ </tex-math></inline-formula> over the symmetric GE channel. For FCFS, the average AoI increases even faster due to the queuing delay, with an additional term related to the packet arrival rate.

AoI Minimization in Energy Harvesting and Spectrum Sharing Enabled 6G Networks

Spectrum sharing is a method to solve the problem of frequency spectrum deficiency. This paper studies a novel spectrum sharing and energy harvesting system using artificial intelligence (AI) in which the freshness of information is guaranteed. The system is based on licensed shared access (LSA) which includes a primary user with access rights to the spectrum and a secondary user. The secondary user is an energy harvesting sensor that intends to use the primary user’s spectrum opportunistically. The problem is formulated as partially observable Markov decision processes (POMDPs) and solved using two methods: 1) a deep Q-network (DQN) and 2) a dueling double deep Q-Network (D3QN) to achieve the optimal policy. The purpose is to choose the best action adaptively in every time slot based on the user’s observations from the environment in overlay and underlay modes to minimize the average AoI of the secondary user. Finally, the simulation analyses are performed to evaluate the effectiveness of the proposed scheme compared to the overlay mode. According to the results, the average age of information (AoI) in the proposed system is less than that of the existing models, including only the overlay mode. The average user access is improved from 30% in the overlay mode to 45% and 48% in the proposed scheme which is implemented using DQN and D3QN, respectively.

On-Demand AoI Minimization in Resource-Constrained Cache-Enabled IoT Networks With Energy Harvesting Sensors

We consider a resource-constrained IoT network, where multiple users make on-demand requests to a cache-enabled edge node to send status updates about various random processes, each monitored by an energy harvesting sensor. The edge node serves users’ requests by deciding whether to command the corresponding sensor to send a fresh status update or retrieve the most recently received measurement from the cache. Our objective is to find the best actions of the edge node to minimize the average age of information (AoI) of the received measurements upon request, i.e., average on-demand AoI, subject to per-slot transmission and energy constraints. First, we derive a Markov decision process model and propose an iterative algorithm that obtains an optimal policy. Then, we develop an asymptotically optimal low-complexity algorithm – termed relax-then-truncate – and prove that it is optimal as the number of sensors goes to infinity. Simulation results illustrate that the proposed relax-then-truncate approach significantly reduces the average on-demand AoI compared to a request-aware greedy policy and a weighted AoI policy, and also depict that it performs close to the optimal solution even for moderate numbers of sensors.

Exact Analytical Model of Age of Information in Multi-Source Status Update Systems With Per-Source Queueing

We study a multisource status update system with Poisson information packet arrivals and exponentially distributed service times. The server is equipped with a waiting room holding the freshest packet from each source referred to as single buffer per-source queueing (SBPSQ). The sources are assumed to be equally important, i.e., (nonweighted) average Age of Information (AoI) or average age violation probability are used as the information freshness metrics to optimize for, and subsequently, two symmetric SBPSQ-based scheduling policies are studied in this article, namely, first source first serve (FSFS) and the earliest served first serve (ESFS) policies. By employing the theory of Markov fluid queues (MFQs), an analytical model is proposed to obtain the exact distribution of the AoI for each source when the FSFS and ESFS policies are employed at the server. Additionally, a benchmark scheduling-free scheme named single buffer with replacement (SBR), which uses a single buffer to hold the freshest packet across all sources, is also studied with a similar but less complex analytical model. We comparatively study the performance of the three policies through numerical examples in terms of the average AoI and the age violation probability averaged across all sources, in a scenario of sources possessing different traffic intensities but sharing a common service time.

Non-Orthogonal Multiple Access-Based Average Age of Information Minimization in LEO Satellite-Terrestrial Integrated Networks

In this paper, we investigate non-orthogonal multiple access (NOMA) based average age of information (AoI) in LEO satellite-terrestrial integrated networks. Two-user NOMA scheme is used in terrestrial networks and several satellites provide orthogonal access for the rest of users to update status packets. Firstly, the closed-form expressions of average AoI in terrestrial networks and satellite channels are derived with average block error rate. Then, the average AoI minimization problem is decomposed into two parts: 1) average AoI minimization in terrestrial networks and 2) average AoI minimization among satellites. In the first part, a base station-terrestrial user selection and transmit power optimization algorithm is proposed for choosing the optimal NOMA pair and transmit power to minimize average AoI in each BS. A coalition formation game (CFG) is formulated to minimize average AoI among satellites in second part. With the proposed Min-AoI order, the CFG is theoretically proven to be an exact potential game, which can achieve total average AoI minimization of all satellite users. A distributed CFG-based algorithm is proposed for SUs grouping with low complexity. Finally, numerical results are provided to verify the correctness of theoretical analysis and evaluate our proposed algorithms, compared with other schemes.

Age-of-Information in First-Come-First-Served Wireless Communications: Upper Bound and Performance Optimization

This article establishes an analytical framework for the upper bound on the average Age-of-Information (AoI) in first-come-first-served (FCFS) wireless communications where a certain level of outage probability is unavoidable. To begin with, we analyze the average AoI and derive a general upper bound for G/G/1 systems with a certain outage probability. Subsequently, for an M/M/1 system with the FCFS scheme, we obtain a concise closed-form expression of the upper bound, and further refine the upper bound after analyzing the relative error. Interestingly, it is found by the analysis that the relative error is independent of the service rate, and the upper bound becomes tighter as the outage probability increases. Based on the refined upper bound, we minimize the average AoI for the communications suffering from block Rayleigh fading. We derive a closed-form expression of the outage probability over a fading channel, and then prove that the refined upper bound is a convex function with respect to the average update generating rate. Consequently, we optimize the AoI performance by solving a convex optimization problem formulated utilizing the refined upper bound expression. The numerical results indicate that the minimum average AoI can be reduced by either increasing the service rate or the transmission power.

Average Age of Information in Wireless Powered Mobile Edge Computing System

Mobile edge computing (MEC) has been recognized as a promising technique to provide enhanced computation services for low-power wireless devices at the network edge. How to evaluate the timeliness of the task and data delivery is critical for the development of MEC applications. Considering a wireless powered MEC system, in this letter we study the average age of information (AoI), which is a crucial performance metric to measure the freshness of information. Specifically, in the considered system, a sensor node harvests energy from an energy transmitter and transmits computation tasks to the MEC server. The charging time of the sensor node’s capacitor, the waiting time and service time at the MEC server are taken into account when calculating the average AoI. The closed-form expression of the average AoI is obtained accordingly and evaluated through numerical simulations.

Information freshness and packet drop rate interplay in a two-user multi-access channel

In this work, we combine the two notions of timely delivery of information to study their interplay; namely, deadline-constrained packet delivery due to latency constraints and freshness of information. More specifically, we consider a two-user multiple access setup with random access, in which user 1 is a wireless device with a buffer and has external bursty traffic which is deadline-constrained, while user 2 monitors a sensor and transmits status updates to the destination. We provide analytical expressions for the throughput and drop probability of user 1. For user 2, we derive in closed form the age of information distribution, the average age of information (AoI), and the probability the AoI to be larger than a certain value for each time slot. The relations reveal a trade-off between the average AoI of user 2 and the drop rate of user 1: the lower the average AoI, the higher the drop rate, and vice versa. Simulations corroborate the validity of our theoretical results.

Performance Analysis of Age of Information in Ultra-Dense Internet of Things (IoT) Systems With Noisy Channels

In this paper, a dense Internet of Things (IoT) monitoring system is studied in which a large number of devices contend for transmitting timely status packets to their corresponding receivers over wireless noisy channels, using a carrier sense multiple access (CSMA) scheme. When each device completes one transmission, due to possible transmission failure, two cases with and without transmission feedback to each device must be considered. Particularly, for the case with no feedback, the device uses policy (I): It will go to an idle state and release the channel regardless of the outcome of the transmission. For the case with perfect feedback, if the transmission succeeds, the device will go to an idle state, otherwise it uses either policy (W), i.e., it will go to a waiting state and re-contend for channel access; or it uses policy (S), i.e., it will stay at a service state and occupy this channel to attempt another transmission. For those three policies, the closed-form expressions of the average age of information (AoI) of each device are characterized under schemes with and without preemption in service. It is shown that, for each policy, the scheme with preemption in service always achieves a smaller average AoI, compared with the scheme without preemption. Then, a mean-field approximation approach with guaranteed accuracy is developed to analyze the asymptotic performance for the considered system with an infinite number of devices and the effects of the system parameters on the average AoI are characterized. Simulation results show that the proposed mean-field approximation is accurate even for a small number of devices. The results also show that olicy (S) achieves the smallest average AoI compared with policies (I) and (W), and the average AoI does not always decrease with the arrival rate for all three policies.