Random Packet Arrivals Research Articles

Downlink Scheduler for Delay Guaranteed Services Using Deep Reinforcement Learning

In this paper, we propose a novel scheduling scheme to guarantee per-packet delay in single-hop wireless networks for delay-critical applications. We consider several classes of packets with different delay requirements, where high-class packets yield high utility after successful transmission. Considering the correla-tionship of delays among competing packets, we apply a delay-laxity concept and introduce a new output gain function for scheduling decisions. Particularly, the selection of a packet takes into account not only its output gain but also the delay-laxity of other packets. In this context, we formulate a multi-objective optimization problem aiming to minimize the average queue length while maximizing the average output gain under the constraint of guaranteeing per-packet delay. However, due to the uncertainty in the environment (e.g., time-varying channel conditions and random packet arrivals), it is difficult and often impractical to solve this problem using traditional optimization techniques. We develop a deep reinforcement learning (DRL)-based framework to solve it. Specifically, we decompose the original optimization problem into a set of scalar optimization subproblems and model each of them as a partially observable Markov Decision Process (POMDP). We then resort to a Double Deep Q Network (DDQN)-based algorithm to learn an optimal scheduling policy for each subproblem, which can overcome the large-scale state space and reduce Q-value overestimation. Simulation results show that our proposed DDQN-based algorithm outperforms the conventional Q-learning algorithm in terms of reward and learning speed. In addition, our proposed scheduling scheme can achieve significant reductions in average delay and delay outage drop rate compared to other benchmark schemes.

Age of Information Cost Minimization with No Buffers, Random Arrivals and Unreliable Channels: A PCL-Indexability Analysis

Over the last decade, the Age of Information has emerged as a key concept and metric for applications where the freshness of sensor-provided data is critical. Limited transmission capacity has motivated research on the design of tractable policies for scheduling information updates to minimize Age of Information cost based on Markov decision models, in particular on the restless multi-armed bandit problem (RMABP). This allows the use of Whittle’s popular index policy, which is often nearly optimal, provided indexability (index existence) is proven, which has been recently accomplished in some models. We aim to extend the application scope of Whittle’s index policy in a broader AoI scheduling model. We address a model with no buffers incorporating random packet arrivals, unreliable channels, and nondecreasing AoI costs. We use sufficient indexability conditions based on partial conservation laws previously introduced by the author to establish the model’s indexability and evaluate its Whittle index in closed form under discounted and average cost criteria. We further use the index formulae to draw insights on how scheduling priority depends on model parameters.

Simple Bounds on Delay-Constrained Capacity and Delay-Violation Probability of Joint Queue and Channel-Aware Wireless Transmissions

The emerging Ultra-Reliable and Low-Latency Communication (URLLC) is expected to meet a hard or probabilistic delay constraint that plays a key role in Time Sensitive Networking (TSN) and Deterministic Networking (DetNet). In this paper, we are interested in simple bounds for delay-constrained wireless communications with deterministic or even random packet arrivals. More specifically, we present a sufficient condition and derive a legitimate data arrival rate, with which the bounded delay can be guaranteed deterministically with an average power constraint. Our derived results will show that the delay-bounded data rate increases with the average power and the tolerated latency normalized to the coherent time. Furthermore, we derive the upper and lower bounds of the delay-violation probability (DVP) when the bounded delay condition cannot be satisfied. It is revealed that in the log coordinate, the DVP as a function of the normalized tolerated latency may enjoy a non-linear decay, with a decay rate that increases with the latency bound. This is in contrast to the large deviation based performance analysis that focuses on the low-latency transmission policies achieving linearly decayed DVP only. Our result demonstrates that the joint channel-aware and queue-aware scheduling may significantly reduce the DVP, compared to the single-layer approaches.

Cross-layer distributed power control: a repeated game formulation to improve the sum energy efficiency

The main objective of this work is to improve the energy-efficiency (EE) of a multiple access channel (MAC) system, through power control, in a distributed manner. In contrast with many existing works on energy-efficient power control, which ignore the possible presence of a queue at the transmitter, we consider a new generalized cross-layer EE metric. This approach is relevant when the transmitters have a non-zero energy cost even when the radiated power is zero and takes into account the presence of a finite packet buffer and packet arrival at the transmitter. As the Nash equilibrium (NE) is an energy-inefficient solution, the present work aims at overcoming this deficit by improving the global energy-efficiency. Indeed, as the considered system has multiple agencies each with their own interest, the performance metric reflecting the individual interest of each decision maker is the global energy-efficiency defined then as the sum over individual energy-efficiencies. Repeated games (RG) are investigated through the study of two dynamic games (finite RG and discounted RG), whose equilibrium is defined when introducing a new operating point (OP), Pareto-dominating the NE and relying only on individual channel state information (CSI). Accordingly, closed-form expressions of the minimum number of stages of the game for finite RG (FRG) and the maximum discount factor of the discounted RG (DRG) were established. The cross-layer model in the RG formulation leads to achieving a shorter minimum number of stages in the FRG even for higher number of users. In addition, the social welfare (sum of utilities) in the DRG decreases slightly with the cross-layer model when the number of users increases while it is reduced considerably with the Goodman model. Finally, we show that in real systems with random packet arrivals, the cross-layer power control algorithm outperforms the Goodman algorithm.

Asymptotic Bounds on the Power-Delay Tradeoff for Fading Point-to-Point Links From Geometric Bounds on the Stationary Distribution of the Queue Length

The optimal power-delay tradeoff is studied for a time-slotted independently and identically distributed fading point-to-point link, with perfect channel state information at both transmitter and receiver, and with random packet arrivals to the transmitter queue. It is assumed that the transmitter can control the number of packets served by controlling the transmit power in the slot. The optimal tradeoff between average power and average delay is analyzed for stationary and monotone transmitter policies. For such policies, an asymptotic lower bound on the minimum average delay of the packets is obtained, when average transmitter power approaches the minimum average power required for transmitter queue stability. The asymptotic lower bound on the minimum average delay is obtained from geometric upper bounds on the stationary distribution of the queue length. This approach, which uses geometric upper bounds, also leads to an intuitive explanation of the asymptotic behavior of average delay. The asymptotic lower bounds, along with previously known asymptotic upper bounds, are used to identify three new cases where the order of the asymptotic behavior differs from that obtained from a previously considered approximate model, in which the transmit power is a strictly convex function of real valued service batch size for every fade state.

Impact of Random Packet Arrival on the Performance of Video Streaming over OFDMA Downlink Systems

Streaming multiple video sessions over wireless networks is challenging due to the scarcity of wireless resources and the channel variations. Orthogonal frequency-division multiple access is an app...

Performance analysis of a Multiuser Multi-Packet Transmission system for WLANs in non-saturation conditions

Multiuser Multi-Packet Transmission (MPT) from an Access Point (AP) equipped with multiple antennas to multiple single-antenna nodes can be achieved by exploiting the spatial dimension of the channel. In this paper we present a queueing model to analytically study such systems from the link-layer perspective, in presence of random packet arrivals, heterogeneous channel conditions and packet errors. The analysis relies on a blind estimation of the number of different destinations among the packets waiting in the queue, which allows for building a simple, but general model for MPT systems with per-node First-In First-Out (FIFO) packet scheduling. Simulation results validate the accuracy of the analytical model and provide further insights on the cross-relations between the channel state, the number of antennas, and the number of active users, as well as how they affect the system performance. The simplicity and accuracy of the model makes it suitable for the evaluation of Medium Access Control (MAC) protocols for Ad Hoc or Wireless Local Area Networks supporting Multiuser MPT in non-saturation conditions, where the queueing dynamics play an important role on the achieved performance, and simple user selection algorithms are required.

Dynamic Index Coding for Wireless Broadcast Networks

We consider a wireless broadcast station that transmits packets to multiple users. The packet requests for each user may overlap, and some users may already have certain packets. This presents a problem of broadcasting in the presence of side information, and is a generalization of the well known (and unsolved) index coding problem of information theory. Rather than achieving the full capacity region, we develop a code-constrained capacity region, which restricts attention to a pre-specified set of coding actions. We develop a dynamic max-weight algorithm that allows for random packet arrivals and supports any traffic inside the code-constrained capacity region. Further, we provide a simple set of codes based on cycles in the underlying demand graph. We show these codes are optimal for a class of broadcast relay problems.

Impact of Cognition and Cooperation on MAC Layer Performance Metrics, Part I: Maximum Stable Throughput

In this paper, we consider a broadband secondary transmitter-receiver pair which interferes with N narrowband primary users and study the effect of cognition and cooperation on the maximum stable throughput. In our study we focus on four transmission protocols as well as two channel types, i.e., flat fading and frequency selective fading. In the cooperative protocols, the broadband transmitter relays the packets of the primary users which have not correctly decoded at the primary receiver. The analysis includes random packet arrivals at the transmitters which may impact on the maximum stable throughput. Moreover, sensing errors at the secondary user are considered. In this paper, we derive the exact stability region for non-cooperative protocols and inner bounds for cooperative ones. The results reveal that depending on the channel states, the cooperative protocols may provide significant performance gains over non-cooperative protocols. Numerical results indicate that in the frequency selective channel, independent and simultaneous parallel transmissions in the available subbands are preferred, while in the flat fading channel, a non-parallel transmission in the total available bandwidth is recommended. In part II of our paper, we investigate the impact of cognition and cooperation on the delay performance of the protocols introduced in this paper.

Buffer-Aware Network Coding for Wireless Networks

Network coding, which can combine various traffic flows or packets via algebraic operations, has the potential of achieving substantial throughput and power efficiency gains in wireless networks. As such, it is considered as a powerful solution to meet the stringent demands and requirements of next-generation wireless systems. However, because of the random and asynchronous packet arrivals, network coding may result in severe delay and packet loss because packets need to wait to be network-coded with each others. To overcome this and guarantee quality of service (QoS), we present a novel cross-layer approach, which we shall refer to as Buffer-Aware Network Coding, or BANC, which allows transmission of some packets without network coding to reduce the packet delay. We shall derive the average delay and power consumption of BANC by presenting a random mapping description of BANC and Markov models of buffer states. A cross-layer optimization problem that minimizes the average delay under a given power constraint is then proposed and analyzed. Its solution will not only demonstrate the fundamental performance limits of BANC in terms of the achievable delay region and delay-power tradeoff, but also obtains the delay-optimal BANC schemes. Simulation results will show that the proposed approach can strike the optimal tradeoff between power efficiency and QoS.

On Coding for Delay—Network Coding for Time-Division Duplexing

In networks with large latency, feedback about received packets may lag considerably the transmission of the original packets, limiting the feedback's usefulness. Moreover, time duplex constraints may entail that receiving feedback may be costly. In this work, we consider tailoring feedback and coding jointly in such settings to reduce the expected delay for successful in order reception of packets. We find that, in certain applications, judicious choices provide results that are close to those that would be obtained with a full-duplex system. We study two cases of data transmission: one-to-all broadcast and all-to-all broadcast. We also analyze important practical considerations weighing the trade off between performance and complexity in applications that rely on random linear network coding. Finally, we study the problem of transmission of information under the large latency and time duplexing constraints in the presence of random packet arrivals. In particular, we analyze the problem of using a batch by batch approach and an online network coding approach with Poisson arrivals. We present numerical results to illustrate the performance under a variety of scenarios and show the benefits of the proposed schemes as compared to typical ARQ and scheduling schemes.

On-demand multicast routing protocol with efficient route discovery

In this paper, we introduce an efficient route discovery mechanism to enhance the performance and multicast efficiency of On-Demand Multicast Routing Protocol (ODMRP). Our framework, called limited flooding ODMRP, improves multicasting mechanism by efficiently managing flooding mechanism based on delay characteristics of the contributing nodes. In our model, only the nodes that satisfy the delay requirements can flood the Join-Query messages. We model the contributing nodes as M/M/1 queuing systems. Our framework considers the significant parameters in delay analysis, including random packet arrival, service process, and random channel access in the relying nodes, and exhibits its best performance results under high traffic load. Simulation results reveal that limited flooding ODMRP drastically reduces the packet overhead under various simulation scenarios as compared to original ODMRP.

Kalman filtering with partial Markovian packet losses

We consider the Kalman filtering problem in a networked environment where there are partial or entire packet losses described by a two state Markovian process. Based on random packet arrivals of the sensor measurements and the Kalman filter updates with partial packet, the statistical properties of estimator error covariance matrix iteration and stability of the estimator are studied. It is shown that to guarantee the stability of the Kalman filter, the communication network is required to provide for each of the sensor measurements an associated throughput, which captures all the rates of the successive sensor measurements losses. We first investigate a general discrete-time linear system with the observation partitioned into two parts and give sufficient conditions of the stable estimator. Furthermore, we extend the results to a more general case where the observation is partitioned into n parts. The results are illustrated with some simple numerical examples.

Low complexity resource allocation and scheduling algorithms for downlink SDMA systems

The downlink zero-forcing beamforming strategy in the case of random packet arrivals is investigated. Under this setting, the relevant fairness criterion is the stabilization of all buffer queues which guarantees a bounded average delay for all users. It has been shown that allocating resources to maximize a queue-length-weighted sum of the rates is a stabilizing policy. However, the high complexity of user selection and the feasible rates determination for optimal scheme may prevent the real-time scheduling operation. Two low complexity algorithms are provided taking the channel state, queue state and orthogonality into account. In particular, the authors pick the first user with the largest product between channel gain and queuing length, and select the remaining users to construct candidate user set based on the greedy user selection method or channel orthogonal user selection method. Then, the power and rate allocation for the selected users are implemented based on the modified water-filling method. The complexity of the proposed algorithms is analyzed. The average delay and average throughput are studied in homogeneous scenarios and heterogeneous scenarios, respectively. Simulation results show that the proposed algorithms can take full advantage of the multi-user diversity gain and provide average delay (or throughput) and fairness improvement compared with channel-aware-only schemes.

Rate control of multi class priority flows with end-to-end delay and rate constraints for QoS networks

To address end-to-end quality of service (QoS) requirements, we derive a novel distributed combined rate and end-to-end delay control in a network serving multi-class flows with priority packet scheduling. We show that the control is globally asymptotically stable without information time lags. The stable flows attain the end-to-end delay requirements and have no packet loss. We also show that by enhancing the network with bandwidth reservation and admission control, minimum rate is also guaranteed. The stability with very long time lags of a discrete time version control with non-greedy flows and random packet arrivals is studied numerically by an NS2 packet-based simulation of the Australian Academic and Research Network.

Stable Throughput of Cognitive Radios With and Without Relaying Capability

A scenario with two single-user links, one licensed to use the spectral resource (primary) and one unlicensed (secondary or cognitive), is considered. According to the cognitive radio principle, the activity of the secondary link is required not to interfere with the performance of the primary. Therefore, in this paper, it is assumed that the cognitive link accesses the channel only when sensed idle. Moreover, the analysis includes: (1) random packet arrivals; (2) sensing errors due to fading at the secondary link; (3) power allocation at the secondary transmitter based on long-term measurements. In this framework, the maximum stable throughput of the cognitive link (in packets/slot) is derived for a fixed throughput selected by the primary link. The model is modified so as to allow the secondary transmitter to act as a ldquotransparentrdquo relay for the primary link. In particular, packets that are not received correctly by the intended destination might be decoded successfully by the secondary transmitter. The latter can, then, queue and forward these packets to the intended receiver. A stable throughput of the secondary link with relaying is derived under the same conditions as before. Results show that benefits of relaying strongly depend on the topology (i.e., average channel powers) of the network.

Joint Beamforming and Scheduling for a Multi-Antenna Downlink with Imperfect Transmitter Channel Knowledge

We consider the downlink of a wireless system where the base-station has M ges 1 antennas and K user terminals have one antenna each. We study the weighted rate sum maximization in the case of non-perfect Channel State Information at the Transmitter (CSIT). Some relevant downlink optimization problems, such as the stabilization of the transmission queues under random packet arrivals and the proportional fair scheduling for infinite backlogged systems, can be solved as special cases of the proposed problem. We restrict the transmitter strategy to be based on Gaussian coding and beamforming. Even under this simplifying condition, the problem at hand is non-convex and it does not appear to lend itself to a simple algorithmic solution. Therefore, we introduce some approximations that yield a definition of signal-to-interference plus noise ratio (SINR) commonly used in the classical array- processing/beamforming literature. For the simpler (but still non-convex) approximated problem, we propose a powerful heuristic solution based on greedy user selection and a gradient iteration that converges to a local maximum of the objective function. This method yields very competitive results with relatively low computational complexity. Extensive simulations show that, in the case of perfect CSIT, the proposed heuristic scheme performs very closely to the optimal (dirty-paper coding) strategy while, in the case of non-perfect CSIT, it significantly outperforms previously proposed suboptimal approaches, such as random beamforming and approximated zero-forcing with greedy user selection.

Transmit Diversity Versus Opportunistic Beamforming in Data Packet Mobile Downlink Transmission

We compare space-time coding (transmit diversity) and random beamforming in a space-division multiple access/time-division multiple access single-cell downlink system with random packet arrivals, correlated block-fading channels, and non-perfect channel state information at the transmitter due to a feedback delay. Our comparison is based on system stability. The ability of accurately predicting the channel signal-to-noise ratio dominates the performance of opportunistic beamforming, even under the optimistic assumption that the sequence of beamforming matrices is perfectly known a priori by the receivers. Our results show that the relative merit of opportunistic beamforming versus space-time coding strongly depends on the channel Doppler bandwidth. Therefore, previous naive conclusions on the fact that transmit diversity always hurts the system performance under multiuser-diversity scheduling should be taken with great care

On the limitation of fluid-based approach for Internet congestion control

Fluid models have been the main tools for Internet congestion control. By capturing how the average rate of each flow evolves, the fluid model proves to be useful as it predicts the equilibrium point to which system trajectory converges and also provides conditions under which the convergence is ensured, i.e., the system is stable. However, due to inherent randomness in the network caused by random packet arrivals or random packet marking, the actual system evolution is always of a stochastic nature. In this paper, we show that we can be better off using a stochastic approach toward the congestion control. We first prove that the equilibrium point of a fluid model can be quite different from the true average rate of the corresponding stochastic system. After we describe the notion of stability for two different approaches, we show that a stable fluid model can impose too much restriction on our choice of system parameters such as buffer size or link utilization. In particular, under fluid models, we show that there exists a fundamental tradeoff between the link utilization and buffer size requirement for large systems, while in a more realistic setting with stochastic models, there is no such tradeoff. This implies that the current congestion control design can be much more flexible, to the benefit of efficient usage of network resources.

An iterative water-filling algorithm for maximum weighted sum-rate of Gaussian MIMO-BC

We consider the maximization of weighted rate sum in Gaussian multiple-input-multiple-output broadcast channels. This problem is motivated by optimal adaptive resource allocation policies in wireless systems with multiple antenna at the base station. In fact, under random packet arrival and transmission queues, the system stability region is achieved by maximizing a weighted rate sum with suitable weights that depend on the queue buffer sizes. Our algorithm is a generalization of the well-known Iterative Multiuser Water-Filling that maximizes the rate sum under a total transmit power constraint and inherits from the latter its simplicity. We propose also a variation on the basic algorithm that makes convergence speed very fast and essentially independent of the number of users