On-off Policy Research Articles

Design and Performance Analysis of RIS-Aided DCSK-WPC System With Energy Buffer

This paper proposes an energy buffer and reconfigurable intelligent surface (RIS) aided differential chaos-shift-keying (DCSK) wireless-powered communication (WPC) system, which includes an access point (AP), a RIS, and a source node equipped with an energy buffer. In such system, the source node first harvests and stores radio-frequency energy from the AP in the downlink, then uses the stored energy to transmit information to the AP in the uplink. A simple time-switching scheme for the proposed system is designed. Two different energy management policies at the source node are considered, i.e., best-effort policy (BEP) and on-off policy (OOP). The bit-error-rate (BER) expressions of the proposed system with BEP and OOP are derived over multipath Rayleigh fading channels. Results demonstrate that the proposed system is able to achieve better BER performance than the conventional DCSK-WPC system and the RIS aided DCSK-WPC buffer-less system, while keeping the non-coherent feature of the DCSK system.

Adaptive Multiuser Cooperative NOMA Scheme With Energy Buffer-Aided Near-Users for High Spectral and Energy Efficiency

In this article, a multiuser cooperative nonorthogonal multiple access (NOMA) network is considered. In order to improve their battery lifetime, the near-user (NU) Internet of Things (IoT) nodes relay information to the far-user (FU) IoT nodes using only the energy harvested from the ambience. The harvested energy at all the NU IoT nodes is stored in energy buffers. To improve the throughput performance, the best NU and best FU (BNBF) user selection scheme is employed. Moreover, unlike conventional orthogonal multiple access (OMA) networks, an adaptive NOMA network is considered in this article that switches between direct NOMA, NOMA with relaying, and OMA modes to maximize throughput. To improve accuracy and reduce the computational complexity, the energy buffer states at the NU IoT nodes are modeled using a continuous state-space Markov chain (CSMC) instead of discrete state-space Markov chain (DSMC). We derive the limiting distributions of the stored energy in energy buffers with either the best effort policy (BEP) or the on–off policy (OOP) applied for buffer energy management. Expressions for throughput are derived for both harvest-store-use (HSU) and harvest-use (HU) architectures. The Monte Carlo simulations are used to validate the derived analytical expressions.

Diffusion Limit of Multi-Server Retrial Queue with Setup Time

In the paper, we consider a multi-server retrial queueing system with setup time which is motivated by applications in power-saving data centers with the ON-OFF policy, where an idle server is immediately turned off and an off server is set up upon arrival of a customer. Customers that find all the servers busy join the orbit and retry for service after an exponentially distributed time. For this model, we derive the stability condition which depends on the setup time and turns out to be more strict than that of the corresponding model with an infinite buffer which is independent of the setup time. We propose asymptotic methods to analyze the system under the condition that the delay in the orbit is extremely long. We show that the scaled-number of customers in the orbit converges to a diffusion process. Using this diffusion limit, we obtain approximations for the steady-state probability distribution of the number of busy servers and that of the number of customers in the orbit. We verify the accuracy of the approximations by simulations and numerical analysis. Numerical results show that the retrial system under the limiting condition consumes more energy than that with an infinite buffer in front of the servers.

Performance of a Cooperative Communication Network With Green Self-Sustaining Nodes

The desire to reduce carbon foot-print and avoid frequent battery replacement has increased interest in green self-sustaining wireless nodes based on energy harvesting. In this article, we analyze the performance of a two-hop cooperative network with green self-sustaining nodes. The source and relay nodes are equipped with energy buffers, and store the energy harvested from the ambience. Signal transmission is powered at both source and relay solely by the stored energy. Both fixed and adaptive-rate transmission schemes are considered, and analytical expressions are presented for throughput and ergodic rate. Unlike all previous works that use discrete-state Markov chains to model the buffer dynamics, we use the more appropriate discrete-time continuous-state space Markov chain. We consider two different energy management policies at the source and relay: best-effort policy (BEP) and on-off policy (OOP). It is observed that for fixed-rate transmission, the better choice of buffer management policy (throughput-wise) at the source and the relay buffers depends on the relay location and target rate. On the other hand, for adaptive rate transmission, using BEP policy at both the energy buffers ensures best performance. The derived analytical expressions are validated by Monte Carlo simulations.

On Stochastic Link and Energy Scheduling for Energy Harvesting Bidirectional Communications

In this paper, a solar-powered bidirectional communication system is studied for a pair of energy harvesting (EH) nodes that intend to communicate with each other over wireless fading channels. The conventional time-division duplex (TDD) transmission is revisited by proposing a stochastic resource scheduling scheme to minimize an average rate outage probability based on a Markov decision process (MDP) design framework. Different from the conventional TDD transmission, the proposed scheme can adjust the link direction and energy expenditure for data transmissions between the two EH nodes, in response to the dynamics of the solar EH, channel fading and battery storage conditions. A downstairs threshold structure is theoretically proved under a special optimal on-off policy, in which two-dimensional thresholds pinpoint the interplay between the transmission actions and the available energy in the batteries of the two nodes. Also the optimal on-off policy at asymptotically high signal-to-noise power ratios (SNRs) is revealed. The outage performance of the proposed stochastic resource scheduling scheme is validated by extensive computer simulations, and it shows that the proposed optimal MDP policy can achieve significant performance gains over the combinations of other compared schemes, including round-robin and battery state-oriented link scheduling schemes, and greedy and conservative energy scheduling schemes.

Performance of a Cooperative Network With an Energy Buffer-Aided Relay

In this paper, we analyze the performance of a cooperative communication network with an energy buffer-aided energy harvesting relay. We consider both fixed and adaptive-rate signalling. The relay is assumed to harvest energy from ambient sources. We consider the harvest-store-use (HSU) protocol employing best-effort or on-off policies, with both finite and infinite-size energy buffers, and analyze the outage probability and ergodic rate performance. Since it is known that using a discrete-state Markov chain to model the energy buffer is inaccurate even for moderate number of levels, we assume use of a discrete-time continuous-state space Markov chain. We do not ignore the direct channel, and assume that the direct and relayed signals are optimally combined. We compare performance of the scheme with that of harvest-use (HU) and direct (relay-less) transmission. Simulation results are presented to validate the derived analytical expressions and bring out useful insights.

Performance of Energy-Buffer Aided Incremental Relaying in Cooperative Networks

In this paper, we consider a two-hop cooperative network with a direct link based on an energy harvesting (EH) decode-and-forward relay. The energy-buffer equipped relay harvests energy from the ambience, and uses the harvest-store-use (HSU) architecture. Since it is known that using a discrete-state Markov chain to model the energy buffer is inaccurate even for moderate number of states, we use a discrete-time continuous-state space Markov chain instead. We derive the limiting distribution of energy for both the incremental on-off policy (IOFP) and the incremental best-effort policy (IBEP), and use them to obtain expressions for outage probability and throughput. The corresponding expressions for non-incremental signaling follow as a special case. We show that the stable buffers using IBEP harness a diversity of two as compared with those using IOFP, which attain a diversity of one. However, while buffers using IBEP are consequently more reliable than those with IOFP, their throughput performance is only marginally superior. The simulation results are presented to validate the derived analytical expressions.

Delay performance of data-center queue with setup policy and abandonment

This paper considers a finite capacity multiserver queue for performance modeling of data centers. One of the most important issues in data centers is to save energy of servers. To this end, a natural policy is to turn off a server immediately once it has no job to process. In this policy, which is called ON–OFF policy, the server must be setup upon the arrival of a new job. During the setup time, the server cannot process jobs but consumes energy. To mitigate the drawback, this paper considers a setup policy, where the number of setup servers at a time is limited. We also consider an extension of the setup policy in which some of servers, but not all of them, are allowed to remain idle for a random amount of time. The main purpose of this paper is to analyze the delay distribution of the multiserver queue with the setup policies. We assume that jobs may abandon the queue without receiving their services. We formulate the queue length process using a two-dimensional continuous-time Markov chain. It can be shown that we cannot apply the distributional Little’s law to obtain the waiting time distribution via the queue length distribution. Therefore, we construct a three-dimensional absorbing Markov chain which describes the virtual waiting time process. We then obtain a phase-type expression of the stationary waiting time distribution. We evaluate the performance of the multiserver queue with the setup policy, and then discuss the optimal control parameters of the policy based on the delay performance as well as the mean power consumption.

Multiserver retrial queue with setup time and its application to data centers

This paper considers a multiserver retrial queue with setup time which is motivated from application in data centers with the ON-OFF policy, where an idle server is immediately turned off. The ON-OFF policy is designed to save energy consumption of idle servers because an idle server still consumes about 60% of its peak consumption processing jobs. Upon arrival, a job is allocated to one of available off-servers and that server is started up. Otherwise, if all the servers are not available upon arrival, the job is blocked and retries in a random time. A server needs some setup time during which the server cannot process a job but consumes energy. We formulate this model using a three-dimensional continuous-time Markov chain obtaining the stability condition via Foster-Lyapunov criteria. Interestingly, the stability condition is different from that of the corresponding non-retrial queue. Furthermore, exploiting the special structure of the Markov chain together with a heuristic technique, we develop an efficient algorithm for computing the stationary distribution. Numerical results reveal that under the ON-OFF policy, allowing retrials is more power-saving than buffering jobs. Furthermore, we obtain a new insight that if the setup time is relatively long, setting an appropriate retrial time could reduce both power consumption and the mean response time of jobs.

Performance Analysis of Near-Optimal Energy Buffer Aided Wireless Powered Communication

In this paper, we consider a wireless powered communication system, where an energy harvesting (EH) node harvests energy from a radio frequency (RF) signal broadcasted by an access point (AP) in the downlink (DL). The node stores the harvested energy in an energy buffer and uses the stored energy to transmit data to the AP in the uplink (UL). We investigate two simple online transmission policies for the EH node, namely a best-effort policy and an on-off policy, which do not require knowledge of the EH profile nor channel knowledge. In particular, for both policies, the EH node transmits in each time slot with a constant desired power if sufficient energy is available in its energy buffer. Otherwise, the node transmits with the maximum possible power in the best-effort policy and remains silent in the on-off policy. For both policies, we use the theory of discrete-time continuous-state Markov chains to analyze the limiting distribution of the stored energy for finite- and infinite-size energy buffers. We provide this limiting distribution in closed form for a Nakagami- $m$ fading DL channel and analyze the outage probability for a Nakagami- $m$ fading UL channel. All derived analytical results are not limited to EH via RF WPT but are applicable for any independent and identically distributed EH process from e.g. solar and wind energy. Our results reveal that, for low-to-medium outage probabilities, the best-effort policy is superior to the on-off policy and the optimal UL transmit power of the EH node that minimizes the outage probability is always less than the average harvested power. The opposite behaviour is observed for high outage probabilities. Furthermore, we show that the minimum outage probability of the two proposed policies is near-optimal.

A stochastic model-based approach to analyse reliable energy-saving rail road switch heating systems

Rail road switch heaters are used to avoid the formation of snow and ice on top of rail road switches during the cold season, in order to guarantee their correct functioning.Effective management of the energy consumption of these devices is important to reduce the costs and minimise the environmental impact. While doing so, it is critical to guarantee the reliability of the system.In this work we analyse reliability and energy consumption indicators for a system of (remotely controlled) rail road switch heaters by developing and solving a stochastic model-based approach based on the Stochastic Activity Networks (SAN) formalism. An on-off policy is considered for heating the switches, with parametric thresholds of activation/deactivation of the heaters and considering different classes of priority.A case study has been developed inspired by a real rail road station, to practically demonstrate the application of the proposed approach to understand the impact of different thresholds and priorities on the indicators under analysis (probability of failure and energy consumption).

Energy Consumption Analysis and Minimization in Multi-Layer Heterogeneous Wireless Systems

Cellular network technologies have traditionally evolved to meet the ever-increasing need for capacity and coverage. Particularly, there has been a significant focus on exploiting the use of small cells and heterogeneous networks (HetNets). In the latter, the economic and environmental impact of the energy consumption is a key concern. Although much research has been done to address the energy consumption in HetNets, existing approaches have failed to capture the key factors affecting it. In this paper, the energy consumption in HetNets is analyzed with a focus on their multi-layer nature and the dependence of the energy consumption on the spatio-temporal traffic demands and the internal base station hardware components. The problem of minimizing the energy consumption is then studied and characterized in terms of a 0-1 Knapsack-like problem. Due to the differences with the classical 0-1 Knapsack problem, an efficient algorithm is introduced to minimize the energy consumption by adjusting the cell-association and the base stations on-off policies. Such algorithm is shown to be applicable to the two- and $m$ -layer HetNet cases. Performance evaluation is provided to identify the achievable energy savings of our algorithm and its effect on the energy consumption, activity, and load across multiple layers.

Data-Driven Stochastic Models and Policies for Energy Harvesting Sensor Communications

Energy harvesting from the surroundings is a promising solution to perpetually power-up wireless sensor communications. This paper presents a data-driven approach of finding optimal transmission policies for a solar-powered sensor node that attempts to maximize net bit rates by adapting its transmission parameters, power levels and modulation types, to the changes of channel fading and battery recharge. We formulate this problem as a discounted Markov decision process (MDP) framework, whereby the energy harvesting process is stochastically quantized into several representative solar states with distinct energy arrivals and is totally driven by historical data records at a sensor node. With the observed solar irradiance at each time epoch, a mixed strategy is developed to compute the belief information of the underlying solar states for the choice of transmission parameters. In addition, a theoretical analysis is conducted for a simple on-off policy, in which a predetermined transmission parameter is utilized whenever a sensor node is active. We prove that such an optimal policy has a threshold structure with respect to battery states and evaluate the performance of an energy harvesting node by analyzing the expected net bit rate. The design framework is exemplified with real solar data records, and the results are useful in characterizing the interplay that occurs between energy harvesting and expenditure under various system configurations. Computer simulations show that the proposed policies significantly outperform other schemes with or without the knowledge of short-term energy harvesting and channel fading patterns.

Delay-optimal Power Control Policies

The delay till success (DTS) is the mean number of transmissions needed, averaged over the fading, until a single packet is successfully received (decoded) over a wireless link. This paper shows that under a mean and a peak power constraint, random power control can significantly reduce the DTS. We derive the optimal power control policies that minimize the DTS at one link of given length. For most commonly used fading distributions, these optimal power control policies are random on-off policies, whose parameters depend on the fading statistics and the link distance. We present two applications of this result in the context of noise-limited wireless networks: minimizing the local delay (mean delay for successful nearest-neighbor communication) and minimizing the local anycast delay (mean delay for a transmission to any node).

Control of Flexible Smart Devices in the Smart Grid

This paper investigates load control and demand response in a smart grid environment where a bidirectional communication link between the operator and the smart flexible devices supports command and data flow. Two control schemes are investigated that can provide energy management, taking into account user's comfort, via binary on-off policies of the smart flexible devices. A dynamic control algorithm is introduced that considers real time network characteristics and initiates command flow when critical parameters exceed predefined thresholds. To sustain fairness in the system, priority based and round robin scheduling algorithms are proposed. A continuous control algorithm is also explored to define the higher bounds of energy savings. To quantify the discomfort of users that participate in this type of services, a heuristic consumer utility metric is proposed and measurements with a flexible device (air conditioning unit) are performed to model empirically possible time intervals of the control scheme. Reciprocal fair energy management schemes are investigated being both operator and user centric. It is shown that great energy and cost savings can be achieved providing the required degrees of freedom to the smart grid to self-adapt during peak hours.

A pattern recognition approach to the solution of optimal singular control problems

Abstract A considerable number of algorithms have been proposed in the literature for the optimization of control policies of nonlinear chemical systems. However, most algorithms do not seem to be flexible and robust enough for their application to be a matter of routine. Rather, each of them presents typical shortcomings that restrict applications to special classes of problems. In particular, bang-bang (or on-off) policies are difficult to optimize, due to control discontinuities. In this paper we propose a strategy based on a combination of pattern recognition theory and non-linear mathematical programming for the computation of optimal singular control problems.