Buffer Management Problem Research Articles

Balancing Project Schedule, Cost, and Value under Uncertainty: A Reinforcement Learning Approach

Industrial projects are plagued by uncertainties, often resulting in both time and cost overruns. This research introduces an innovative approach, employing Reinforcement Learning (RL), to address three distinct project management challenges within a setting of uncertain activity durations. The primary objective is to identify stable baseline schedules. The first challenge encompasses the multimode lean project management problem, wherein the goal is to maximize a project’s value function while adhering to both due date and budget chance constraints. The second challenge involves the chance-constrained critical chain buffer management problem in a multimode context. Here, the aim is to minimize the project delivery date while considering resource constraints and duration-chance constraints. The third challenge revolves around striking a balance between the project value and its net present value (NPV) within a resource-constrained multimode environment. To tackle these three challenges, we devised mathematical programming models, some of which were solved optimally. Additionally, we developed competitive RL-based algorithms and verified their performance against established benchmarks. Our RL algorithms consistently generated schedules that compared favorably with the benchmarks, leading to higher project values and NPVs and shorter schedules while staying within the stakeholders’ risk thresholds. The potential beneficiaries of this research are project managers and decision-makers who can use this approach to generate an efficient frontier of optimal project plans.

An optimal algorithm for 2-bounded delay buffer management with lookahead

The bounded delay buffer management problem, which was proposed by Kesselman et al. (STOC 2001 and SIAM Journal on Computing 33(3), 2004), is an online problem focusing on buffer management of a switch supporting Quality of Service (QoS). The problem definition is as follows: Packets arrive to a buffer over time and each packet is specified by the release time, deadline and value. An algorithm can transmit at most one packet from the buffer at each integer time and can gain its value as the profit if transmitting the packet by its deadline after its release time. The objective of this problem is to maximize the gained profit. We say that an instance of the problem is s-bounded if for any packet, an algorithm has at most s chances to transmit it. For any s≥2, Hajek (CISS 2001) showed that the competitive ratio of any deterministic algorithm is at least (1+5)/2≥1.618. Recently, Veselý et al. (SODA 2019) designed an online algorithm matching the lower bound.Böhm et al. (ISAAC 2016 and Theoretical Computer Science, 2019) introduced the lookahead ability to an online algorithm. At a time t, the algorithm obtains information about packets arriving at time t+1, and showed that for s=2, there is an algorithm which achieves the competitive ratio of (−1+13)/2≤1.303. Also, they showed that the competitive ratio of any deterministic algorithm is at least (1+17)/4≥1.280. In this paper, for the 2-bounded model with lookahead, we design an algorithm with a matching competitive ratio of (1+17)/4.

A Comparative Analysis of Buffer Management Algorithms for Delay Tolerant Wireless Sensor Networks

One of the challenges in Delay Tolerant Wireless Sensor Networks (DT-WSN), is to handle situations where the available buffer space is insufficient- the buffer management problem. Although several buffer management algorithms have been proposed for DT-WSNs, to the best of our knowledge, there is no comprehensive study on the effects different factors have on their performance, and which evaluates the relative performance of these algorithms in different contexts. This paper evaluates in a fixed-factor factorial experiment the performance in terms of latency and Quality of Information (QoI) of four representative buffer management algorithms for DT-WSNs; two traditional, FiFO and Random, and two QoI-based algorithms- one proposed by Humber and Ngai and the SmartGap algorithm. The evaluation suggests that the buffer management algorithm in combination with employed routing protocol and the sensor node buffer sizes have a significant impact on latency, while the obtained QoI rather depends on the characteristics of the transported data and the routing protocol, provided a single-copy routing protocol is used. Moreover, the evaluation suggests that QoI-based buffer management algorithms do offer improved QoI, with an 31% improvement in MAE for SmartGap compared to FIFO. However, they do so at the expense of higher latency, with SmartGap giving a 60% higher latency than FIFO on average.

Randomized two-valued bounded delay online buffer management

In the bounded delay buffer management problem unit size packets arrive online to be sent over a network link. The objective is to maximize the total weight of packets sent before their deadline. In this paper we are interested in the two-valued variant of the problem, where every packet has either low (1) or high priority weight (α>1). We show that the optimal randomized competitive ratio against an oblivious adversary is 1+(α−1)∕(α2+α).

An Optimal Randomized Online Algorithm for QoS Buffer Management

The QoS buffer management problem, with significant and diverse computer applications, e.g., in online cloud resource allocation problems, is a classic online admission control problem in the presence of resource constraints. In its basic setting, packets with different values, arrive in online fashion to a switching node with limited buffer size. Then, the switch needs to make an immediate decision to either admit or reject the incoming packet based on the value of the packet and its buffer availability. The objective is to maximize the cumulative profit of the admitted packets, while respecting the buffer constraint. Even though the QoS buffer management problem was proposed more than a decade ago, no optimal online solution has been proposed in the literature. This paper proposes an optimal randomized online algorithm for this problem.

Online buffer management for transmitting packets with processing cycles

We study an online buffer management problem under the model introduced by Azar and Gilon (2015) [5] recently. Unit-sized packets arrive and are kept in a First-In-First-Out buffer of size B in an online fashion at a network server. Each packet is associated with an arrival time, a value and a processing cycle time in the buffer. The density of a packet is defined to be the ratio of its value to processing time. It is assumed that every packet can be transmitted only after its processing cycle is completed and only the packet at the head of the buffer can be processed. A packet is allowed to be preempted and then discarded from the buffer. But, the value of a packet is attained only if it is successfully transmitted. Under the model, the objective of online buffer management is to maximize the total value of transmitted packets. This model finds applications to packet scheduling in communication networks.In this study, we consider the model with constant density from a theoretical perspective. We first propose some lower bounds for the problem. Azar and Gilon obtained a 4-competitive algorithm for the online buffer management problem for packets with constant density. Here, we present a (2+1B−1)-competitive algorithm for the case B>1 as well as its generalization to the multi-buffer model. Moreover, we prove that the competitive ratio of a deterministic online algorithm is at least four when the buffer size is one. We also conduct experiments to demonstrate the superior performance of the proposed online algorithm against the previous approach.

An Optimal Randomized Online Algorithm for QoS Buffer Management

The QoS (Quality of Service) buffer management problem, with significant and diverse computer applications, e.g., in online cloud resource allocation problems, is a classic online admission control problem in the presence of resource constraints. In its basic setting, packets with different values according to their QoS requirements, arrive in online fashion to a switching node with limited buffer size. Then, the switch needs to make an immediate decision to either admit or reject the incoming packet based on the value of the packet and its buffer availability. The objective is to maximize the cumulative profit of the admitted packets, while respecting the buffer constraint. Even though the QoS buffer management problem was proposed more than a decade ago, no optimal online solution has been proposed in the literature. This paper contributes to this problem by proposing: 1) A fixed threshold-based online algorithm with smaller competitive ratio than the existing results; 2) an optimal deterministic online algorithm under fractional admission model in which a packet could be admitted partially; and 3) an optimal randomized online algorithm for the general problem. We consider the last result being the main contribution of this paper.

Competitive buffer management for multi-queue switches in QoS networks using packet buffering algorithms

In this paper, we consider the online buffer management problem, which formulates the problem of managing network switches supporting Quality of Service guarantee. We improve competitive ratios of the 2-value multi-queue switch model, where the value of a packet is restricted to 1 or α(≥1). We use a similar approach as Azar and Richter (STOC 2003 and Algorithmica 43(1-2), 2005) did for the multi-value multi-queue switch model. Namely, we show that the competitive ratio of “the relaxed model” of the 2-value multi-queue switch model is at most x=min⁡{c+2−cα(2−c)+c−1,cα}, if the competitive ratio of an online algorithm for the unit-value multi-queue switch model is at most c. Azar and Richter's technique implies that if the competitive ratio of the 2-value single-queue switch model is x′, then the competitive ratio of the 2-value multi-queue switch model is at most xx′. We obtain several results using known c and x′.

Better bounds for online k-frame throughput maximization in network switches

We consider a variant of the online buffer management problem in network switches, called the k-frame throughput maximization problem (k-FTM). This problem models the situation where a large frame is fragmented into k packets and transmitted through the Internet, and the receiver can reconstruct the frame only if he/she accepts all the k packets. Kesselman et al. introduced this problem and showed that its competitive ratio is unbounded even when k=2. They also introduced an “order-respecting” variant of k-FTM, called k-OFTM, where inputs are restricted in some natural way. They proposed an online algorithm and showed that its competitive ratio is at most 2kB⌊B/k⌋+k for any B≥k, where B is the size of the buffer. They also gave a lower bound of B⌊2B/k⌋ for deterministic online algorithms when 2B≥k and k is a power of 2.In this paper, we improve upper and lower bounds on the competitive ratio of k-OFTM. Our main result is to improve an upper bound of O(k2) by Kesselman et al. to 5B+⌊B/k⌋−4⌊B/2k⌋=O(k) for B≥2k. Note that this upper bound is tight up to a multiplicative constant factor since the lower bound given by Kesselman et al. is Ω(k). We also give two lower bounds. First we give a lower bound of 2B⌊B/(k−1)⌋+1 on the competitive ratio of deterministic online algorithms for any k≥2 and any B≥k−1, which improves the previous lower bound of B⌊2B/k⌋ by a factor of almost four. Next, we present the first nontrivial lower bound on the competitive ratio of randomized algorithms. Specifically, we give a lower bound of k−1 against an oblivious adversary for any k≥3 and any B. Since a deterministic algorithm, as mentioned above, achieves an upper bound of about 10k, this indicates that randomization does not help too much.

Reordering buffer management with advice

In the reordering buffer management problem, a sequence of colored items arrives at a service station to be processed. Each color change between two consecutively processed items generates some cost. A reordering buffer of capacity k items can be used to preprocess the input sequence in order to decrease the number of color changes. The goal is to find a scheduling strategy that, using the reordering buffer, minimizes the number of color changes in the given sequence of items. We consider the problem in the setting of online computation with advice. In this model, the color of an item becomes known only at the time when the item enters the reordering buffer. Additionally, together with each item entering the buffer, we get a fixed number of advice bits, which can be seen as information about the future or as information about an optimal solution (or an approximation thereof) for the whole input sequence. We show that for any $$\varepsilon > 0$$ there is a $$(1+\varepsilon )$$ -competitive algorithm for the problem which uses only a constant (depending on $$\varepsilon $$ ) number of advice bits per input item. This also immediately implies a $$(1+\varepsilon )$$ -approximation algorithm which has $$2^{O(n\log 1/\varepsilon )}$$ running time (this should be compared to the trivial optimal algorithm which has a running time of $$k^{O(n)}$$ ). We complement the above result by presenting a lower bound of $$\varOmega (\log k)$$ bits of advice per request for any 1-competitive algorithm.

An effective buffer management in delay tolerant networks based on a partially observed Markov decision process framework

The delay tolerant networks work in communication environments that are a subject to delays and distributions. Mobile nodes in DTN can communicate with each other even though there is no permanent link between the source and the destination. To have a better rate of delivery of messages in DTN, the long-term storage and message replication require an efficient buffer management. In this paper, we propose a partially observed Markov decision process framework to solve the problem of buffer management. The node uses messages' transmissions number to decide which message to drop. We look for an efficient strategy that a node can follow to maximise its average reward, while taking into account the energy penalties in different levels of cooperation. For this, we have analytically determined the optimal policy's interval of our proposal. Furthermore, we use ONE simulator to compare our findings to traditional buffer management policies under different mobility models.

Optimal buffer management for 2-frame throughput maximization

We consider a variant of the online buffer management problem in network switches, called the k-frame throughput maximization problem (k-FTM). Large data, called frames, carried on the Internet are split into small k packets by a sender, and the receiver can reconstruct each frame only if he/she accepts all the k constituent packets of the frame. Packets pass through network switches on the Internet, and each switch is equipped with a FIFO buffer to temporarily store arriving packets. Since the size of the buffer is bounded, some packets must be discarded if it is full. It is impossible to reconstruct frames including discarded packets any more. Our goal is to maximize the number of reconstructed frames. Kesselman et al. proposed this problem, and showed that any online algorithm has an unbounded competitive ratio even when k=2. Hence, they considered the “order-respecting” variant of k-FTM. They showed that the competitive ratio of their algorithm is at most (2kB⌊B/k⌋+k) for any B ≥ k, where B is the size of the buffer. Also, they gave a lower bound of B⌊2B/k⌋ on the competitive ratio when 2B ≥ k and k is a power of 2. Furthermore, they proved that the competitive ratio of a greedy algorithm is at most (11+8B−1) for any B ≥ 2 and k=2.We analyze a greedy algorithm for k=2, and show that its competitive ratio is at most 3 for any B, improving the previous upper bound of 4B⌊B/2⌋+2(≥10). Moreover, we show that the competitive ratio of any deterministic algorithm is at least 3 for any B if k=2, which matches our upper bound.

A Statistical Analysis Based Probabilistic Routing for Resource-Constrained Delay Tolerant Networks

The nonexistence of end-to-end path between the sender and the receiver poses great challenges to the successful message transmission in delay tolerant networks. Probabilistic routing provides an efficient scheme to route messages, but most existing probabilistic routing protocols do not consider whether a message has enough time-to-live to reach its destination. In this paper, we propose an improved probabilistic routing algorithm that fully takes into account message's time-to-live when predicting the delivery probability. Based on statistical analysis, we compute and update the expected intermeeting times between nodes. And then the probability for a message to be delivered within its time-to-live is computed based on the assumed exponential distribution. We further propose an optimal message schedule policy, by modeling the buffer management problem as 0-1 knapsack, of which the maximum delivery probability sum can be achieved by resorting to the back track technique. Extensive simulations are conducted and the results show that the proposed algorithm can greatly enhance routing performance in terms of message delivery probability, overhead ratio, and average hop count.

A [formula omitted]-competitive algorithm for collecting items with increasing weights from a dynamic queue

The bounded-delay packet scheduling (or buffer management) problem is to schedule transmissions of packets arriving in a buffer of a network link. Each packet has a deadline and a weight associated with it. The objective is to maximize the weight of packets that are transmitted before their deadlines, assuming that only one packet can be transmitted in one time step. Online packet scheduling algorithms have been extensively studied. It is known that no online algorithm can achieve a competitive ratio better than ϕ≈1.618 (the golden ratio), while the currently best upper bound on the competitive ratio is 22−1≈1.824. Closing the gap between these bounds remains a major open problem.The above mentioned lower bound of ϕ uses instances where item weights increase exponentially over time. In fact, all lower bounds for various versions of buffer management problems involve instances of this type. In this paper, we design an online algorithm for packet scheduling with competitive ratio ϕ when packet weights are increasing, thus matching this lower bound. Our algorithm applies, in fact, to a much more general version of packet scheduling, where only the relative order of the deadlines is known, not their exact values.

Considering Suppressed Packets Improves Buffer Management in Quality of Service Switches

The following buffer management problem arises in network switches providing different levels of services: At the beginning of each time step, one packet can be sent, and afterward an arbitrary number of new packets arrive. Packets that are not sent can be stored in a buffer. Each packet has a deadline, and a packet is automatically deleted from the buffer if it is still stored in the buffer by the end of its deadline. Additionally, each packet has a value which reflects its importance. A buffer management strategy determines the packet to be sent in each time step. The goal of a buffer management strategy is to maximize the sum of the values of sent packets. We introduce the concept of suppressed packets and present a deterministic strategy that is based on this concept. We show that this strategy achieves a competitive ratio of $2 \sqrt{2} - 1 \approx 1.828$, which is the best known competitive ratio in the deterministic case. In addition, we present a memoryless version of this strategy that achieves a...

Randomized competitive algorithms for online buffer management in the adaptive adversary model

In the problem of buffer management with bounded delay, packets with weights and deadlines arrive at a network switch over time, and the goal is to send those packets on the outgoing link while maximizing the total weight of the packets that are sent before their deadlines. We present a study of randomized algorithms that are competitive against an adaptive adversary. Previous studies considered only the oblivious adversary model that does not capture dependency of network traffic on the packet scheduling algorithm. We give a new analysis of a previously known algorithm, which shows that it remains e / ( e − 1 ) -competitive even against an adaptive adversary. We complement this with a 4/3 lower bound on the competitive ratio on 2-bounded instances, in which each packet has a lifespan of one or two steps. We also study more restricted 2-uniform instances, in which every packet has a lifespan of exactly two steps. For such instances we give a 1.2 lower bound on the competitive ratio of arbitrary algorithms and 4/3 lower bound on the competitive ratio of memoryless scale-invariant algorithms. Finally, we devise a 4/3-competitive memoryless scale-invariant algorithm for 2-bounded instances, matching two of these lower bounds.

A Tight Upper Bound on Online Buffer Management for Multi-Queue Switches with Bicodal Buffers

The online buffer management problem formulates the problem of queuing policies of network switches supporting QoS (Quality of Service) guarantee. In this paper, we consider one of the most standard models, called multi-queue switches model. In this model, Albers et al. gave a lower bound $\frac{e}{e-1}$, and Azar et al. gave an upper bound $\frac{e}{e-1}$ on the competitive ratio when m, the number of input ports, is large. They are tight, but there still remains a gap for small m. In this paper, we consider the case where m = 2, namely, a switch is equipped with two ports, which is called a bicordal buffer model. We propose an online algorithm called Segmental Greedy Algorithm (SG) and show that its competitive ratio is at most $\frac{16}{13} (\simeq 1.231)$, improving the previous upper bound by $\frac{9}{7} (\simeq 1.286)$. This matches the lower bound given by Schmidt.

A Tight Bound on Online Buffer Management for Two-Port Shared-Memory Switches

The online buffer management problem formulates the problem of queueing policies of network switches supporting QoS (Quality of Service) guarantee. For this problem, several models are considered. In this paper, we focus on shared memory switches with preemption. We prove that the competitive ratio of the Longest Queue Drop (LQD) policy is 4M-4/3M-2 in the case of N = 2, where N is the number of output ports in a switch and M is the size of the buffer. This matches the lower bound given by Hahne, Kesselman and Mansour. Also, in the case of arbitrary N, we improve the competitive ratio of LQD from 2 to 2-1/M minK = 1, 2, ₀, N { ⌊M/K⌋ + K-1}.

Dynamic buffer management using optimal control of hybrid systems

This paper studies a dynamic buffer management problem with one buffer inserted between two interacting components. The component to be controlled is assumed to have multiple power modes corresponding to different data processing rates. The overall system is modeled as a hybrid system and the buffer management problem is formulated as an optimal control problem. Different from many previous studies, the objective function of the proposed problem depends on the switching cost and the size of the continuous state space, making its solution much more challenging. By exploiting some particular features of the problem, the best mode sequence and the optimal switching instants are characterized analytically using a variational approach. Simulation results based on real data shows that the proposed method can significantly reduce the energy consumption compared with another heuristic scheme in several typical situations.

Improved online algorithms for buffer management in QoS switches

We consider the following buffer management problem arising in QoS networks: Packets with specified weights and deadlines arrive at a network switch and need to be forwarded so that the total weight of forwarded packets is maximized. Packets not forwarded before their deadlines are lost. The main result of the article is an online 64/33 ≈ 1.939-competitive algorithm, the first deterministic algorithm for this problem with competitive ratio below 2. For the 2-uniform case we give an algorithm with ratio ≈ 1.377 and a matching lower bound.