State Aggregation Technique Research Articles

Efficient Algorithm for Proportional Lumpability and Its Application to Selfish Mining in Public Blockchains

This paper explores the concept of proportional lumpability as an extension of the original definition of lumpability, addressing the challenges posed by the state space explosion problem in computing performance indices for large stochastic models. Lumpability traditionally relies on state aggregation techniques and is applicable to Markov chains demonstrating structural regularity. Proportional lumpability extends this idea, proposing that the transition rates of a Markov chain can be modified by certain factors, resulting in a lumpable new Markov chain. This concept facilitates the derivation of precise performance indices for the original process. This paper establishes the well-defined nature of the problem of computing the coarsest proportional lumpability that refines a given initial partition, ensuring a unique solution exists. Additionally, a polynomial time algorithm is introduced to solve this problem, offering valuable insights into both the concept of proportional lumpability and the broader realm of partition refinement techniques. The effectiveness of proportional lumpability is demonstrated through a case study that consists of designing a model to investigate selfish mining behaviors on public blockchains. This research contributes to a better understanding of efficient approaches for handling large stochastic models and highlights the practical applicability of proportional lumpability in deriving exact performance indices.

Inventory Management with Advance Booking Information: The Case of Surgical Supplies and Elective Surgeries

Problem definition: Medical operations require a large volume and variety of consumable supplies that are kept in hospital inventory and replenished on a regular basis. Stringent requirements on the availability of these supplies, together with high variability in their daily usage, contribute to the high inventory costs of the surgical departments in hospitals. We investigate the value of utilizing Advance Booking Information (ABI) on elective surgeries—which are often booked up to months in advance—in reducing inventory costs. Methodology/results: We study a single-item, periodic-review, stochastic inventory control problem, where the item demand in each period is driven by the number and type of surgeries requiring the item, and with the available information on elective surgeries integrated into the ordering decisions. Given that item usage from each case is uncertain and only realized after the surgery, ABI provides imperfect information on future demand. Through exact analysis of a simplified version of the problem, as well as extensive numerical experiments using synthetic and real data, enabled using a state aggregation technique, we provide insights on and quantify the value of using ABI as a function of the number of periods of ABI integrated into the ordering decisions. We identify a relevant parameter regime—namely, high backlog (relative to holding) costs and when surgeries are booked sufficiently in advance—where the value of using ABI could be significant and the majority of the benefits can be gained through incorporating only one period of ABI beyond the order lead time. In a case study conducted using real data, we observe up to 26% reduction in average inventory levels, without violating the service levels. Managerial implications: By incorporating readily available elective surgery schedules into replenishment decisions of surgical supplies, hospitals could significantly reduce inventory costs without compromising the availability of the supplies. Funding: This work was partially funded by The Ontario Ministry of Government and Consumer Services (MGCS). The views expressed in the paper are the views of the authors and do not necessarily reflect those of the Province. Supplemental Material: The e-companion is available at https://doi.org/10.1287/msom.2021.0063 .

Link Analysis for Solving Multiple-Access MDPs With Large State Spaces

Wireless communication networks can be well-modeled by Markov Decision Processes (MDPs). While traditional dynamic programming algorithms such as value and policy iteration have lower complexity than brute force strategies, they still suffer from complexity issues for large state spaces. In this paper, the development of moderate complexity algorithms with high performance is sought for wireless network control. To this end, the approximate value function can be computed by projecting the original value function into a lower-dimensional subspace with the careful choice of basis vectors using tools from graph signal processing (GSP). Although GSP theory mainly focuses on undirected graphs, the transition graphs of MDPs for wireless networks are generally directed. For this reason, graph symmetrization based on the co-link method is considered due to its ability to preserve multi-hop dependencies in the graph. Given the multiple-access model under consideration, key properties of the transition probability matrix are exploited to determine the optimal subspace for the approximate value function with low complexity. The numerical results for a multiple-access channel show that the proposed algorithm can find the optimal basis vectors with high accuracy, and furthermore, the approach is robust to changes in system parameters. It is also shown that the projected equation method outperforms the state aggregation technique in producing higher accuracy with a lower runtime complexity.

Proportional lumpability and proportional bisimilarity

In this paper, we deal with the lumpability approach to cope with the state space explosion problem inherent to the computation of the stationary performance indices of large stochastic models. The lumpability method is based on a state aggregation technique and applies to Markov chains exhibiting some structural regularity. Moreover, it allows one to efficiently compute the exact values of the stationary performance indices when the model is actually lumpable. The notion of quasi-lumpability is based on the idea that a Markov chain can be altered by relatively small perturbations of the transition rates in such a way that the new resulting Markov chain is lumpable. In this case, only upper and lower bounds on the performance indices can be derived. Here, we introduce a novel notion of quasi-lumpability, named proportional lumpability, which extends the original definition of lumpability but, differently from the general definition of quasi-lumpability, it allows one to derive exact stationary performance indices for the original process. We then introduce the notion of proportional bisimilarity for the terms of the performance process algebra PEPA. Proportional bisimilarity induces a proportional lumpability on the underlying continuous-time Markov chains. Finally, we prove some compositionality results and show the applicability of our theory through examples.

An Operational State Aggregation Technique for Transmission Expansion Planning Based on Line Benefits

This paper provides a novel technique to represent in a reduced, or compact, way temporal variability in transmission expansion planning (TEP). This reduction is handled by means of “snapshot selection.” Instead of taking into account all the possible operational states and their associated optimal power flow, a reduced group of them is selected that is representative of all the states that should have an influence on investment decisions. Considering this reduced group of operational states should lead to the same investment decisions as if all the snapshots in the target year were considered. Original operational states are compared in the space of the benefits produced by potential reinforcements considered, which are relevant drivers for investment decisions. The benefits produced by these potential reinforcements are computed based on the incremental change in operation costs resulting from their installation. A clustering algorithm is used to group together those operational states where similar line benefits are realized. Our algorithm has been tested on a modified version of the standard IEEE 24 bus system. The method produces promising results and proves to be more efficient than other snapshot selection methods used until now in computing an accurate enough selection of snapshots representing system operation variability in TEP.

Optimal Power Allocation and Scheduling Under Jamming Attacks

In this paper, we consider a jammed wireless scenario where a network operator aims to schedule users to maximize network performance while guaranteeing a minimum performance level to each user. We consider the case where no information about the position and the triggering threshold of the jammer is available. We show that the network performance maximization problem can be modeled as a finite-horizon joint power control and user scheduling problem, which is NP-hard. To find the optimal solution of the problem, we exploit dynamic programming techniques. We show that the obtained problem can be decomposed, i.e., the power control problem and the user scheduling problem can be sequentially solved at each slot. We investigate the impact of uncertainty on the achievable performance of the system and we show that such uncertainty leads to the well-known exploration-exploitation tradeoff. Due to the high complexity of the optimal solution, we introduce an approximation algorithm by exploiting state aggregation techniques. We also propose a performance-aware online greedy algorithm to provide a low-complexity sub-optimal solution to the joint power control and user scheduling problem under minimum quality-of-service requirements. The efficiency of both solutions is evaluated through extensive simulations, and our results show that the proposed solutions outperform other traditional scheduling policies.

Discrete-time chaotic systems synchronization based on vector norms approach

Suitable stabilization conditions obtained for continuous chaotic systems are generalized to discrete-time chaotic systems. The proposed approach, leading to these conditions for complete synchronization is based on the use of state feedback and aggregation techniques for stability studies associated with the arrow form matrix for system description. The results are successfully applied for two identical discrete-time hyper chaotic Henon maps with different orders and also for non-identical discrete-time chaotic systems with same order namely the Lozi and the Ushio maps.

Mean-Field Linear-Quadratic-Gaussian (LQG) Games for Stochastic Integral Systems

In this paper we discuss a class of mean field linear-quadratic-Gaussian (LQG) games for large population system which has never been addressed by existing literature. The features of our works are sketched as follows. First of all, our state is modeled by stochastic Volterra-type equation which leads to some new study on stochastic "integral" system. This feature makes our setup significantly different from the previous mean field games where the states always follow some stochastic "differential" equations. Actually, our stochastic integral system is rather general and can be viewed as natural generalization of stochastic differential equations. In addition, it also includes some types of stochastic delayed systems as its special cases. Second, some new techniques are explored to tackle our mean-field LQG games due to the special structure of integral system. For example, unlike the Riccati equation in linear controlled differential system, some Fredholm-type equations are introduced to characterize the consistency condition of our integral system via the resolvent kernels. Third, based on the state aggregation technique, the Nash certainty equivalence (NCE) equation is derived and the set of associated decentralized controls are verified to satisfy the $\epsilon$-Nash equilibrium property. To this end, some new estimates of stochastic Volterra equations are developed which also have their own interests.

MEC--a near-optimal online reinforcement learning algorithm for continuous deterministic systems.

In this paper, the first probably approximately correct (PAC) algorithm for continuous deterministic systems without relying on any system dynamics is proposed. It combines the state aggregation technique and the efficient exploration principle, and makes high utilization of online observed samples. We use a grid to partition the continuous state space into different cells to save samples. A near-upper Q operator is defined to produce a near-upper Q function using samples in each cell. The corresponding greedy policy effectively balances between exploration and exploitation. With the rigorous analysis, we prove that there is a polynomial time bound of executing nonoptimal actions in our algorithm. After finite steps, the final policy reaches near optimal in the framework of PAC. The implementation requires no knowledge of systems and has less computation complexity. Simulation studies confirm that it is a better performance than other similar PAC algorithms.

Wireless channel model using stochastic high‐level Petri nets for cross‐layer performance analysis in orthogonal frequency‐division multiplexing system

In this study, the authors form a wireless channel model for orthogonal frequency-division multiplexing (OFDM) systems with stochastic high-level Petri net (SHLPN) formalism in order to simplify the cross-layer performance analysis of modern wireless systems. Compared with existing finite state Markov channel model whose state space grows exponentially with the number of OFDM subchannels, the author's proposed SHLPN model uses state aggregation technique to deal with this problem. Closed-form expressions to calculate the transition probabilities among the compound markings of the SHLPN model are provided. When applied to derive the performance measures for OFDM system in terms of the average throughput, average delay and packet dropping probability, the SHLPN model can accurately capture the correlated time-varying nature of wireless channels. Simulation is performed to show that the numerical results offered by the proposed model are more accurate compared with other simplified channel models for avoiding state space complexity.

A new parallel algorithm for solving large-scale Markov chains

In this paper, we propose a new parallel sparse iterative method (PPSIA) for computing the stationary distribution of large-scale Markov chains. The PPSIA method is based on Markov chain state isolation and aggregation techniques. The parallel method conserves as much as possible the benefits of aggregation, and Gauss---Seidel effects contained in the sequential algorithm (SIA) using a pipelined technique. Both SIA and PPSIA exploit sparse matrix representation in order to solve large-scale Markov chains. Some Markov chains have been tested to compare the performance of SIA, PPSIA algorithms with other techniques such as the power method, and the generalized minimal residual GMRES method. In all the tested models, PPSIA outperforms the other methods and shows a super-linear speed-up.

A bi-level approach for the design of event-triggered control systems over a shared network

An efficient usage of available resources is a substantial requirement for the successful design of networked control systems. Recent results indicate major benefits of event-based control compared to conventional designs, when resources such as communication, energy, and computation, are sparse. This paper considers multiple entities of heterogeneous control systems whose feedback loops are coupled through a common communication medium. The design of the decentralized event-triggering control system is formulated as an average-cost problem that aims at the minimization of a social cost criterion. A state aggregation technique is used to develop a bi-level design method, which divides into a local average-cost problem within every subsystem and a global resource allocation problem assigning optimal transmission rates to every subsystem. Stability conditions are derived that guarantee stochastic stability of the aggregate system. Under these conditions, it is shown that the design approach is asymptotically optimal as the number of subsystems increases.

State aggregation-based model of asynchronous multi-fiber optical switching with shared wavelength converters

This paper proposes new analytical models to study optical packet switching architectures with multi-fiber interfaces and shared wavelength converters. The multi-fiber extension of the recently proposed Shared-Per-Input-Wavelength (SPIW) scheme is compared against the multi-fiber Shared-Per-Node (SPN) scheme in terms of cost and performance for asynchronous traffic. In addition to using Markov chains and fixed-point iterations for modeling the mono-fiber case, a novel state aggregation technique is proposed to evaluate the packet loss in asynchronous multi-fiber scenario. The accuracy of the performance models is validated by comparison with simulations in a wide variety of scenarios with both balanced and imbalanced input traffic. The proposed analytical models are shown to remarkably capture the actual system behavior in all scenarios we tested. The adoption of multi-fiber interfaces is shown to achieve remarkable savings in the number of wavelength converters employed and their range. In addition, the SPIW solution allows to save, in particular conditions, a significant number of optical gates compared to the SPN solution. Indeed, SPIW allows, if properly dimensioned, potential complexity and cost reduction compared to SPN, while providing similar performance.

Exact Decomposition Approaches for Markov Decision Processes: A Survey

As classical methods are intractable for solving Markov decision processes (MDPs) requiring a large state space, decomposition and aggregation techniques are very useful to cope with large problems. These techniques are in general a special case of the classic Divide-and-Conquer framework to split a large, unwieldy problem into smaller components and solving the parts in order to construct the global solution. This paper reviews most of decomposition approaches encountered in the associated literature over the past two decades, weighing their pros and cons. We consider several categories of MDPs (average, discounted, and weighted MDPs), and we present briefly a variety of methodologies to find or approximate optimal strategies.

QoS‐aware packet forwarding in MIMO sensor networks: a cross‐layer approach

AbstractMultiple‐input multiple‐output (MIMO) enabled wireless sensor networks (WSNs) are becoming increasingly important since significant performance enhancement can be realized. In this paper, we propose a packet forward strategy for MIMO sensor networks by jointly considering channel coding, rate adaptation, and power allocation. Each sensor node has multiple antennas and uses orthogonal space time block codes (OSTBC) to exploit both spatial and temporal diversities. The objective is to determine the optimal routing path that achieves the minimum symbol error rate (SER) subject to the source‐to‐destination (S‐D) energy consumption constraint. This SER‐based quality‐of‐service (QoS) aware packet forwarding problem is formulated into the framework of dynamic programming (DP). We then propose a low‐complexity and near‐optimal approach to considerably reduce the computation complexity, which includes state space partition and state aggregation techniques. Simulations indicate that the proposed protocol significantly outperforms traditional algorithms. Further still, the performance gain increases with tighter S‐D energy constraint. Copyright © 2009 John Wiley & Sons, Ltd.

Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash Equilibria

We consider linear quadratic Gaussian (LQG) games in large population systems where the agents evolve according to nonuniform dynamics and are coupled via their individual costs. A state aggregation technique is developed to obtain a set of decentralized control laws for the individuals which possesses an epsiv-Nash equilibrium property. A stability property of the mass behavior is established, and the effect of inaccurate population statistics on an isolated agent is also analyzed by variational techniques.

Population aggregation based on fitness

It is known that modelling a finite population genetic algorithm as a Markov chain requires a prohibitively large number of states. In an attempt to resolve this problem, a number of state aggregation techniques have been proposed. We consider two different strategies for aggregating populations, one using equal average fitness and the other using equal best fitness. We examine how the approximation scales with population size, in addition to studying the effects of other parameters (such as mutation rate). We find that a large reduction in the number of states is possible, sometimes with surprisingly small loss of accuracy.