Non-classical Information Structure Research Articles

On Team Decision Problems with Nonclassical Information Structures

In this paper, we consider sequential dynamic team decision problems with nonclassical information structures. First, we address the problem from the point of view of a “manager” who seeks to derive the optimal strategy of the team in a centralized process. We derive structural results that yield an information state for the team which does not depend on the control strategy, and thus it can lead to a dynamic programming decomposition where the optimization problem is over the space of the team's decisions. We, then, derive structural results for each team member that yield an information state which does not depend on their control strategy, and thus it can lead to a dynamic programming decomposition where the optimization problem for each team member is over the space of their decisions. Finally, we show that the solution of each team member is the same as the one derived by the manager. We present an illustrative example of a dynamic team with a delayed sharing information structure.

Deep Teams: Decentralized Decision Making With Finite and Infinite Number of Agents

Inspired by the concepts of deep learning in artificial intelligence and fairness in behavioural economics, we introduce deep teams in this paper. In such systems, agents are partitioned into a few sub-populations so that the dynamics and cost of agents in each sub-population is invariant to the indexing of agents. The goal of agents is to minimize a common cost function in such a manner that the agents in each sub-population are not discriminated or privileged by the way they are indexed. Two non-classical information structures are studied. In the first one, each agent observes its local state as well as the empirical distribution of the states of agents in each sub-population, called deep state, whereas in the second one, the deep states of a subset (possibly all) of sub-populations are not observed. Novel dynamic programs are developed to identify globally optimal and sub-optimal solutions for the first and second information structures, respectively. The computational complexity of finding the optimal solution in both space and time is polynomial (rather than exponential) with respect to the number of agents in each sub-population and is linear (rather than exponential) with respect to the control horizon. This complexity is further reduced in time by introducing a forward equation, that we call deep Chapman-Kolmogorov equation, described by multiple convolutional layers of Binomial probability distributions. Two different prices are defined for computation and communication, and it is shown that under mild assumptions they converge to zero as the quantization level and the number of agents tend to infinity. In addition, the main results are extended to the infinite-horizon discounted model and arbitrarily asymmetric cost function. Finally, a service-management example with 200 users is presented.

Optimal Local and Remote Controllers With Unreliable Uplink Channels: An Elementary Proof

Recently, a model of a decentralized control system with local and remote controllers connected over unreliable channels was presented in [1] . The model has a nonclassical information structure that is not partially nested. Nonetheless, it is shown in [1] that the optimal control strategies are linear functions of the state estimate (which is a nonlinear function of the observations). Their proof is based on a fairly sophisticated dynamic programming argument. In this article, we present an alternative and elementary proof of the result which uses common information-based conditional independence and completion of squares.

A Convex Information Relaxation for Constrained Decentralized Control Design Problems

We describe a convex programming approach to the calculation of lower bounds on the minimum cost of constrained decentralized control problems with nonclassical information structures. The class of problems that we consider entail the decentralized output feedback control of a linear time-varying system over a finite horizon, subject to polyhedral constraints on the state and input trajectories, and sparsity constraints on the controller's information structure. As the determination of optimal control policies for such systems is known to be computationally intractable in general, considerable effort has been made in the literature to identify efficiently computable, albeit suboptimal, feasible control policies. The construction of computationally tractable bounds on their suboptimality is the primary motivation for the techniques developed in this note. Specifically, given a decentralized control problem with nonclassical information, we characterize an expansion of the given information structure, which ensures its partial nestedness, while maximizing the optimal value of the resulting decentralized control problem under the expanded information structure. The resulting decentralized control problem is cast as an infinite-dimensional convex program, which is further relaxed via a partial dualization and a restriction to affine dual control policies. The resulting problem is a finite-dimensional conic program whose optimal value is a provable lower bound on the minimum cost of the original constrained decentralized control problem.

MinMax Mean-Field Team Approach for a Leader–Follower Network: A Saddle-Point Strategy

This letter investigates a soft-constrained MinMax control problem of a leader-follower network. The network consists of one leader and an arbitrary number of followers that wish to reach consensus with minimum energy consumption in the presence of external disturbances. The leader and followers are coupled in the dynamics and cost function. Two non-classical information structures are considered: 1) mean-field sharing and 2) intermittent mean-field sharing, where the mean-field refers to the aggregate state of the followers. In mean-field sharing, every follower observes its local state, the state of the leader and the mean field while in the intermittent mean-field sharing, the mean-field is only observed at some (possibly no) time instants. A social welfare cost function is defined, and it is shown that a unique saddle-point strategy exists which minimizes the worst-case value of the cost function under mean-field sharing information structure. The solution is obtained by two scalable Riccati equations, which depend on a prescribed attenuation parameter, serving as a robustness factor. For the intermittent mean-field sharing information structure, an approximate saddle-point strategy is proposed, and its converges to the saddle-point is analyzed. Two numerical examples are provided to demonstrate the efficacy of the obtained results.

An Optimizer's Approach to Stochastic Control Problems With Nonclassical Information Structures

We present a general optimization-based framework for stochastic control problems with nonclassical information structures. We cast these problems equivalently as optimization problems on joint distributions. The resulting problems are necessarily nonconvex. Our approach to solving them is through convex relaxation . We solve the instance solved by Bansal and Başar (“Stochastic teams with nonclassical information revisited: When is an affine law optimal?”, IEEE Trans. Automatic Control, 1987) with a particular application of this approach that uses the data processing inequality for constructing the convex relaxation. Using certain f-divergences, we obtain a new, larger set of inverse optimal cost functions for such problems. Insights are obtained on the relation between the structure of cost functions and of convex relaxations for inverse optimal control.

Optimal Control Strategies in Delayed Sharing Information Structures

The $n$-step delayed sharing information structure is investigated. This information structure comprises of $K$ controllers that share their information with a delay of $n$ time steps. This information structure is a link between the classical information structure, where information is shared perfectly between the controllers, and a non-classical information structure, where there is no "lateral" sharing of information among the controllers. Structural results for optimal control strategies for systems with such information structures are presented. A sequential methodology for finding the optimal strategies is also derived. The solution approach provides an insight for identifying structural results and sequential decomposition for general decentralized stochastic control problems.

Optimal Design of Sequential Real-Time Communication Systems

Optimal design of sequential real-time communication of a Markov source over a noisy channel is investigated. In such a system, the delay between the source output and its reconstruction at the receiver should equal a fixed prespecified amount. An optimal communication strategy must minimize the total expected symbol-by-symbol distortion between the source output and its reconstruction. Design techniques or performance bounds for such real-time communication systems are unknown. In this paper a systematic methodology, based on the concepts of information structures and information states, to search for an optimal real-time communication strategy is presented. This methodology trades off complexity in communication length (linear in contrast to doubly exponential) with complexity in alphabet sizes (doubly exponential in contrast to exponential). As the communication length is usually order of magnitudes bigger than the alphabet sizes, the proposed methodology simplifies the search for an optimal communication strategy. In spite of this simplification, the resultant optimality equations cannot be solved efficiently using existing algorithmic techniques. The main idea is to formulate a zero-delay communication problem as a dynamic team with nonclassical information structure. Then, an appropriate choice of information states converts the dynamic team problem into a centralized stochastic control problem in function space. Thereafter, Markov decision theory is used to derive nested optimality equations for choosing an optimal design. For infinite horizon problems, these optimality equations give rise to a fixed point functional equation. Communication systems with fixed finite delay constraint, a higher-order Markov source, and channels with memory are treated in the same manner after an appropriate expansion of the state space. Thus, this paper presents a comprehensive methodology to study different variations of real-time communication.

Optimal Performance of Networked Control Systems with Nonclassical Information Structures

A discrete time stochastic feedback control system consisting of a nonlinear plant, a sensor, a controller, and a noisy communication channel between the sensor and the controller is considered. The sensor has limited memory and, at each time, it transmits an encoded symbol over the channel and updates its memory. The controller receives a noise-corrupted copy of the transmitted symbol. It generates a control action based on all its past observations and all its past actions. This control action is fed back to the plant. At each time instant the system incurs an instantaneous cost depending on the state of the plant and the control action. The objective is to choose encoding, memory update, and control strategies to minimize an expected total cost over a finite horizon, or an expected discounted cost over an infinite horizon, or an average cost per unit time over an infinite horizon. A solution methodology for obtaining a sequential decomposition of the global optimization problem is developed. This solution methodology is extended to the case when the sensor makes an imperfect observation of the state of the plant.

Information Reduction in Optimally Controlling Composite Systems

The objective of this paper is to study possibilities of eliminating redundancies of informations available to controllers in optimally controlling composite systems. First, in a general framework of stochastic optimal control problem with nonclassical information structure, a new type of information reduction problem is formulated and a sufficient condition for information reductions is presented. Second, this condition is applied to the cases with some special structures of composite systems and cost functionals, and the features of couplings bearing information reductions are characterized. In particular, the case of composite systems with multiplicatively noisy couplings is elaborated on.

Another look at the nonclassical information structure problem

In this paper, we provide another view of the basic difficulty of a nonclassical information structure decision problem. Computational considerations led us to the result that, under the assumption of linear information structure, the partially nested structure is the only class of structures which are equivalent to a static one.

On Decentralized Multicriteria Linear-Quadratic Stochastic Control Problems

In the framework of linear stochastic differential systems and quadratic cost functionals, 2-member 2-criteria control problems are discussed and the unique Nash equilibrium solution is derived under some nonclassical information structures, i.e., “stochastic open loop” and “strongly partially nested” information structures, with observation delays. It is shown that this unique solution can be realized by the linear functions of the state estimates. As a special case, the team control problem is elaborated on. These results does not require Gaussian properties of the initial state and the noises.

Multi-Agent Linear-Quadratic-Gaussian Systems with “Poor” Observations

Very few solutions are available for multi-agent control problems which have a nonclassical information structure. This is due to the fact that the agents can use their control actions not only for control itself, but also to pass information to other agents. We consider decentralized control situations in which information transfer is comparitively difficult because of large noise on some of the observation variables. By obtaining appropriate limits to this information transfer, linear control laws derived by disregarding the “poor” observations are shown to be “close” to optimal. It is then possible to derive other linear laws, from a similar situation again with static information structure, which are an order of magnitude closer, both in respect of the laws themselves and the optimized costs. It is shown however that iteration of this process in a natural way does not produce substantially better results.