Relative Entropy Constraint Research Articles

Generalization Analysis of Machine Learning Algorithms via the Worst-Case Data-Generating Probability Measure

In this paper, the worst-case probability measure over the data is introduced as a tool for characterizing the generalization capabilities of machine learning algorithms. More specifically, the worst-case probability measure is a Gibbs probability measure and the unique solution to the maximization of the expected loss under a relative entropy constraint with respect to a reference probability measure. Fundamental generalization metrics, such as the sensitivity of the expected loss, the sensitivity of the empirical risk, and the generalization gap are shown to have closed-form expressions involving the worst-case data-generating probability measure. Existing results for the Gibbs algorithm, such as characterizing the generalization gap as a sum of mutual information and lautum information, up to a constant factor, are recovered. A novel parallel is established between the worst-case data-generating probability measure and the Gibbs algorithm. Specifically, the Gibbs probability measure is identified as a fundamental commonality of the model space and the data space for machine learning algorithms.

Training Autoencoders Using Relative Entropy Constraints

Autoencoders are widely used for dimensionality reduction and feature extraction. The backpropagation algorithm for training the parameters of the autoencoder model suffers from problems such as slow convergence. Therefore, researchers propose forward propagation algorithms. However, the existing forward propagation algorithms do not consider the characteristics of the data itself. This paper proposes an autoencoder forward training algorithm based on relative entropy constraints, called relative entropy autoencoder (REAE). When solving the feature map parameters, REAE imposes different constraints on the average activation value of the hidden layer outputs obtained by the feature map for different data sets. In the experimental section, different forward propagation algorithms are compared by applying the features extracted from the autoencoder to an image classification task. The experimental results on three image classification datasets show that the classification performance of the classification model constructed by REAE is better than that of the classification model constructed by other forward propagation algorithms.

Optimised radar waveform design based on the relative entropy constraint

A new strategy to optimise the waveform of multiple-input multiple-output (MIMO) radar under the relative entropy constraint and signal-to-noise ratio (SNR) constraint is presented. The Kullback–Leibler divergence (KLD) is exploited between the two probability density functions of binary hypothesis testing as the design metric. A particular analysis of detection performance is made here to illuminate the balance between the false alarm and missing alarm probabilities. The results of the simulation demonstrate that the waveform design process constrained by relative entropy outperforms the process employing an explicit SNR constraint when the SNR is relatively low (i.e., within the interval [−2 dB, 0.5 dB]). The enhanced performance is attributable to the capability of the proposed optimisation process to focus the energy of the transmission waveform onto components where the eigenvalues of the target signal are large and the eigenvalues of the noise are small.

Robust Classical-Impulse Stochastic Control Problems in an In finite Horizon

This paper establishes a general analytical framework for classical and impulse stochastic control problems in the presence of model uncertainty. We consider a set of dominated models, which are induced by the measures equivalent to that of a reference model. The state process under the reference model is a multidimensional Markov process with multidimensional Brownian motion, controlled by continuous and impulse control variates. We propose quasi-variational inequalities (QVI) associated with the value function of the control problem and prove a veri cation theorem for the solution to the QVI. With the relative entropy constraints and piecewise linear intervention penalty, we show that the QVI can be degenerated to the non-robust case and it can be solved via the solution to a free boundary problem. To illustrate the tractability of the proposed framework, we apply it to a linearquadratic setting, which covers a broad class of problems including robust mean-reverting inventory controls.

Event-triggered minimax state estimation with a relative entropy constraint

In this paper, we consider an event-triggered minimax state estimation problem for uncertain systems subject to a relative entropy constraint. This minimax estimation problem is formulated as an equivalent event-triggered linear exponential quadratic Gaussian problem. It is then shown that this problem can be solved via dynamic programming and a newly defined information state. As the solution to this dynamic programming problem is computationally intractable, a one-step event-triggered minimax estimation problem is further formulated and solved, where an a posteriori relative entropy is introduced as a measure of the discrepancy between probability measures. The resulting estimator is shown to evolve in recursive closed-form expressions. For the multi-sensor system scenario, a one-step event-triggered minimax estimator is also presented in a sequential fusion way. Finally, comparative simulation examples are provided to illustrate the performance of the proposed one-step event-triggered minimax estimators.

Data‐Driven Robust Credit Portfolio Optimization for Investment Decisions in P2P Lending

Peer‐to‐Peer (P2P) lending has attracted increasing attention recently. As an emerging micro‐finance platform, P2P lending plays roles in removing intermediaries, reducing transaction costs, and increasing the benefits of both borrowers and lenders. However, for the P2P lending investment, there are two major challenges, the deficiency of loans’ historical observations about the certain borrower and the ambiguity problem of estimated loans’ distribution. In order to solve the difficulties, this paper proposes a data‐driven robust model of portfolio optimization with relative entropy constraints based on an “instance‐based” credit risk assessment framework. The model exploits a nonparametric kernel approach to estimate P2P loans’ expected return and risk under the condition that the historical data of the same borrower is unavailable. Furthermore, we construct a robust mean–variance optimization problem based on relative entropy method for P2P loan investment decision. Using the real‐world dataset from a notable P2P lending platform, Prosper, we validate the proposed model. Empirical results reveal that our model provides better investment performances than the existing model.

State-Regularized Policy Search for Linearized Dynamical Systems

Trajectory-Centric Reinforcement Learning and Trajectory Optimization methods optimize a sequence of feedback-controllers by taking advantage of local approximations of model dynamics and cost functions. Stability of the policy update is a major issue for these methods, rendering them hard to apply for highly nonlinear systems. Recent approaches combine classical Stochastic Optimal Control methods with information-theoretic bounds to control the step-size of the policy update and could even be used to train nonlinear deep control policies. These methods bound the relative entropy between the new and the old policy to ensure a stable policy update. However, despite the bound in policy space, the state distributions of two consecutive policies can still differ significantly, rendering the used local approximate models invalid. To alleviate this issue we propose enforcing a relative entropy constraint not only on the policy update, but also on the update of the state distribution, around which the dynamics and cost are being approximated. We present a derivation of the closed-form policy update and show that our approach outperforms related methods on two nonlinear and highly dynamic simulated systems.

Optimal asset allocation: Risk and information uncertainty

In asset allocation problem, the distribution of the assets is usually assumed to be known in order to identify the optimal portfolio. In practice, we need to estimate their distribution. The estimations are not necessarily accurate and it is known as the uncertainty problem. Many researches show that most people are uncertainty aversion and this affects their investment strategy. In this article, we consider risk and information uncertainty under a common asset allocation framework. The effects of risk premium and covariance uncertainty are demonstrated by the worst scenario in a set of measures generated by a relative entropy constraint. The nature of the uncertainty and its impacts on the asset allocation are discussed.

Robust State Space Filtering Under Incremental Model Perturbations Subject to a Relative Entropy Tolerance

This paper considers robust filtering for a nominal Gaussian state-space model, when a relative entropy tolerance is applied to each time increment of a dynamical model. The problem is formulated as a dynamic minimax game where the maximizer adopts a myopic strategy. This game is shown to admit a saddle point whose structure is characterized by applying and extending results presented earlier in “Robust least-squares estimation with a relative entropy constraint” (B. C. Levy and R. Nikoukhah, IEEE Trans. Inf. Theory, vol. 50, no. 1, 89-104, Jan. 2004) for static least-squares estimation. The resulting minimax filter takes the form of a risk-sensitive filter with a time varying risk sensitivity parameter, which depends on the tolerance bound applied to the model dynamics and observations at the corresponding time index. The least-favorable model is constructed and used to evaluate the performance of alternative filters. Simulations comparing the proposed risk-sensitive filter to a standard Kalman filter show a significant performance advantage when applied to the least-favorable model, and only a small performance loss for the nominal model.

Stability and reliable data reconstruction of uncertain dynamic systems over finite capacity channels

In this paper we present an encoder, decoder and a stabilizing controller for reliable data reconstruction and robust stability of uncertain dynamic systems controlled over Additive White Gaussian Noise (AWGN) channels. The uncertainty in the dynamic system is described by a relative entropy constraint. Such an uncertainty description is a natural stochastic generalization of the sum quadratic uncertainty description. This paper complements the results of Farhadi and Charalambous (2008) by showing that the necessary condition presented there can be tight. This is shown by designing an encoder, decoder and a stabilizing controller.

Nonlinear Estimation for a Class of Systems

This paper considers nonlinear estimation problems for classes of models, and employs relative entropy to describe the uncertainty classes. Two optimization problems are formulated on general Banach spaces, and their solutions are sought: 1) when the transition probability between the signal to be estimated X and the measurement Y or stochastic kernel is unknown, and 2) when the joint probability induced by the random variables (RVs) X, Y is unknown. For both problems, the uncertainty is described by a relative entropy constraint between the unknown distribution and a fixed nominal distribution. The results include existence of the optimal measures using weak convergence techniques, and properties associated with the estimate of the true distribution. Classical examples are chosen to illustrate the applicability of the results.

Robust coding for a class of sources: Applications in control and reliable communication over limited capacity channels

This paper is concerned with the control of a class of dynamical systems over finite capacity communication channels. Necessary conditions for reliable data reconstruction and stability of a class of dynamical systems are derived. The methodology is information theoretic. It introduces the notion of entropy for a class of sources, which is defined as a maximization of the Shannon entropy over a class of sources. It also introduces the Shannon information transmission theorem for a class of sources, which states that channel capacity should be greater or equal to the mini-max rate distortion (maximization is over the class of sources) for reliable communication. When the class of sources is described by a relative entropy constraint between a class of source densities, and a given nominal source density, the explicit solution to the maximum entropy, is given, and its connection to Rényi entropy is illustrated. Furthermore, this solution is applied to a class of controlled dynamical systems to address necessary conditions for reliable data reconstruction and stability of such systems.

Finite Horizon Robust State Estimation for Uncertain Finite-Alphabet Hidden Markov Models with Conditional Relative Entropy Constraints

We consider a robust state estimation problem for time-varying uncertain discrete-time, homogeneous, first-order, finite-state finite-alphabet hidden Markov models (HMMs). A class of time-varying uncertain HMMs is considered in which the uncertainty is sequentially described by a conditional relative entropy constraint on perturbed conditional probability measures given a realized observation sequence. For this class of uncertain HMMs, the robust state estimation problem is formulated as a constrained optimization problem. Using a Lagrange multiplier technique and a variational formula for conditional relative entropy, the above problem is converted into an unconstrained optimization problem and a problem related to partial information risk-sensitive filtering. A measure transformation technique and an information state method are employed to solve this equivalent problem related to risk-sensitive filtering. A characterization of the solution to the robust state estimation problem is also presented.

Relative entropy and spectral constraints: some invariance properties of the ARMA class

Abstract. We determine the form of spectral densities of multidimensional scalar processes which minimize a relative entropy under a finite number of general moment‐type constraints. The obtained theoretical results are applied to spectral densities of weakly stationary processes under covariances, inverse covariances and cepstral or impulse response constraints. Invariance properties of the class of autoregressive moving‐average (ARMA) processes are shown to hold under the relative entropy minimization principle for many choices of entropy.

Stochastic Uncertain Systems Subject to Relative Entropy Constraints: Induced Norms and Monotonicity Properties of Minimax Games

Entropy and relative entropy are fundamental concepts on which information theory is founded on, and in general, telecommunication systems design. On the other hand, dissipation inequalities, minimax strategies, and induced norms are the basic concepts on which robustness of uncertain control and estimation of systems are founded on. In this paper, the precise relation between these notions is investigated. In particular, it will be shown that the higher the dissipation the higher the entropy of the system, which has implications in computing the induced norm associated with robustness. These connections are obtained by considering stochastic optimal uncertain control systems, in which uncertainty is described by a relative entropy constraint between the nominal and uncertain measures, while the pay-off is a linear functional of the uncertain measure. This is a minimax game, in which the controller measure seeks to minimize the pay-off, while the disturbance measure aims at maximizing the pay-off. Salient properties of the minimax solution are derived, including a characterization of the optimal sensitivity reduction, computation of the induced norm, monotonicity properties of minimax solution, and relations between dissipation and relative entropy of the system. The theory is developed in an abstract setting and then applied to nonlinear partially observable continuous-time uncertain controlled systems, in which the nominal and uncertain systems are described by conditional distributions. In addition, existence of the optimal control policy among the class of policies known as wide-sense control laws is shown, and an explicit formulae for the worst case conditional measure is derived. The results are applied to linear-quadratic-Gaussian problems

On the worst-case disturbance of minimax optimal control

We consider a minimax optimal control problem for uncertain stochastic systems. The uncertainty in the underlying stochastic system is formulated in terms of probability measure perturbations satisfying a relative entropy constraint. By characterizing the worst-case measure for a related stochastic minimax game, it is shown that the worst-case uncertain system can be represented in the form of a parametric perturbation of the nominal system. A numerical example is presented to illustrate theoretical results developed in this paper.

Equilibrium Asset Pricing with Time-varying Pessimism

We study the equilibrium pricing effects of a sentiment for pessimism. Pessimism has the form of Knightian model uncertainty aversion for a neighborhood of indistinguishable model specifications that are constrained in their relative entropy from a given reference model. We fully characterise the equilibrium of a pessimistic, representative agent, exchange economy with intertemporal consumption, stochastic opportunity set, and a relative entropy constraint that can depend on the state of the economy. We find that Knightian pessimism generates substantial First Order Risk Aversion (FORA) effects that enhance excess equity returns by pushing riskfree rates down. However, we find that the structure of equity returns is virtually unaffected by a Knightian concern for model uncertainty. We compute and calibrate explicit equilibrium examples of a pessimistic economy with an amount of pessimism associated to an 11% upper probability bound of confusing the relevant worst-case model and the given reference model. Relative entropy is the key in fixing such a realistic amount of pessimism in our calibrations. Even for log utility, such small amount of pessimism generates some 55 basis points more of unconditional equity premium. Knightian pessimism provides an economically and observationally different description of excess equity returns. Our findings show that realistic amounts of both pessimism and standard risk aversion yield substantial equity premia and low riskfree rates.

Robust filtering of stochastic uncertain systems on an infinite time horizon

In this paper, we consider a robust filtering problem for continuous time stochastic uncertain systems.The uncertainty in the system is characterized in terms of an uncertain probability distribution on the noise input. This uncertainty is assumed to satisfy a certain relative entropy constraint. The solution to a specially parameterized risk-sensitive stochastic filtering problem is used to construct a filter for the original uncertain system which guarantees an optimal worst-case filtering error. The corresponding minimax optimal filter is obtained by solving a pair of algebraic Riccati equations.

GUARANTEED ERROR ROBUST FIR EQUALIZATION FOR UNCERTAIN CHANNELS

The equalization problem is a problem of estimating a signal transmitted over a telecommunication channel. This problem has a range of applications in telecommunications. In this paper, we consider a version of the problem in which the channel model contains an uncertainty. Our goal is to design a robust finite impulse response (FIR) equalizer which guarantees certain bound on the worst-case estimation error. The design procedure presented in the paper relies on the recent progress in maxdet convex optimization. Another result used in the paper concerns minimax optimal control and filtering of uncertain systems with relative entropy constraints on the uncertainty.

Minimax LQG Control of Stochastic Partially Observed Uncertain Systems

We consider an infinite-horizon linear-quadratic minimax optimal control problem for stochastic uncertain systems with output measurement. A new description of stochastic uncertainty is introduced using a relative entropy constraint. For the stochastic uncertain system under consideration, a connection between the worst-case control design problem and a specially parametrized risk-sensitive stochastic control problem is established. Using this connection, a minimax optimal LQG controller is constructed which is based on a pair of algebraic matrix Riccati equations arising in risk-sensitive control. It is shown that this minimax optimal controller absolutely stabilizes the stochastic uncertain system.