Convex Potential Function Research Articles

Variational Principles for Mirror Descent and Mirror Langevin Dynamics

Mirror descent, introduced by Nemirovski and Yudin in the 1970s, is a primal-dual convex optimization method that can be tailored to the geometry of the optimization problem at hand through the choice of a strongly convex potential function. It arises as a basic primitive in a variety of applications, including large-scale optimization, machine learning, and control. This paper proposes a variational formulation of mirror descent and of its stochastic variant, mirror Langevin dynamics. The main idea, inspired by the classic work of Brezis and Ekeland on variational principles for gradient flows, is to show that mirror descent emerges as a closed-loop solution for a certain optimal control problem, and the Bellman value function is given by the Bregman divergence between the initial condition and the global minimizer of the objective function.

On Ricci solitons whose potential is convex

In this paper, we consider the Ricci curvature of a Ricci soliton. In particular, we have showed that a complete gradient Ricci soliton with non-negative Ricci curvature possessing a non-constant convex potential function having finite weighted Dirichlet integral satisfying an integral condition is Ricci flat and also it isometrically splits a line. We have also proved that a gradient Ricci soliton with non-constant concave potential function and bounded Ricci curvature is non-shrinking and hence the scalar curvature has at most one critical point.

Soft clustering by convex electoral model

In this paper, we suggest a new technique for soft clustering of multidimensional data. It is based on a new convex voting model, where each voter chooses a party with certain probability depending on the divergence between his/her preferences and the position of the party. The parties can react on the results of polls by changing their positions. We prove that under some natural assumptions this system has a unique fixed point, providing a unique solution for soft clustering. The solution of our model can be found either by imitation of the sequential elections, or by direct minimization of a convex potential function. In both cases, the methods converge linearly to the solution. We provide our methods with worst-case complexity bounds. To the best of our knowledge, these are the first polynomial-time complexity results in this field.

Tatonnement beyond gross substitutes? Gradient descent to the rescue

We identify a class of economies for which tatonnement is equivalent to gradient descent. This is the class of economies for which there is a convex potential function whose gradient is always equal to the negative of the excess demand. Among other consequences, we show that a discrete version of tatonnement converges to the equilibrium for the following economies of complementary goods.i. Fisher economies in which all buyers have complementary CES utilities, with a linear rate of convergence. (In Fisher economies all agents are either buyers or sellers of non-numeraire goods, but not both.)This shows that tatonnement converges for the entire range of Fisher economies when buyers have complementary CES utilities, in contrast to prior work, which could analyze only the substitutes range, together with a small portion of the complementary range.ii. Fisher economies in which all buyers have Leontief utilities, with an O(1/t) rate of convergence.

Learning Generalized Nash Equilibria in a Class of Convex Games

We consider multiagent decision making where each agent optimizes its convex cost function subject to individual and coupling constraints. The constraint sets are compact convex subsets of a Euclidean space. To learn Nash equilibria, we propose a novel distributed payoff-based algorithm, where each agent uses information only about its cost value and the constraint value with its associated dual multiplier. We prove convergence of this algorithm to a Nash equilibrium, under the assumption that the game admits a strictly convex potential function. In the absence of coupling constraints, we prove convergence to Nash equilibria under significantly weaker assumptions, not requiring a potential function. Namely, strict monotonicity of the game mapping is sufficient for convergence. We also derive the convergence rate of the algorithm for strongly monotone game maps.

Sensitivity to Cumulative Perturbations for a Class of Piecewise Constant Hybrid Systems

We consider a class of continuous-time hybrid dynamical systems that correspond to subgradient flows of a piecewise linear and convex potential function with finitely many pieces, and which includes the fluid-level dynamics of the Max-Weight scheduling policy as a special case. We study the effect of an external disturbance/perturbation on the state trajectory, and establish that the magnitude of this effect can be bounded by a constant multiple of the integral of the perturbation.

Attitude-constrained reorientation control for spacecraft based on extended state observer

This article investigates the spacecraft attitude reorientation control problem in the presence of attitude constraint, actuator saturation, parametric uncertainty, and external disturbance. First, a nonlinear tracking law based on a strictly convex potential function is proposed to generate the virtual control angular velocity which has only one global minimum. Then, utilizing the auxiliary system governed by the difference between the upper bound of actuator torque and the untreated command torque, a novel backstepping controller is presented, which is able to satisfy the constraint of actuator saturation and guarantee the stability of control system. In addition, an extended state observer with the uniformly ultimately bounded estimation error and finite-time stability is put forward to realize the real-time compensation of the compound disturbance consisting of parametric uncertainty and external disturbance. Therefore, it enhances the robustness and improves the accuracy of the extended state observer–based backstepping controller. Finally, simulation results validate the effectiveness and reliability of the proposed schemes.

Generalized mirror descents with non-convex potential functions in atomic congestion games: Continuous time and discrete time

When playing certain specific classes of no-regret algorithms such as multiplicative updates and replicator dynamics in atomic congestion games, some previous convergence analyses were done with the standard Rosenthal potential function in terms of mixed strategy profiles (i.e., probability distributions on atomic flows), which could be non-convex. In several other works, the convergence, when playing the mirror-descent algorithm (a more general family of no-regret algorithms including multiplicative updates, gradient descents, etc.), was guaranteed with a convex potential function in terms of nonatomic flows as an approximation of the Rosenthal one. The convexity of the potential function provides convenience for analysis. One may wonder if the convergence of mirror descents can still be guaranteed directly with the non-convex Rosenthal potential function. In this paper, we answer the question affirmatively for discrete-time generalized mirror descents with the smoothness property (similarly adopted in many previous works for congestion games and markets) and for continuous-time generalized mirror descents with the separability of regularization functions.

Image Regularity and Fidelity Measure with a Two-Modality Potential Function

We define a strictly convex smooth potential function and use it to measure the data fidelity as well as the regularity for image denoising and cartoon-texture decomposition. The new model has several advantages over the well-known ROF or TV-L2 and the TV-L1 model. First, due to the two-modality property of the new potential function, the new regularity has strong regularizing properties in all directions and thus encourages removing noise in smooth areas, while, near edges, it smoothes the edge mainly along the tangent direction and thus can well preserve the edges. Second, the new potential function is very close to the L1 norm; thus using it to measure the data fidelity makes the new model perform very well in removing impulse noise and preserving the contrast. Lastly, the proposed fidelity and regularization term is strictly convex and smooth and thus allows a unique global minimizer and it can be solved by using the steepest descent method. Numerical experiments show that the proposed model outperforms TV-L2 and TV-L1 in removing impulse noise and mixed noise. It also outperforms some state-of-the-art methods specially designed for impulse noise. Tests on cartoon-texture decomposition show that our method is effective and performs better than TV-L1.

CONTACT FORCE MODEL FOR A BEAM WITH DISCRETELY SPACED GAP SUPPORTS AND ITS APPROXIMATED SOLUTION

This paper proposes an approximated contact force model to identify the nonlinear behavior of a fuel rod with gap supports; also, the numerical prediction of interfacial forces in the mechanical contact of fuel rods with gap supports is studied. The Newmark integration method requires the current status of the contact force, but the contact force is not given a priori. Taylor's expansion can be used to predict the unknown contact force; therefore, it should be guaranteed that the first derivative of the contact force is continuous. This work proposes a continuous and differentiable contact force model with the ability to estimate the current state of the contact force. An approximated convex and differentiable potential function for the contact force is described, and a variational formulation is also provided. A numerical example that considers the particularly stiff supports has been studied, and a fuel rod with hardening supports was also examined for a realistic simulation. An approximated proper solution can be obtained using the results, and abrupt changes from the contacting state to non-contacting state, or vice versa, can be relieved. It can also be seen that not only the external force but also the developed contact force affects the response.

Random classification noise defeats all convex potential boosters

A broad class of boosting algorithms can be interpreted as performing coordinate-wise gradient descent to minimize some potential function of the margins of a data set. This class includes AdaBoost, LogitBoost, and other widely used and well-studied boosters. In this paper we show that for a broad class of convex potential functions, any such boosting algorithm is highly susceptible to random classification noise. We do this by showing that for any such booster and any nonzero random classification noise rate ?, there is a simple data set of examples which is efficiently learnable by such a booster if there is no noise, but which cannot be learned to accuracy better than 1/2 if there is random classification noise at rate ?. This holds even if the booster regularizes using early stopping or a bound on the L 1 norm of the voting weights. This negative result is in contrast with known branching program based boosters which do not fall into the convex potential function framework and which can provably learn to high accuracy in the presence of random classification noise.

Thermal-stress concentration near inclusions in viscoelastic random composites

The determination of thermal-stress concentrations near inclusions in viscoelastic random composites is concerned with the prediction of the overall response of random nonlinear viscoelastic multi-component media. The continuum considered here is assumed to be subjected to a finite deformation. First Piola’s stress tensor and deformation gradient are used as conjugate field variables in a fixed reference state. A nonlinear problem is investigated in a second-order approximation theory when the gradient deformation terms higher than second order are neglected. A convex potential function in a thermo-elastic problem and time functionals in a viscoelastic one are used to construct overall constitutive relations. The technique of surface operators developed by R. Hill and others is used to determine stress concentrations near inclusions for nonlinear matrix creep.

Multicommodity flow problems with a bounded number of paths: A flow deviation approach

AbstractWe propose a modified version of the Flow Deviation method of Fratta, Gerla, and Kleinrock to solve multicommodity problems with minimal congestion and a bounded number of active paths. We discuss the approximation of the min–max objective function by a separable convex potential function and give a mixed‐integer nonlinear model for the constrained routing problem. Heuristic control of the path generation is then embedded in the original algorithm based on the concepts of cleaning the quasi inactive paths and reducing the flow width for each commodity. Numerical experiments show the validity of the approach for realistic medium‐sized networks associated with routing problems in broadband communication networks with multiple protocols and label‐switched paths (MPLS). © 2006 Wiley Periodicals, Inc. NETWORKS, Vol. 49(1), 80–89 2007

Parameter estimation in Bayesian reconstruction of SPECT images: An aid in nuclear medicine diagnosis

AbstractDespite the adequacy of Bayesian methods to reconstruct nuclear medicine SPECT (single‐photon emission computed tomography) images, they are rarely used in everyday medical practice. This is primarily because of their computational cost and the need to appropriately select the prior model hyperparameters. We propose a simple procedure for the estimation of these hyperparameters and the reconstruction of the original image and test the procedure on both synthetic and real SPECT images. The experimental results demonstrate that the proposed hyperparameter estimation method produces satisfactory reconstructions. Although we have used generalized Gaussian Markov random fields (GGMRF) as prior models, the proposed estimation method can be applied to any priors with convex potential and tractable partition function with respect to the scale hyperparameter. © 2004 Wiley Periodicals, Inc. Int J Imaging Syst Technol 14, 21–27, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.20003

Amoebas, Monge-Ampàre measures, and triangulations of the Newton polytope

The amoeba of a holomorphic function f is, by definition, the image in R n of the zero locus of f under the simple mapping that takes each coordinate to the logarithm of its modulus. The terminology was introduced in the 1990s by the famous (biologist and) mathematician Israel Gelfand and his coauthors Kapranov and Zelevinsky (GKZ). In this paper we study a natural convex potential function N f with the property that its Monge-Ampére mass is concentrated to the amoeba of f We obtain results of two kinds; by approximating N f with a piecewise linear function, we get striking combinatorial information regarding the amoeba and the Newton polytope of f; by computing the Monge-Ampére measure, we find sharp bounds for the area of amoebas in ℝ . We also consider systems of functions f 1 ,…, f n and prove a local version of the classical Bernstein theorem on the number of roots of systems of algebraic equations.

Validation of new Gibbs priors for Bayesian tomographic reconstruction using numerical studies and physically acquired data

The variety of Bayesian MAP approaches to emission tomography proposed in recent years can both stabilize reconstructions and lead to improved bias and variance. In the authors' previous work (S.J. Lee et al., IEEE Trans. Med. Imaging, vol. MI-14, no. 4, p. 669-80, 1995; S.J. Lee et al., IEEE Trans. Nuclear Science, vol. NS-44, no. 3, p. 1381-7, 1997), the authors showed that the thin-plate (TP) prior, which is less sensitive to variations in first spatial derivatives than the conventional membrane (MM) prior, yields improved reconstructions in the sense of low bias. In spite of the several advantages of such quadratic smoothing priors, they are still less than ideal due to their limitations in edge preservation. Here, the authors use a convex but nonquadratic (CNQ) potential function, which provides a degree of edge preservation. As in the case of quadratic priors, a class of two-dimensional smoothing splines with first and second partial derivatives are applied to the new potential function. In order to reduce difficulties such as oversmoothing for MM and edge overshooting for TP, the authors also generalize the prior energy definition to that of a linear combination of MM and TP using a control parameter, and observe its transition between the two extreme cases. To validate advantages of their new priors, the authors first perform extensive numerical studies using a digital phantom to compare the bias/variance behavior of CNQ priors with that of quadratic priors. They also use physically acquired PET emission and transmission data from phantoms to observe the efficacies of their new priors. The authors' numerical studies and results using physical phantoms show that a combination of first and second partial derivatives applied to the CNQ potential yields improved quantitative results in terms of scalar metrics of image quality computed from independent noise trials and good qualitative results for both emission and transmission images.

Bounds of third order for the overall response of nonlinear composites

Composites whose response can be described in terms of a convex potential function are discussed. Bounds are constructed for the overall, or effective, potential of the composite, given the individual potentials of its constituents. Steady-state creep is considered explicitly but the results apply equally well to physically nonlinear elasticity, or deformation-theory plasticity, if strain-rate is reinterpreted as infinitesimal strain. Earlier work employed a linear “comparison” medium. This permitted the construction of only one bound—either an upper bound or a lower bound—or even in some cases no bound at all. Use of a nonlinear comparison medium removes this restriction but at the expense of requiring detailed exploration of the properties of the trial fields that are employed. The fields used here—and previously—have the property of “bounded mean oscillation”; the use of a theorem that applies to such fields permits the construction of the bounds that were previously inaccessible. Illustrative results, which allow for three-point correlations, are presented for an isotropic two-phase composite, each component of which is isotropic, incompressible and conforms to a power-law relation between equivalent stress and equivalent strain-rate. Generalized Hashin-Shtrikman-type bounds follow by allowing the parameter corresponding to the three-point correlations to take its extreme values.

Micromechanics and homogenization of inelastic composite materials with growing cracks

A homogenization scheme is employed to derive the effective constitutive equations of an elastoplastic composite system with growing damage. The homogenization procedure followed herein is based on the thermodynamics of dissipative media. It is shown that when damage consists of sharps microcracks the macroscopic constitutive behavior is that of a so-called generalized standard material. The latter is a general dissipative medium whose constitutive equations are completely characterized by a single scalar convex potential function of the chosen state variables and whose evolution is completely characterized by a single convex dissipation potential function of the thermodynamic forces conjugate to the chosen internal state variables. The analysis presented is valid under the assumption that the evolution of the representative volume element at hand is unique and stable. The results of the theoretical analysis are then employed for formulating an approximate method for practically deriving the macroscopic constitutive equations. Computer software development for the application of said method is currently ongoing. A simple example of the numerical results obtained so far is presented.

Convex Potentials and their Conjugates in Analog Mean-Field Optimization

This paper deals with the problem of mapping hybrid (i.e., both discrete and continuous) constrained optimization problems onto analog networks. The saddle-point paradigm of mean-field methods in statistical physics provides a systematic procedure for finding such a mapping via the notion of effective energy. Specifically, it is shown that within this paradigm, to each closed bounded constraint set is associated a smooth convex potential function. Using the conjugate (or the Legendre-Fenchel transform) of the convex potential, the effective energy can be transformed to yield a cost function that is a natural generalization of the analog Hopfield energy. Descent dynamics and deterministic annealing can then be used to find the global minimum of the original minimization problem. When the conjugate is hard to compute explicitly, it is shown that a minimax dynamics, similar to that of Arrow and Hurwicz in Lagrangian optimization, can be used to find the saddle points of the effective energy. As an illustration of its wide applicability, the effective energy framework is used to derive Hopfield-like energy functions and descent dynamics for two classes of networks previously considered in the literature, winner-take-all networks and rotor networks, even when the cost function of the original optimization problem is not quadratic.

Upper and lower bounds for the overall properties of a nonlinear composite dielectric. I. Random microgeometry

The direct extension of the Hashin-Shtrikman methodology to nonlinear composite problems generally produces at most one new bound - either an upper bound or a lower bound - and in some cases produces no new bound at all. This paper is devoted to the construction of bounds, of generalized Hashin-Shtrikman type, for any nonlinear composite whose behaviour can be characterized in terms of a convex potential function. The construction relies on the use of a nonlinear comparison medium’ and trial fields with the property of ‘bounded mean oscillation’. This permits the exercise of control over the size of the penalty incurred from the use of a nonlinear, as opposed to linear, comparison medium. In cases where a linear comparison medium is adequate, the already established bounds of Hashin-Shtrikman type are reproduced. The exposition is presented in the context of bounding the properties of a nonlinear dielectric, for which a single bound was obtained previously by one of the authors. The approach, however, is applicable more generally.