Class Of Convex Optimization Problems Research Articles

Certifying Polynomial Nonnegativity via Hyperbolic Optimization

We describe a new approach to certifying the global nonnegativity of multivariate polynomials by solving hyperbolic optimization problems---a class of convex optimization problems that generalize s...

Fusion Estimation for Discrete Time-Varying Systems With Bounded Nonlinearities

This paper studies the distributed fusion estimation problem for a class of time-varying systems with bounded nonlinearities, where nonlinear terms are considered under two different cases. By using matrix analysis approach, each local estimator with time-varying gain is designed by minimizing an upper bound of the square error of the estimator. Based on the obtained local estimators, the distributed fusion estimators are designed by establishing a class of convex optimization problems. Notice that the designed local estimators and fusion estimators are stable, and the established convex optimization problems can be easily solved by standard software packages. Localization of a mobile robot is given to demonstrate the effectiveness of the proposed methods.

Tight Global Linear Convergence Rate Bounds for Operator Splitting Methods

In this paper, we establish necessary and sufficient conditions for global linear convergence rate bounds in operator splitting methods for a general class of convex optimization problems, where the associated fixed-point operator is strongly quasi-nonexpansive. We also provide a tight bound on the achievable convergence rate. Most existing results establishing global linear convergence in such methods require restrictive assumptions regarding strong convexity and smoothness of the constituent functions in the optimization problem. However, there are several examples in the literature showing that linear convergence is possible even when these properties do not hold. We provide a unifying analysis method for establishing global linear convergence based on linear regularity and show that many existing results are special cases of our approach. Moreover, we propose a novel linearly convergent splitting method for linear programming.

Prediction-Correction Interior-Point Method for Time-Varying Convex Optimization

In this paper, we develop an interior-point method for solving a class of convex optimization problems with time-varying objective and constraint functions. Using log-barrier penalty functions, we propose a continuous-time dynamical system for tracking the (time-varying) optimal solution with an asymptotically vanishing error. This dynamical system is composed of two terms: a correction term consisting of a continuous-time version of Newton's method, and a prediction term able to track the drift of the optimal solution by taking into account the time-varying nature of the objective and constraint functions. Using appropriately chosen time-varying slack and barrier parameters, we ensure that the solution to this dynamical system globally asymptotically converges to the optimal solution at an exponential rate. We illustrate the applicability of the proposed method in two applications: a sparsity promoting least squares problem and a collision-free robot navigation problem.

Robust ${\mathscr {H}}_{\infty }$ Filtering for a Class of Two-Dimensional Uncertain Fuzzy Systems With Randomly Occurring Mixed Delays

This paper is concerned with the robust ${\mathscr {H}}_{\infty }$ filtering problem for a class of 2-D uncertain fuzzy systems with randomly occurring mixed delays (ROMDs). The underlying 2-D systems are described by the Fornasini–Marchesini model, and the uncertainty is expressed in a linear fraction form. An improved Takagi–Sugeno (T–S) fuzzy model corresponding to the spatial promise variables is adopted to represent the complicated 2-D nonlinear system. The mixed delays consisting of both discrete and distributed delays are allowed to appear in a random manner governed by two sets of Bernoulli distributed white sequences with known probability. A full-order fuzzy filter is constructed to estimate the output signal such that, in the presence of parameter uncertainties and ROMDs, the dynamics of the estimation errors is asymptotically stable with a prescribed ${\mathscr {H}}_{\infty }$ disturbance attenuation level. Based on the stochastic analysis technique and the Lyapunov-like functional, sufficient conditions are established to ensure the existence of the desired filters, and the explicit expressions of such filters are derived by means of the solution to a class of convex optimization problems that can be solved via standard software packages. A numerical example is provided to demonstrate the effectiveness of the developed filter design algorithms, and the filter performances with and without fuzzy rules are also compared.

Distributed optimization of first-order discrete-time multi-agent systems with event-triggered communication

This paper focuses on the event-triggered distributed subgradient algorithms for solving a class of convex optimization problems based on first-order discrete-time multi-agent systems over undirected networks. The communication process of the whole network is controlled by a set of trigger conditions monitored by each agent. The trigger condition and event-triggered distributed subgradient optimization algorithm for each agent are completely decentralized and just rest with each agent's and its neighboring agents’ individual states at the event-triggered sequence of themselves as well as each agent's local objective function. At each time instant, each agent updates its state by employing its own objective function and the states collected from itself and its neighboring agents at their separate event-triggered time instants. A sufficient condition for ensuring the consensus and reaching the optimization solution is established under the condition that the undirected network topology is connected and the design parameters are properly designed. Theoretical analysis shows that the event-triggered distributed subgradient algorithm is capable of steering the whole network of agents asymptotically converge to an optimal solution of the convex optimization problem. Simulation results validate effectiveness of the introduced algorithm and demonstrate feasibility of the theoretical analysis.

Duality in Countably Infinite Monotropic Programs

Finite-dimensional monotropic programs form a class of convex optimization problems that includes linear programs, convex minimum cost flow problems on networks and hypernetworks, and separable convex programs with linear constraints. Countably infinite monotropic programs arise, for example, in infinite-horizon sequential decision problems and in robust optimization. Their applications encompass (i) countably infinite linear programs such as the shortest path formulations of infinite-horizon nonstationary as well as countable-state Markov decision processes; and (ii) convex minimum cost flow problems on countably infinite networks and hypernetworks. Duality results for finite-dimensional monotropic programs are as powerful as those available for linear programs. On the contrary, applicable duality results for countably infinite monotropic programs are currently nonexistent owing to several mathematical pathologies in infinite-dimensional sequence spaces. This paper overcomes this hurdle by first embeddin...

Randomized Block Proximal Damped Newton Method for Composite Self-Concordant Minimization

In this paper we consider the composite self-concordant (CSC) minimization problem, which minimizes the sum of a self-concordant function $f$ and a (possibly nonsmooth) proper closed convex function $g$. The CSC minimization is the cornerstone of the path-following interior point methods for solving a broad class of convex optimization problems. It has also found numerous applications in machine learning. The proximal damped Newton (PDN) methods have been well studied in the literature for solving this problem that enjoy a nice iteration complexity. Given that at each iteration these methods typically require evaluating or accessing the Hessian of $f$ and also need to solve a proximal Newton subproblem, the cost per iteration can be prohibitively high when applied to large-scale problems. Inspired by the recent success of block coordinate descent methods, we propose a randomized block proximal damped Newton (RBPDN) method for solving the CSC minimization. Compared to the PDN methods, the computational cos...

$H_\infty $ Control for 2-D Fuzzy Systems With Interval Time-Varying Delays and Missing Measurements.

In this paper, we consider the H∞ control problem for a class of 2-D Takagi-Sugeno fuzzy described by the second Fornasini-Machesini local state-space model with time-delays and missing measurements. The state delays are allowed to be time-varying within a known interval. The measurement output is subject to randomly intermittent packet dropouts governed by a random sequence satisfying the Bernoulli distribution. The purpose of the addressed problem is to design an output-feedback controller such that the closed-loop system is globally asymptotically stable in the mean square and the prescribed H∞ performance index is satisfied. By employing a combination of the intensive stochastic analysis and the free weighting matrix method, several delay-range-dependent sufficient conditions are presented that guarantee the existence of the desired controllers for all possible time-delays and missing measurements. The explicit expressions of such controllers are derived by means of the solution to a class of convex optimization problems that can be solved via standard software packages. Finally, a numerical simulation example is given to demonstrate the applicability of the proposed control scheme.

Linear Convergence of the Alternating Direction Method of Multipliers for a Class of Convex Optimization Problems

The numerical success of the alternating direction method of multipliers (ADMM) inspires much attention in analyzing its theoretical convergence rate. While there are several results on the iterative complexity results implying sublinear convergence rate for the general case, there are only a few results for the special cases such as linear programming, quadratic programming, and nonlinear programming with strongly convex functions. In this paper, we consider the convergence rate of ADMM when applying to the convex optimization problems that the subdifferentials of the underlying functions are piecewise linear multifunctions, including LASSO, a well-known regression model in statistics, as a special case. We prove that due to its inherent polyhedral structure, a recent global error bound holds for this class of problems. Based on this error bound, we derive the linear rate of convergence for ADMM. We also consider the proximal based ADMM and derive its linear convergence rate.

SIGEST

The SIGEST paper in this issue, “Optimization in High Dimensions via Accelerated, Parallel, and Proximal Coordinate Descent,” by Olivier Fercoq and Peter Richtárik, is an expanded version of their paper “Accelerated, Parallel, and Proximal Coordinate Descent” from the SIAM Journal on Optimization. Coordinate descent methods, which search for a lower value of the objective function by varying one or a small number of variables, are useful for problems so large that operations on full-sized vectors are impractical. If the vector of design variables will not fit in main memory, for example, then a full vector operation would require disk access. The paper considers a special class of convex optimization problems which has a structure that favors coordinate descent methods. The authors exploit this structure with a method they call APPROX (accelerated, parallel, and proximal). The structure of the problem facilitates parallelism. The method is accelerated in the sense that the convergence rate is proportional to the reciprocal of the square of the iteration number, rather than the reciprocal. Proximal methods penalize large differences between iterations. The authors have added a section on applications of the results in this expanded version. This new material appeared after the original paper and illustrates the utility of the theory very well. Interestingly, the section on applications is not at the end, but is a new second section which appears before the description of the algorithm. The advantage of this is that the reader gets an early view of how the problem class maps to applications, making it easier to appreciate the paper.

A globally and quadratically convergent smoothing Newton method for solving second-order cone optimization

Second-order cone optimization (denoted by SOCO) is a class of convex optimization problems and it contains the linear optimization problem, convex quadratic optimization problem and quadratically constrained convex quadratic optimization problem as special cases. In this paper, we propose a new smoothing Newton method for solving the SOCO based on a non-symmetrically perturbed smoothing Fischer–Burmeister function. At each iteration, a system of linear equations is solved only approximately by using the inexact Newton method. It is shown that any accumulation point of the iteration sequence generated by the proposed algorithm is a solution of the SOCO. Furthermore, we prove that the generated sequence is bounded and hence it has at least one accumulation point. Under the assumption of nonsingularity, we establish the local quadratic convergence of the proposed algorithm without strict complementarity condition. Numerical experiments indicate that our method is effective.

On the proximal Landweber Newton method for a class of nonsmooth convex problems

We consider a class of nonsmooth convex optimization problems where the objective function is a convex differentiable function regularized by the sum of the group reproducing kernel norm and $$\ell _1$$l1-norm of the problem variables. This class of problems has many applications in variable selections such as the group LASSO and sparse group LASSO. In this paper, we propose a proximal Landweber Newton method for this class of convex optimization problems, and carry out the convergence and computational complexity analysis for this method. Theoretical analysis and numerical results show that the proposed algorithm is promising.

Interior-point algorithms for a class of convex optimization problems

In this paper we consider interior-point methods (IPM) for the nonlinear, convex optimization problem where the objective function is a weighted sum of reciprocals of variables subject to linear constraints (SOR). This problem appears often in various applications such as statistical stratified sampling and entropy problems, to mention just few examples. The SOR is solved using two IPMs. First, a homogeneous IPM is used to solve the Karush-Kuhn-Tucker conditions of the problem which is a standard approach. Second, a homogeneous conic quadratic IPM is used to solve the SOR as a reformulated conic quadratic problem. As far as we are aware of it, this is a novel approach not yet considered in the literature. The two approaches are then numerically tested on a set of randomly generated problems using optimization software MOSEK. They are compared by CPU time and the number of iterations, showing that the second approach works better for problems with higher dimensions. The main reason is that although the first approach increases the number of variables, the IPM exploits the structure of the conic quadratic reformulation much better than the structure of the original problem.

A first order method for finding minimal norm-like solutions of convex optimization problems

We consider a general class of convex optimization problems in which one seeks to minimize a strongly convex function over a closed and convex set which is by itself an optimal set of another convex problem. We introduce a gradient-based method, called the minimal norm gradient method, for solving this class of problems, and establish the convergence of the sequence generated by the algorithm as well as a rate of convergence of the sequence of function values. The paper ends with several illustrating numerical examples.

Learning with tensors: a framework based on convex optimization and spectral regularization

We present a framework based on convex optimization and spectral regularization to perform learning when feature observations are multidimensional arrays (tensors). We give a mathematical characterization of spectral penalties for tensors and analyze a unifying class of convex optimization problems for which we present a provably convergent and scalable template algorithm. We then specialize this class of problems to perform learning both in a transductive as well as in an inductive setting. In the transductive case one has an input data tensor with missing features and, possibly, a partially observed matrix of labels. The goal is to both infer the missing input features as well as predict the missing labels. For induction, the goal is to determine a model for each learning task to be used for out of sample prediction. Each training pair consists of a multidimensional array and a set of labels each of which corresponding to related but distinct tasks. In either case the proposed technique exploits precise low multilinear rank assumptions over unknown multidimensional arrays; regularization is based on composite spectral penalties and connects to the concept of Multilinear Singular Value Decomposition. As a by-product of using a tensor-based formalism, our approach allows one to tackle the multi-task case in a natural way. Empirical studies demonstrate the merits of the proposed methods.

Stochastic second-order cone programming: Applications models

Second-order cone programs are a class of convex optimization problems. We refer to them as deterministic second-order cone programs (DSCOPs) since data defining them are deterministic. In DSOCPs we minimize a linear objective function over the intersection of an affine set and a product of second-order (Lorentz) cones. Stochastic programs have been studied since 1950s as a tool for handling uncertainty in data defining classes of optimization problems such as linear and quadratic programs. Stochastic second-order cone programs (SSOCPs) with recourse is a class of optimization problems that defined to handle uncertainty in data defining DSOCPs. In this paper we describe four application models leading to SSOCPs.

A SIMPLE STRATEGY TO TACKLE MUTUAL COUPLING AND PLATFORM EFFECTS IN SURVEILLANCE SYSTEMS

This paper presents an optimal power pattern synthesis procedure able to tackle the mutual coupling and platform efiects even for electrically large arrays. The novelty of the approach is due to its capability to account in the synthesis procedure for two difierent aspects at the same time: the coupling between the array radiating elements and the coupling between these elements and the array platform. The mutual coupling evaluation is based on the active element pattern method, and the active element pattern is numerically computed. The kind of synthesis problems here addressed belongs to the class of convex optimization problems. Therefore, the solution is found by means of very e-cient convex programming tools, without requiring global optimization schemes, thus saving time and costs. The extension of the overall tool to adaptive arrays is also considered.

Phase retrieval of diffraction from highly strained crystals

An important application of phase retrieval methods is to invert coherent x-ray diffraction measurements to obtain real-space images of nanoscale crystals. The phase information is currently recovered from reciprocal-space amplitude measurements by the application of iterative projective algorithms that solve the nonlinear and nonconvex optimization problem. Various algorithms have been developed each of which apply constraints in real and reciprocal space on the reconstructed object. In general, these methods rely on experimental data that is oversampled above the Nyquist frequency. To date, support-based methods have worked well, but are less successful for highly strained structures, defined as those which contain (real-space) phase information outside the range of ±?/2. As a direct result the acquired experimental data is, in general, inadvertently subsampled below the Nyquist frequency. In recent years, a new theory of “compressive sensing” has emerged, which dictates that an appropriately subsampled (or compressed) signal can be recovered exactly through iterative reconstruction and various routes to minimizing the ?1 norm or total variation in that signal. This has proven effective in solving several classes of convex optimization problems. Here we report on a “density-modification” phase reconstruction algorithm that applies the principles of compressive sensing to solve the nonconvex phase retrieval problem for highly strained crystalline materials. The application of a nonlinear operator in real-space minimizes the ?1 norm of the amplitude by a promotion-penalization (or “propenal”) operation that confines the density bandwidth. This was found to significantly aid in the reconstruction of highly strained nanocrystals. We show how this method is able to successfully reconstruct phase information that otherwise could not be recovered

A fast optimization algorithm for multicriteria intensity modulated proton therapy planning

To describe a fast projection algorithm for optimizing intensity modulated proton therapy (IMPT) plans and to describe and demonstrate the use of this algorithm in multicriteria IMPT planning. The authors develop a projection-based solver for a class of convex optimization problems and apply it to IMPT treatment planning. The speed of the solver permits its use in multicriteria optimization, where several optimizations are performed which span the space of possible treatment plans. The authors describe a plan database generation procedure which is customized to the requirements of the solver. The optimality precision of the solver can be specified by the user. The authors apply the algorithm to three clinical cases: A pancreas case, an esophagus case, and a tumor along the rib cage case. Detailed analysis of the pancreas case shows that the algorithm is orders of magnitude faster than industry-standard general purpose algorithms (MOSEK'S interior point optimizer, primal simplex optimizer, and dual simplex optimizer). Additionally, the projection solver has almost no memory overhead. The speed and guaranteed accuracy of the algorithm make it suitable for use in multicriteria treatment planning, which requires the computation of several diverse treatment plans. Additionally, given the low memory overhead of the algorithm, the method can be extended to include multiple geometric instances and proton range possibilities, for robust optimization.