Separable Optimization Problems Research Articles

On the Distributed Method of Multipliers for Separable Convex Optimization Problems

In this paper, we present a novel method for convex optimization in distributed networks called the distributed method of multipliers (DMM). The proposed method is based on a combination of a particular dual lifting and classic monotone operator splitting approaches to produce an algorithm with guaranteed asymptotic convergence in undirected networks. The proposed method allows any separable convex problem with linear constraints to be solved in undirected networks. In contrast to typical distributed approaches, the structure of the network does not restrict the types of problems that can be solved. Furthermore, the solver can be applied to general separable problems, those with separable convex objectives and constraints, via the use of an additional primal lifting approach. Finally, we demonstrate the use of DMM in solving a number of classic signal processing problems including beamforming, channel capacity maximization and portfolio optimization.

A framework for petrophysically and geologically guided geophysical inversion using a dynamic Gaussian mixture model prior

SUMMARYWe propose a new framework for incorporating petrophysical and geological information into voxel-based geophysical inversion. By developing the geophysical inverse problem from a probabilistic perspective, we redesign the objective function and the iteration steps as a suite of cyclic optimization problems in which three separate MAP optimization problems are solved using geophysical, petrophysical and geological data, respectively. By quantitatively linking these data into a single framework, we recover a final inverted model that reproduces the observed, or desired, petrophysical and geological features while fitting the geophysical data. To achieve our goal we replace the Gaussian prior, used in the Tikhonov inversion approach, by a Gaussian mixture model. After each geophysical model update, the mixture parameters (means, variances and proportions) are determined by the geophysical model and the expected characteristics of the lithologies through another optimization process using the expectation–maximization algorithm. We then classify the model cells into rock units according to the petrophysical and geological information. These two additional steps over the petrophysical and geological data result in a dynamic update of the reference model and associated weights and guide the inversion towards reproducing the expected petrophysical and geological characteristics. The resulting geophysical objective function does not require extra terms to include the additional petrophysical and geological information; this is an important distinction between our work and previous frameworks that carry out joint geophysical and petrophysical data inversion. We highlight different capabilities of our methodology by inverting magnetotelluric and direct-current resistivity data in 1-D and 2-D, respectively. Finally, we apply our framework to inverting airborne frequency domain data, acquired in Australia, for the detection and characterization of saline contamination of freshwater.

Convex Relaxations of the Network Flow Problem Under Cycle Constraints

In this paper, we consider the problem of optimizing the flows in a lossless flow network with additional nonconvex cycle constraints on nodal variables; such constraints appear in several applications, including electric power, water distribution, and natural gas networks. This problem is a nonconvex version of the minimum-cost network flow problem (NFP), and to solve it, we propose three different approaches. One approach is based on solving a convex approximation of the problem, obtained by augmenting the cost function with an entropy-like term to relax the nonconvex constraints. We show that the approximation error, for which we give an upper bound, can be made small enough for practical use. An alternative approach is to solve the classical NFP, that is, without the cycle constraints, and solve a separate optimization problem afterwards in order to recover the actual flows satisfying the cycle constraints; the solution of this separate problem maximizes the individual entropies of the cycles. The third approach is based on replacing the nonconvex constraint set with a convex inner approximation, which yields a suboptimal solution for the cyclic networks with each edge belonging to at most two cycles. We validate the practical usefulness of the theoretical results through numerical examples, in which we study the standard test systems for water and electric power distribution networks.

An LQP-Based Symmetric Alternating Direction Method of Multipliers with Larger Step Sizes

Symmetric alternating direction method of multipliers (ADMM) is an efficient method for solving a class of separable convex optimization problems. This method updates the Lagrange multiplier twice with appropriate step sizes at each iteration. However, such step sizes were conservatively shrunk to guarantee the convergence in recent studies. In this paper, we are devoted to seeking larger step sizes whenever possible. The logarithmic-quadratic proximal (LQP) terms are applied to regularize the symmetric ADMM subproblems, allowing the constrained subproblems to then be converted to easier unconstrained ones. Theoretically, we prove the global convergence of such LQP-based symmetric ADMM by specifying a larger step size domain. Moreover, the numerical results on a traffic equilibrium problem are reported to demonstrate the advantage of the method with larger step sizes.

Regularized Jacobi-type ADMM-methods for a class of separable convex optimization problems in Hilbert spaces

We consider a regularized version of a Jacobi-type alternating direction method of multipliers (ADMM) for the solution of a class of separable convex optimization problems in a Hilbert space. The analysis shows that this method is equivalent to the standard proximal-point method applied in a Hilbert space with a transformed scalar product. The method therefore inherits the known convergence results from the proximal-point method and allows suitable modifications to get a strongly convergent variant. Some additional properties are also shown by exploiting the particular structure of the ADMM-type solution method. Applications and numerical results are provided for the domain decomposition method and potential (generalized) Nash equilibrium problems in a Hilbert space setting.

Inexact alternating direction methods of multipliers for separable convex optimization

Inexact alternating direction multiplier methods (ADMMs) are developed for solving general separable convex optimization problems with a linear constraint and with an objective that is the sum of smooth and nonsmooth terms. The approach involves linearized subproblems, a back substitution step, and either gradient or accelerated gradient techniques. Global convergence is established. The methods are particularly useful when the ADMM subproblems do not have closed form solution or when the solution of the subproblems is expensive. Numerical experiments based on image reconstruction problems show the effectiveness of the proposed methods.

A framework for parallel second order incremental optimization algorithms for solving partially separable problems

We propose Hessian Approximated Multiple Subsets Iteration (HAMSI), which is a generic second order incremental algorithm for solving large-scale partially separable convex and nonconvex optimization problems. The algorithm is based on a local quadratic approximation, and hence, allows incorporating curvature information to speed-up the convergence. HAMSI is inherently parallel and it scales nicely with the number of processors. We prove the convergence properties of our algorithm when the subset selection step is deterministic. Combined with techniques for effectively utilizing modern parallel computer architectures, we illustrate that a particular implementation of the proposed method based on L-BFGS updates converges more rapidly than a parallel gradient descent when both methods are used to solve large-scale matrix factorization problems. This performance gain comes only at the expense of using memory that scales linearly with the total size of the optimization variables. We conclude that HAMSI may be considered as a viable alternative in many large scale problems, where first order methods based on variants of gradient descent are applicable.

The Proximal Alternating Minimization Algorithm for Two-Block Separable Convex Optimization Problems with Linear Constraints

The Alternating Minimization Algorithm has been proposed by Paul Tseng to solve convex programming problems with two-block separable linear constraints and objectives, whereby (at least) one of the components of the latter is assumed to be strongly convex. The fact that one of the subproblems to be solved within the iteration process of this method does not usually correspond to the calculation of a proximal operator through a closed formula affects the implementability of the algorithm. In this paper, we allow in each block of the objective a further smooth convex function and propose a proximal version of the algorithm, which is achieved by equipping the algorithm with proximal terms induced by variable metrics. For suitable choices of the latter, the solving of the two subproblems in the iterative scheme can be reduced to the computation of proximal operators. We investigate the convergence of the proposed algorithm in a real Hilbert space setting and illustrate its numerical performances on two applications in image processing and machine learning.

A Collaborative Data Collection Scheme Based on Optimal Clustering for Wireless Sensor Networks.

In recent years, energy-efficient data collection has evolved into the core problem in the resource-constrained Wireless Sensor Networks (WSNs). Different from existing data collection models in WSNs, we propose a collaborative data collection scheme based on optimal clustering to collect the sensed data in an energy-efficient and load-balanced manner. After dividing the data collection process into the intra-cluster data collection step and the inter-cluster data collection step, we model the optimal clustering problem as a separable convex optimization problem and solve it to obtain the analytical solutions of the optimal clustering size and the optimal data transmission radius. Then, we design a Cluster Heads (CHs)-linking algorithm based on the pseudo Hilbert curve to build a CH chain with the goal of collecting the compressed sensed data among CHs in an accumulative way. Furthermore, we also design a distributed cluster-constructing algorithm to construct the clusters around the virtual CHs in a distributed manner. The experimental results show that the proposed method not only reduces the total energy consumption and prolongs the network lifetime, but also effectively balances the distribution of energy consumption among CHs. By comparing it o the existing compression-based and non-compression-based data collection schemes, the average reductions of energy consumption are 17.9% and 67.9%, respectively. Furthermore, the average network lifetime extends no less than 20-times under the same comparison.

Dual Optimization of Contact-Aided Compliant Mechanisms for Passive Dynamic Shape Change

A dual optimization method for optimizing contact-aided compliant mechanism design parameters and their spatial distribution in a dynamic ornithopter wing structure for increased pitch agility is presented. This methodology separates the task into two separate optimization problems where, first, a computationally efficient rigid-body dynamics model is used to determine the optimal stiffness and spatial distribution of the compliant mechanisms and, second, a detailed compliant mechanism design is developed and optimized that achieves the desired nonlinear stiffness. A rigid-body mechanics model of the wing structure is used to find the location and stiffness of a contact-aided compliant mechanism that will induce the forward sweep passively by coupling lift loads to forward sweep. The forward-swept compliant mechanism is then developed and optimized to achieve the desirable coupling. The free-flight pitch agility of an ornithopter is shown to increase via sweeping the wings forward during downstroke. A free-flight experiment is performed, and the novel contact-aided compliant mechanism is shown to induce the desired forward sweep during downstroke.

A note on the sufficient initial condition ensuring the convergence of directly extended 3-block ADMM for special semidefinite programming

ABSTRACTIn this note, we consider three types of problems, H-weighted nearest correlation matrix problem and two types of important doubly non-negative semidefinite programming, derived from the binary integer quadratic programming and maximum cut problem. The dual of these three types of problems is a 3-block separable convex optimization problem with a coupling linear equation constraint. It is known that, the directly extended 3-block alternating direction method of multipliers (ADMM3d) is more efficient than many of its variants for solving these convex optimization, but its convergence is not guaranteed. By choosing initial points properly, we obtain the convergence of ADMM3d for solving the dual of these three types of problems. Furthermore, we simplify the iterative scheme of ADMM3d and show the equivalence of ADMM3d to the 2-block semi-proximal ADMM for solving the dual's reformulation, under these initial conditions.

Relaxed inertial proximal Peaceman-Rachford splitting method for separable convex programming

The strictly contractive Peaceman-Rachford splitting method is one of effective methods for solving separable convex optimization problem, and the inertial proximal Peaceman-Rachford splitting method is one of its important variants. It is known that the convergence of the inertial proximal Peaceman-Rachford splitting method can be ensured if the relaxation factor in Lagrangian multiplier updates is underdetermined, which means that the steps for the Lagrangian multiplier updates are shrunk conservatively. Although small steps play an important role in ensuring convergence, they should be strongly avoided in practice. In this article, we propose a relaxed inertial proximal Peaceman-Rachford splitting method, which has a larger feasible set for the relaxation factor. Thus, our method provides the possibility to admit larger steps in the Lagrangian multiplier updates. We establish the global convergence of the proposed algorithm under the same conditions as the inertial proximal Peaceman-Rachford splitting method. Numerical experimental results on a sparse signal recovery problem in compressive sensing and a total variation based image denoising problem demonstrate the effectiveness of our method.

Generalized Lagrangian duality for nonconvex polynomial programs with polynomial multipliers

In this paper, under the existence of a certificate of nonnegativity of the objective function over the given constraint set, we present saddle-point global optimality conditions and a generalized Lagrangian duality theorem for (not necessarily convex) polynomial optimization problems, where the Lagrange multipliers are polynomials. We show that the nonnegativity certificate together with the archimedean condition guarantees that the values of the Lasserre hierarchy of semidefinite programming (SDP) relaxations of the primal polynomial problem converge asymptotically to the common primal–dual value. We then show that the known regularity conditions that guarantee finite convergence of the Lasserre hierarchy also ensure that the nonnegativity certificate holds and the values of the SDP relaxations converge finitely to the common primal–dual value. Finally, we provide classes of nonconvex polynomial optimization problems for which the Slater condition guarantees the required nonnegativity certificate and the common primal–dual value with constant multipliers and the dual problems can be reformulated as semidefinite programs. These classes include some separable polynomial programs and quadratic optimization problems with quadratic constraints that admit certain hidden convexity. We also give several numerical examples that illustrate our results.

O(1/t) complexity analysis of the generalized alternating direction method of multipliers

Owing to its efficiency in solving some types of large-scale separable optimization problems with linear constraints, the convergence rate of the alternating direction method of multipliers (ADMM for short) has recently attracted significant attention. In this paper, we consider the generalized ADMM (G-ADMM), which incorporates an acceleration factor and is more efficient. Instead of using a solution measure that depends on a bounded set and cannot be easily estimated, we propose using the original ϵ-optimal solution measure, under which we prove that the G-ADMM converges at a rate of O(1/t). The new bound depends on the penalty parameter and the distance between the initial point and solution set, which is more reasonable than the previous bound.

Multitarget Joint Delay and Doppler-Shift Estimation in Bistatic Passive Radar

We study the problem of jointly estimating the delays and Doppler shifts of multiple targets in a bistatic passive radar system when the unknown signal from the illuminator of opportunity (IO) is modeled as a correlated stochastic process. We treat all the delays and Doppler shifts as continuous-valued parameters to avoid straddle loss due to discretization. A computationally efficient method based on the expectation maximization (EM) algorithm is proposed, which breaks up the complex multidimensional maximum likelihood estimation problem into several separate optimization problems, one optimization per target. The algorithm provides estimates of all the targets’ delays and Doppler shifts along with the estimate of the IO signal and each target's component signal in the surveillance channel. The Cramer–Rao lower bound (CRLB) is also derived. Simulation results are presented, where we compare the proposed algorithm with the conventional cross-correlation (CC) estimator, modified CC (MCC) estimator, EM with successive interference cancellation (EM-SIC) estimator, and CRLB. It is shown that when the targets are well separated on the delay-Doppler shift plane, the proposed method outperforms all the estimators. Moreover, when the targets are close in this plane, the CC and EM-SIC estimators fails to distinguish the targets, whereas the proposed method successfully estimates the delays and Doppler shifts and outperforms the MCC in this case as well. Finally as the signal-to-noise ratio increases, the performance of the proposed method approaches that of the CRLB.

Distributed resource allocation over random networks based on stochastic approximation

In this paper, we study a resource allocation problem in which a group of agents cooperatively optimize a separable optimization problem with a linear network resource constraint and allocation feasibility constraints, where the global objective function is the sum of agents’ local objective functions. Each agent can only get noisy observations of its local gradient function and its local resource, which cannot be shared by other agents or transmitted to a center. There also exist communication uncertainties such as time-varying topologies (described by random graphs) and additive channel noises. To solve the resource allocation with uncertainties, we propose a stochastic approximation based distributed algorithm, and prove that agents can collaboratively achieve the optimal allocation with probability one by virtue of the ordinary differential equation (ODE) method for stochastic approximation. Finally, simulations related to the demand response management in power systems verify the effectiveness of the proposed algorithm.

ОБОСНОВАНИЕ ОПТИМАЛЬНОСТИ ОПЕРАЦИИ ДЕЛЕНИЯ ОЦЕНОК ШУМОВЫХ ТЕМПЕРАТУР НА СМЕЖНЫХ ИНТЕРВАЛАХ НАБЛЮДЕНИЯ В РАДИОМЕТРЕ С НЕСТАБИЛЬНЫМ КОЭФФИЦИЕНТОМ УСИЛЕНИЯ

В статье обоснована оптимальность операции деления оценок шумовых температур на смежных интервалах наблюдения в радиометре с нестабильным коэффициентом усиления. Задача раздельной оптимизации статистических оценок шумовых температур случайных процессов, наблюдаемых на двух смежных интервалах времени равной длительности при открытом и закрытом входе модуляционного радиометра с нестабильным коэффициентом усиления, решена методом максимального правдоподобия. Показано, что одной из существенных операций синтезированного алгоритма первичной обработки сигналов является операция деления оценок их средних мощностей, измеренных на соседних временных интервалах. Выведены аналитические выражения для предельных погрешностей оценок шумовой температуры исследуемого источника и нестабильности коэффициента усиления. Эти погрешности зависят от времени наблюдения, ширины полосы рабочих частот по входу, отношения сигнал/шум и нестабильности коэффициента усиления. Полученный алгоритм обработки сигналов целесообразно реализовывать в радиометрах, в состав додетекторных трактов которых входят недорогие низкостабильные усилители.

Dynamic Locomotion Through Online Nonlinear Motion Optimization for Quadrupedal Robots

This letter presents a realtime motion planning and control method that enables a quadrupedal robot to execute dynamic gaits including trot, pace, and dynamic lateral walk, as well as gaits with full flight phases such as jumping, pronking, and running trot. The proposed method also enables smooth transitions between these gaits. Our approach relies on an online zero-moment point based motion planner which continuously updates the reference motion trajectory as a function of the contact schedule and the state of the robot. The reference footholds for each leg are obtained by solving a separate optimization problem. The resulting optimized motion plans are tracked by a hierarchical whole-body controller. Our framework has been tested in simulation and on ANYmal, a fully torque-controllable quadrupedal robot, both in simulation and on the actual robot.

Exact Second-Order Cone Programming Relaxations for Some Nonconvex Minimax Quadratic Optimization Problems

In this paper, we study, for the first time, nonconvex minimax separable quadratic optimization problems with multiple separable quadratic constraints and their second-order cone programming (SOCP)...

Sensitivity Analysis for Convex Separable Optimization Over Integral Polymatroids

We study the sensitivity of optimal solutions of convex separable optimization problems over an integral polymatroid base polytope with respect to parameters determining both the cost of each eleme...