Constrained Convex Optimization Problem Research Articles

Tikhonov regularized second-order plus first-order primal-dual dynamical systems with asymptotically vanishing damping for linear equality constrained convex optimization problems

In this paper, in the setting of Hilbert spaces, we consider a Tikhonov regularized second-order plus first-order primal-dual dynamical system with asymptotically vanishing damping for a linear equality constrained convex optimization problem. It is shown that convergence properties of the proposed inertial dynamical system depend upon the choice of the Tikhonov regularization parameter. When the Tikhonov regularization parameter decays rapidly to zero, we derive the fast convergence rates of the primal-dual gap, the objective residual, the feasibility violation and the gradient norm of the objective function along the generated trajectory. When the Tikhonov regularization parameter decreases slowly to zero, we prove the strong convergence of the primal trajectory of the Tikhonov regularized dynamical system to the minimal norm solution of the linear equality constrained convex optimization problem. Numerical experiments are performed to illustrate the efficiency of our approach.

The Perturbation Method and Regularization of the Lagrange Multiplier Rule in Convex Constrained Optimization Problems

The Perturbation Method and Regularization of the Lagrange Multiplier Rule in Convex Constrained Optimization Problems

Ensemble Clustering via Co-Association Matrix Self-Enhancement.

Ensemble clustering integrates a set of base clustering results to generate a stronger one. Existing methods usually rely on a co-association (CA) matrix that measures how many times two samples are grouped into the same cluster according to the base clusterings to achieve ensemble clustering. However, when the constructed CA matrix is of low quality, the performance will degrade. In this article, we propose a simple, yet effective CA matrix self-enhancement framework that can improve the CA matrix to achieve better clustering performance. Specifically, we first extract the high-confidence (HC) information from the base clusterings to form a sparse HC matrix. By propagating the highly reliable information of the HC matrix to the CA matrix and complementing the HC matrix according to the CA matrix simultaneously, the proposed method generates an enhanced CA matrix for better clustering. Technically, the proposed model is formulated as a symmetric constrained convex optimization problem, which is efficiently solved by an alternating iterative algorithm with convergence and global optimum theoretically guaranteed. Extensive experimental comparisons with 12 state-of-the-art methods on ten benchmark datasets substantiate the effectiveness, flexibility, and efficiency of the proposed model in ensemble clustering. The codes and datasets can be downloaded at https://github.com/Siritao/EC-CMS.

Optimization of thruster configuration and control allocation for a spherical magnetic coupling thrusters actuated underwater robot

Propeller-based reconfigurable magnetic coupling thrusters are a promising propulsion system for underwater robots, due to their vectored thrust and watertightness. This paper presents a lightweight and compact 3-DOF spherical reconfigurable magnetic coupling thruster (SRMCT-II), and focuses on its thruster configuration and control allocation, considering its intrinsic characteristics. To enhance robot agility, we develop an optimization problem for the configuration of SRMCT-II without relying on the engineers' experience or intuition and use a genetic algorithm to determine the optimal layout of three SRMCT-IIs for an underwater robot prototype. To assign motion control commands to the SRMCT-IIs, we formulate the control allocation as a constrained convex optimization problem and solve it using a practical augmented Lagrangian method. In the experiments, a customized force-sensing testbed is fabricated to evaluate the performance of the SRMCT-II and the robot's control allocation with its optimal thruster layout. The results verify the vectored characteristics of the SRMCT-II and demonstrate the computational efficiency and accuracy of the control allocation. The validation of thrust attainability reveals that the optimized thruster configuration achieves complete decoupling of the generalized control forces and torques. This indicates that the underwater robot can control its thrust and torque independently in all three dimensions.

Dynamical System Approach for Time-Varying Constrained Convex Optimization Problems

Dynamical System Approach for Time-Varying Constrained Convex Optimization Problems

Accelerated Primal-Dual Projection Neurodynamic Approach With Time Scaling for Linear and Set Constrained Convex Optimization Problems

Accelerated Primal-Dual Projection Neurodynamic Approach With Time Scaling for Linear and Set Constrained Convex Optimization Problems

Discrete-time Euler-smoothing methods for time-varying convex constrained optimization

In this paper, we propose two kinds of prediction–correction methods, discrete-time Euler-smoothing methods, for solving the general time-varying convex constrained optimization problem. By using the complementarity function and smoothing technique, the Karush–Kuhn–Tucker (KKT) trajectory of the problem can be reformulated as a system of smooth equations and considering the dynamics of the system of smooth equations in a discrete constant time-sampling periods scheme. The proposed methods mainly consist of the prediction step derived by Euler approximation and the correction step generated by one or multiple Newton iterations. Under some reasonable conditions, the approximate solution trajectories produced by the methods achieve the asymptotic error bound behaving as O(h4) with respect to optimal trajectory both in the primal and dual space as the smoothing parameter approaches to zero, where h is the sampling period. Finally, the experimental results show that our methods are efficient, and the steady-state tracking errors are much smaller than the other prediction–correction methods under the same conditions.

AN-SPS: adaptive sample size nonmonotone line search spectral projected subgradient method for convex constrained optimization problems

We consider convex optimization problems with a possibly nonsmooth objective function in the form of a mathematical expectation. The proposed framework (AN-SPS) employs Sample Average Approximations (SAA) to approximate the objective function, which is either unavailable or too costly to compute. The sample size is chosen in an adaptive manner, which eventually pushes the SAA error to zero almost surely (a.s.). The search direction is based on a scaled subgradient and a spectral coefficient, both related to the SAA function. The step size is obtained via a nonmonotone line search over a predefined interval, which yields a theoretically sound and practically efficient algorithm. The method retains feasibility by projecting the resulting points onto a feasible set. The a.s. convergence of AN-SPS method is proved without the assumption of a bounded feasible set or bounded iterates. Preliminary numerical results on Hinge loss problems reveal the advantages of the proposed adaptive scheme. In addition, a study of different nonmonotone line search strategies in combination with different spectral coefficients within AN-SPS framework is also conducted, yielding some hints for future work.

Optimization of nonequilibrium free energy harvesting illustrated on bacteriorhodopsin

Harvesting free energy from the environment is essential for the operation of many biological and artificial systems. We use techniques from stochastic thermodynamics to investigate the maximum rate of harvesting achievable by optimizing a set of reactions in a Markovian system, possibly under various kinds of topological, kinetic, and thermodynamic constraints. This question is relevant for the optimal design of new harvesting devices as well as for quantifying the efficiency of existing systems. We first demonstrate that the maximum harvesting rate can be expressed as a constrained convex optimization problem. We illustrate it on bacteriorhodopsin, a light-driven proton pump from Archaea, which we find is close to optimal under realistic conditions. In our second result, we solve the optimization problem in closed-form in three physically meaningful limiting regimes. These closed-form solutions are illustrated on two idealized models of unicyclic harvesting systems. Published by the American Physical Society 2024

Distributed Online Constrained Optimization With Feedback Delays.

We investigate multiagent distributed online constrained convex optimization problems with feedback delays, where agents make sequential decisions before being aware of the cost and constraint functions. The main purpose of the distributed online constrained convex optimization problem is to cooperatively minimize the sum of time-varying local cost functions subject to time-varying coupled inequality constraints. The feedback information of the distributed online optimization problem is revealed to agents with time delays, which is common in practice. Every node in the system can interact with neighbors through a time-varying sequence of directed communication topologies, which is uniformly strongly connected. The distributed online primal-dual bandit push-sum algorithm that generates primal and dual variables with delayed feedback is used for the presented problem. Expected regret and expected constraint violation are proposed for measuring the performance of the algorithm, and both of them are shown to be sublinear with respect to the total iteration span T in this article. In the end, the optimization problem for the power grid is simulated to justify the proposed theoretical results.

Decomposition of air conditioning electricity consumption based on effective duration.

As the share of air conditioning electricity consumption within total grid electricity consumption grows, the decomposition of such consumption becomes increasingly crucial for assessing electricity usage patterns, devising consumption scheduling strategies, and maintaining the stability of the power grid. Although there is a strong correlation between apparent temperature and air conditioning electricity consumption, the literature currently available seldom explores the impact of apparent temperature on this consumption. Moreover, there is a scarcity of effective assessment indices to evaluate the efficacy of air conditioning electricity consumption breakdown. This study introduces a method for decomposing electricity consumption from air conditioning units, utilizing effective duration as a basis to tackle these issues. By employing an apparent temperature model as a constraint, this approach identifies the effective operating time of air conditioning and constructs a constrained convex optimization problem to estimate air conditioning power usage. Additionally, a novel evaluation index for the effectiveness of air conditioning electricity consumption decomposition is proposed, which includes penalties for negative decomposed consumption, alongside the traditional consistency index. Comparative experiments are conducted using real electricity consumption data from Fujian Province. Empirical results indicate that the methodology for air conditioning electricity consumption decomposition presented in this paper aligns more closely with actual conditions. Furthermore, the evaluation metrics introduced for the decomposition of air conditioning electricity consumption are adept at precisely gauging the quality of the air conditioning electricity consumption data.

Accelerated Projection Algorithm Based on Smoothing Approximation for Distributed Nonsmooth Optimization

In this paper, a distributed smoothing accelerated projection algorithm (DSAPA) is proposed to address constrained non-smooth convex optimization problems over undirected multi-agent networks in a distributed manner, where the objective function is free of the assumption of Lipschitz gradient or strong convexity. First, based on a distributed exact penalty method, the original optimization problem is translated to a problem of standard assignment without consensus constraints. Then, a novel DSAPA by combining the smoothing approximation with Nesterov's accelerated schemes is proposed. In addition, we provide a systematic analysis to derive an upper bound on the convergence rate in terms of the objective function based on penalty function and to choose the optimal step size accordingly. Our results demonstrate that the proposed DSAPA can reach <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$O(\frac{\log (k)}{k})$</tex-math></inline-formula> when the optimal step size is chosen. Finally, the effectiveness and correctness of the proposed algorithm are verified by numerical and practical application examples.

A Low-Cost Alternating Projection Approach for a Continuous Formulation of Convex and Cardinality Constrained Optimization

We consider convex constrained optimization problems that also include a cardinality constraint. In general, optimization problems with cardinality constraints are difficult mathematical programs which are usually solved by global techniques from discrete optimization. We assume that the region defined by the convex constraints can be written as the intersection of a finite collection of convex sets, such that it is easy and inexpensive to project onto each one of them (e.g., boxes, hyper-planes, or half-spaces). Taking advantage of a recently developed continuous reformulation that relaxes the cardinality constraint, we propose a specialized penalty gradient projection scheme combined with alternating projection ideas to compute a solution candidate for these problems, i.e., a local (possibly non-global) solution. To illustrate the proposed algorithm, we focus on the standard mean-variance portfolio optimization problem for which we can only invest in a preestablished limited number of assets. For these portfolio problems with cardinality constraints, we present a numerical study on a variety of data sets involving real-world capital market indices from major stock markets. In many cases, we observe that the proposed scheme converges to the global solution. On those data sets, we illustrate the practical performance of the proposed scheme to produce the effective frontiers for different values of the limited number of allowed assets.

Convergence rates of mixed primal-dual dynamical systems with Hessian driven damping

For a linear equality constrained convex optimization problem, we initially propose a mixed primal-dual dynamical system with Hessian driven damping. This dynamical system comprises a second-order ordinary differential equation (ODE) with damping and Hessian driven damping for the primal variable, and a first-order ODE for the dual variable. Utilizing the Lyapunov analysis approach, we prove that all the Lagrangian residual, the objective residual and the feasibility violation enjoy fast convergence under general scaling coefficients. We further analyse the convergence rate results under specific scaling coefficients. Our mixed dynamical system is extended to solve multi-block optimization problems, and we also consider the discrete case of the mixed dynamical system. This is the first study to explore primal-dual dynamical systems with Hessian driven damping for linear equality constrained problems.

Inexact generalized ADMM with relative error criteria for linearly constrained convex optimization problems

Inexact generalized ADMM with relative error criteria for linearly constrained convex optimization problems

Accelerated Dual Averaging Methods for Decentralized Constrained Optimization

In this article, we study decentralized convex constrained optimization problems in networks. We focus on the dual averaging-based algorithmic framework that is well-documented to be superior in handling constraints and complex communication environments simultaneously. Two new decentralized dual averaging (DDA) algorithms are proposed. In the first one, a second-order dynamic average consensus protocol is tailored for DDA-type algorithms, which equips each agent with a provably more accurate estimate of the global dual variable than conventional schemes. We rigorously prove that the proposed algorithm attains <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(1/t)$</tex-math></inline-formula> convergence for general convex and smooth problems, for which existing DDA methods were only known to converge at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(1/\sqrt{t})$</tex-math></inline-formula> prior to our work. In the second one, we use the extrapolation technique to accelerate the convergence of DDA. Compared to existing accelerated algorithms, where typically two different variables are exchanged among agents at each time, the proposed algorithm only seeks consensus on local gradients. Then, the extrapolation is performed based on two sequences of primal variables, which are determined by the accumulations of gradients at two consecutive time instants, respectively. The algorithm is proved to converge at <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {O}(1)\left(\frac{1}{t^{2}}+\frac{1}{t(1-\beta)^{2}}\right)$</tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\beta$</tex-math></inline-formula> denotes the second largest singular value of the mixing matrix. We remark that the condition for the algorithmic parameter to guarantee convergence does not rely on the spectrum of the mixing matrix, making itself easy to satisfy in practice. Finally, numerical results are presented to demonstrate the efficiency of the proposed methods.

Totally Asynchronous Primal-Dual Convex Optimization in Blocks

In this article, we present a parallelized primal-dual algorithm for solving constrained convex optimization problems. The algorithm is “block-based,” in which vectors of primal and dual variables are partitioned into blocks, each of which is updated only by a single processor. We consider four behaviors that could be asynchronous: 1) updates to primal variables; 2) updates to dual variables; 3) communications of primal variables; and 4) communications of dual variables. We show that any amount of asynchrony in the communications of dual variables can preclude convergence, though the other forms of asynchrony are permitted. A first-order primal-dual update law is then developed and shown to be robust to these other forms of asynchrony. We next derive convergence rates to an approximate Lagrangian saddle point in terms of the operations agents execute, without specifying any timing or pattern with which they must be executed. The distance between the approximate solution we obtain and the exact solution is explicitly bounded. Convergence rates include an “asynchrony penalty” that we quantify and present ways to mitigate. Numerical results illustrate these developments.

Convex Optimization Problems on Differentiable Sets

Given a closed convex set in a Banach space , motivated by the continuity and Fréchet differentiability of introduced, respectively, in [D. Gale and V. Klee, Math. Scand., 7 (1959), pp. 379–391] and [X. Y. Zheng, SIAM J. Optim., 30 (2020), pp. 490–512], this paper considers the -differentiability, subdifferentiability, and Gâteaux differentiability of . Using the technique of variational analysis, it is proved that is -differentiable (resp., subdifferentiable or Gâteaux differentiable) if and only if for every continuous convex function with the corresponding constrained optimization problem is 2/ -order-well-posed solvable (resp., generalized well-posed solvable or weak well-posed solvable). It is also proved that if the conjugate function of a continuous convex function on is -differentiable on , then for every closed convex set in with the corresponding optimization problem is -order-well-posed solvable. As a byproduct, every constrained convex optimization problem with a strongly convex quadratic objective function is proved to be globally second-order-well-posed solvable. Our main results are new even in the case of finite dimensional spaces.

A Primal-Dual Forward-Backward Splitting Algorithm for Distributed Convex Optimization

Motivated by modern large-scale information processing problems in engineering, this paper concentrates on studying distributed constrained convex optimization problems over a connected undirected network. The problem involves a sum of a differentiable convex function with Lipschitz continuous gradient and two non-smooth convex functions with a linear operator. To solve such a problem, we propose a novel distributed primal-dual forward-backward splitting algorithm, called D-PDFBS. Each agent locally computes the Lipschitz continuous gradient and two proximal operators, and exchanges information with its neighbors. D-PDFBS adopts non-identical stepsizes, and we reveal the relationship between selection of stepsizes and parameters of objective functions. The simulation results verify the feasibility of D-PDFBS and the correctness of theoretical findings.

Event-triggered distributed online convex optimization with delayed bandit feedback

This paper is concerned with an online distributed convex-constrained optimization problem over a multi-agent network, where the limited network bandwidth and potential feedback delay caused by network communication are considered. To cope with the limited network bandwidth, an event-triggered communication scheme is introduced in information exchange. Then, based on the delayed (i.e., single-point and two-point) bandit feedback, two event-triggered distributed online convex optimization algorithms are developed by utilizing the Bregman divergence in the projection step. Meanwhile, the convergence of the two developed algorithms is analyzed according to the provided static regret bounds achieved by the algorithm. The obtained results show that a sublinear static regret with respect to the time horizon T can be ensured if the triggering threshold gradually approaches zero. In this case, the corresponding order of the regret bounds is also determined by choosing suitable triggering thresholds. Finally, a distributed online regularized linear regression problem is provided as an example to illustrate the effectiveness of the proposed two algorithms.