Inertial Accelerated Primal–Dual Algorithms for Non-smooth Convex Optimization Problems with Linear Equality Constraints

We study an optimal control problem governed by an elliptic partial differential equation with pointwise constraints on the state and Neumann boundary conditions. The simultaneous presence of these constraints and Neumann conditions poses challenges for analysis and discretization, particularly on polygonal domains that are not necessarily convex. In particular, we develop and analyze two finite element methods for this problem. The first method uses a standard variational formulation, while the second introduces a mass condensation technique that produces a diagonal mass matrix, thus facilitating the use of efficient primal-dual algorithms.

The primal-dual hybrid gradient (PDHG) algorithm is a well-established method for solving linearly constrained convex programming problems. However, in general conditions, convergence is not ensured. In this paper, we propose a convex combination of proximal centers PDHG (CCPDHG) algorithm by integrating convex combinations of the iterates into proximal centers. Specifically, the convex combination factor α 1 plays a vital role in regulating the proximal centre of the primal variable, it can be relaxed to ( 0 , 3 − 3 2 ) . Under mild assumptions, we demonstrate the global convergence and an O ( 1 / K ) ergodic sublinear convergence rate of the proposed algorithm. Furthermore, when the objective function is strongly convex, we propose the accelerated CCPDHG algorithm, which achieves an O ( 1 / K 2 ) ergodic convergence rate. Finally, we verify the effectiveness of the proposed algorithms through a series of numerical experiments.

A separate preconditioned primal-dual splitting algorithm for composite monotone inclusion problems

In this paper, we consider a class of three-composite nonconvex optimization problems, in which the nonsmooth function is further composed with a linear operator. This problem has many applications such as sparse signal recovery, image processing and machine learning. Based on the conjugate duality theory, we present an accelerated preconditioned primal-dual gradient algorithm for this problem. Compared with the existing algorithms, our algorithm only needs to calculate the proximal mapping of the conjugate function $h^∗$ which is always convex and lower semicontinuous and it does not need to calculate the proximal mapping of nonconvex functions. This may significantly reduce the computation load. We prove that the sequence generated by the proposed algorithm globally converges to a critical point when the function satisfies the Kurdyka- Lojasiewicz property. We also obtain the convergence rate of the proposed algorithm. Finally, numerical results on sparse signal recovery and image processing illustrate the efficiency and competitiveness of the proposed algorithm.

A Primal-Dual Interior-Point algorithm for nonlinear ACOPF with adaptively reformulated Volt-Var constraints

Single-pixel imaging (SPI) is a novel computational imaging technique. Recently, deep learning has achieved remarkable improvements in SPI reconstruction in both quality and speed. However, the rapidly increasing computational costs at large image sizes remain a key obstacle to practical SPI. In this work, we propose the K-CPP Net, a gradient-descent-based deep unfolding network that couples a Kronecker-based SPI model with a Chambolle-Pock-inspired primal-dual iterative optimization algorithm for efficient SPI reconstruction. Specifically, we avoid explicit vectorization of the large sensing matrix by representing the matrix as the Kronecker product of two learnable subsampling matrices, and design a lightweight denoising module based on residual convolutional blocks and efficient channel-wise self-attention. This method substantially reduces the computational cost, enabling full-image SPI training and inference while effectively suppressing distortion and blur and preserving fine structural details. Moreover, we implement K-CPP Net on an FPGA via an optimized accelerator architecture, providing a reference design for real-world SPI applications. Experimental results demonstrate that the proposed method achieves superior reconstruction quality while offering markedly improved computational efficiency.

Primal-dual algorithm for weakly convex functions under sharpness conditions

A class of one-dimensional dynamic impact models is investigated with respect to non-smooth velocities using variational inequalities and space-time finite element approximation. For the problem of collision of a rigid obstacle by an elastic bar in the gravitational field, a benchmark based on particular solutions to the wave equation is constructed on a partition of rectangle domains. The full discretization of the collision problem is carried out over a uniform space-time triangulation and extended to distorted meshes. For the solution of the corresponding variational inequality, a semi-smooth Newton-based primal-dual active set algorithm is applied. Numerical experiments demonstrate advantages over time-step approximation: a high-precision numerical solution is computed in a few iterations without any spurious oscillations.

We derive a convex relaxation principle for a large class of non convex variational problems where the functional to be minimized involves a one homogeneous gradient energy. This applies directly to free boundary or multiphase problems in the case of the classical total variation or of some anisotropic variants. The underlying argument is an exclusion principle which states that any global minimizer avoids taking values in the intervals where the lower order potential is nonconvex. This allows using duality methods and deriving a saddle point characterization of the global minimizers. A numerical validation of our principle is presented in the case of several free boundary and multiphase problems that we treat through a primal-dual algorithm. The accuracy of the interfaces and the convergence of the algoritm benefit in a large way of a new epigraphical projection method that we introduced to tackle the non differentiability of the convexified Lagrangian.

We introduce a new interior-point method for solving convex quadratic programming under full-Newton step. The method involves an equivalent algebraic transformation applied to the system defining the central path. This approach provides an efficient search direction for the considered algorithm. Furthermore, we show that the introduced method produces an optimal solution within polynomial time. The established numerical tests conclude that the newly proposed algorithm is not only polynomial but requires a number of iterations clearly lower than that obtained theoretically.

This paper addresses the problem of image denoising for images corrupted by mixed Gaussian and impulse noise, formulated as an inverse problem. We introduce a new PDE-constrained optimization model that simultaneously reconstructs a clean image and identifies impulse noise. The well-posedness of the direct problem is presented by proving the existence and uniqueness of a weak solution, while the existence of a weak solution for the adjoint problem, arising in the optimality conditions, is shown using Galerkin's method. For the numerical implementation, we use a finite difference discretization combined with a primal-dual optimization algorithm. Experimental results on standard test images proved that the proposed method consistently outperforms existing approaches in denoising quality, robustness to impulse noise, and runtime performance, making it a promising framework for mixed-noise image restoration.

Representing and exploiting multivariate signals requires capturing relations between variables, which we can represent by graphs. Graph dictionaries allow to describe complex relational information as a sparse sum of simpler structures, but no prior model exists to infer such underlying structure elements from data. We define a novel Graph-Dictionary signal model, where a finite set of graphs characterizes relationships in data distribution as filters on the weighted sum of their Laplacians. We propose a framework to infer the graph dictionary representation from observed node signals, which allows to include a priori knowledge about signal properties, and about underlying graphs and their coefficients. We introduce a bilinear generalization of the primal-dual splitting algorithm to solve the learning problem. We show the capability of our method to reconstruct graphs from signals in multiple synthetic settings, where our model outperforms popular baselines. Then, we exploit graph-dictionary representations in an illustrative motor imagery decoding task on brain activity data, where we classify imagined motion better than standard methods relying on many more features. Our graph-dictionary model bridges a gap between sparse representations of multivariate data and a structured decomposition of sample-varying relationships into a sparse combination of elementary graph atoms.

New primal-dual algorithm for convex-concave saddle point problems

This article focuses on nonconvex distributed composite optimization over time-varying multiagent networks, where each agent possesses a local objective function, composed of a nonconvex and smooth function plus a nonsmooth function. The network aims to minimize the sum of all local functions subject to local set constraints and global nonconvex coupled inequality constraints. The inherent nonconvex and nonlinear characteristics of the objective and constraint functions pose formidable challenges in developing efficient distributed algorithms with convergence guarantees. To tackle this intricate problem, a novel distributed linearized augmented primal-dual algorithm is designed by incorporating distributed tracking and dynamic consensus techniques. It is theoretically shown that, with appropriately chosen parameters, the proposed algorithm can find an $\epsilon $ -Karush-Kuhn-Tucker (KKT) point. Specifically, the sequences of average optimality, constraint violation, and complementary slackness measure converge to zero at sublinear rates. Finally, a numerical application is presented to validate the effectiveness of the proposed algorithm.

A double extrapolations primal-dual algorithm with linesearch and applications to image denoising

In this paper, we propose a new primal-dual algorithmic framework for a class of convex-concave saddle point problems frequently arising from image processing and machine learning. Our algorithmic framework updates the primal variable between the twice calculations of the dual variable, thereby appearing a symmetric iterative scheme, which is accordingly called the symmetric primal-dual algorithm (SPIDA). It is noteworthy that the subproblems of our SPIDA are equipped with Bregman proximal regularization terms, which make SPIDA versatile in the sense that it enjoys an algorithmic framework to understand the iterative schemes of some existing algorithms, such as the classical augmented Lagrangian method (ALM), linearized ALM, and Jacobian splitting algorithms for linearly constrained optimization problems. Besides, our algorithmic framework allows us to derive some customized versions so that SPIDA works as efficiently as possible for structured optimization problems. Theoretically, under some mild conditions, we prove the global convergence of SPIDA and estimate the linear convergence rate under a generalized error bound condition defined by Bregman distance. Finally, a series of numerical experiments on the basis pursuit, robust principal component analysis, and image restoration demonstrate that our SPIDA works well on synthetic and real-world datasets.

An accelerated preconditioned primal-dual gradient algorithm for structured nonconvex optimization problems

Practical proximal primal-dual algorithms for structured saddle point problems

This paper addresses the problem of distributed online generalized Nash equilibrium (GNE) learning for multi-cluster games with delayed function feedback. Specifically, each agent in the game is assumed to be informed of a sequence of local cost functions and constraint functions, which are known to the agent with time-varying delays subsequent to decision-making at each round. The objective of each agent within a cluster is to collaboratively optimize the cluster’s cost function, subject to time-varying coupled inequality constraints and local constraint sets over time. Additionally, it is assumed that each agent is required to estimate the decisions of all other agents through interactions with its neighbors, rather than directly accessing the decisions of all agents, i.e., each agent needs to make decisions under partial-decision information. To solve such a challenging problem, a novel distributed online delay-tolerant GNE learning algorithm is developed based upon the primal-dual algorithm with an aggregation gradient mechanism. The system-wise regret and the constraint violation are formulated to measure the performance of the algorithm, demonstrating sublinear growth with respect to the time horizon <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\boldsymbol {T}$ </tex-math></inline-formula> under certain conditions. Finally, numerical results are presented to verify the effectiveness of the proposed algorithm.