Forward-backward Splitting Research Articles

Convergence analysis and applications of the inertial algorithm solving inclusion problems

Nonlinear operator theory is an important area of nonlinear functional analysis. This area encompasses diverse nonlinear problems in many areas of mathematics, the physical sciences and engineering such as monotone operator equations, fixed point problems and more.In this work we are concern with the problem of finding a common solution of a monotone operator equation and fixed point of a nonexpansive mapping in real Hilbert spaces. Derived from dynamical systems, a simple inertial forward-backward splitting method for solving the problem is presented and analyzed under mild and standard assumptions. Some numerical examples in real-world and comparisons with related works, illustrate the theoretical advantages as well the potential applicability of the proposed scheme.

A Dynamical Splitting Method for Minimizing the Sum of Three Convex Functions

In this paper, we develop a dynamical splitting method based on a nonlinear dynamical system with two proximal operators for solving the minimum value problem of the sum of three convex functions. The method endows some existing classical splitting methods such as the Forward-Backward splitting method and the Douglas-Rachford splitting method with continuous time behaviors. We demonstrate that the proposed dynamical system converges globally to a point whose image, imaged by each of the two proximal operators, is a solution to the minimum value problem. We also shows that both the residual of the two proximal operators and the objective error enjoy the convergence rates of where t denotes time. This rate can be improved to by using an ergodic average technique. Moreover, under certain smoothness and metric subregularity conditions, we further establish the global exponential convergence of the dynamical splitting method. Finally, we apply the dynamical splitting method to deal with the problem of compressive sensing.

Low-Gain Stability of Projected Integral Control for Input-Constrained Discrete-Time Nonlinear Systems

We consider the problem of zeroing an error output of a nonlinear discrete-time system in the presence of constant exogenous disturbances, subject to hard convex constraints on the input signal. The design specification is formulated as a variational inequality, and we adapt a forward-backward splitting algorithm to act as an integral controller which ensures that the input constraints are met at each time step. We establish a low-gain stability result for the closed-loop system when the plant is exponentially stable, generalizing previously known results for integral control of discrete-time systems. Specifically, it is shown that if the composition of the plant equilibrium input-output map and the integral feedback gain is strongly monotone, then the closed-loop system is exponentially stable for all sufficiently small integral gains. The method is illustrated via application to a four-tank process.

On Robustness in Optimization-Based Constrained Iterative Learning Control

Iterative learning control (ILC) is a control strategy for repetitive tasks wherein information from previous runs is leveraged to improve future performance. Optimization-based ILC (OB-ILC) is a powerful design framework for constrained ILC where measurements from the process are integrated into an optimization algorithm to provide robustness against noise and modelling error. This paper proposes a robust ILC controller for constrained linear processes based on the forward-backward splitting algorithm. It demonstrates how structured uncertainty information can be leveraged to ensure constraint satisfaction and provides a rigorous stability analysis in the iteration domain by combining concepts from monotone operator theory and robust control. Numerical simulations of a precision motion stage support the theoretical results.

Stochastic generalized Nash equilibrium seeking under partial-decision information

We consider for the first time a stochastic generalized Nash equilibrium problem, i.e., with expected-value cost functions and joint feasibility constraints, under partial-decision information, meaning that the agents communicate only with some trusted neighbors. We propose several distributed algorithms for network games and aggregative games that we show being special instances of a preconditioned forward–backward splitting method. We prove that the algorithms converge to a generalized Nash equilibrium when the forward operator is restricted cocoercive by using the stochastic approximation scheme with variance reduction to estimate the expected value of the pseudogradient.

Convergence analysis of modified inertial forward–backward splitting scheme with applications

In the settings of real Hilbert spaces, this paper proposes modified forward– backward scheme with inertial extrapolation step for solving inverse problems. In our proposed method, it is possible for the inertial factor to be chosen as 1 unlike many previous inertial forward–backward splitting methods available in the literature. Weak and strong convergence of our proposed method are obtained under some standard conditions. In order to show the numerical advantage gained when the inertial factor is chosen as 1, a number of numerical implementations are performed on two different inverse problems: LASSO and SCAD penalty problems.

Primal-Dual Splitting Algorithms for Solving Structured Monotone Inclusion with Applications

This work proposes two different primal-dual splitting algorithms for solving structured monotone inclusion containing a cocoercive operator and the parallel-sum of maximally monotone operators. In particular, the parallel-sum is symmetry. The proposed primal-dual splitting algorithms are derived from two approaches: One is the preconditioned forward–backward splitting algorithm, and the other is the forward–backward–half-forward splitting algorithm. Both algorithms have a simple calculation framework. In particular, the single-valued operators are processed via explicit steps, while the set-valued operators are computed by their resolvents. Numerical experiments on constrained image denoising problems are presented to show the performance of the proposed algorithms.

L1-L2 norm regularization via forward-backward splitting for fluorescence molecular tomography.

Fluorescent molecular tomography (FMT) is a highly sensitive and noninvasive imaging approach for providing three-dimensional distribution of fluorescent marker probes. However, owing to its light scattering effect and the ill-posedness of inverse problems, it is challenging to develop an efficient reconstruction algorithm that can achieve the exact location and morphology of the fluorescence source. In this study, therefore, in order to satisfy the need for early tumor detection and improve the sparsity of solution, we proposed a novel L 1-L 2 norm regularization via the forward-backward splitting method for enhancing the FMT reconstruction accuracy and the robustness. By fully considering the highly coherent nature of the system matrix of FMT, it operates by splitting the objective to be minimized into simpler functions, which are dealt with individually to obtain a sparser solution. An analytic solution of L 1-L 2 norm proximal operators and a forward-backward splitting algorithm were employed to efficiently solve the nonconvex L 1-L 2 norm minimization problem. Numerical simulations and an in-vivo glioma mouse model experiment were conducted to evaluate the performance of our algorithm. The comparative results of these experiments demonstrated that the proposed algorithm obtained superior reconstruction performance in terms of spatial location, dual-source resolution, and in-vivo practicability. It was believed that this study would promote the preclinical and clinical applications of FMT in early tumor detection.

Computational inverse imaging method by machine learning-informed physical model for electrical capacitance tomography

The solution of the imaging inversion problem is an important step in the electrical capacitance tomography developed for process parameter measurements. Many studies have been carried out to improve the reconstruction quality, but the discrepancy between ground-truth imaging prototypes and recovered tomograms is still significant. To address the challenge, the regularization by denoising (RED) is introduced in this work, turning the denoising algorithm into a regularizer. Measurement physics, RED and sparsity prior are coupled into a new imaging model. A new numerical method is developed to solve the established imaging model by integrating the split Bregman algorithm and the forward backward splitting technique. To improve the performance of RED, the multiple output least squares support vector machine is combined with the low-dimensional representation method, and the training problem is solved by a new distributed computational method. The nonnegative matrix factorization method is extended into a new low-dimensional representation method, and a powerful optimizer is developed to solve the model. The performance evaluations clearly imply that the new method achieves more significant reconstruction performance gain and better robustness than popular imaging algorithms. This study improves the measurement physics based imaging method by machine learning techniques, and provides new perspectives and insights into the development of the image reconstruction paradigm.

A Nonlinearly Preconditioned Forward-Backward Splitting Method and Applications

In this paper, we prove the weak convergence of the iterates generated by the nonlinearly preconditioned forward-backward splitting method for the sum of a maximally hypermonotone operator A and a hypercocoercive operator B under several suitable conditions. We provide several choices of the nonlinear preconditioners for solving nonlinearly composed inclusions. In particular, the backward-forward splitting method is recovered by the nonlinearly preconditioned forward-backward splitting method with a special choice of the nonlinear preconditioner.

L1-L2 Minimization Via A Proximal Operator For Fluorescence Molecular Tomography.

Fluorescent Molecular Tomography (FMT) is a highly sensitive and noninvasive imaging method that provides three-dimensional distribution of biomarkers by noninvasive detection of fluorescent marker probes. However, due to the light scattering effect and ill-posedness of inverse problems, it is challenging to develop an efficient construction method that can provide the exact location and morphology of the fluorescence distribution. In this paper, we proposed L1-L2 norm regularization to improve FMT reconstruction. In our research, proximal operators of non-convex L1 -L2 norm and forward-backward splitting method was adopted to solve the inverse problem of FMT. Simulation results on heterogeneous mouse model demonstrated that the proposed FBS method is superior to IVTCG, DCA and IRW-L1/2 reconstruction methods in location accuracy and other aspects.

An improved fast iterative shrinkage thresholding algorithm with an error for image deblurring problem

In this paper, we introduce a new iterative forward-backward splitting method with an error for solving the variational inclusion problem of the sum of two monotone operators in real Hilbert spaces. We suggest and analyze this method under some mild appropriate conditions imposed on the parameters such that another strong convergence theorem for these problem is obtained. We also apply our main result to improve the fast iterative shrinkage thresholding algorithm (IFISTA) with an error for solving the image deblurring problem. Finally, we provide numerical experiments to illustrate the convergence behavior and show the effectiveness of the sequence constructed by the inertial technique to the fast processing with high performance and the fast convergence with good performance of IFISTA.

Strong convergence of the forward–backward splitting algorithms via linesearches in Hilbert spaces

Many real-world problems in applied sciences, engineering and economics can be reformulated as the convex minimization problem of two objective functions. In order to solve this problem, the forward–backward splitting algorithm and its modifications have been invented and used to investigate the convergence analysis. In this work, we discuss strong convergence of the sequence generated by the forward–backward algorithm involving the viscosity approximation method. The stepsizes studied in this paper are defined by two different kinds of linesearches which do not require the Lipschitz continuity condition on the gradient. Finally, we provide numerical experiments to show the efficiency of the proposed methods in signal recovery. It is reported that our algorithms have a good convergence behavior in practice without information of Lipschitz constants of the gradient of functions.

Coordinating distributed MPC efficiently on a plantwide scale: The Lyapunov envelope algorithm

The model predictive control (MPC) of large-scale systems should adopt a distributed optimization approach, where controllers for the constituent subsystems optimize their control actions and iterations are used to coordinate their decisions. The real-time implementation of MPC, however, usually allows very limited time for computation and inevitably needs to be terminated early. In this work, we propose a splitting algorithm for distributed optimization analogous to forward-backward splitting (FBS), where ℓ1 and quadratic penalties are imposed on the violation of interconnecting relations among the subsystems. By designing the involved parameters based on dissipative analysis, the iterations result in the monotonic decrease of a plant-wide Lyapunov function, which we call Lyapunov envelope, thus maintaining closed-loop stability under distributed MPC despite early termination and yielding improving control performance as the allowed computational time or number of iterations increases. The proposed Lyapunov envelope algorithm is tested on an industrial-scale vinyl acetate monomer process.

Convolutional proximal neural networks and Plug-and-Play algorithms

In this paper, we introduce convolutional proximal neural networks (cPNNs), which are by construction averaged operators. For filters with full length, we propose a stochastic gradient descent algorithm on a submanifold of the Stiefel manifold to train cPNNs. In case of filters with limited length, we design algorithms for minimizing functionals that approximate the orthogonality constraints imposed on the operators by penalizing the least squares distance to the identity operator. Then, we investigate how scaled cPNNs with a prescribed Lipschitz constant can be used for denoising signals and images, where the achieved quality depends on the Lipschitz constant. Finally, we apply cPNN based denoisers within a Plug-and-Play framework and provide convergence results for the corresponding PnP forward-backward splitting algorithm based on an oracle construction.

English

The aim of this paper is to propose two modified forward-backward splitting algorithms for zeros of the sum of a maximal monotone operator and a Bregman inverse strongly monotone operator in reflexive Banach spaces. We prove weak and strong convergence theorems of the generated sequences by the proposed methods under some suitable conditions. We apply our results to study the variational inequality problem and the equilibrium problem. Finally, a numerical example is given to illustrate the proposed methods. The results presented in this paper improve and generalize many known results in recent literature.

On the Strong Convergence of Forward-Backward Splitting in Reconstructing Jointly Sparse Signals

We consider the problem of reconstructing an infinite set of sparse, finite-dimensional vectors, that share a common sparsity pattern, from incomplete measurements. This is in contrast to the work (Daubechies et al., Pure Appl. Math. 57(11), 1413–1457, 2004), where the single vector signal can be infinite-dimensional, and (Fornasier and Rauhut, SIAM J. Numer. Anal. 46(2), 577613, 2008), which extends the aforementioned work to the joint sparse recovery of finite number of infinite-dimensional vectors. In our case, to take account of the joint sparsity and promote the coupling of nonvanishing components, we employ a convex relaxation approach with mixed norm penalty ℓ2,1. This paper discusses the computation of the solutions of linear inverse problems with such relaxation by a forward-backward splitting algorithm. However, since the solution matrix possesses infinitely many columns, the arguments of Daubechies et al. (Pure Appl. Math. 57(11), 1413–1457, 2004) no longer apply. As such, we establish new strong convergence results for the algorithm, in particular when the set of jointly sparse vectors is infinite.

Generalized Hybrid Viscosity-Type Forward-Backward Splitting Method with Application to Convex Minimization and Image Restoration Problems

Let E be a uniformly convex and q-uniformly smooth real Banach space. Let be an α- inverse strongly accretive mapping of order q, be a set-valued m- accretive mapping and be a nonexpansive mapping. In this article, a viscosity-type forward-backward splitting method for approximating a zero of (A + B) which is also a fixed point of S is introduced studied. Strong convergence theorem of the method is proved under suitable conditions. Furthermore, the convergence result obtained is applied to convex minimization and image restoration problems. Finally, numerical illustrations are presented to compare the convergence of the sequence of our algorithm and that of some recent important algorithms.

Memory-Efficient Training for Fully Unrolled Deep Learned PET Image Reconstruction with Iteration-Dependent Targets.

We propose a new version of the forward-backward splitting expectation-maximisation network (FBSEM-Net) along with a new memory-efficient training method enabling the training of fully unrolled implementations of 3D FBSEM-Net. FBSEM-Net unfolds the maximum a posteriori expectation-maximisation algorithm and replaces the regularisation step by a residual convolutional neural network. Both the gradient of the prior and the regularisation strength are learned from training data. In this new implementation, three modifications of the original framework are included. First, iteration-dependent networks are used to have a customised regularisation at each iteration. Second, iteration-dependent targets and losses are introduced so that the regularised reconstruction matches the reconstruction of noise-free data at every iteration. Third, sequential training is performed, making training of large unrolled networks far more memory efficient and feasible. Since sequential training permits unrolling a high number of iterations, there is no need for artificial use of the regularisation step as a leapfrogging acceleration. The results obtained on 2D and 3D simulated data show that FBSEM-Net using iteration-dependent targets and losses improves the consistency in the optimisation of the network parameters over different training runs. We also found that using iteration-dependent targets increases the generalisation capabilities of the network. Furthermore, unrolled networks using iteration-dependent regularisation allowed a slight reduction in reconstruction error compared to using a fixed regularisation network at each iteration. Finally, we demonstrate that sequential training successfully addresses potentially serious memory issues during the training of deep unrolled networks. In particular, it enables the training of 3D fully unrolled FBSEM-Net, not previously feasible, by reducing the memory usage by up to 98% compared to a conventional end-to-end training. We also note that the truncation of the backpropagation (due to sequential training) does not notably impact the network’s performance compared to conventional training with a full backpropagation through the entire network.

A fast viscosity forward-backward algorithm for convex minimization problems with an application in image recovery

The purpose of this paper is to invent an accelerated algorithm for the convex minimization problem which can be applied to the image restoration problem. Theoretically, we first introduce an algorithm based on viscosity approximation method with the inertial technique for finding a common fixed point of a countable family of nonexpansive operators. Under some suitable assumptions, a strong convergence theorem of the proposed algorithm is established. Subsequently, we utilize our proposed algorithm to solving a convex minimization problem of the sum of two convex functions. As an application, we apply and analyze our algorithm to image restoration problems. Moreover, we compare convergence behavior and efficiency of our algorithm with other well-known methods such as the forward-backward splitting algorithm and the fast iterative shrinkage-thresholding algorithm. By using image quality metrics, numerical experiments show that our algorithm has a higher efficiency than the mentioned algorithms.