Proximal Splitting Algorithm Research Articles

Improving BFS measurement accuracy of BOTDR based on Cauchy proximal splitting

A novel approach based on the Cauchy proximal splitting (CPS) algorithm is proposed to improve Brillouin frequency shift (BFS) measurement accuracy in Brillouin optical time domain reflectometry system. The CPS algorithm utilizes proximal splitting to handle the data and the penalty function based on Cauchy distribution to accurately estimate the target signal to promote sparsity, achieving significant improvement in signal-to-noise ratio (SNR). Experimental results confirm that with a SNR increase of 12.7 dB, an increase in BFS measurement accuracy from 4.78 MHz to 0.43 MHz is achieved over a 10.3 km sensing fiber. The denoising effects of the CPS, wavelet denoising (WD) and non-local mean (NLM) algorithms are further compared, demonstrating that the CPS algorithm has the lowest root mean square error (0.43 MHz) and no deterioration in spatial resolution rather than the WD and NLM algorithms does.

A Perturbation Framework for Convex Minimization and Monotone Inclusion Problems with Nonlinear Compositions

We introduce a framework based on Rockafellar’s perturbation theory to analyze and solve general nonsmooth convex minimization and monotone inclusion problems involving nonlinearly composed functions as well as linear compositions. Such problems have been investigated only from a primal perspective and only for nonlinear compositions of smooth functions in finite-dimensional spaces in the absence of linear compositions. In the context of Banach spaces, the proposed perturbation analysis serves as a foundation for the construction of a dual problem and of a maximally monotone Kuhn–Tucker operator, which is decomposable as the sum of simpler monotone operators. In the Hilbertian setting, this decomposition leads to a block-iterative primal-dual algorithm that fully splits all the components of the problem and appears to be the first proximal splitting algorithm for handling nonlinear composite problems. Various applications are discussed. Funding: The work of L. M. Briceño-Arias was supported by Agencia Nacional de Investigación y Desarrollo-Chile [Grant Fondo Nacional de Desarrollo Científico y Tecnológico 1190871, Grant Centro de Modelamiento Matemático ACE210010, Grant Centro de Modelamiento Matemático FB210005, and basal Funds for Centers of Excellence], and the work of P. L. Combettes was supported by the National Science Foundation [Grant DMS-1818946].

Sliced optimal transport on the sphere

Sliced optimal transport reduces optimal transport on multi-dimensional domains to transport on the line. More precisely, sliced optimal transport is the concatenation of the well-known Radon transform and the cumulative density transform, which analytically yields the solutions of the reduced transport problems. Inspired by this concept, we propose two adaptions for optimal transport on the 2-sphere. Firstly, as counterpart to the Radon transform, we introduce the vertical slice transform, which integrates along all circles orthogonal to a given direction. Secondly, we introduce a semicircle transform, which integrates along all half great circles with an appropriate weight function. Both transforms are generalized to arbitrary measures on the sphere. While the vertical slice transform can be combined with optimal transport on the interval and leads to a sliced Wasserstein distance restricted to even probability measures, the semicircle transform is related to optimal transport on the circle and results in a different sliced Wasserstein distance for arbitrary probability measures. The applicability of both novel sliced optimal transport concepts on the sphere is demonstrated by proof-of-concept examples dealing with the interpolation and classification of spherical probability measures. The numerical implementation relies on the singular value decompositions of both transforms and fast Fourier techniques. For the inversion with respect to probability measures, we propose the minimization of an entropy-regularized Kullback–Leibler divergence, which can be numerically realized using a primal-dual proximal splitting algorithm.

Fast Compressed Sensing of 3D Radial T1 Mapping with Different Sparse and Low-Rank Models.

Knowledge of the relative performance of the well-known sparse and low-rank compressed sensing models with 3D radial quantitative magnetic resonance imaging acquisitions is limited. We use 3D radial T1 relaxation time mapping data to compare the total variation, low-rank, and Huber penalty function approaches to regularization to provide insights into the relative performance of these image reconstruction models. Simulation and ex vivo specimen data were used to determine the best compressed sensing model as measured by normalized root mean squared error and structural similarity index. The large-scale compressed sensing models were solved by combining a GPU implementation of a preconditioned primal-dual proximal splitting algorithm to provide high-quality T1 maps within a feasible computation time. The model combining spatial total variation and locally low-rank regularization yielded the best performance, followed closely by the model combining spatial and contrast dimension total variation. Computation times ranged from 2 to 113 min, with the low-rank approaches taking the most time. The differences between the compressed sensing models are not necessarily large, but the overall performance is heavily dependent on the imaged object.

Tikhonov regularized iterative methods for nonlinear problems

ABSTRACT We consider the monotone inclusion problems in real Hilbert spaces. Proximal splitting algorithms are very popular technique to solve it and generally achieve weak convergence under mild assumptions. Researchers assume the strong conditions like strong convexity or strong monotonicity on the considered operators to prove strong convergence of the algorithms. Mann iteration method and normal S-iteration method are popular methods to solve fixed point problems. We propose a new common fixed point algorithm based on normal S-iteration method using Tikhonov regularization to find common fixed point of non-expansive operators and prove strong convergence of the generated sequence to the set of common fixed points without assuming strong convexity and strong monotonicity. Based on proposed fixed point algorithm, we propose a forward–backward-type algorithm and a Douglas–Rachford algorithm in connection with Tikhonov regularization to find the solution of monotone inclusion problems. Further, we consider the complexly structured monotone inclusion problems which are very popular these days. We also propose a strongly convergent forward–backward-type primal–dual algorithm and a Douglas–Rachford-type primal–dual algorithm to solve the monotone inclusion problems. Finally, we conduct a numerical experiment to solve image deblurring problems.

Proximal Splitting Algorithms for Convex Optimization: A Tour of Recent Advances, with New Twists

Convex nonsmooth optimization problems, whose solutions live in very high dimensional spaces, have become ubiquitous. To solve them, the class of first-order algorithms known as proximal splitting algorithms is particularly adequate: they consist of simple operations, handling the terms in the objective function separately. In this overview, we demystify a selection of recent proximal splitting algorithms: we present them within a unified framework, which consists in applying splitting methods for monotone inclusions in primal-dual product spaces, with well-chosen metrics. Along the way, we easily derive new variants of the algorithms and revisit existing convergence results, extending the parameter ranges in several cases. In particular, we emphasize that when the smooth term in the objective function is quadratic, e.g., for least-squares problems, convergence is guaranteed with larger values of the relaxation parameter than previously known. Such larger values are usually beneficial for the convergence speed in practice.

DHOSGR: lncRNA-disease Association Prediction Based on Decay High-order Similarity and Graph-regularized Matrix Completion

Background: It has been shown in numerous recent studies that long non-coding RNAs (lncRNAs) play a vital role in the regulation of various biological processes, as well as serve as a basis for understanding the causes of human illnesses. Thus, many researchers have developed matrix completion approaches to infer lncRNA–disease connections and enhance prediction performance by using similarity information. Objective: Most matrix completion approaches are solely based on the first-order or second-order similarity between nodes, and higher-order similarity is rarely considered. In view of this, we developed a computational method to incorporate higher-order similarity information into the similarity network with different weights using a decay function designed by a random walk with restart (DHOSGR). Methods: First, considering that the information will decay as the distance increases during network propagation, we defined a novel decay high-order similarity by combining the similarity matrix and its high-order similarity information through a decay function to construct a similarity network. Then, we applied the similarity network to the objective function as a graph regularization term. Finally, a proximal splitting algorithm was used to perform matrix completion to infer relationships between diseases and lncRNAs. Results: In the experiment, DHOSGR achieves a superior performance in leave-one-out cross validation (LOOCV) and 100 times 5-fold cross validation (5-fold-CV), with AUC values of 0.9459 and 0.9334±0.0016, respectively, which are better than other five previous models. Moreover, case studies of three diseases (leukemia, lymphoma, and squamous cell carcinoma) demonstrated that DHOSGR can reliably predict associated lncRNAs. Conclusion: DHOSGR can serve as a high efficiency calculation model for predicting lncRNA-disease associations.

A Generalized GraphEM for Sparse Time-Varying Dynamical Systems

We consider the problem of joint parameter estimation and smoothing in structured linear systems using the expectation maximization (EM) framework. Specifically, we explore how partially known sparsity structures in the estimation model can be leveraged to improve the computation speed and performance of the considered EM approaches. We use these ideas to generalize a recently proposed GraphEM algorithm to a linear time-varying setting, where the sparsity structures may vary in time. We obtain a biconvex form of the majorizing function in the M-step, which is minimized subject to an l1-regularization using a Douglas-Rachford proximal splitting algorithm. Numerical results using a satellite positioning example shows significant improvements in the estimation errors and an F1-score that quantifies model sparsity.

Decentralized proximal splitting algorithms for composite constrained convex optimization

This paper concentrates on a class of decentralized convex optimization problems subject to local feasible sets, equality and inequality constraints, where the global objective function consists of a sum of locally smooth convex functions and non-smooth regularization terms. To address this problem, a synchronous full-decentralized primal-dual proximal splitting algorithm (Syn-FdPdPs) is presented, which avoids the unapproximable property of the proximal operator with respect to inequality constraints via logarithmic barrier functions. Following the proposed decentralized protocol, each agent carries out local information exchange without any global coordination and weight balancing strategies introduced in most consensus algorithms. In addition, a randomized version of the proposed algorithm (Rand-FdPdPs) is conducted through subsets of activated agents, which further removes the global clock coordinator. Theoretically, with the help of asymmetric forward-backward-adjoint (AFBA) splitting technique, the convergence results of the proposed algorithms are provided under the same local step-size conditions. Finally, the effectiveness and practicability of the proposed algorithms are demonstrated by numerical simulations on the least-square and least absolute deviation problems.

Distributed Proximal Splitting Algorithms with Rates and Acceleration

We analyze several generic proximal splitting algorithms well suited for large-scale convex nonsmooth optimization. We derive sublinear and linear convergence results with new rates on the function value suboptimality or distance to the solution, as well as new accelerated versions, using varying stepsizes. In addition, we propose distributed variants of these algorithms, which can be accelerated as well. While most existing results are ergodic, our nonergodic results significantly broaden our understanding of primal–dual optimization algorithms.

A constrained LiGME model and its proximal splitting algorithm under overall convexity condition

A constrained LiGME model and its proximal splitting algorithm under overall convexity condition

Robust partial Fourier reconstruction for diffusion-weighted imaging using a recurrent convolutional neural network.

To develop an algorithm for robust partial Fourier (PF) reconstruction applicable to diffusion-weighted (DW) images with non-smooth phase variations. Based on an unrolled proximal splitting algorithm, a neural network architecture is derived, which alternates between data consistency operations and regularization implemented by recurrent convolutions. In order to exploit correlations, multiple repetitions of the same slice are jointly reconstructed under consideration of permutation-equivariance. The algorithm is trained on DW liver data of 60 volunteers and evaluated on retrospectively and prospectively subsampled data of different anatomies and resolutions. The proposed method is able to significantly outperform conventional PF techniques on retrospectively subsampled data in terms of quantitative measures as well as perceptual image quality. In this context, joint reconstruction of repetitions as well as the particular type of recurrent network unrolling are found to be beneficial with respect to reconstruction quality. On prospectively PF-sampled data, the proposed method enables DW imaging with higher signal without sacrificing image resolution or introducing additional artifacts. Alternatively, it can be used to counter the TE increase in acquisitions with higher resolution. Furthermore, generalizability can be shown to prospective brain data exhibiting anatomies and contrasts not present in the training set. This work demonstrates that robust PF reconstruction of DW data is feasible even at strong PF factors in anatomies prone to phase variations. Since the proposed method does not rely on smoothness priors of the phase but uses learned recurrent convolutions instead, artifacts of conventional PF methods can be avoided.

A Linearly Involved Generalized Moreau Enhancement of ℓ2,1-Norm with Application to Weighted Group Sparse Classification

This paper proposes a new group-sparsity-inducing regularizer to approximate ℓ2,0 pseudo-norm. The regularizer is nonconvex, which can be seen as a linearly involved generalized Moreau enhancement of ℓ2,1-norm. Moreover, the overall convexity of the corresponding group-sparsity-regularized least squares problem can be achieved. The model can handle general group configurations such as weighted group sparse problems, and can be solved through a proximal splitting algorithm. Among the applications, considering that the bias of convex regularizer may lead to incorrect classification results especially for unbalanced training sets, we apply the proposed model to the (weighted) group sparse classification problem. The proposed classifier can use the label, similarity and locality information of samples. It also suppresses the bias of convex regularizer-based classifiers. Experimental results demonstrate that the proposed classifier improves the performance of convex ℓ2,1 regularizer-based methods, especially when the training data set is unbalanced. This paper enhances the potential applicability and effectiveness of using nonconvex regularizers in the frame of convex optimization.

Data-Driven Sparse Sensor Selection Based on A-Optimal Design of Experiment With ADMM

The present study proposes a sensor selection method based on the proximal splitting algorithm and the A-optimal design of experiment using the alternating direction method of multipliers (ADMM) algorithm. The performance of the proposed method was evaluated with a random sensor problem and compared with the previously proposed methods such as the greedy method and the convex relaxation. The performance of the proposed method is better than an existing method in terms of the A-optimality criterion. In addition, the proposed method requires longer computational time than the greedy method but it is quite shorter than the convex relaxation in large-scale problems. The proposed method was applied to the data-driven sparse-sensor-selection problem. A data set adopted is the NOAA OISST V2 mean sea surface temperature set. At the number of sensors larger than that of the latent state variables, the proposed method showed similar and better performances compared with previously proposed methods in terms of the A-optimality criterion and reconstruction error.

Enhanced total generalized variation method based on moreau envelope

Image restoration with total generalized variation (TGV) regularization has been proved to be a creditable method and used broadly, but this method is imperfect in retaining image details. This paper proposes a non-convex and non-separable regularization term based on TGV enhancement, which can maintain the strict convexity of the total cost function and avoid the underestimation of the TGV method. The new regularization is obtained by subtracting the Moreau envelope of TGV from the TGV term, in which the former is defined by infimal convolution. To minimize the cost function, the proximal forward-backward splitting (FBS) algorithm is applied, and the alternating direction method of multipliers with adaptive parameter estimation (APE-ADMM), which avoids the drawbacks of FBS algorithm, is proposed by applying the variable splitting and the augment Lagrangian method (ALM). This proposed algorithm can update the regularization parameter adaptively. Experiments varify that the proposed method is more accurate in solution estimation than the other promoted ones, and the new ADMM algorithm with new regularization shows improved effects of image restoration.

Stochastic proximal splitting algorithm for composite minimization

Supported by the recent contributions in multiple domains, the first-order splitting became algorithms of choice for structured nonsmooth optimization. The large-scale noisy contexts make available stochastic information on the objective function and thus, the extension of proximal gradient schemes to stochastic oracles is heavily based on the tractability of the proximal operator corresponding to nonsmooth component, which has been highly exploited in the literature. However, some questions remained about the complexity of the composite models with proximal untractable terms. In this paper we tackle composite optimization problems, assuming only the access to stochastic information on both smooth and nonsmooth components, with a stochastic proximal first-order scheme with stochastic proximal updates. We provide sublinear \(\mathcal {O}\left( \frac{1}{k} \right) \) convergence rates (in expectation of squared distance to the optimal set) under the strong convexity assumption on the objective function. Also, linear convergence is achieved for convex feasibility problems. The empirical behavior is illustrated by numerical tests on parametric sparse representation models.

Optimal Control of the Principal Coefficient in a Scalar Wave Equation

We consider optimal control of the scalar wave equation where the control enters as a coefficient in the principal part. Adding a total variation penalty allows showing existence of optimal controls, which requires continuity results for the coefficient-to-solution mapping for discontinuous coefficients. We additionally consider a so-called multi-bang penalty that promotes controls taking on values pointwise almost everywhere from a specified discrete set. Under additional assumptions on the data, we derive an improved regularity result for the state, leading to optimality conditions that can be interpreted in an appropriate pointwise fashion. The numerical solution makes use of a stabilized finite element method and a nonlinear primal–dual proximal splitting algorithm.

The Proximal Alternating Direction Method of Multipliers in the Nonconvex Setting: Convergence Analysis and Rates

We propose two numerical algorithms in the fully nonconvex setting for the minimization of the sum of a smooth function and the composition of a nonsmooth function with a linear operator. The iterative schemes are formulated in the spirit of the proximal alternating direction method of multipliers and its linearized variant, respectively. The proximal terms are introduced via variable metrics, a fact that allows us to derive new proximal splitting algorithms for nonconvex structured optimization problems, as particular instances of the general schemes. Under mild conditions on the sequence of variable metrics and by assuming that a regularization of the associated augmented Lagrangian has the Kurdyka–Łojasiewicz property, we prove that the iterates converge to a Karush–Kuhn–Tucker point of the objective function. By assuming that the augmented Lagrangian has the Łojasiewicz property, we also derive convergence rates for both the augmented Lagrangian and the iterates.

Linearly involved generalized Moreau enhanced models and their proximal splitting algorithm under overall convexity condition

The convex envelopes of the direct discrete measures, for the sparsity of vectors or for the low-rankness of matrices, have been utilized extensively as practical penalties in order to compute a globally optimal solution of the corresponding regularized least-squares models. Motivated mainly by the ideas in Zhang (2010 Ann. Stat. 38 894–942; Selesnick 2017 IEEE Trans. Signal Process. 65 4481–94; Yin et al 2019 IEEE Trans. Signal Process. 67 2595–607) to exploit nonconvex penalties in the regularized least-squares models without losing their overall convexities, this paper presents the linearly involved generalized Moreau enhanced (LiGME) model as a unified extension of such utilizations of nonconvex penalties. The proposed model can admit multiple nonconvex penalties without losing its overall convexity and thus is applicable to much broader scenarios in the sparsity-rank-aware signal processing. Under the general overall-convexity condition of the LiGME model, we also present a novel proximal splitting type algorithm of guaranteed convergence to a globally optimal solution. Numerical experiments in typical examples of the sparsity-rank-aware signal processing demonstrate the effectiveness of the LiGME models and the proposed proximal splitting algorithm.

Convergence Guarantees for Non-Convex Optimisation With Cauchy-Based Penalties

In this paper, we propose a proximal splitting methodology with a non-convex penalty function based on the heavy-tailed Cauchy distribution. We first suggest a closed-form expression for calculating the proximal operator of the Cauchy prior, which then makes it applicable in generic proximal splitting algorithms. We further derive the condition required for guaranteed convergence to a solution in optimisation problems involving the Cauchy based penalty function. Setting the system parameters by satisfying the proposed condition ensures convergence even though the overall cost function is non-convex when minimisation is performed via a proximal splitting algorithm. The proposed method based on Cauchy regularisation is evaluated by solving generic signal processing examples, i.e. 1D signal denoising in the frequency domain, two image reconstruction tasks including de-blurring and denoising, and error recovery in a multiple-antenna communication system. We experimentally verify the proposed convergence conditions for various cases, and show the effectiveness of the proposed Cauchy based non-convex penalty function over state-of-the-art penalty functions such as $L_1$ and total variation ($TV$) norms.