Proximal Operator Research Articles

A subgradient proximal method for solving a class of monotone multivalued variational inequality problems

It is well known that the algorithms with using a proximal operator can be not convergent for monotone variational inequality problems in the general case. Malitsky (Optim. Methods Softw. 33 (1) 140–164, ??) proposed a proximal extrapolated gradient algorithm ensuring convergence for the problems, where the constraints are a finite-dimensional vector space. Based on this proximal extrapolated gradient techniques, we propose a new subgradient proximal iteration method for solving monotone multivalued variational inequality problems with the closed convex constraint. At each iteration, two strongly convex subprograms are required to solve separately by using proximal operators. Then, the algorithm is convergent for monotone and Lipschitz continuous cost mapping. We also use the proposed algorithm to solve a jointly constrained Cournot-Nash equilibirum model. Some numerical experiment and comparison results for convex nonlinear programming confirm efficiency of the proposed modification.

Gradient-Based Markov Chain Monte Carlo for Bayesian Inference With Non-differentiable Priors

The use of nondifferentiable priors in Bayesian statistics has become increasingly popular, in particular in Bayesian imaging analysis. Current state-of-the-art methods are approximate in the sense that they replace the posterior with a smooth approximation via Moreau-Yosida envelopes, and apply gradient-based discretized diffusions to sample from the resulting distribution. We characterize the error of the Moreau-Yosida approximation and propose a novel implementation using underdamped Langevin dynamics. In misson-critical cases, however, replacing the posterior with an approximation may not be a viable option. Instead, we show that piecewise-deterministic Markov processes (PDMP) can be used for exact posterior inference from distributions satisfying almost everywhere differentiability. Furthermore, in contrast with diffusion-based methods, the suggested PDMP-based samplers place no assumptions on the prior shape, nor require access to a computationally cheap proximal operator, and consequently have a much broader scope of application. Through detailed numerical examples, including a nondifferentiable circular distribution and a nonconvex genomics model, we elucidate the relative strengths of these sampling methods on problems of moderate to high dimensions, underlining the benefits of PDMP-based methods when accurate sampling is decisive. Supplementary materials for this article are available online.

How it's done: search tools and techniques for major bibliographic databases

This article explains how to write an effective search plan using simple steps. The article takes you through the tools and techniques that are widely used in major bibliographic databases such as MEDLINE and CINAHL to conduct searches. These include Boolean logic, truncation and wildcards, in-field searching, proximity operators, limits and subject thesauri. Each process is illustrated with an example to help you apply them to your own searches. The process of using these tools and techniques to either narrow (find fewer results) or broaden (find more results) is described and summarised in an easy-to-use table.

Nonnegative Blind Source Separation for Ill-Conditioned Mixtures via John Ellipsoid.

Nonnegative blind source separation (nBSS) is often a challenging inverse problem, namely, when the mixing system is ill-conditioned. In this work, we focus on an important nBSS instance, known as hyperspectral unmixing (HU) in remote sensing. HU is a matrix factorization problem aimed at factoring the so-called endmember matrix, holding the material hyperspectral signatures, and the abundance matrix, holding the material fractions at each image pixel. The hyperspectral signatures are usually highly correlated, leading to a fast decay of the singular values (and, hence, high condition number) of the endmember matrix, so HU often introduces an ill-conditioned nBSS scenario. We introduce a new theoretical framework to attack such tough scenarios via the John ellipsoid (JE) in functional analysis. The idea is to identify the maximum volume ellipsoid inscribed in the data convex hull, followed by affinely mapping such ellipsoid into a Euclidean ball. By applying the same affine mapping to the data mixtures, we prove that the endmember matrix associated with the mapped data has condition number 1, the lowest possible, and that these (preconditioned) endmembers form a regular simplex. Exploiting this regular structure, we design a novel nBSS criterion with a provable identifiability guarantee and devise an algorithm to realize the criterion. Moreover, for the first time, the optimization problem for computing JE is exactly solved for a large-scale instance; our solver employs a split augmented Lagrangian shrinkage algorithm with all proximal operators solved by closed-form solutions. The competitiveness of the proposed method is illustrated by numerical simulations and real data experiments.

An accelerated IRNN-Iteratively Reweighted Nuclear Norm algorithm for nonconvex nonsmooth low-rank minimization problems

Low-rank minimization problems arise in numerous important applications such as recommendation systems, machine learning, network analysis, and so on. The problems however typically consist of minimizing a sum of a smooth function and nonconvex nonsmooth composite functions, which solving them remains a big challenge. In this paper, we take inspiration from the Nesterov’s acceleration technique to accelerate an iteratively reweighted nuclear norm algorithm for the considered problems ensuring that every limit point is a critical point. Our algorithm iteratively computes the proximal operator of a reweighted nuclear norm which has a closed-form solution by performing the SVD of a smaller matrix instead of the full SVD. This distinguishes our work from recent accelerated proximal gradient algorithms that require an expensive computation of the proximal operator of nonconvex nonsmooth composite functions. We also investigate the convergence rate with the Kurdyka–Łojasiewicz assumption. Numerical experiments are performed to demonstrate the efficiency of our algorithm and its superiority over well-known methods.

Deep learning reconstruction of digital breast tomosynthesis images for accurate breast density and patient-specific radiation dose estimation.

The two-dimensional nature of mammography makes estimation of the overall breast density challenging, and estimation of the true patient-specific radiation dose impossible. Digital breast tomosynthesis (DBT), a pseudo-3D technique, is now commonly used in breast cancer screening and diagnostics. Still, the severely limited 3rd dimension information in DBT has not been used, until now, to estimate the true breast density or the patient-specific dose. This study proposes a reconstruction algorithm for DBT based on deep learning specifically optimized for these tasks. The algorithm, which we name DBToR, is based on unrolling a proximal-dual optimization method. The proximal operators are replaced with convolutional neural networks and prior knowledge is included in the model. This extends previous work on a deep learning-based reconstruction model by providing both the primal and the dual blocks with breast thickness information, which is available in DBT. Training and testing of the model were performed using virtual patient phantoms from two different sources. Reconstruction performance, and accuracy in estimation of breast density and radiation dose, were estimated, showing high accuracy (density <±3%; dose <±20%) without bias, significantly improving on the current state-of-the-art. This work also lays the groundwork for developing a deep learning-based reconstruction algorithm for the task of image interpretation by radiologists.

Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization

&lt;p style='text-indent:20px;'&gt;We study the problem of minimizing the sum of two functions. The first function is the average of a large number of nonconvex component functions and the second function is a convex (possibly nonsmooth) function that admits a simple proximal mapping. With a diagonal Barzilai-Borwein stepsize for updating the metric, we propose a variable metric proximal stochastic variance reduced gradient method in the mini-batch setting, named VM-SVRG. It is proved that VM-SVRG converges sublinearly to a stationary point in expectation. We further suggest a variant of VM-SVRG to achieve linear convergence rate in expectation for nonconvex problems satisfying the proximal Polyak-Łojasiewicz inequality. The complexity of VM-SVRG is lower than that of the proximal gradient method and proximal stochastic gradient method, and is the same as the proximal stochastic variance reduced gradient method. Numerical experiments are conducted on standard data sets. Comparisons with other advanced proximal stochastic gradient methods show the efficiency of the proposed method.&lt;/p&gt;

A Golden Ratio Primal–Dual Algorithm for Structured Convex Optimization

We design, analyze and test a golden ratio primal–dual algorithm (GRPDA) for solving structured convex optimization problem, where the objective function is the sum of two closed proper convex functions, one of which involves a composition with a linear transform. GRPDA preserves all the favorable features of the classical primal–dual algorithm (PDA), i.e., the primal and the dual variables are updated in a Gauss–Seidel manner, and the per iteration cost is dominated by the evaluation of the proximal point mappings of the two component functions and two matrix-vector multiplications. Compared with the classical PDA, which takes an extrapolation step, the novelty of GRPDA is that it is constructed based on a convex combination of essentially the whole iteration trajectory. We show that GRPDA converges within a broader range of parameters than the classical PDA, provided that the reciprocal of the convex combination parameter is bounded above by the golden ratio, which explains the name of the algorithm. An $$\mathcal {O}(1/N)$$ ergodic convergence rate result is also established based on the primal–dual gap function, where N denotes the number of iterations. When either the primal or the dual problem is strongly convex, an accelerated GRPDA is constructed to improve the ergodic convergence rate from $$\mathcal {O}(1/N)$$ to $$\mathcal {O}(1/N^2)$$ . Moreover, we show for regularized least-squares and linear equality constrained problems that the reciprocal of the convex combination parameter can be extended from the golden ratio to 2 and meanwhile a relaxation step can be taken. Our preliminary numerical results on LASSO, nonnegative least-squares and minimax matrix game problems, with comparisons to some state-of-the-art relative algorithms, demonstrate the efficiency of the proposed algorithms.

Proximal sensor-enhanced soil mapping in complex soil-landscape areas of Brazil

Portable X-ray fluorescence (pXRF) spectrometry and magnetic susceptibility (MS) via magnetometer have been increasingly used with terrain variables for digital soil mapping. However, this methodology is still emerging in many countries with tropical soils. The objective of this study was to use proximal soil sensor data associated with terrain variables at varying spatial resolutions to predict soil classes using the Random Forest (RF) algorithm. The study was conducted on a 316-ha area featuring highly variable soil classes and complex soil-landscape relationships in Minas Gerais State, Brazil. The overall accuracy and Kappa index were evaluated using soils that were classified at 118 sites, with 90 being used for modeling and 28 for validation. Digital elevation models (DEMs) were created at 5-, 10-, 20-, and 30-m resolutions using contour lines from two sources. The resulting DEMs were processed to generate 12 terrain variables. Total Fe, Ti, and SiO2 contents were obtained using pXRF, with MS determined via a magnetometer. Soil class prediction was performed using the RF algorithm. The quality of the soil maps improved when using only the five most important covariates and combining proximal sensor data with terrain variables at different spatial resolutions. The finest spatial resolution did not always provide the most accurate maps. The high soil complexity in the area prevented highly accurate predictions. The most important variables influencing the soil mapping were MS, Fe, and Ti. Proximal sensor data associated with terrain information were successfully used to map Brazilian soils at variable spatial resolutions.

Optimization-inspired deep learning high-resolution inversion for seismic data

Seismic high-resolution processing plays a critical role in reservoir target detection. As one of the most common approaches, regularization can achieve a high-resolution inversion result. However, the performance of regularization depends on the settings of the associated parameters and constraint functions. Further, it is difficult to solve an objective function with complex constraints, and it requires designing an optimization algorithm. In addition, existing algorithms have high computational complexity, which impedes the inversion of the large data volume. To address these problems, an optimization-inspired deep learning inversion solver is proposed to solve the blind high-resolution inverse (BHRI) problems of various seismic wavelets rapidly, called BHRI-Net. The method builds on ideas from classic regularization theory and recent advances in deep learning, and it makes full use of prior information encoded in the forward operator and noise model to learn an accurate mapping relationship. It unrolls the alternating iterative BHRI algorithm into a deep neural network, and it applies the convolutional neural network to learn proximal mappings, in which all parameters of the BHRI algorithm are learned from training data. Further, the proposed network can be split into two parts and incorporate the transfer learning strategy to invert field data, which increases the flexibility of the proposed network and reduces training time. Finally, the tests on synthetic and field data show that the proposed method can effectively invert the high-resolution data and seismic wavelet from observation data with improved accuracy and high computational efficiency.

BCOR modulates transcriptional activity of a subset of glucocorticoid receptor target genes involved in cell growth and mobility

Glucocorticoid (GC) receptor (GR) is a key transcription factor (TF) that regulates vital metabolic and anti-inflammatory processes. We have identified BCL6 corepressor (BCOR) as a dexamethasone-stimulated interaction partner of GR. BCOR is a component of non-canonical polycomb repressor complex 1.1 (ncPCR1.1) and linked to different developmental disorders and cancers, but the role of BCOR in GC signaling is poorly characterized. Here, using ChIP-seq we show that, GC induces genome-wide redistribution of BCOR chromatin binding towards GR-occupied enhancers in HEK293 cells. As assessed by RNA-seq, depletion of BCOR altered the expression of hundreds of GC-regulated genes, especially the ones linked to TNF signaling, GR signaling and cell migration pathways. Biotinylation-based proximity mapping revealed that GR and BCOR share several interacting partners, including nuclear receptor corepressor NCOR1. ChIP-seq showed that the NCOR1 co-occurs with both BCOR and GR on a subset of enhancers upon GC treatment. Simultaneous depletion of BCOR and NCOR1 influenced GR target gene expression in a combinatorial and gene-specific manner. Finally, we show using live cell imaging that the depletion of BCOR together with NCOR1 markedly enhances cell migration. Collectively, our data suggest BCOR as an important gene and pathway selective coregulator of GR transcriptional activity.

An Optimal High-Order Tensor Method for Convex Optimization

This paper is concerned with finding an optimal algorithm for minimizing a composite convex objective function. The basic setting is that the objective is the sum of two convex functions: the first function is smooth with up to the dth-order derivative information available, and the second function is possibly nonsmooth, but its proximal tensor mappings can be computed approximately in an efficient manner. The problem is to find—in that setting—the best possible (optimal) iteration complexity for convex optimization. Along that line, for the smooth case (without the second nonsmooth part in the objective), Nesterov proposed an optimal algorithm for the first-order methods ([Formula: see text]) with iteration complexity [Formula: see text], whereas high-order tensor algorithms (using up to general dth-order tensor information) with iteration complexity [Formula: see text] were recently established. In this paper, we propose a new high-order tensor algorithm for the general composite case, with the iteration complexity of [Formula: see text], which matches the lower bound for the dth-order methods as previously established and hence is optimal. Our approach is based on the accelerated hybrid proximal extragradient (A-HPE) framework proposed by Monteiro and Svaiter, where a bisection procedure is installed for each A-HPE iteration. At each bisection step, a proximal tensor subproblem is approximately solved, and the total number of bisection steps per A-HPE iteration is shown to be bounded by a logarithmic factor in the precision required.

Peridynamics enabled learning partial differential equations

This study presents an approach to discover the significant terms in partial differential equations (PDEs) that describe particular phenomena based on the measured data. The relationship between the known field data and its continuous representation of PDEs is achieved through a linear regression model. It specifically employs the peridynamic differential operator (PDDO) and sparse linear regression learning algorithm. The PDEs are approximated by constructing a feature matrix, velocity vector and unknown coefficient vector. Each candidate term (derivatives) appearing in the feature matrix is evaluated numerically by using the PDDO. The solution to the regression model with regularization is achieved through Douglas-Rachford (D-R) algorithm which is based on proximal operators. This coupling performs well due to their robustness to noisy data and the calculation of accurate derivatives. Its effectiveness is demonstrated by considering several fabricated data associated with challenging nonlinear PDEs such as Burgers, Swift-Hohenberg (S-H), Korteweg-de Vries (KdV), Kuramoto-Sivashinsky (K-S), nonlinear Schrödinger (NLS) and Cahn-Hilliard (C-H) equations.

Quantitative Characterization of Tumor Proximity to Stem Cell Niches: Implications on Recurrence and Survival in GBM Patients.

Emerging evidence has linked glioblastoma multiforme (GBM) recurrence and survival to stem cell niches (SCNs). However, the traditional tumor-ventricle distance is insufficiently powered for an accurate prediction. We aimed to use a novel inverse distance map for improved prediction. Two T1-magnetic resonance imaging data sets were included for a total of 237 preoperative scans for prognostic stratification and 55 follow-up scans for recurrent pattern identification. SCN, including the subventricular zone (SVZ) and subgranular zone (SGZ), were manually defined on a standard template. A proximity map was generated using the summed inverse distances to all SCN voxels. The mean and maximum proximity scores (PSm-SCN and PSmax-SCN) were calculated for each primary/recurrent tumor, deformably transformed into the template. The prognostic capacity of proximity score (PS)-derived metrics was assessed using Cox regression and log-rank tests. To evaluate the impact of SCNs on recurrence patterns, we performed group comparisons of PS-derived metrics between the primary and recurrent tumors. For comparison, the same analyses were conducted on PS derived from SVZ alone and traditional edge/center-to-ventricle metrics. Among all SCN-derived features, PSm-SCN was the strongest survival predictor (P < .0001). PSmax-SCN was the best in risk stratification, using either evenly sorted (P = .0001) or k-means clustering methods (P = .0045). PS metrics based on SVZ only also correlated with overall survival and risk stratification, but to a lesser degree of significance. In contrast, edge/center-to-ventricle metrics showed weak to no prediction capacities in either task. Moreover, PSm-SCN,PSm-SVZ, and center-to-ventricle metrics revealed a significantly closer SCN distribution of recurrence than primary tumors. We introduced a novel inverse distance-based metric to comprehensively capture the anatomic relationship between GBM tumors and SCN zones. The derived metrics outperformed traditional edge or center distance-based measurements in overall survival prediction, risk stratification, and recurrent pattern differentiation. Our results reveal the potential role of SGZ in recurrence aside from SVZ.

Tensor Recovery via $*_L$-Spectral $k$-Support Norm

Unlike traditional tensor decompositions which model low-rankness in the original domain, the recently proposed tensor <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*L</sub> -Singular Value Decomposition ( <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*L</sub> -SVD) casts a new light on tensor analysis by exploiting low-rankness in the spectral domain. For convexized rank minimization within the framework of <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*L</sub> -SVD, the tensor <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*L</sub> -Tubal Nuclear Norm ( <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*L</sub> -TNN) out-performs classical tensorial extensions of matrix nuclear norm in many applications. In this paper, we first generalize <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*L</sub> -TNN to the <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> *L</sub> -Spectral k-Support Norm ( <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*L</sub> -SpSN-k)as a new low-rank regularizer, and then adopt it to formulate two estimators for tensor recovery from noisy linear observations. Further, statistical performance of the proposed estimators is analyzed by establishing both deterministic and non-asymptotic upper bounds on the estimation error, which indicates that the proposed tensor Dantzig selector enjoys near-optimal estimation error for tensor compressive sensing and tensor completion. Moreover, proximal operator of the proposed norm is derived and embeded in an efficient algorithm to compute the estimators. Experiments on synthetic datasets verify correctness of the proposed error bounds and image inpainting results demonstrate superiority of the proposed estimators.

Inundation Assessment of the 2019 Typhoon Hagibis in Japan Using Multi-Temporal Sentinel-1 Intensity Images

Typhoon Hagibis passed through Japan on October 12, 2019, bringing heavy rainfall over half of Japan. Twelve banks of seven state-managed rivers collapsed, flooding a wide area. Quick and accurate damage proximity maps are helpful for emergency responses and relief activities after such disasters. In this study, we propose a quick analysis procedure to estimate inundations due to Typhoon Hagibis using multi-temporal Sentinel-1 SAR intensity images. The study area was Ibaraki Prefecture, Japan, including two flooded state-managed rivers, Naka and Kuji. First, the completely flooded areas were detected by two traditional methods, the change detection and the thresholding methods. By comparing the results in a part of the affected area with our field survey, the change detection was adopted due to its higher recall accuracy. Then, a new index combining the average value and the standard deviation of the differences was proposed for extracting partially flooded built-up areas. Finally, inundation maps were created by merging the completely and partially flooded areas. The final inundation map was evaluated via comparison with the flooding boundary produced by the Geospatial Information Authority (GSI) and the Ministry of Land, Infrastructure, Transport, and Tourism (MLIT) of Japan. As a result, 74% of the inundated areas were able to be identified successfully using the proposed quick procedure.

Inexact proximal memoryless quasi-Newton methods based on the Broyden family for minimizing composite functions

This study considers a proximal Newton-type method to solve the minimization of a composite function that is the sum of a smooth nonconvex function and a nonsmooth convex function. In general, the method uses the Hessian matrix of the smooth portion of the objective function or its approximation. The uniformly positive definiteness of the matrix plays an important role in establishing the global convergence of the method. In this study, an inexact proximal memoryless quasi-Newton method is proposed based on the memoryless Broyden family with the modified spectral scaling secant condition. The proposed method inexactly solves the subproblems to calculate scaled proximal mappings. The approximation matrix is shown to retain the uniformly positive definiteness and the search direction is a descent direction. Using these properties, the proposed method is shown to have global convergence for nonconvex objective functions. Furthermore, the R-linear convergence for strongly convex objective functions is proved. Finally, some numerical results are provided.

A proximal neurodynamic model for solving inverse mixed variational inequalities

This paper proposes a proximal neurodynamic model (PNDM) for solving inverse mixed variational inequalities (IMVIs) based on the proximal operator. It is shown that the PNDM has a unique continuous solution under the condition of Lipschitz continuity (L-continuity). It is also shown that the equilibrium point of the proposed PNDM is asymptotically stable or exponentially stable under some mild conditions. Finally, three numerical examples are presented to illustrate effectiveness of the proposed PNDM.

New convergence analysis of a primal-dual algorithm with large stepsizes

We consider a primal-dual algorithm for minimizing $f(x)+h\square l(Ax)$ with Fr\'echet differentiable $f$ and $l^*$. This primal-dual algorithm has two names in literature: Primal-Dual Fixed-Point algorithm based on the Proximity Operator (PDFP$^2$O) and Proximal Alternating Predictor-Corrector (PAPC). In this paper, we prove its convergence under a weaker condition on the stepsizes than existing ones. With additional assumptions, we show its linear convergence. In addition, we show that this condition (the upper bound of the stepsize) is tight and can not be weakened. This result also recovers a recently proposed positive-indefinite linearized augmented Lagrangian method. In addition, we apply this result to a decentralized consensus algorithm PG-EXTRA and derive the weakest convergence condition.

Stochastic proximal splitting algorithm for composite minimization

Supported by the recent contributions in multiple domains, the first-order splitting became algorithms of choice for structured nonsmooth optimization. The large-scale noisy contexts make available stochastic information on the objective function and thus, the extension of proximal gradient schemes to stochastic oracles is heavily based on the tractability of the proximal operator corresponding to nonsmooth component, which has been highly exploited in the literature. However, some questions remained about the complexity of the composite models with proximal untractable terms. In this paper we tackle composite optimization problems, assuming only the access to stochastic information on both smooth and nonsmooth components, with a stochastic proximal first-order scheme with stochastic proximal updates. We provide sublinear \(\mathcal {O}\left( \frac{1}{k} \right) \) convergence rates (in expectation of squared distance to the optimal set) under the strong convexity assumption on the objective function. Also, linear convergence is achieved for convex feasibility problems. The empirical behavior is illustrated by numerical tests on parametric sparse representation models.