Quadratic Regularization Research Articles

Direct Reconstruction of CT-based Attenuation Correction Images for PET with Cluster-Based Penalties.

Extremely low-dose CT acquisitions used for PET attenuation correction have high levels of noise and potential bias artifacts due to photon starvation. This work explores the use of a priori knowledge for iterative image reconstruction of the CT-based attenuation map. We investigate a maximum a posteriori framework with cluster-based multinomial penalty for direct iterative coordinate decent (dICD) reconstruction of the PET attenuation map. The objective function for direct iterative attenuation map reconstruction used a Poisson log-likelihood data fit term and evaluated two image penalty terms of spatial and mixture distributions. The spatial regularization is based on a quadratic penalty. For the mixture penalty, we assumed that the attenuation map may consist of four material clusters: air+background, lung, soft tissue, and bone. Using simulated noisy sinogram data, dICD reconstruction was performed with different strengths of the spatial and mixture penalties. The combined spatial and mixture penalties reduced the RMSE by roughly 2 times compared to a weighted least square and filtered backprojection reconstruction of CT images. The combined spatial and mixture penalties resulted in only slightly lower RMSE compared to a spatial quadratic penalty alone. For direct PET attenuation map reconstruction from ultra-low dose CT acquisitions, the combination of spatial and mixture penalties offers regularization of both variance and bias and is a potential method to reconstruct attenuation maps with negligible patient dose. The presented results, using a best-case histogram suggest that the mixture penalty does not offer a substantive benefit over conventional quadratic regularization and diminishes enthusiasm for exploring future application of the mixture penalty.

The Use of Quadratic Regularization with a Cubic Descent Condition for Unconstrained Optimization

Cubic-regularization and trust-region methods with worst-case first-order complexity $O(\varepsilon^{-3/2})$ and worst-case second-order complexity $O(\varepsilon^{-3})$ have been developed in the last few years. In this paper it is proved that the same complexities are achieved by means of a quadratic-regularization method with a cubic sufficient-descent condition instead of the more usual predicted-reduction based descent. Asymptotic convergence and order of convergence results are also presented. Finally, some numerical experiments comparing the new algorithm with a well-established quadratic regularization method are shown.

Functional error estimators for the adaptive discretization of inverse problems

So-called functional error estimators provide a valuable tool for reliably estimating the discretization error for a sum of two convex functions. We apply this concept to Tikhonov regularization for the solution of inverse problems for partial differential equations, not only for quadratic Hilbert space regularization terms but also for nonsmooth Banach space penalties. Examples include the measure-space norm (i.e., sparsity regularization) or the indicator function of an ball (i.e., Ivanov regularization). The error estimators can be written in terms of residuals in the optimality system that can then be estimated by conventional techniques, thus leading to explicit estimators. This is illustrated by means of an elliptic inverse source problem with the above-mentioned penalties, and numerical results are provided for the case of sparsity regularization.

Second-order TGV model for Poisson noise image restoration.

Restoring Poissonian noise images have drawn a lot of attention in recent years. There are many regularization methods to solve this problem and one of the most famous methods is the total variation model. In this paper, by adding a quadratic regularization on TGV regularization part, a new image restoration model is proposed based on second-order total generalized variation regularization. Then the split Bregman iteration algorithm was used to solve this new model. The experimental results show that the proposed model and algorithm can deal with Poisson image restoration problem well. What’s more, the restoration model performance is significantly improved both in visual effect and objective evaluation indexes.

Adaptive Elastic Echo State Network for Multivariate Time Series Prediction.

Echo state network (ESN) is a new kind of recurrent neural network with a randomly generated reservoir structure and an adaptable linear readout layer. It has been widely employed in the field of time series prediction. However, when high-dimensional reservoirs are utilized to predict multivariate time series, there may be a collinearity problem. In this paper, to overcome the collinearity problem and obtain a sparse solution, we propose a new model-adaptive elastic ESN, in which adaptive elastic net algorithm is used to calculate the unknown weights. It combines the strengths of the quadratic regularization and the adaptively weighted lasso shrinkage. Hence, the proposed model can deal with the collinearity problem and enjoy the oracle property with an unbiased estimation. We exhibit the merits of our model on two benchmark multivariate chaotic datasets and two real-world applications. Experimental results substantiate the effectiveness and characteristics of the proposed model.

Fast Variance Prediction for Iteratively Reconstructed CT Images With Locally Quadratic Regularization

Predicting noise properties of iteratively reconstructed CT images is useful for analyzing reconstruction methods; for example, local noise power spectrum (NPS) predictions may be used to quantify the detectability of an image feature, to design regularization methods, or to determine dynamic tube current adjustment during a CT scan. This paper presents a method for fast prediction of reconstructed image variance and local NPS for statistical reconstruction methods using quadratic or locally quadratic regularization. Previous methods either require impractical computation times to generate an approximate map of the variance of each reconstructed voxel, or are restricted to specific CT geometries. Our method can produce a variance map of the entire image, for locally shift-invariant CT geometries with sufficiently fine angular sampling, using a computation time comparable to a single back-projection. The method requires only the projection data to be used in the reconstruction, not a reconstruction itself, and is reasonably accurate except near image edges where edge-preserving regularization behaves highly nonlinearly. We evaluate the accuracy of our method using reconstructions of both simulated CT data and real CT scans of a thorax phantom.

Hanke-Raus heuristic rule for variational regularization in Banach spaces

We generalize the heuristic parameter choice rule of Hanke-Raus for quadratic regularization to general variational regularization for solving linear as well as nonlinear ill-posed inverse problems in Banach spaces. Under source conditions formulated as variational inequalities, we obtain a posteriori error estimates in term of the Bregman distance. By imposing certain conditions on the random noise, we establish four convergence results; one relies on the source conditions and the other three do not depend on any source conditions. Numerical results are presented to illustrate the performance.

New second order Mumford–Shah model based on Γ-convergence approximation for image processing

In this paper, a second order variational model named the Mumford–Shah total generalized variation (MSTGV) is proposed for simultaneously image denoising and segmentation, which combines the original Γ-convergence approximated Mumford–Shah model with the second order total generalized variation (TGV). For image denoising, the proposed MSTGV can eliminate both the staircase artefact associated with the first order total variation and the edge blurring effect associated with the quadratic H1 regularization or the second order bounded Hessian regularization. For image segmentation, the MSTGV can obtain clear and continuous boundaries of objects in the image. To improve computational efficiency, the implementation of the MSTGV does not directly solve its high order nonlinear partial differential equations and instead exploits the efficient split Bregman algorithm. The algorithm benefits from the fast Fourier transform, analytical generalized soft thresholding equation, and Gauss–Seidel iteration. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of the proposed model.

Bezdek-Type Fuzzified Co-Clustering Algorithm

In this study, two co-clustering algorithms based on Bezdek-type fuzzification of fuzzy clustering are proposed for categorical multivariate data. The two proposed algorithms are motivated by the fact that there are only two fuzzy co-clustering methods currently available – entropy regularization and quadratic regularization – whereas there are three fuzzy clustering methods for vectorial data: entropy regularization, quadratic regularization, and Bezdek-type fuzzification. The first proposed algorithm forms the basis of the second algorithm. The first algorithm is a variant of a spherical clustering method, with the kernelization of a maximizing model of Bezdek-type fuzzy clustering with multi-medoids. By interpreting the first algorithm in this way, the second algorithm, a spectral clustering approach, is obtained. Numerical examples demonstrate that the proposed algorithms can produce satisfactory results when suitable parameter values are selected.

Monocular Concurrent Recovery of Structure and Motion Scene Flow

This paper describes a variational method of joint three-dimensional structure and motion scene flow recovery from a single image sequence. A basic scheme is developed by minimizing a functional with a term of conformity of scene flow and depth to the image sequence spatiotemporal variations, and quadratic smoothness regularization terms. The data term follows by re-writing optical velocity in the optical flow gradient constraint in terms of scene flow and depth. As a result, this problem statement is analogous to the classical Horn and Schunck optical flow formulation except that it involves scene flow and depth rather than image motion. When discretized, the Euler-Lagrange equations give a large scale sparse system of linear equations in the unknowns of the scene flow three coordinates and depth. The equations can be ordered in such a way that its matrix is symmetric positive definite such that they can be solved efficiently by Gauss-Seidel iterations. Experiments are shown to verify the scheme’s validity and efficiency.

Dualization and discretization of linear-quadratic control problems with bang–bang solutions

We consider linear-quadratic (LQ) control problems, where the control variable appears linearly and is box-constrained. It is well-known that these problems exhibit bang–bang and singular solutions. We assume that the solution is of bang–bang type, which is computationally challenging to obtain. We employ a quadratic regularization of the LQ control problem by embedding the $$L^2$$ -norm of the control variable into the cost functional. First, we find a dual problem guided by the methodology of Fenchel duality. Then we prove strong duality and the saddle point property, which together ensure that the primal solution can be recovered from the dual solution. We propose a discretization scheme for the dual problem, under which a diagram depicting the relations between the primal and dual problems and their discretization commutes. The commuting diagram ensures that, given convergence results for the discrete primal variables, discrete dual variables also converge to a solution of the dual problem with a similar error bound. We demonstrate via a simple but illustrative example that significant computational savings can be achieved by solving the dual, rather than the primal, problem.

Cross-domain, soft-partition clustering with diversity measure and knowledge reference

Conventional, soft-partition clustering approaches, such as fuzzy c-means (FCM), maximum entropy clustering (MEC) and fuzzy clustering by quadratic regularization (FC-QR), are usually incompetent in those situations where the data are quite insufficient or much polluted by underlying noise or outliers. In order to address this challenge, the quadratic weights and Gini-Simpson diversity based fuzzy clustering model (QWGSD-FC), is first proposed as a basis of our work. Based on QWGSD-FC and inspired by transfer learning, two types of cross-domain, soft-partition clustering frameworks and their corresponding algorithms, referred to as type-I/type-II knowledge-transfer-oriented c-means (TI-KT-CM and TII-KT-CM), are subsequently presented, respectively. The primary contributions of our work are four-fold: (1) The delicate QWGSD-FC model inherits the most merits of FCM, MEC and FC-QR. With the weight factors in the form of quadratic memberships, similar to FCM, it can more effectively calculate the total intra-cluster deviation than the linear form recruited in MEC and FC-QR. Meanwhile, via Gini-Simpson diversity index, like Shannon entropy in MEC, and equivalent to the quadratic regularization in FC-QR, QWGSD-FC is prone to achieving the unbiased probability assignments, (2) owing to the reference knowledge from the source domain, both TI-KT-CM and TII-KT-CM demonstrate high clustering effectiveness as well as strong parameter robustness in the target domain, (3) TI-KT-CM refers merely to the historical cluster centroids, whereas TII-KT-CM simultaneously uses the historical cluster centroids and their associated fuzzy memberships as the reference. This indicates that TII-KT-CM features more comprehensive knowledge learning capability than TI-KT-CM and TII-KT-CM consequently exhibits more perfect cross-domain clustering performance and (4) neither the historical cluster centroids nor the historical cluster centroid based fuzzy memberships involved in TI-KT-CM or TII-KT-CM can be inversely mapped into the raw data. This means that both TI-KT-CM and TII-KT-CM can work without disclosing the original data in the source domain, i.e. they are of good privacy protection for the source domain. In addition, the convergence analyses regarding both TI-KT-CM and TII-KT-CM are conducted in our research. The experimental studies thoroughly evaluated and demonstrated our contributions on both synthetic and real-life data scenarios.

An Inverse Problems Approach to MR-EPT Image Reconstruction.

Magnetic Resonance-Electrical Properties Tomography (MR-EPT) is an imaging modality that maps the spatial distribution of the electrical conductivity and permittivity using standard MRI systems. The presence of a body within the scanner alters the RF field, and by mapping these alterations it is possible to recover the electrical properties. The field is time-harmonic, and can be described by the Helmholtz equation. Approximations to this equation have been previously used to estimate conductivity and permittivity in terms of first or second derivatives of RF field data. Using these same approximations, an inverse approach to solving the MR-EPT problem is presented here that leverages a forward model for describing the magnitude and phase of the field within the imaging domain, and a fitting approach for estimating the electrical properties distribution. The advantages of this approach are that 1) differentiation of the measured data is not required, thus reducing noise sensitivity, and 2) different regularization schemes can be adopted, depending on prior knowledge of the distribution of conductivity or permittivity, leading to improved image quality. To demonstrate the developed approach, both Quadratic (QR) and Total Variation (TV) regularization methods were implemented and evaluated through numerical simulation and experimentally acquired data. The proposed inverse approach to MR-EPT reconstruction correctly identifies contrasts and accurately reconstructs the geometry in both simulations and experiments. The TV regularized scheme reconstructs sharp spatial transitions, which are difficult to reconstruct with other, more traditional approaches.

On the connection and equivalence of two methods for solving an ill-posed inverse problem based on FRAP data

Two methods for solving an ill-posed inverse problem based on Fickian diffusion equation and spatio-temporal data from FRAP measurements are presented. The most usual method is the Tikhonov regularization. Nevertheless, in our specific problem we have detected difficulties residing in determination of the optimal regularization parameter α. Hence, an equivalent method based on least squares with a quadratic constraint regularization is proposed. This latter approach naturally takes into account the noise level in the data and corresponds to Morozov’s discrepancy principle as well. The equivalence of both methods is rigorously proven and on a simple numerical example with synthetic input data practically documented.

Adaptive Elastic Net GMM Estimation with Many Invalid Moment Conditions: Simultaneous Model and Moment Selection

Abstract This paper develops a new estimator. An adaptive elastic-net GMM estimator with possibly many invalid moment conditions is shown. We allow for the number of structural parameters (p_{0}) as well as the number of moment conditions increase with the sample size (n). We do simultaneous model and moment selection. We estimate the structural parameters along with parameters attached with invalid moments. The basic idea is to conduct the standard GMM with combining two penalty terms: the quadratic regularization and the adaptively weighted lasso shrinkage. Given many orthogonality restrictions, including the invalid ones, the new estimator uses information only from the valid moment conditions to achieve the semiparametric efficiency bound. The estimator is thus very useful in practice since it conducts the consistent moment selection and efficient estimation of the structural parameters simultaneously. We also establish the order of magnitude for the smallest local to zero coefficient to be selected as nonzero. We apply the new estimation procedure to dynamic panel data models, where both the time and cross section dimensions are large. The new estimator is robust to possible serial correlations in the regression error term.

Second Order and Stability Analysis for Optimal Sparse Control of the FitzHugh--Nagumo Equation

Optimal sparse control problems are considered for the FitzHugh--Nagumo system including the so-called Schlogl model. The nondifferentiable objective functional of tracking type includes a quadratic Tikhonov regularization term and the $L^1$-norm of the control that accounts for the sparsity. Though the objective functional is not differentiable, a theory of second order sufficient optimality conditions is established for Tikhonov regularization parameter $\nu>0$ and also for the case $\nu = 0$. In this context, also local minima are discussed that are strong in the sense of the calculus of variations. The second order conditions are used as the main assumption for proving the stability of locally optimal solutions with respect to $\nu \to 0$ and with respect to perturbations of the desired state functions. The theory is confirmed by numerical examples that are resolved with high precision to confirm that the optimal solution obeys the system of necessary optimality conditions.

Linearly Constrained Non-Lipschitz Optimization for Image Restoration

Nonsmooth nonconvex optimization models have been widely used in the restoration and reconstruction of real images. In this paper, we consider a linearly constrained optimization problem with a non-Lipschitz regularization term in the objective function which includes the $l_p$ norm ($0<p<1$) of the gradient of the underlying image in the $l_2$-$l_p$ problem as a special case. We prove that any cluster point of $\epsilon$ scaled first order stationary points satisfies a first order necessary condition for a local minimizer of the optimization problem as $\epsilon $ goes to $0$. We propose a smoothing quadratic regularization (SQR) method for solving the problem. At each iteration of the SQR algorithm, a new iterate is generated by solving a strongly convex quadratic problem with linear constraints. Moreover, we show that the SQR algorithm can find an $\epsilon$ scaled first order stationary point in at most $O(\epsilon^{-2})$ iterations from any starting point. Numerical examples are given to show good performance of the SQR algorithm for image restoration.

Поиск глобального минимума методом точной квадратичной регуляризации

We offer a new method of exact quadratic regularization for search of a global minimum the functions with constraints. The method includes nonlinear transformations of the functions, local search and a dichotomy. This method has allowed to solve set of difficult test and applied problems of global optimization. Comparative numerical experiments have shown that new method is the best for the solution of this class of problems.

A note on the smoothing quadratic regularization method for non-Lipschitz optimization

We present a new smaller upper bound for all the elements in the associate generalized Hessian used in the smoothing quadratic regularization (SQR) algorithm proposed by Bian and Chen (SIAM J. Optim. 23: 1718-1741, 2013). We modify the SQR algorithm by making use of the new upper bound. Numerical results show that our new upper bound improves the performance of the SQR algorithm significantly.

Quadratic regularization projected Barzilai–Borwein method for nonnegative matrix factorization

In this paper, based on the alternating nonnegative least squares framework, we present a new efficient method for nonnegative matrix factorization that uses a quadratic regularization projected Barzilai---Borwein (QRPBB) method to solve the subproblems. At each iteration, the QRPBB method first generates a point by solving a strongly convex quadratic minimization problem, which has a simple closed-form solution that is inexpensive to calculate, and then applies a projected Barzilai---Borwein method to update the solution of NMF. Global convergence result is established under mild conditions. Numerical comparisons of methods on both synthetic and real-world datasets show that the proposed method is efficient.