Optimization Problem In Hilbert Space Research Articles

Improved Accuracy Estimation of the Tikhonov Method for Ill-Posed Optimization Problems in Hilbert Space

Improved Accuracy Estimation of the Tikhonov Method for Ill-Posed Optimization Problems in Hilbert Space

Exact augmented Lagrangians for constrained optimization problems in Hilbert spaces I: theory

In this two-part study, we develop a general theory of the so-called exact augmented Lagrangians for constrained optimization problems in Hilbert spaces. In contrast to traditional nonsmooth exact penalty functions, these augmented Lagrangians are continuously differentiable for smooth problems and do not suffer from the Maratos effect, which makes them especially appealing for applications in numerical optimization. Our aim is to present a detailed study of various theoretical properties of exact augmented Lagrangians and discuss several applications of these functions to constrained variational problems, problems with PDE constraints, and optimal control problems. The first paper is devoted to a theoretical analysis of an exact augmented Lagrangian for optimization problems in Hilbert spaces. We obtain several useful estimates of this augmented Lagrangian and its gradient, and present several types of sufficient conditions for KKT-points of a constrained problem corresponding to locally/globally optimal solutions to be local/global minimizers of the exact augmented Lagrangian.

A stochastic regularized second-order iterative scheme for optimal control and inverse problems in stochastic partial differential equations.

Numerous applied models used in the study of optimal control problems, inverse problems, shape optimization, machine learning, fractional programming, neural networks, image registration and so on lead to stochastic optimization problems in Hilbert spaces. Under a suitable convexity assumption on the objective function, a necessary and sufficient optimality condition for stochastic optimization problems is a stochastic variational inequality. This article presents a new stochastic regularized second-order iterative scheme for solving a variational inequality in a stochastic environment where the primary operator is accessed by employing sampling techniques. The proposed iterative scheme, which fits within the general framework of the stochastic approximation approach, has its almost-sure convergence analysis given in a Hilbert space. We test the feasibility and the efficacy of the proposed stochastic approximation approach for a stochastic optimal control problem and a stochastic inverse problem, both associated with a second-order stochastic partial differential equation. This article is part of the theme issue 'Non-smooth variational problems and applications'.

Application of physics-constrained data-driven reduced-order models to shape optimization

This study proposes a novel approach for building surrogate models, in the form of reduced-order models(ROMs), for partial differential equation constrained optimization. A physics-constrained data-driven framework is developed to transform large-scale nonlinear optimization problems into ROM-constrained optimization problems. Unlike conventional methods, which suffer from instability of the forward sensitivity function, the proposed approach maps optimization problems to system dynamics optimization problems in Hilbert space to improve stability, reduce memory requirements, and lower computational cost. The utility of this approach is demonstrated for aerodynamic optimization of an NACA 0012 airfoil at $Re = 1000$ . A drag reduction of 9.35 % is obtained at an effective angle of attack of eight degrees, with negligible impact on lift. Similarly, a drag reduction of 20 % is obtained for fully separated flow at an angle of attack of $25^{\circ }$ . Results from these two optimization problems also reveal relationships between optimization in physical space and optimization of dynamical behaviours in Hilbert space.

Interior quasi-subgradient method with non-Euclidean distances for constrained quasi-convex optimization problems in hilbert spaces

Interior quasi-subgradient method with non-Euclidean distances for constrained quasi-convex optimization problems in hilbert spaces

Stability analysis of conically perturbed linearly constrained least-squares problems by optimizing the regularized trajectories

This paper studies linearly constrained least-square optimization problems in Hilbert spaces for which the KKT system is not necessarily available to analyze and compute the solution. The primary objective is to develop new qualitative and quantitative stability estimates for the regularization error in the conical regularization approach. To attain this goal, we associate the notion of stability with the solvability of some scalar and vector optimization problems defined in terms of the regularized trajectory on the domain space and the regularized state trajectory on the constraint space. We analyze three optimization formulations. The first formulation minimizes a scalar objective function over the regularized trajectory. The second formulation consists of vector optimizing the regularized trajectory on the domain space for a specific Bishop–Phelps cone. The third formulation results in vector optimizing the regularized state trajectory for the constraint cone. We provide numerical examples to illustrate the efficacy of the developed framework.

Strong convergence of over-relaxed multi-parameter proximal scaled gradient algorithm and superiorization

ABSTRACT In this paper, we propose an over-relaxed proximal scaled gradient algorithm for solving the non-smooth composite optimization problem in Hilbert space. We prove the strong convergence and the bounded perturbation resilience of this algorithm under some mild conditions. We also list the superiorized version of the algorithm. We apply the algorithms to the problem to illustrate the performance and validity. The numerical examples show that the over-relaxed algorithm can achieve fewer iteration steps and less running times in comparison to the under-relaxed algorithm, and that the superiorization algorithm has the minimum iterative steps among the three algorithms.

Regularized Jacobi-type ADMM-methods for a class of separable convex optimization problems in Hilbert spaces

We consider a regularized version of a Jacobi-type alternating direction method of multipliers (ADMM) for the solution of a class of separable convex optimization problems in a Hilbert space. The analysis shows that this method is equivalent to the standard proximal-point method applied in a Hilbert space with a transformed scalar product. The method therefore inherits the known convergence results from the proximal-point method and allows suitable modifications to get a strongly convergent variant. Some additional properties are also shown by exploiting the particular structure of the ADMM-type solution method. Applications and numerical results are provided for the domain decomposition method and potential (generalized) Nash equilibrium problems in a Hilbert space setting.

Gradient methods with regularization for constrained optimization problems and their complexity estimates

We suggest simple implementable modifications of conditional gradient and gradient projection methods for smooth convex optimization problems in Hilbert spaces. Usually, the custom methods attain only weak convergence. We prove strong convergence of the new versions and establish their complexity estimates, which appear similar to the convergence rate of the weakly convergent versions. Preliminary results of computational tests confirm efficiency of the proposed modification.

A generalized proximal-point method for convex optimization problems in Hilbert spaces

We deal with a generalization of the proximal-point method and the closely related Tikhonov regularization method for convex optimization problems. The prime motivation behind this is the well-known connection between the classical proximal-point and augmented Lagrangian methods, and the emergence of modified augmented Lagrangian methods in recent years. Our discussion includes a formal proof of a corresponding connection between the generalized proximal-point method and the modified augmented Lagrange approach in infinite dimensions. Several examples and counterexamples illustrate the convergence properties of the generalized proximal-point method and indicate that the corresponding assumptions are sharp.

On the convergence of the forward–backward splitting method with linesearches

In this paper we focus on the convergence analysis of the forward–backward splitting method for solving nonsmooth optimization problems in Hilbert spaces when the objective function is the sum of two convex functions. Assuming that one of the functions is Fréchet differentiable and using two new linesearches, the weak convergence is established without any Lipschitz continuity assumption on the gradient. Furthermore, we obtain many complexity results of cost values at the iterates when the stepsizes are bounded below by a positive constant.

On Proximal Subgradient Splitting Method for Minimizing the sum of two Nonsmooth Convex Functions

In this paper we present a variant of the proximal forward-backward splitting iteration for solving nonsmooth optimization problems in Hilbert spaces, when the objective function is the sum of two nondifferentiable convex functions. The proposed iteration, which will be called Proximal Subgradient Splitting Method, extends the classical subgradient iteration for important classes of problems, exploiting the additive structure of the objective function. The weak convergence of the generated sequence was established using different stepsizes and under suitable assumptions. Moreover, we analyze the complexity of the iterates.

Stability of Saddle Points Via Explicit Coderivatives of Pointwise Subdifferentials

We derive stability criteria for saddle points of a class of nonsmooth optimization problems in Hilbert spaces arising in PDE-constrained optimization, using metric regularity of infinite-dimensional set-valued mappings. A main ingredient is an explicit pointwise characterization of the regular coderivative of the subdifferential of convex integral functionals. This is applied to several stability properties for parameter identification problems for an elliptic partial differential equation with non-differentiable data fitting terms.

An indirect pseudospectral method for the solution of linear-quadratic optimal control problems with infinite horizon

We consider a class of linear-quadratic infinite horizon optimal control problems in Lagrange form involving the Lebesgue integral in the objective. The key idea is to introduce weighted Sobolev spaces as state spaces and weighted Lebesgue spaces as control spaces into the problem setting. Then, the problem becomes an optimization problem in Hilbert spaces. We use the weight functions in our consideration. This problem setting gives us the possibility to extend the admissible set and simultaneously to be sure that the adjoint variable belongs to a Hilbert space too. For the class of problems proposed, existence results as well as a Pontryagin-type Maximum Principle, as necessary and sufficient optimality condition, can be shown. Based on this principle we develop a Galerkin method, coupled with the Gauss–Laguerre quadrature formulas as discretization scheme, to solve the problem numerically. Results are presented for the introduced model and different weight functions.

A General Approximation Method for a Kind of Convex Optimization Problems in Hilbert Spaces

The constrained convex minimization problem is to find a pointx∗with the property thatx∗∈C, andh(x∗)=min h(x),∀x∈C, whereCis a nonempty, closed, and convex subset of a real Hilbert spaceH,h(x)is a real-valued convex function, andh(x)is not Fréchet differentiable, but lower semicontinuous. In this paper, we discuss an iterative algorithm which is different from traditional gradient-projection algorithms. We firstly construct a bifunctionF1(x,y)defined asF1(x,y)=h(y)−h(x). And we ensure the equilibrium problem forF1(x,y)equivalent to the above optimization problem. Then we use iterative methods for equilibrium problems to study the above optimization problem. Based on Jung’s method (2011), we propose a general approximation method and prove the strong convergence of our algorithm to a solution of the above optimization problem. In addition, we apply the proposed iterative algorithm for finding a solution of the split feasibility problem and establish the strong convergence theorem. The results of this paper extend and improve some existing results.

Lagrange Multiplier Rules for Weak Approximate Pareto Solutions of Constrained Vector Optimization Problems in Hilbert Spaces

In the Hilbert space case, in terms of proximal normal cone and proximal coderivative, we establish a Lagrange multiplier rule for weak approximate Pareto solutions of constrained vector optimization problems. In this case, our Lagrange multiplier rule improves the main result on vector optimization in Zheng and Ng (SIAM J. Optim. 21: 886---911, 2011). We also introduce a notion of a fuzzy proximal Lagrange point and prove that each Pareto (or weak Pareto) solution is a fuzzy proximal Lagrange point.

Evolution Differential Inclusion with Projection for Solving Constrained Nonsmooth Convex Optimization in Hilbert Space

This paper introduces a projection subgradient system modeled by an evolution differential inclusion to solve a class of hierarchical optimization problems in Hilbert space. Basing on the Moreau–Yosida approximation, we prove the global existence and uniqueness of the solution of the proposed evolution differential inclusion with projection and the unique solution of the proposed system is just its “slow solution” when the constrained set is defined by the affine equalities. When the outer layer objective function ψ is strongly convex, any solution of the proposed system is strongly convergent to the unique minimizer of the constrained optimization problem, while, the strongly convergence is also given when the inner layer objective function ϕ is strongly convex. Furthermore, we present some other optimization problem models, which can be solved by the proposed system. All the results obtained are new not only in the infinite dimensional Hilbert space framework but also in the finite dimensional space.

Inertial proximal point regularization algorithm for unconstrained vector convex optimization problems

The purpose of this paper is to investigate an iterative regularization method of proximal point type for solving ill posed vector convex optimization problems in Hilbert spaces. Applications to the convex feasibility problems and the problem of common fixed points for nonexpansive potential mappings are also given.

Existence of exact penalty for constrained optimization problems in Hilbert spaces

In this paper we use the penalty approach in order to study two constrained minimization problems in Hilbert spaces. A penalty function is said to have the exact penalty property if there is a penalty coefficient for which a solution of an unconstrained penalized problem is a solution of the corresponding constrained problem. In this paper we establish a sufficient condition for the exact penalty property.

Regular Lagrange Multipliers for Control Problems with Mixed Pointwise Control-State Constraints

A class of quadratic optimization problems in Hilbert spaces is considered, where pointwise box-constraints and constraints of bottleneck type are given. The main focus is on proving the existence of regular Lagrange multipliers in $L^2$-spaces. This question is solved by investigating the solvability of a Lagrange dual quadratic problem. The theory is applied to different optimal control problems for elliptic and parabolic partial differential equations with mixed pointwise control-state constraints.