Constrained Convex Minimization Problem Research Articles

A Regularized High-Dimensional Positive Definite Covariance Estimator with High-Frequency Data

This paper proposes a novel large-dimensional positive definite covariance estimator for high-frequency data under a general factor model framework. We demonstrate an appealing connection between the proposed estimator and a weighted group least absolute shrinkage and selection operator (LASSO) penalized least-squares estimator. The proposed estimator improves on traditional principal component analysis by allowing for weak factors, whose signal strengths are weak relative to idiosyncratic components. Despite the presence of microstructure noises and asynchronous trading, the proposed estimator achieves guarded positive definiteness without sacrificing the convergence rate. To make our method fully operational, we provide an extended simultaneous alternating direction method of multipliers algorithm to solve the resultant constrained convex minimization problem efficiently. Empirically, we study the monthly high-frequency covariance structure of the stock constituents of the S&P 500 index from 2008 to 2016, using all traded stocks from the NYSE, AMEX, and NASDAQ stock markets to construct the high-frequency Fama-French four and extended eleven economic factors. We further examine the out-of-sample performance of the proposed method through vast portfolio allocations, which deliver significantly reduced out-of-sample portfolio risk and enhanced Sharpe ratios. The success of our method supports the usefulness of machine learning techniques in finance. This paper was accepted by Agostino Capponi, finance. Funding: This work was supported by the Research Grants Council, University Grants Committee [Grants 11500119, 11505522, 11505721, and 21504818] and the National Natural Science Foundation of China (NSFC) Basic Scientific Center Project [Grant 71988101], entitled as “Econometric Modelling and Economic Policy Studies”, as well as NSFC [Grants 71803166 and 72173104]. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2022.04138 .

An augmented Lagrangian approach for cardinality constrained minimization applied to variable selection problems

To solve convex constrained minimization problems, that also include a cardinality constraint, we propose an augmented Lagrangian scheme combined with alternating projection ideas. Optimization problems that involve a cardinality constraint are NP-hard mathematical programs and typically very hard to solve approximately. Our approach takes advantage of a recently developed and analyzed continuous formulation that relaxes the cardinality constraint. Based on that formulation, we solve a sequence of smooth convex constrained minimization problems, for which we use projected gradient-type methods. In our setting, the convex constraint region can be written as the intersection of a finite collection of convex sets that are easy and inexpensive to project. We apply our approach to a variety of over and under determined constrained linear least-squares problems, with both synthetic and real data that arise in variable selection, and demonstrate its effectiveness.

A hybrid direct search and projected simplex gradient method for convex constrained minimization

We propose a new Derivative-free Optimization (DFO) approach for solving convex constrained minimization problems. The feasible set is assumed to be the non-empty intersection of a finite collection of closed convex sets, such that the projection onto each of these individual convex sets is simple and inexpensive to compute. Our iterative approach alternates between steps that use Directional Direct Search (DDS), considering adequate poll directions, and a Spectral Projected Gradient (SPG) method, replacing the real gradient by a simplex gradient, under a DFO approach. In the SPG steps, if the convex feasible set is simple, then a direct projection is computed. If the feasible set is the intersection of finitely many convex simple sets, then Dykstra's alternating projection method is applied. Convergence properties are established under standard assumptions usually associated to DFO methods. Some preliminary numerical experiments are reported to illustrate the performance of the proposed algorithm, in particular by comparing it with a classical DDS method. Our results indicate that the hybrid algorithm is a robust and effective approach for derivative-free convex constrained optimization.

New Convergence Theorems for Pseudomonotone Variational Inequality on Hadamard Manifolds

In this paper, we propose an efficient viscosity type subgradient extragradient algorithm for solving pseudomonotone variational inequality on Hadamard manifolds which is of symmetrical characteristic. Under suitable conditions, we obtain the convergence of the iteration sequence generated by the proposed algorithm to a solution of a pseudomonotone variational inequality on Hadamard manifolds. We also employ our main result to solve a constrained convex minimization problem and present a numerical experiment to illustrate the asymptotic behavior of the algorithm. Our results develop and improve some recent results.

New subgradient extragradient algorithm for solving variational inequalities in Hadamard manifold

This paper introduces a schematic approximation method for a solution to a variational inequality problem in the framework of Hadamard manifold. The method is a combination of the subgradient extragradient technique and Popov extragradient method. Using this method, some convergence algorithms were proved when the cost operators are pseudomonotone and strongly pseudomonotone, respectively. In the construction of this method, the dependence on Lipschitz constants of the operators is dispensed with by the use of a monotone decreasing step size. We give an application of our main result to the constrained convex minimization problem. Finally, we report some numerical examples to illustrate the efficiency and applicability of the method.

On the Arrow–Hurwicz differential system for linearly constrained convex minimization

ABSTRACT In a real Hilbert space setting, we reconsider the classical Arrow–Hurwicz differential system in view of solving linearly constrained convex minimization problems. We investigate the asymptotic properties of the differential system and provide conditions for which its solutions converge towards a saddle point of the Lagrangian associated with the convex minimization problem. Our convergence analysis mainly relies on a ‘Lagrangian identity’ which naturally extends on the well-known descent property of the classical continuous steepest descent method. In addition, we present asymptotic estimates on the decay of the solutions and the primal-dual gap function measured in terms of the Lagrangian. These estimates are further refined to the ones of the classical damped harmonic oscillator provided that second-order information on the objective function of the convex minimization problem is available. Finally, we show that our results directly translate to the case of solving structured convex minimization problems. Numerical experiments further illustrate our theoretical findings.

Novel Algorithms with Inertial Techniques for Solving Constrained Convex Minimization Problems and Applications to Image Inpainting

In this paper, we propose two novel inertial forward–backward splitting methods for solving the constrained convex minimization of the sum of two convex functions, φ1+φ2, in Hilbert spaces and analyze their convergence behavior under some conditions. For the first method (iFBS), we use the forward–backward operator. The step size of this method depends on a constant of the Lipschitz continuity of ∇φ1, hence a weak convergence result of the proposed method is established under some conditions based on a fixed point method. With the second method (iFBS-L), we modify the step size of the first method, which is independent of the Lipschitz constant of ∇φ1 by using a line search technique introduced by Cruz and Nghia. As an application of these methods, we compare the efficiency of the proposed methods with the inertial three-operator splitting (iTOS) method by using them to solve the constrained image inpainting problem with nuclear norm regularization. Moreover, we apply our methods to solve image restoration problems by using the least absolute shrinkage and selection operator (LASSO) model, and the results are compared with those of the forward–backward splitting method with line search (FBS-L) and the fast iterative shrinkage-thresholding method (FISTA).

An intermixed method for solving the combination of mixed variational inequality problems and fixed-point problems

In this paper, we introduce an intermixed algorithm with viscosity technique for finding a common solution of the combination of mixed variational inequality problems and the fixed-point problem of a nonexpansive mapping in a real Hilbert space. Moreover, we propose the mathematical tools related to the combination of mixed variational inequality problems in the second section of this paper. Utilizing our mathematical tools, a strong convergence theorem is established for the proposed algorithm. Furthermore, we establish additional conclusions concerning the split-feasibility problem and the constrained convex-minimization problem utilizing our main result. Finally, we provide numerical experiments to illustrate the convergence behavior of our proposed algorithm.

On New Generalized Viscosity Implicit Double Midpoint Rule for Hierarchical Problem

The implicit midpoint rules are employed as a powerful numerical technique, and in this article we attend a class of viscosity iteration approximations on hierarchical problems for the implicit double midpoint rules. We prove the strong convergence theorem to the unique solution on hierarchical problem of this technique is established under some favorable conditions imposed on the control parameters in Hilbert spaces. Furthermore, we propose some applications to the constrained convex minimization problem, nonlinear Fredholm integral equation and variational inequality on fixed point problem. Moreover, some numerical examples are also presented to illustrate the different proposed methods and convergence results. Our results modified the implicit double midpoint rules with the hierarchical problem.

The Ball-Relaxed Gradient-Projection Algorithm for Split Feasibility Problem

In this paper, we concern with the split feasibility problem (SFP) whenever the convex sets involved are composed of level sets. By applying Gradient-projection algorithm which is used to solve constrained convex minimization problem of a real valued convex function, we construct two new algorithms for the split feasibility problem and prove that both of them are convergent weakly to a solution of the feasibility problem. In the end, as an application, we obtain a new algorithm for solving the split equality problem.

New Iterative Algorithm for Solving Constrained Convex Minimization Problem and Split Feasibility Problem

The purpose of this paper is to introduce a new iterative algorithm to approximate the fixed points of almost contraction mappings and generalized α-nonexpansive mappings. Also, we show that our proposed iterative algorithm converges weakly and strongly to the fixed points of almost contraction mappings and generalized α-nonexpansive mappings. Furthermore, it is proved analytically that our new iterative algorithm converges faster than one of the leading iterative algorithms in the literature for almost contraction mappings. Some numerical examples are also provided and used to show that our new iterative algorithm has better rate of convergence than all of S, Picard-S, Thakur and M iterative algorithms for almost contraction mappings and generalized α-nonexpansive mappings. Again, we show that the proposed iterative algorithm is stable with respect to T and data dependent for almost contraction mappings. Some applications of our main results and new iterative algorithm are considered. The results in this article are improvements, generalizations and extensions of several relevant results existing in the literature.

A Common Solution of Equilibrium, Constrained Convex Minimization, and Fixed Point Problems

In this paper, we propose a new iterative scheme with the help of the gradient- projection algorithm (GPA) for finding a common solution of an equilibrium problem, a constrained convex minimization problem, and a fixed point problem. Then, we prove some strong convergence theorems which improve and extend some recent results. Moreover, we give a numerical result to show the validity of our main theorem.

A new iterative method for generalized equilibrium and constrained convex minimization problems

The gradient-projection algorithm (GPA) plays an important role in solving constrained convex minimization problems. In this paper, we combine the GPA and averaged mapping approach to propose an explicit composite iterative scheme for finding a common solution of a generalized equilibrium problem and a constrained convex minimization problem. Then, we prove a strong convergence theorem which improves and extends some recent results.

Convergence study on strictly contractive Peaceman–Rachford splitting method for nonseparable convex minimization models with quadratic coupling terms

The alternating direction method of multipliers (ADMM) and Peaceman Rachford splitting method (PRSM) are two popular splitting algorithms for solving large-scale separable convex optimization problems. Though problems with nonseparable structure appear frequently in practice, researches on splitting methods for these problems remain to be scarce. Very recently, Chen et al. (Math Program 173(1–2):37–77, 2019) extended the 2-block ADMM to linearly constrained nonseparable models with quadratic coupling terms and established its convergence. However, theoretical researches about nonseparable PRSM or its variants are still lacking. To fill the gap, in this paper we focus on the strictly contractive PRSM (SC-PRSM) applied to 2-block linearly constrained convex minimization problems with quadratic coupling objective functions. Under mild conditions, we prove the convergence of our proposed SC-PRSM and establish its o(1/k) convergence rate. Moreover, we implement the SC-PRSM to solve a problem of calculating the Euclidian distance between two ellipsoids, and compare its performance with three ADMM type algorithms. The results show the nonseparable SC-PRSM outperforms the other three algorithms in terms of both the iteration numbers and CPU time.

Parallel Nesterov Domain Decomposition Method for Elliptic Partial Differential Equations

We study a parallel non-overlapping domain decomposition method, based on the Nesterov accelerated gradient descent, for the numerical approximation of elliptic partial differential equations. The problem is reformulated as a constrained (convex) minimization problem with the interface continuity conditions as constraints. The resulting domain decomposition method is an accelerated projected gradient descent with convergence rate [Formula: see text]. At each iteration, the proposed method needs only one matrix/vector multiplication. Numerical experiments show that significant (standard and scaled) speed-ups can be obtained.

Inertial Haugazeau's hybrid subgradient extragradient algorithm for variational inequality problems in Banach spaces

The variational inequality problem plays an important role in nonlinear analysis and optimization. It is a generalization of the nonlinear complementarity problem. For a variational inequality problem in a Hilbert space, the extragradient algorithm with inertial effects has been studied. For a variational inequality problem in a Banach space, Nakajo introduced Haugazeau's hybrid method and Liu introduced the Halpern subgradient extragradient method. In this paper, we construct a new inertial iterative method for solving variational inequality problems in Banach spaces based on the work we mentioned above. We propose a strong convergence theorem. As applications, our result can be used to solve constrained convex minimization problems.

New iterative schemes for solving variational inequality and fixed points problems involving demicontractive and quasi-nonexpansive mappings in Banach spaces

Abstract In this paper, we suggest and analyze a new iterative method for finding a common element of the set of fixed points of a quasi-nonexpansive mapping and the set of fixed points of a demicontractive mapping which is the unique solution of some variational inequality problems involving accretive operators in a Banach space. We prove the strong convergence of the proposed iterative scheme without imposing any compactness condition on the mapping or the space. Finally, applications of our theorems to some constrained convex minimization problems are given.

A general framework for ADMM acceleration

The Alternating Direction Multipliers Method (ADMM) is a very popular algorithm for computing the solution of convex constrained minimization problems. Such problems are important from the application point of view, since they occur in many fields of science and engineering. ADMM is a powerful numerical tool, but unfortunately its main drawback is that it can exhibit slow convergence. Several approaches for its acceleration have been proposed in the literature and in this paper we present a new general framework devoted to this aim. In particular, we describe an algorithmic framework that makes possible the application of any acceleration step while still having the guarantee of convergence. This result is achieved thanks to a guard condition that ensures the monotonic decrease of the combined residual. The proposed strategy is applied to image deblurring problems. Several acceleration techniques are compared; to the best of our knowledge, some of them are investigated for the first time in connection with ADMM. Numerical results show that the proposed framework leads to a faster convergence with respect to other acceleration strategies recently introduced for ADMM.

On perturbed hybrid steepest descent method with minimization or superiorization for subdifferentiable functions

For finding the minimum value of differentiable functions over a nonempty closed convex subset of a Hilbert space, the hybrid steepest descent method (HSDM) can be applied. In this work, we study perturbed algorithms in line with a generalized HSDM and discuss how some selections of perturbations enable us to increase the convergence speed. When we specialize these results to constrained minimization then the perturbations become bounded perturbations used in the superiorization methodology (SM). We show usefulness of the SM in studying the constrained convex minimization problem for subdifferentiable functions and proceed with the study of the computational efficiency of the SM compared with the HSDM. In the computational experiment comparing the HSDM with superiorization, the latter seems to be advantageous for the specific experiment.

Strong convergence result for solution of stochastic convex minimization problem generated by generalized random iterative scheme

Stochastic approximation procedures have been extensively used in solving stochastic optimization problems. Gradient projection algorithm (GPA) plays an important role in solving constrained convex minimization problems. In this paper, strong convergence result for approximate solution of a constrained stochastic convex minimization problem in Hilbert space is proved. A generalized random iterative scheme based on skewed gradient projection algorithm and a strictly pseudo-contractive mapping is used to generate the solution.