Convex Minimization Research Articles

An algorithm for approximating a common solution of some nonlinear problems in Banach spaces with an application

In this paper, we construct a new Halpern-type subgradient extragradient iterative algorithm. The sequence generated by this algorithm converges strongly to a common solution of a variational inequality, an equilibrium problem, and a J-fixed point of a continuous J-pseudo-contractive map in a uniformly smooth and two-uniformly convex real Banach space. Also, the theorem is applied to approximate a common solution of a variational inequality, an equilibrium problem, and a convex minimization problem. Moreover, a numerical example is given to illustrate the implementability of our algorithm. Finally, the theorem proved complements, improves, and unifies some related recent results in the literature.

Naive Bayesian fusion based deep learning networks for multisegmented classification of fishes in aquaculture industries

Fish classification is an essential requirement for biomass estimation, disease identification, and quality analysis. In aquaculture industries, fish classification is carried out in the processing unit. The fishes are out of water, subjecting them to structural deformation and orientation misalignments, makes classification challenging. A multisegmented fish classification technique using deep learning networks with naive Bayesian type fusion is proposed in this work to address these challenges. Fish images are acquired using an overhead camera. The fish head is identified by observing a minimal convexity deficiency region to facilitate segmentation. A multi-stage exhaustive enumerative optimization method is used to adjust the orientation, which can minimize unwanted background region in the image segment. Fish head, scales, and body are segmented from the fish image. For each fish segment, AlexNet is trained by using the transfer learning approach. A naive Bayesian fusion layer is introduced to fuse these trained deep learning networks and enhance classification accuracy. Experimental results illustrate a classification accuracy of 98.64% for ‘Fish-Pak’ image dataset with six different fish species and 98.94% for BYU fish dataset with four species. Comparative analysis with standard networks and ablation study demonstrates the accuracy and robustness of the proposed fusion architecture, respectively. Various fusion layers have also been analyzed, and observations illustrate the accuracy of the proposed NBC layer. Significant improvements in other classification performance metrics were also observed.

A Common Solution of Equilibrium, Constrained Convex Minimization, and Fixed Point Problems

In this paper, we propose a new iterative scheme with the help of the gradient- projection algorithm (GPA) for finding a common solution of an equilibrium problem, a constrained convex minimization problem, and a fixed point problem. Then, we prove some strong convergence theorems which improve and extend some recent results. Moreover, we give a numerical result to show the validity of our main theorem.

Terminal‐cost design for model predictive control with linear stage‐costs: A set‐theoretic method

AbstractThis article presents a method for designing stabilizing terminal‐costs for linear model predictive controllers whose stage‐costs are a mixture of weighted 1‐norms and ∞‐norms of the system states and inputs. This problem is abstracted by replacing the 1/∞‐norm stage‐costs with convex Minkowski functions. We show that a convex Minkowski terminal‐cost is a control Lyapunov function if and only if the dual of its defining set satisfies a set‐inclusion condition. Furthermore, we show that this set‐inclusion is satisfied by all robust positive invariant sets of a certain dual system. In particular, we show that the minimal convex set that satisfies the set‐inclusion produces the optimal terminal‐cost which is the cost‐to‐go of the corresponding unconstrained infinite‐horizon optimal control problem with convex Minkowski stage‐costs. To reduce the number of constraints in the MPC, we show that the defining set of the Minkowski function can be used as an invariant terminal‐constraint. Finally, we demonstrate our terminal‐cost design procedure for three case studies; a double‐integrator, a galvanometer laser scanner, and a nanosatellite.

Gorilla Abundance Estimations within North-East Moukalaba-Doudou National Park, Gabon

Accurate measures of animal population densities are essential to evaluate conservation status and implement action plans to ensure species survival. Gorilla numbers were assessed using the recce survey method within Moukalaba-Doudou National Park (MDNP) in Gabon using fresh nest counts of up to 1 week old. We walked 3,592 km within a 23.01-km² study site totalling a sampling effort of 297 days. Encounter rate was 0.12 fresh nests per kilometre, and gorilla density estimates generated by home range sizes (by minimal convex polygon) ranged between 1.14 and 1.48 gorillas/km². Gorillas preferred mixed forest for nesting over other habitats (Cheason index value 1.31). Results showed that gorilla density values within the study area concurred with previous studies that used line transect methodologies. We conclude that the choice of sampling design is dependent on environmental conditions characterised by each habitat type and target species.

Gradient methods with memory

ABSTRACT In this paper, we consider gradient methods for minimizing smooth convex functions, which employ the information obtained at the previous iterations in order to accelerate the convergence towards the optimal solution. This information is used in the form of a piece-wise linear model of the objective function, which provides us with much better prediction abilities as compared with the standard linear model. To the best of our knowledge, this approach was never really applied in Convex Minimization to differentiable functions in view of the high complexity of the corresponding auxiliary problems. However, we show that all necessary computations can be done very efficiently. Consequently, we get new optimization methods, which are better than the usual Gradient Methods both in the number of oracle calls and in the computational time. Our theoretical conclusions are confirmed by preliminary computational experiments.

Regularized based implicit Lagrangian twin extreme learning machine in primal for pattern classification

In this paper, we suggest a novel approach termed as regularized based implicit Lagrangian twin extreme learning machine in primal as a pair of unconstrained convex minimization problem (RILTELM) where regularization term is added to follow the structural risk minimization principle. Here, we consider 2-norm of the slack vector of variables to make the problem strongly convex which results in a unique solution. Since it has non-smooth plus functions in their objective function, so we find an approximate solution by replacing the non-smooth plus function with smooth approximation function because to find an approximation solution in primal space is always superior to its dual. Due to non-smooth plus function, we solve the problem by either smooth approximation approach or generalized derivative approach. In addition, a functional iterative scheme is also suggested to find the optimal solution. Hence, no external optimization toolbox is required unlike in twin extreme learning machine (TELM) and twin support vector machine (TWSVM). The numerical experiments are demonstrated on artificial and real-world datasets and compared with TWSVM, ELM, TELM and LSTELM to establish the efficacy and applicability of proposed RILTELM.

A New Gradient Projection Algorithm for Convex Minimization Problem and its Application to Split Feasibility Problem

In this paper, we study convergence analysis of a new gradient projection algorithm for solving convex minimization problems in Hilbert spaces. We observe that the proposed gradient projection algorithm weakly converges to a minimum of convex function f which is defined from a closed and convex subset of a Hilbert space to $\mathbb {R}$ . Also, we give a nontrivial example to illustrate our result in an infinite dimensional Hilbert space. We apply our result to solve the split feasibility problem.

An iterative algorithm for solving variational inequality, generalized mixed equilibrium, convex minimization and zeros problems for a class of nonexpansive-type mappings

In this paper, we study a classical monotone and Lipschitz continuous variational inequality and fixed point problems defined on a level set of a convex function in the framework of Hilbert spaces. First, we introduce a new iterative scheme which combines the inertial subgradient extragradient method with viscosity technique and with self-adaptive stepsize. Unlike in many existing subgradient extragradient techniques in literature, the two projections of our proposed algorithm are made onto some half-spaces. Furthermore, we prove a strong convergence theorem for approximating a common solution of the variational inequality and fixed point of an infinite family of nonexpansive mappings under some mild conditions. The main advantages of our method are: the self-adaptive stepsize which avoids the need to know a priori the Lipschitz constant of the associated monotone operator, the two projections made onto some half-spaces, the strong convergence and the inertial technique employed which accelerates convergence rate of the algorithm. Second, we apply our theorem to solve generalised mixed equilibrium problem, zero point problems and convex minimization problem. Finally, we present some numerical examples to demonstrate the efficiency of our algorithm in comparison with other existing methods in literature. Our results improve and extend several existing works in the current literature in this direction.

A Convex Collaborative Filtering Framework for Global Market Return Prediction

A new convex collaborative filtering framework for global market return prediction is presented. The prediction problem is first phrased as a matrix completion problem assuming that the matrix of market returns is low rank. Using historical data, we examine the experimental conditions of the matrix of market returns that lends to a low rank matrix and low value for subspace incoherence in view of convex matrix completion theory. The method then uses convex nuclear norm minimization to predict returns for the missing future markets based on this analysis. We evaluate prediction performance of the technique for global market return prediction through several simulation experiments and compare performance against user based collaborative filtering and singular value decomposition impute. We also conduct several trading experiments to assess the practical value of the predictions using basic momentum, reverse momentum, naive random and minimum variance trading strategies. Our experiments demonstrate that collaborative filtering via nuclear norm minimization consistently achieves better prediction performance compared with the two other collaborative filtering techniques tested. Furthermore, the trading experiments that incorporate these predictions demonstrate improvement in average return by 0.41%.

Globally Convergent Gradient Projection Type Algorithms for a Class of Robust Hypothesis Testings

This paper considers the popular minimax robust hypothesis testing problem—seeking the optimal decision rule with a minimum error probability for the least favorable distributions (LFDs) lying within an uncertainty set, which is characterized by an upper bound on the distance between actual and nominal densities. First, we convert the minimax robust hypothesis testing problem to a convex minimization problem. By leveraging Danskin's theorem, the gradient of the objective function of the transformed problem is derived as a function of LFDs. Then, we propose the gradient projection algorithm (GPA) and the hybrid gradient projection algorithm (HGPA) to solve the transformed problem. In particular, when the distance is chosen to be the Kullback-Leibler (KL) or $\alpha$ -divergence, each LFD only relies on two unknown parameters which can be determined efficiently. In these two cases, the decision rule sequences generated by the GPA and the HGPA are respectively proved to converge weakly and strongly to the global minimizer under some mild conditions. To the best of our knowledge, these decision rules are the first to be guaranteed to globally converge towards the optimal solution for this type of robust hypothesis testing problems. We further propose an accelerated gradient projection algorithm (AGPA) to improve the efficiency of the GPA when the observation space of the robust hypothesis testing only contains finitely many points. Several simulations illustrate that the proposed GPA, HGPA and AGPA can obtain the globally optimal solution with high efficiency.

Second-Order Dynamics with Hessian-Driven Damping for Linearly Constrained Convex Minimization

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 5 March 2020Accepted: 14 June 2021Published online: 18 October 2021Keywordsdissipative inertial dynamics, Hessian-driven damping, Lyapunov analysis, asymptotic properties, Bregman distance, convex minimizationAMS Subject Headings37N40, 46N10, 49M30, 65K05, 90C25Publication DataISSN (print): 0363-0129ISSN (online): 1095-7138Publisher: Society for Industrial and Applied MathematicsCODEN: sjcodc

Convex duality for principal frequencies

<abstract><p>We consider the sharp Sobolev-Poincaré constant for the embedding of $ W^{1, 2}_0(\Omega) $ into $ L^q(\Omega) $. We show that such a constant exhibits an unexpected dual variational formulation, in the range $ 1 < q < 2 $. Namely, this can be written as a convex minimization problem, under a divergence–type constraint. This is particularly useful in order to prove lower bounds. The result generalizes what happens for the torsional rigidity (corresponding to $ q = 1 $) and extends up to the case of the first eigenvalue of the Dirichlet-Laplacian (i.e., to $ q = 2 $).</p></abstract>

On the Convergence of Projected-Gradient Methods with Low-Rank Projections for Smooth Convex Minimization over Trace-Norm Balls and Related Problems

Smooth convex minimization over the unit trace-norm ball is an important optimization problem in machine learning, signal processing, statistics, and other fields that underlies many tasks in which one wishes to recover a low-rank matrix given certain measurements. While first-order methods for convex optimization enjoy optimal convergence rates, they require in the worst-case to compute a full-rank SVD on each iteration, in order to compute the Euclidean projection onto the trace-norm ball. These full-rank SVD computations, however, prohibit the application of such methods to large-scale problems. A simple and natural heuristic to reduce the computational cost of such methods is to approximate the Euclidean projection using only a low-rank SVD. This raises the question if, and under what conditions, this simple heuristic can indeed result in provable convergence to the optimal solution. In this paper we show that any optimal solution is a center of a Euclidean ball inside which the projected-gradient mapping admits a rank that is at most the multiplicity of the largest singular value of the gradient vector at this optimal point. Moreover, the radius of the ball scales with the spectral gap of this gradient vector. We show how this readily implies the local convergence (i.e., from a “warm-start" initialization) of standard first-order methods such as the projected-gradient method and accelerated gradient methods, using only low-rank SVD computations. We also quantify the effect of “over-parameterization," i.e., using SVD computations with higher rank, on the radius of this ball, showing it can increase dramatically with moderately larger rank. We extend our results also to the setting of smooth convex minimization with trace-norm regularization and smooth convex optimization over bounded-trace positive semidefinite matrices. Our theoretical investigation is supported by concrete empirical evidence that demonstrates the correct convergence of first-order methods with low-rank projections for the matrix completion task on real-world datasets.

Unstabilized Hybrid High-order Method for a Class of Degenerate Convex Minimization Problems

The relaxation in the calculus of variation motivates the numerical analysis of a class of degenerate convex minimization problems with non-strictly convex energy densities with some convexity control and two-sided $p$-growth. The minimizers may be non-unique in the primal variable but lead to a unique stress $\sigma \in H(\operatorname{div},\Omega;\mathbb{M})$. Examples include the p-Laplacian, an optimal design problem in topology optimization, and the convexified double-well problem. The approximation by hybrid high-order methods (HHO) utilizes a reconstruction of the gradients with piecewise Raviart-Thomas or BDM finite elements without stabilization on a regular triangulation into simplices. The application of this HHO method to the class of degenerate convex minimization problems allows for a unique $H(\operatorname{div})$ conforming stress approximation $\sigma_h$. The main results are a~priori and a posteriori error estimates for the stress error $\sigma-\sigma_h$ in Lebesgue norms and a computable lower energy bound. Numerical benchmarks display higher convergence rates for higher polynomial degrees and include adaptive mesh-refining with the first superlinear convergence rates of guaranteed lower energy bounds.

An accelerated forward-backward algorithm with a new linesearch for convex minimization problems and its applications

We study and investigate a convex minimization problem of the sum of two convex functions in the setting of a Hilbert space. By assuming the Lipschitz continuity of the gradient of the function, many optimization methods have been invented, where the stepsizes of those algorithms depend on the Lipschitz constant. However, finding such a Lipschitz constant is not an easy task in general practice. In this work, by using a new modification of the linesearches of Cruz and Nghia [ 7 ] and Kankam et al. [ 14 ] and an inertial technique, we introduce an accelerated algorithm without any Lipschitz continuity assumption on the gradient. Subsequently, a weak convergence result of the proposed method is established. As applications, we apply and analyze our method for solving an image restoration problem and a regression problem. Numerical experiments show that our method has a higher efficiency than the well-known methods in the literature.

Modified Tseng's splitting algorithms for the sum of two monotone operators in Banach spaces

<abstract> In this work, we introduce two modified Tseng's splitting algorithms with a new non-monotone adaptive step size for solving monotone inclusion problem in the framework of Banach spaces. Under some mild assumptions, we establish the weak and strong convergence results of the proposed algorithms. Moreover, we also apply our results to variational inequality problem, convex minimization problem and signal recovery, and provide several numerical experiments including comparisons with other related algorithms. </abstract>

Проксимальні алгоритми для дворівневих задач опуклої оптимізації

In this paper, problems of bi-level convex minimization in a Hilbert space are considered. The bi-level convex minimization problem is to minimize the first convex function on the set of minima of the second convex function. This setting has many applications, but the implicit constraints generated by the internal problem make it difficult to obtain optimality conditions and construct algorithms. Multilevel optimization problems are formulated in a similar way, the source of which is the operation research problems (optimization according to sequentially specified criteria or lexicographic optimization). Attention is focused on problem solving using two proximal methods. The main theoretical results are theorems on the convergence of methods in various situations. The first of the methods is obtained by combining the penalty function method and the proximal method. Strong convergence is proved in the case of strong convexity of the function of the exterior problem. In the general case, only weak convergence has been proved. The second, the so-called proximal-gradient method, is a combination of one of the variants of the fast proximal-gradient algorithm with the method of penalty functions. The rates of convergence of the proximal-gradient method and its weak convergence are proved.

A new iterative method for generalized equilibrium and constrained convex minimization problems

The gradient-projection algorithm (GPA) plays an important role in solving constrained convex minimization problems. In this paper, we combine the GPA and averaged mapping approach to propose an explicit composite iterative scheme for finding a common solution of a generalized equilibrium problem and a constrained convex minimization problem. Then, we prove a strong convergence theorem which improves and extends some recent results.

Efficient numerical methods to solve sparse linear equations with application to PageRank

Over the last two decades, the PageRank problem has received increased interest from the academic community as an efficient tool to estimate web-page importance in information retrieval. Despite numerous developments, the design of efficient optimization algorithms for the PageRank problem is still a challenge. This paper proposes three new algorithms with a linear time complexity for solving the problem over a bounded-degree graph. The idea behind them is to set up the PageRank as a convex minimization problem over a unit simplex, and then solve it using iterative methods with small iteration complexity. Our theoretical results are supported by an extensive empirical justification using real-world and simulated data.