General Constrained Optimization Problems Research Articles

Asymptotic analysis of scalarization functions and applications

<p style='text-indent:20px;'>In this paper, we consider two common scalarization functions and their applications via asymptotic analysis. We mainly analyze the recession and asymptotic properties of translation invariant function and oriented distance function, and discuss their monotonicity and Lipschitz continuity in terms of recession functions. Finally, we apply these scalarization functions to the characterization of the nonemptiness and boundedness of the solution set for a general constrained nonconvex optimization problem.</p>

Dynamic Analysis and Path Planning of a Turtle-Inspired Amphibious Spherical Robot.

A dynamic path-planning algorithm based on a general constrained optimization problem (GCOP) model and a sequential quadratic programming (SQP) method with sensor input is proposed in this paper. In an unknown underwater space, the turtle-inspired amphibious spherical robot (ASR) can realise the path-planning control movement and achieve collision avoidance. Due to the special underwater environments, thrusters and diamond parallel legs (DPLs) are installed in the lower hemisphere to realise accurate motion control. A propulsion model for a novel water-jet thruster based on experimental analysis and a modified Denavit-Hartenberg (MDH) algorithm are developed for multiple degrees of freedom (MDOF) to realize high-precision and high-speed motion control. Simulations and experiments verify that the effectiveness of the GCOP and SQP algorithms can realize reasonable path planning and make it possible to improve the flexibility of underwater movement with a small estimation error.

Asymptotic properties of dual averaging algorithm for constrained distributed stochastic optimization

Considering the constrained stochastic optimization problem over a time-varying random network, where the agents are to collectively minimize a sum of objective functions subject to a common constraint set, we investigate asymptotic properties of a distributed algorithm based on dual averaging of gradients. Different from most existing works on distributed dual averaging algorithms that are mainly focused on their non-asymptotic properties, we prove not only almost sure convergence and rate of almost sure convergence, but also asymptotic normality and asymptotic efficiency of the algorithm. Firstly, for general constrained convex optimization problem distributed over a random network, we prove that almost sure consensus can be achieved and the estimates of agents converge to the same optimal point. For the case of linear constrained convex optimization, we show that the mirror map of the averaged dual sequence identifies the active constraints of the optimal solution with probability 1, which helps us to prove the almost sure convergence rate and then establish asymptotic normality of the algorithm. Furthermore, we also verify that the algorithm is asymptotically optimal. To the best of our knowledge, it is the first asymptotic normality result for constrained distributed optimization algorithms. Finally, a numerical example is provided to justify the theoretical analysis.

A Framework for Constrained Static State Estimation in Unbalanced Distribution Networks

State estimation plays a key role in the transition from the passive to the active operation of distribution systems, as it allows to monitor the networks and provides the necessary information to perform control actions. However, designing state estimators for distribution systems is challenging, due to the characteristics of the networks, such as limited measurement availability. Furthermore, the features of the distribution system present significant local variations, e.g., voltage level and number and type of customers, which makes it hard to design a “one-size-fits-all” state estimator. This paper introduces a unifying framework that allows to easily implement and compare diverse unbalanced static state estimation models. This is achieved by formulating state estimation as a general constrained optimization problem. The advantages of this approach are described and supported by numerical illustration on a large set of real distribution feeders. The framework is also implemented in software and made available open-source.

Sharp Lagrange multipliers for set-valued optimization problems

In this paper, we give a comparison among some notions of weak sharp minima introduced in Amahroq et al. [Le matematiche J. 73 (2018) 99–114], Durea and Strugariu [Nonlinear Anal. 73 (2010) 2148–2157] and Zhu et al. [Set-Valued Var. Anal. 20 (2012) 637–666] for set-valued optimization problems. Besides, we establish sharp Lagrange multiplier rules for general constrained set-valued optimization problems involving new scalarization functionals based on the oriented distance function. Moreover, we provide sufficient optimality conditions for the considered problems without any convexity assumptions.

Asset Allocation via Machine Learning

In this paper, we document a novel machine learning-based numerical framework to solve static and dynamic portfolio optimization problems, with, potentially, an extremely large number of assets. The framework proposed applies to general constrained optimization problems and overcomes many major difficulties arising in current literature. We not only empirically test our methods in U.S. and China A-share equity markets, but also run a horse-race comparison of some optimization schemes documented in (Homescu, 2014). We record significant excess returns, relative to the selected benchmarks, in both U.S. and China equity markets using popular schemes solved by our framework, where the conditional expected returns are obtained via machine learning regression, inspired by (Gu, Kelly & Xiu, 2020) and (Leippold, Wang & Zhou, 2021), of future returns on pricing factors carefully chosen.

Efficient ptychographic phase retrieval via a matrix-free Levenberg-Marquardt algorithm.

The phase retrieval problem, where one aims to recover a complex-valued image from far-field intensity measurements, is a classic problem encountered in a range of imaging applications. Modern phase retrieval approaches usually rely on gradient descent methods in a nonlinear minimization framework. Calculating closed-form gradients for use in these methods is tedious work, and formulating second order derivatives is even more laborious. Additionally, second order techniques often require the storage and inversion of large matrices of partial derivatives, with memory requirements that can be prohibitive for data-rich imaging modalities. We use a reverse-mode automatic differentiation (AD) framework to implement an efficient matrix-free version of the Levenberg-Marquardt (LM) algorithm, a longstanding method that finds popular use in nonlinear least-square minimization problems but which has seen little use in phase retrieval. Furthermore, we extend the basic LM algorithm so that it can be applied for more general constrained optimization problems (including phase retrieval problems) beyond just the least-square applications. Since we use AD, we only need to specify the physics-based forward model for a specific imaging application; the first and second-order derivative terms are calculated automatically through matrix-vector products, without explicitly forming the large Jacobian or Gauss-Newton matrices typically required for the LM method. We demonstrate that this algorithm can be used to solve both the unconstrained ptychographic object retrieval problem and the constrained "blind" ptychographic object and probe retrieval problems, under the popular Gaussian noise model as well as the Poisson noise model. We compare this algorithm to state-of-the-art first order ptychographic reconstruction methods to demonstrate empirically that this method outperforms best-in-class first-order methods: it provides excellent convergence guarantees with (in many cases) a superlinear rate of convergence, all with a computational cost comparable to, or lower than, the tested first-order algorithms.

A general constrained optimization framework for the eco-routing problem: Comparison and analysis of solution strategies for hybrid electric vehicles

Vehicles electrification marks a very important step towards sustainable mobility. However, energy efficiency and driving range of electrified vehicles are nowadays a major concern. From an algorithmic perspective, eco-routing opens up new possibilities regarding the strategies and tools aimed at improving energy efficiency by finding an energy-minimal route under different constraints coming from vehicle characteristics (powertrain, battery capacity, etc.) and user preferences (travel time, etc.). In this work, a powertrain-independent speed prediction model is presented. This model is then used to derive a fast numerical solution of the powertrain energy management for hybrid electric vehicles. Furthermore, a new general formulation is derived for the minimum-energy navigation problem, with a focus on the specific complexity introduced by electrified vehicles. The general constrained optimization problem is reformulated in several alternative ways in order to achieve a solution in limited computation time. The most commonly used approaches nowadays in the literature (integer programming and shortest path algorithms on directed graphs) are compared and benchmarked in terms of solution accuracy and computational effort. The objective is to identify best-practices in accurately and efficiently solving the constrained eco-routing problem.

A merit function approach for evolution strategies

• A globally convergent evolution strategy to solve general constrained optimization problems. • Quantifiable relaxable constraints are handled using a merit function approach combined with a specific restoration procedure. • The proposed approach is very competitive compared to existing derivative-free optimization solvers on a large set of problems. In this paper, we extend a class of globally convergent evolution strategies to handle general constrained optimization problems. The proposed framework handles quantifiable relaxable constraints using a merit function approach combined with a specific restoration procedure. The unrelaxable constraints, when present, can be treated either by using the extreme barrier function or through a projection approach. Under reasonable assumptions, the introduced extension guarantees to the regarded class of evolution strategies global convergence properties for first order stationary constraints. Numerical experiments are carried out on a set of problems from the CUTEst collection as well as on known global optimization problems.

Perturbation of Image and conjugate duality for vector optimization

<p style='text-indent:20px;'>This paper aims at employing the image space approach to investigate the conjugate duality theory for general constrained vector optimization problems. We introduce the concepts of conjugate map and subdifferential by using two types of maximums. We also construct the conjugate duality problems via a perturbation method. Moreover, the separation condition is proposed by means of vector weak separation functions. Then, it is proved to be a new sufficient condition, which ensures the strong duality theorem. This separation condition is different from the classical regular conditions in the literature. Simultaneously, the application to a nonconvex multi-objective optimization problem is shown to verify our main results.</p>

The Stationary Point Set Map in General Parametric Optimization Problems

The present paper shows how the linear independence constraint qualification (LICQ) can be combined with some conditions put on the first-order and second-order derivatives of the objective function and the constraint functions to ensure the Robinson stability and the Lipschitz-like property of the stationary point set map of a general C2-smooth parametric constrained optimization problem. So, a part of the results in two preceding papers of the authors [J. Optim. Theory Appl. 180 (2019), 91–116 (Part 1); 117–139 (Part 2)], which were obtained for a problem with just one inequality constraint, now has an adequate extension for problems having finitely many equality and inequality constraints. Our main tool is an estimate of B. S. Mordukhovich and R. T. Rockafellar [SIAM J. Optim. 22 (2012), 953–986; Theorem 3.3] for a second-order partial subdifferential of a composite function. The obtained results are illustrated by three examples.

A Computation-Efficient Decentralized Algorithm for Composite Constrained Optimization

This paper focuses on solving the problem of composite constrained convex optimization with a sum of smooth convex functions and non-smooth regularization terms (ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm) subject to locally general constraints. Motivated by the modern large-scale information processing problems in machine learning (the samples of a training dataset are randomly decentralized across multiple computing nodes), each of the smooth objective functions is further considered as the average of several constituent functions. To address the problem in a decentralized fashion, we propose a novel computation-efficient decentralized stochastic gradient algorithm, which leverages the variance reduction technique and the decentralized stochastic gradient projection method with constant step-size. Theoretical analysis indicates that if the constant step-size is less than an explicitly estimated upper bound, the proposed algorithm can find the exact optimal solution in expectation when each constituent function (smooth) is strongly convex. Concerning the existing decentralized schemes, the proposed algorithm not only is suitable for solving the general constrained optimization problems but also possesses low computation cost in terms of the total number of local gradient evaluations. Furthermore, the proposed algorithm via differential privacy strategy can effectively mask the privacy of each constituent function, which is more practical in applications involving sensitive messages, such as military affairs or medical treatment. Finally, numerical evidence is provided to demonstrate the appealing performance of the proposed algorithm.

A Quantum-Behaved Neurodynamic Approach for Nonconvex Optimization with Constraints

This paper presents a quantum-behaved neurodynamic swarm optimization approach to solve the nonconvex optimization problems with inequality constraints. Firstly, the general constrained optimization problem is addressed and a high-performance feedback neural network for solving convex nonlinear programming problems is introduced. The convergence of the proposed neural network is also proved. Then, combined with the quantum-behaved particle swarm method, a quantum-behaved neurodynamic swarm optimization (QNSO) approach is presented. Finally, the performance of the proposed QNSO algorithm is evaluated through two function tests and three applications including the hollow transmission shaft, heat exchangers and crank–rocker mechanism. Numerical simulations are also provided to verify the advantages of our method.

Penalty functions and two-step selection procedure based DIRECT-type algorithm for constrained global optimization

Applied optimization problems often include constraints. Although the well-known derivative-free global-search DIRECT algorithm performs well solving box-constrained global optimization problems, it does not naturally address constraints. In this article, we develop a new algorithm DIRECT-GLce for general constrained global optimization problems incorporating two-step selection procedure and penalty function approach in our recent DIRECT-GL algorithm. The proposed algorithm effectively explores hyper-rectangles with infeasible centers which are close to boundaries of feasibility and may cover feasible regions. An extensive experimental investigation revealed the potential of the proposed approach compared with other existing DIRECT-type algorithms for constrained global optimization problems, including important engineering problems.

Constrained Multiobjective Nonlinear Optimization: A User Preference Enabling Method.

A novel user preference enabling (UPE) method is developed to solve general constrained nonlinear multiple objective optimization (MOO) problems. User wish lists on the preferred range of each objective function are introduced and incorporated into the MOO formulation to form a user-preferred (UP) MOO problem. A theoretical foundation of the UP feasible region of MOO problems is developed. The developed theoretical work leads to the development of a UPE method for effectively solving the UP-MOO problems. Distinguishing features of the proposed method include its ability to compute a targeted Pareto solution and to serve as a complement to existing MOO methods, in the sense that the proposed method assists the existing MOO methods in computing the Pareto front by providing feasible solutions and UP feasible solutions. Both proposed UPE method and derived theoretical developments are evaluated on several test systems with promising results.

Programmable Logic Controller for Embedded Implementation of Input-Constrained Systems

Programmable logic controllers (PLCs) have the benefit of being able to withstand a variety of environments while maintaining high reliability compared to computers and other controllers. Traditionally, PLCs have been used for simple ON-OFF control schemes or for proportional-integral-derivative (PID) control. However, recent developments in technology have allowed even low-cost PLCs to implement more advanced and efficient algorithms. This paper investigates a PLC implementation of the specialized Projected Gauss-Seidel (PGS) algorithm. The PGS algorithm is able to quickly and efficiently solve both symmetric and asymmetric mathematical programming problems. The algorithm can be further utilized to solve more complicated problems such as model predictive control (MPC). The focus of this paper is to solve a general constrained optimization problem on a PLC and to expand the capability to solve a MPC example. The DirectLogic Do-More PLC is exploited both for computational efficiency and for memory utilization.

A Proximal Point Analysis of the Preconditioned Alternating Direction Method of Multipliers

We study preconditioned algorithms of alternating direction method of multipliers type for nonsmooth optimization problems. The alternating direction method of multipliers is a popular first-order method for general constrained optimization problems. However, one of its drawbacks is the need to solve implicit subproblems. In various applications, these subproblems are either easily solvable or linear, but nevertheless challenging. We derive a preconditioned version that allows for flexible and efficient preconditioning for these linear subproblems. The original and preconditioned version is written as a new kind of proximal point method for the primal problem, and the weak (strong) convergence in infinite (finite) dimensional Hilbert spaces is proved. Various efficient preconditioners with any number of inner iterations may be used in this preconditioned framework. Furthermore, connections between the preconditioned version and the recently introduced preconditioned Douglas---Rachford method for general nonsmooth problems involving quadratic---linear terms are established. The methods are applied to total variation denoising problems, and their benefits are shown in numerical experiments.

Theoretical and Practical Convergence of a Self-Adaptive Penalty Algorithm for Constrained Global Optimization

This paper proposes a self-adaptive penalty function and presents a penalty-based algorithm for solving nonsmooth and nonconvex constrained optimization problems. We prove that the general constrained optimization problem is equivalent to a bound constrained problem in the sense that they have the same global solutions. The global minimizer of the penalty function subject to a set of bound constraints may be obtained by a population-based meta-heuristic. Further, a hybrid self-adaptive penalty firefly algorithm, with a local intensification search, is designed, and its convergence analysis is established. The numerical experiments and a comparison with other penalty-based approaches show the effectiveness of the new self-adaptive penalty algorithm in solving constrained global optimization problems.

Separation Functions and Optimality Conditions in Vector Optimization

In this paper, we propose weak separation functions in the image space for general constrained vector optimization problems on strong and weak vector minimum points. Gerstewitz function is applied to construct a special class of nonlinear separation functions as well as the corresponding generalized Lagrangian functions. By virtue of such nonlinear separation functions, we derive Lagrangian-type sufficient optimality conditions in a general context. Especially for nonconvex problems, we establish Lagrangian-type necessary optimality conditions under suitable restriction conditions, and we further deduce Karush–Kuhn–Tucker necessary conditions in terms of Clarke subdifferentials.

Smoothing projected Barzilai–Borwein method for constrained non-Lipschitz optimization

We present a smoothing projected Barzilai---Borwein (SPBB) algorithm for solving a class of minimization problems on a closed convex set, where the objective function is nonsmooth nonconvex, perhaps even non-Lipschitz. At each iteration, the SPBB algorithm applies the projected gradient strategy that alternately uses the two Barzilai---Borwein stepsizes to the smooth approximation of the original problem. Nonmonotone scheme is adopted to ensure global convergence. Under mild conditions, we prove convergence of the SPBB algorithm to a scaled stationary point of the original problem. When the objective function is locally Lipschitz continuous, we consider a general constrained optimization problem and show that any accumulation point generated by the SPBB algorithm is a stationary point associated with the smoothing function used in the algorithm. Numerical experiments on $$\ell _2$$l2-$$\ell _p$$lp problems, image restoration problems, and stochastic linear complementarity problems show that the SPBB algorithm is promising.