Nonconvex Constrained Optimization Problems Research Articles

A primal–dual penalty method via rounded weighted-ℓ1 Lagrangian duality

We propose a new duality scheme based on a sequence of smooth minorants of the weighted- penalty function, interpreted as a parametrized sequence of augmented Lagrangians, to solve non-convex constrained optimization problems. For the induced sequence of dual problems, we establish strong asymptotic duality properties. Namely, we show that (i) the sequence of dual problems is convex and (ii) the dual values monotonically increase to the optimal primal value. We use these properties to devise a subgradient based primal–dual method, and show that the generated primal sequence accumulates at a solution of the original problem. We illustrate the performance of the new method with three different types of test problems: A polynomial non-convex problem, large-scale instances of the celebrated kissing number problem, and the Markov–Dubins problem. Our numerical experiments demonstrate that, when compared with the traditional implementation of a well-known smooth solver, our new method (using the same well-known solver in its subproblem) can find better quality solutions, i.e. ‘deeper’ local minima, or solutions closer to the global minimum. Moreover, our method seems to be more time efficient, especially when the problem has a large number of constraints.

A mixture varying-gain dynamic learning network for solving nonlinear and nonconvex constrained optimization problems

Nonlinear and nonconvex optimization problem (NNOP) is a challenging problem in control theory and applications. In this paper, a novel mixture varying-gain dynamic learning network (MVG-DLN) is proposed to solve NNOP with inequality constraints. To do so, first, this NNOP is transformed into some equations through Karush–Kuhn–Tucker (KKT) conditions and projection theorem, and the neuro-dynamics function can be obtained. Second, the time varying convergence parameter is utilized to obtain a faster convergence speed. Third, an integral term is used to strengthen the robustness. Theoretical analysis proves that the proposed MVG-DLN has global convergence and good robustness. Three numerical simulation comparisons between FT-FP-CDNN and MVG-DLN substantiate the faster convergence performance and greater robustness of the MVG-DLN in solving the nonlinear and nonconvex optimization problems.

Design of Optimal Static Output Feedback Controllers for Linear Control Systems Subject to General Structural Constraints

This work considers the design of optimal static output feedback controllers with priori structural constraints for linear systems. The structural constraints impose individual elements of the feedback matrix to satisfy certain equality or inequality constraints. The barrier method is used to address an auxiliary minimization problem to attain an approximate solution to the original nonconvex constrained optimization problem. The Lagrange multiplier method is applied to derive necessary conditions for the optimal solution. An easy-to-implement algorithm based on the gradient descent method is developed to solve the auxiliary minimization problem, and the convergence of the algorithm is proven. The effectiveness of the proposed methodology is validated using two numerical examples.

Computationally Efficient Approximations Using Adaptive Weighting Coefficients for Solving Structural Optimization Problems

With rapid development of advanced manufacturing technologies and high demands for innovative lightweight constructions to mitigate the environmental and economic impacts, design optimization has attracted increasing attention in many engineering subjects, such as civil, structural, aerospace, automotive, and energy engineering. For nonconvex nonlinear constrained optimization problems with continuous variables, evaluations of the fitness and constraint functions by means of finite element simulations can be extremely expensive. To address this problem by algorithms with sufficient accuracy as well as less computational cost, an extended multipoint approximation method (EMAM) and an adaptive weighting-coefficient strategy are proposed to efficiently seek the optimum by the integration of metamodels with sequential quadratic programming (SQP). The developed EMAM stems from the principle of the polynomial approximation and assimilates the advantages of Taylor’s expansion for improving the suboptimal continuous solution. Results demonstrate the superiority of the proposed EMAM over other evolutionary algorithms (e.g., particle swarm optimization technique, firefly algorithm, genetic algorithm, metaheuristic methods, and other metamodeling techniques) in terms of the computational efficiency and accuracy by four well-established engineering problems. The developed EMAM reduces the number of simulations during the design phase and provides wealth of information for designers to effectively tailor the parameters for optimal solutions with computational efficiency in the simulation-based engineering optimization problems.

Joint Power Allocation and Beamforming for Energy-Efficient Design in Multiuser Distributed MIMO Systems

Energy efficiency (EE) optimization of joint power allocation (PA) and beamforming (BF) is studied for multiuser distributed multiple-input multiple-output (D-MIMO) systems with maximum ratio combining. Subject to power budget constraints, a non-convex constrained optimization problem is firstly formulated to maximize EE of single-user D-MIMO. We propose a near-optimal joint design scheme with the block coordinate descent method, which has an effective iterative algorithm to find optimal BF for fixed PA, and an iterative algorithm based on the Dinkelbach methods to find optimal PA for fixed BF. We also develop a low-complexity suboptimal joint scheme, which has closed-form BF and PA by maximizing an effective signal-to-noise ratio for fixed PA and a derived EE upper bound for fixed BF, respectively. It has lower complexity than the near-optimal scheme with slight performance loss. With these results, two joint design schemes are developed for multiuser D-MIMO. The first scheme can achieve the best performance, while the second one based on closed-form PA and BF has slight performance loss in comparing to that of the former, but with lower complexity. Simulation results verify the effectiveness of the developed schemes over some existing schemes.

Ghost Penalties in Nonconvex Constrained Optimization: Diminishing Stepsizes and Iteration Complexity

We consider nonconvex constrained optimization problems and propose a new approach to the convergence analysis based on penalty functions. We make use of classical penalty functions in an unconventional way, in that penalty functions only enter in the theoretical analysis of convergence while the algorithm itself is penalty free. Based on this idea, we are able to establish several new results, including the first general analysis for diminishing stepsize methods in nonconvex, constrained optimization, showing convergence to generalized stationary points, and a complexity study for sequential quadratic programming–type algorithms.

A QCQP-based splitting SQP algorithm for two-block nonconvex constrained optimization problems with application

This paper discusses a class of two-block nonconvex smooth optimization problems with nonlinear constraints. Based on a quadratically constrained quadratic programming (QCQP) approximation, an augmented Lagrangian function (ALF), and a Lagrangian splitting technique into small-scale subproblems, we propose a novel sequential quadratic programming (SQP) algorithm. First, inspired by the augmented Lagrangian method (ALM), we penalize the quadratic equality constraints associated with the QCQP approximation subproblem in its objective by means of the ALF, and then split the resulting subproblem into two small-scale ones, but both of them are not quadratic programming (QP) due to the square of the quadratic equality constraints in the objective. Second, by ignoring the three-order infinitesimal arising from the squared term, the two small-scale subproblems are reduced to two standard QP subproblems, which can yield an improved search direction. Third, taking the ALF of the discussed problem as a merit function, the next iterate point is generated by the Armijo line search. As a result, a new SQP method, called QCQP-based splitting SQP method, is proposed. Under suitable conditions, the global convergence, strong convergence, iteration complexity and convergence rate of the proposed method are analyzed and obtained. Finally, preliminary numerical experiments and applications were carried out, and these show that the proposed method is promising.

Optimal SWIPT in RIS-Aided MIMO Networks

The use of simultaneous wireless information and power transfer (SWIPT) is a key enabler of achieving convenience and prolonging the energy supply lifetime of wireless networks. To address the low efficiencies of far-field power transfer, a reconfigurable intelligent surface (RIS) is adopted to enhance the energy harvesting (EH) performance, which can construct a favorable wireless propagation environment. In this paper, we consider an RIS-assisted SWIPT system with wireless transfer from the access point (AP) to multiple-antenna receivers, which include information receivers (IRs) and energy receivers (ERs). First, we formulate the problem of maximizing the minimum rates of the IRs as a nonconvex constrained optimization problem. For ideal and nonideal channels, we propose two different solutions. Second, we simplify the objective function and decompose the problem into several subproblems by using sorting and iterative optimization algorithms. Moreover, under optimal boundary and Karush-Kuhn-Tucker (KKT) conditions, we successfully solve this problem. The simulation results illustrate a promising approach for wireless communication by comparing ideal RIS and unsatisfactory situations with no RIS cases under various conditions.

Reconstructing Point Sets From Distance Distributions

We address the problem of reconstructing a set of points on a line or a loop from their unassigned noisy pairwise distances. When the points lie on a line, the problem is known as the turnpike; when they are on a loop, it is known as the beltway. We approximate the problem by discretizing the domain and representing the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> points via an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> -hot encoding, which is a density supported on the discretized domain. We show how the distance distribution is then simply a collection of quadratic functionals of this density and propose to recover the point locations so that the estimated distance distribution matches the measured distance distribution. This can be cast as a constrained nonconvex optimization problem which we solve using projected gradient descent with a suitable spectral initializer. We derive conditions under which the proposed distance distribution matching approach locally converges to a global optimizer at a linear rate. Compared to the conventional backtracking approach, our method jointly reconstructs all the point locations and is robust to noise in the measurements. We substantiate these claims with state-of-the-art performance across a number of numerical experiments. Our method is the first practical approach to solve the large-scale noisy beltway problem where the points lie on a loop.

Semidefinite relaxation method for unified near-Field and far-Field localization by AOA

This paper investigates the unified near-field and far-field localization using angle-of-arrival measurements, where the knowledge that the source is in the near-field or far-field is not required. We propose a semidefinite relaxation method to solve this problem by using the modified polar representation of the source. By approximating the measurement model, we formulate the localization problem as a weighted least squares problem, which is a non-convex constrained optimization problem. This problem is then relaxed to a convex semidefinite program (SDP) by applying semidefinite relaxation. The relaxed SDP problem is tightened by adding a set of second-order cone constraints to reach a tighter mixed semidefinite and second-order cone program. The proposed method provides an accurate position estimate for the near-field source, and yields a precise direction of arrival estimate for the far-field source. To reduce the estimation bias introduced by the approximation of the measurement model, we further derive the analytical expression of the bias and subtract it from the solution to obtain an approximately unbiased estimate. Both simulations and theoretical analysis show that the proposed method reaches the Cramer-Rao lower bound when the noise is not large under Gaussian noise, and has a comparable bias with the maximum likelihood estimator.

A Reference Vector-Based Simplified Covariance Matrix Adaptation Evolution Strategy for Constrained Global Optimization.

During the last two decades, the notion of multiobjective optimization (MOO) has been successfully adopted to solve the nonconvex constrained optimization problems (COPs) in their most general forms. However, such works mainly utilized the Pareto dominance-based MOO framework while the other successful MOO frameworks, such as the reference vector (RV) and the decomposition-based ones, have not drawn sufficient attention from the COP researchers. In this article, we utilize the concepts of the RV-based MOO to design a ranking strategy for the solutions of a COP. We first transform the COP into a biobjective optimization problem (BOP) and then solve it by using the covariance matrix adaptation evolution strategy (CMA-ES), which is arguably one of the most competitive evolutionary algorithms of current interest. We propose an RV-based ranking strategy to calculate the mean and update the covariance matrix in CMA-ES. Besides, the RV is explicitly tuned during the optimization process based on the characteristics of COPs in a RV-based MOO framework. We also propose a repair mechanism for the infeasible solutions and a restart strategy to facilitate the population to escape from the infeasible region. We test the proposal extensively on two well-known benchmark suites comprised of 36 and 112 test problems (at different scales) from the IEEE CEC (Congress on Evolutionary Computation) 2010 and 2017 competitions along with a real-world problem related to power flow. Our experimental results suggest that the proposed algorithm can meet or beat several other state-of-the-art constrained optimizers in terms of the performance on a wide variety of problems.

Low-rank matrix completion and denoising under Poisson noise

Abstract This paper considers the problem of estimating a low-rank matrix from the observation of all or a subset of its entries in the presence of Poisson noise. When we observe all entries, this is a problem of matrix denoising; when we observe only a subset of the entries, this is a problem of matrix completion. In both cases, we exploit an assumption that the underlying matrix is low-rank. Specifically, we analyse several estimators, including a constrained nuclear-norm minimization program, nuclear-norm regularized least squares and a non-convex constrained low-rank optimization problem. We show that for all three estimators, with high probability, we have an upper error bound (in the Frobenius norm error metric) that depends on the matrix rank, the fraction of the elements observed and the maximal row and column sums of the true matrix. We furthermore show that the above results are minimax optimal (within a universal constant) in classes of matrices with low-rank and bounded row and column sums. We also extend these results to handle the case of matrix multinomial denoising and completion.

A filter proximal bundle method for nonsmooth nonconvex constrained optimization

A filter proximal bundle algorithm is presented for nonsmooth nonconvex constrained optimization problems. The new algorithm is based on the proximal bundle method and utilizes the improvement function to regularize the constraint. At every iteration by solving a convex piecewise-linear subproblem a trial point is obtained. The process of the filter technique is employed either to accept the trial point as a serious iterate or to reject it as a null iterate. Under some mild and standard assumptions and for every possible choice of a starting point, it is shown that every accumulation point of the sequence of serious iterates is feasible. In addition, there exists at least one accumulation point which is stationary for the improvement function. Finally, some encouraging numerical results show that the proposed algorithm is effective.

Optimal Protection Coordination of Nonstandard Overcurrent Relays Using Hybrid QCQP Method

The optimal protection coordination (OPC) problem of directional overcurrent relays (OcR) in power system networks is a complex, highly constrained nonconvex optimization problem because of the nonlinear inverse nature of the OcR characteristics. Finding the best OcR settings by solving OPC is a challenging task because of its nonconvex nature. Although various heuristic, hybrid heuristic, and traditional optimization-based solution methods are suggested in the literature, such methods are just far enough to provide better optimal solutions to the OPC problem. Besides, the standard inverse characteristics of the OcRs with fixed characteristic coefficients limit the OPC problem from finding the best settings. As digital relaying technology has the feasibility to embed more relay settings, nonstandard relay characteristics can be developed to achieve minimum relay operating times. To find the best OcR operating times, a nonstandard characteristic based OPC problem is formulated in this work by parameterizing the OcR characteristic coefficients along with the OcR settings. By setting two variables of the proposed OPC using particle swarm optimization (PSO), the rest of OPC is formulated as a quadratic programming problem with quadratic constraints (QCQP). The proposed method is developed in MATLAB programming platform, and results are found to be satisfactory.

Optimality conditions for efficient solutions of nonconvex constrained multiobjective optimization problems via image space analysis

This paper mainly focuses on optimality conditions for efficient solutions of a nonconvex constrained multiobjective optimization problem via image space analysis. By virtue of the Gerstewitz function and the sum of the components of the objective function, a nonlinear regular weak separation function for efficient solutions, is constructed. In order to illustrate the importance of the sum of the components of the objective function in the nonlinear regular weak separation function for efficient solutions, a nonlinear regular weak separation function for weak efficient solutions, is also given. Moreover, a nonlinear strong separation function for efficient and weak efficient solutions, is introduced. Then, some theorems of the weak and strong alternative and a global necessary and sufficient optimality condition for efficient solutions are derived by means of the nonlinear regular weak and strong separation functions for efficient solutions. A saddle point sufficient optimality condition for efficient solutions in terms of a generalized Lagrangian function associated with the nonlinear regular weak separation function, is established. Finally, under some suitable assumptions, a saddle point necessary optimality condition is obtained, which further yields a Karush/Kuhn–Tucker necessary optimality condition for efficient solutions by the Clarke subdifferential.

A test-suite of non-convex constrained optimization problems from the real-world and some baseline results

Real-world optimization problems have been comparatively difficult to solve due to the complex nature of the objective function with a substantial number of constraints. To deal with such problems, several metaheuristics as well as constraint handling approaches have been suggested. To validate the effectiveness and strength, performance of a newly designed approach should be benchmarked by using some complex real-world problems, instead of only the toy problems with synthetic objective functions, mostly arising from the area of numerical analysis. A list of standard real-life problems appears to be the need of the time for benchmarking new algorithms in an efficient and unbiased manner. In this study, a set of 57 real-world Constrained Optimization Problems (COPs) are described and presented as a benchmark suite to validate the COPs. These problems are shown to capture a wide range of difficulties and challenges that arise from the real life optimization scenarios. Three state-of-the-art constrained optimization methods are exhaustively tested on these problems to analyze their hardness. The experimental outcomes reveal that the selected problems are indeed challenging to these algorithms, which have been shown to solve many synthetic benchmark problems easily.

Estimating individualized treatment rules with risk constraint.

Individualized treatment rules (ITRs) recommend treatments based on patient-specific characteristics in order to maximize the expected clinical outcome. At the same time, the risks caused by various adverse events cannot be ignored. In this paper, we propose a method to estimate an optimal ITR that maximizes clinical benefit while having the overall risk controlled at a desired level. Our method works for a general setting of multi-category treatment. The proposed procedure employs two shifted ramp losses to approximate the 0-1 loss in the objective function and constraint, respectively, and transforms the estimation problem into a difference of convex functions (DC) programming problem. A relaxed DC algorithm is used to solve the nonconvex constrained optimization problem. Simulations and a real data example are used to demonstrate the finite sample performance of the proposedmethod.

A class of exact penalty functions and penalty algorithms for nonsmooth constrained optimization problems

In this paper, a class of smoothing penalty functions is proposed for optimization problems with equality, inequality and bound constraints. It is proved exact, under the condition of weakly generalized Mangasarian–Fromovitz constraint qualification, in the sense that each local optimizer of the penalty function corresponds to a local optimizer of the original problem. Furthermore, necessary and sufficient conditions are discussed for the inverse proposition of exact penalization. Based on the theoretical results in this paper, a class of smoothing penalty algorithms with feasibility verification is presented. Theories on the penalty exactness, feasibility verification and global convergence of the proposed algorithm are presented. Numerical results show that this algorithm is effective for nonsmooth nonconvex constrained optimization problems.

Joint Waveform and Receiver Design for Co-Channel Hybrid Active-Passive Sensing With Timing Uncertainty

We consider a hybrid active-passive radar system that employs a wireless source as a passive illuminator of opportunity (IO) and a co-channel active radar transmitter operating in the same frequency band to seek spectral efficiency. The hybrid system can take advantage of the strengths of passive radar (e.g., energy efficiency, bi-/multi-static configuration, and spatial diversity) as well as those of active radar (dedicated transmitter, flexible transmit beam steering, waveform optimized for sensing, etc.). To mitigate the mutual interference and location-induced timing uncertainty between the radar and communication signals, we propose two designs for the joint optimization of the radar waveform and receive filters. The first is a max-min (MM) criterion that optimizes a worst-case performance metric over a timing uncertainty interval, and the other a weighted-sum (WS) criterion that forms a weighted sum of the performance metric at each delay within the delay uncertainty interval. Both design criteria result in nonconvex constrained optimization problems that are solved by sequential convex programming methods. When timing uncertainty vanishes, the two designs become identical and admit a simpler solution. Numerical results are presented to demonstrate the performance of the proposed hybrid schemes in comparison with conventional active-only and passive-only radar systems.

Optimality Conditions for Nonconvex Constrained Optimization Problems

We present optimality conditions for a class of nonsmooth and nonconvex constrained optimization problems. To achieve this aim, various well-known constraint qualifications are extended based on the concept of tangential subdifferential and the relations between them are investigated. Moreover, local and global necessary and sufficient optimality conditions are derived in the absence of convexity of the feasible set. In addition to the theoretical results, several examples are provided to illustrate the advantage of our outcomes.