Nonconvex Sets Research Articles

Distributed Multi-Target Rotating Encirclement Control of Second-Order Multi-Agent Systems With Nonconvex Input Constraints

This paper investigates the collective multi-target rotating encirclement formation problem of second-order multi-agent systems, where the inputs are constrained in nonconvex sets. The objective requires that all agents rotate around targets’ geometric center with the desired radius and angular velocity. Firstly, the targets’ geometric center and rotating radius are calculated by two distributed fixed-time estimators based on the neighbors’ information. Then, with the complex domain theory, we construct two consensus vectors complying with the conditions of the multi-target rotating encirclement formation, and propose a distributed multi-target rotating encirclement control scheme to force all agents achieve the desired formation structure. Moreover, a sufficient condition is provided for choosing design parameters with the aid of Lyapunov theory. In order to handle the problem caused by the nonconvex input constraints, a constraint operator is introduced to ensure all control inputs always lie in the corresponding nonconvex sets. Finally, numerical simulation results are presented to demonstrate the effectiveness and correctness of our theoretical control scheme.

Precise 3-D GNSS Attitude Determination Based on Riemannian Manifold Optimization Algorithms

In the past few years, Global Navigation Satellite Systems (GNSS) based attitude determination has been widely used thanks to its high accuracy, low cost, and real-time performance. This paper presents a novel 3-D GNSS attitude determination method based on Riemannian optimization techniques. The paper first exploits the antenna geometry and baseline lengths to reformulate the 3-D GNSS attitude determination problem as an optimization over a non-convex set. Since the solution set is a manifold, in this manuscript we formulate the problem as an optimization over a Riemannian manifold. The study of the geometry of the manifold allows the design of efficient first and second order Riemannian algorithms to solve the 3-D GNSS attitude determination problem. Despite the non-convexity of the problem, the proposed algorithms are guaranteed to globally converge to a critical point of the optimization problem. To assess the performance of the proposed framework, numerical simulations are provided for the most challenging attitude determination cases: the unaided, single-epoch, and single-frequency scenarios. Numerical results reveal that the proposed algorithms largely outperform state-of-the-art methods for various system configurations with lower complexity than generic non-convex solvers, e.g., interior point methods.

Douglas--Rachford Splitting and ADMM for Nonconvex Optimization: Tight Convergence Results

Although originally designed and analyzed for convex problems, the alternating direction method of multipliers (ADMM) and its close relatives, Douglas-Rachford splitting (DRS) and Peaceman-Rachford splitting (PRS), have been observed to perform remarkably well when applied to certain classes of structured nonconvex optimization problems. However, partial global convergence results in the nonconvex setting have only recently emerged. In this paper we show how the Douglas-Rachford envelope (DRE), introduced in 2014, can be employed to unify and considerably simplify the theory for devising global convergence guarantees for ADMM, DRS and PRS applied to nonconvex problems under less restrictive conditions, larger prox-stepsizes and over-relaxation parameters than previously known. In fact, our bounds are tight whenever the over-relaxation parameter ranges in $(0,2]$. The analysis of ADMM uses a universal primal equivalence with DRS that generalizes the known duality of the algorithms.

High-Dimensional Nonconvex Stochastic Optimization by Doubly Stochastic Successive Convex Approximation

In this paper, we consider supervised learning problems over training sets in which the number of training examples and the dimension of feature vectors are both large. We focus on the case where the loss function defining the quality of the parameter we wish to estimate may be non-convex, but also has a convex regularization. We propose a Doubly Stochastic Successive Convex approximation scheme (DSSC) able to handle non-convex regularized expected risk minimization. The method operates by decomposing the decision variable into blocks and operating on random subsets of blocks at each step (fusing the merits of stochastic approximation with block coordinate methods), and then implements successive convex approximation. In contrast to many stochastic convex methods whose almost sure behavior is not guaranteed in non-convex settings, DSSC attains almost sure convergence to a stationary solution of the problem. Moreover, we show that the proposed DSSC algorithm achieves stationarity at a rate of O((log t)/t <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1/4</sup> ). Numerical experiments on a non-convex variant of a lasso regression problem show that DSSC performs favorably in this setting. We then apply this method to the task of dictionary learning from high-dimensional visual data collected from a ground robot, and observe reliable convergence behavior for a difficult non-convex stochastic program.

Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach

We study the problem of recovery of matrices that are simultaneously low rank and row and/or column sparse. Such matrices appear in recent applications in cognitive neuroscience, imaging, computer vision, macroeconomics, and genetics. We propose a GDT (Gradient Descent with hard Thresholding) algorithm to efficiently recover matrices with such structure, by minimizing a bi-convex function over a nonconvex set of constraints. We show linear convergence of the iterates obtained by GDT to a region within statistical error of an optimal solution. As an application of our method, we consider multi-task learning problems and show that the statistical error rate obtained by GDT is near optimal compared to minimax rate. Experiments demonstrate competitive performance and much faster running speed compared to existing methods, on both simulations and real data sets.

On the behavior of Lagrange multipliers in convex and nonconvex infeasible interior point methods

We analyze sequences generated by interior point methods (IPMs) in convex and nonconvex settings. We prove that moving the primal feasibility at the same rate as the barrier parameter $\mu$ ensures the Lagrange multiplier sequence remains bounded, provided the limit point of the primal sequence has a Lagrange multiplier. This result does not require constraint qualifications. We also guarantee the IPM finds a solution satisfying strict complementarity if one exists. On the other hand, if the primal feasibility is reduced too slowly, then the algorithm converges to a point of minimal complementarity; if the primal feasibility is reduced too quickly and the set of Lagrange multipliers is unbounded, then the norm of the Lagrange multiplier tends to infinity. Our theory has important implications for the design of IPMs. Specifically, we show that IPOPT, an algorithm that does not carefully control primal feasibility has practical issues with the dual multipliers values growing to unnecessarily large values. Conversely, the one-phase IPM of \citet*{hinder2018one}, an algorithm that controls primal feasibility as our theory suggests, has no such issue.

Monge–Ampère of Pac-Man

We show that the Monge–Ampere density of the extremal function $$V_P$$ for a non-convex Pac-Man set $$P\subset {{\mathbb {R}}}^2$$ tends to a finite limit as we approach the vertex p of P along lines but with a value that may vary with the line. On the other hand, along a tangential approach to p, the Monge–Ampere density becomes unbounded. This partially mimics the behavior of the Monge–Ampere density of the union of two quarter disks S of Sigurdsson and Snaebjarnarson (Ann Pol Math 123:481–504, 2019). We also recover their formula for $$V_S$$ by elementary methods.

Generating $\alpha $-dense curves in non-convex sets to solve a class of non-smooth constrained global optimization

This paper deals with the dimensionality reduction approach to study multi-dimensional constrained global optimization problems where the objective function is non-differentiable over a general compact set $D$ of $\mathbb{R}^{n}$ and H\{o}lderian. The fundamental principle is to provide explicitly a parametric representation $x_{i}=\ell _{i}(t),1\leq i\leq n$ of $\alpha $-dense curve $\ell_{\alpha }$ in the compact $D$, for $t$ in an interval $\mathbb{I}$ of $\mathbb{R}$, which allows to convert the initial problem to a one dimensional H\{o}lder unconstrained one. Thus, we can solve the problem by using an efficient algorithm available in the case of functions depending on a single variable. A relation between the parameter $\alpha $ of the curve $\ell _{\alpha }$ and the accuracy of attaining the optimal solution is given. Some concrete $\alpha $ dense curves in a non-convex feasible region $D$ are constructed. The numerical results show that the proposed approach is efficient.

Core-Selecting Mechanisms in Electricity Markets

Due to its theoretical virtues, several recent works propose the use of the incentive-compatible Vickrey-Clarke-Groves (VCG) mechanism for electricity markets. Coalitions of participants, however, can influence the VCG outcome to obtain higher collective profit. To address this issue, we propose core-selecting mechanisms for their coalition-proofness. We show that core-selecting mechanisms generalize the economic rationale of the locational marginal pricing (LMP) mechanism. Namely, these mechanisms are the exact class of mechanisms that ensure the existence of a competitive equilibrium in linear/nonlinear prices. This implies that the LMP mechanism is also core-selecting, and hence coalition-proof. In contrast to the LMP mechanism, core-selecting mechanisms exist for a broad class of electricity markets, such as ones involving nonconvex costs and nonconvex constraint sets. In addition, they can approximate truthfulness without the price-taking assumption of the LMP mechanism. Finally, we show that they are also budget-balanced. Our results are verified with case studies based on optimal power flow test systems and the Swiss reserve market.

ByRDiE: Byzantine-Resilient Distributed Coordinate Descent for Decentralized Learning

Distributed machine learning algorithms enable learning of models from datasets that are distributed over a network without gathering the data at a centralized location. While efficient distributed algorithms have been developed under the assumption of faultless networks, failures that can render these algorithms nonfunctional occur frequently in the real world. This paper focuses on the problem of Byzantine failures, which are the hardest to safeguard against in distributed algorithms. While Byzantine fault tolerance has a rich history, existing work does not translate into efficient and practical algorithms for high-dimensional learning in fully distributed (also known as decentralized) settings. In this paper, an algorithm termed Byzantine-resilient distributed coordinate descent (ByRDiE) is developed and analyzed that enables distributed learning in the presence of Byzantine failures. Theoretical analysis (convex settings) and numerical experiments (convex and nonconvex settings) highlight its usefulness for high-dimensional distributed learning in the presence of Byzantine failures.

Energy Efficiency Maximization via Joint Sub-Carrier Assignment and Power Control for OFDMA Full Duplex Networks

In this paper, we develop an energy efficient resource allocation scheme for orthogonal frequency division multiple access (OFDMA) networks with in-band full-duplex (IBFD) communication between the base station and user equipments (UEs) considering a realistic self-interference (SI) model. Our primary aim is to maximize the system energy efficiency (EE) through a joint power control and sub-carrier assignment in both the downlink (DL) and uplink (UL), where the quality of service requirements of the UEs in DL and UL are guaranteed. The formulated problem is non-convex due to the non-linear fractional objective function and the non-convex feasible set which is generally intractable. In order to handle this difficulty, we first use fractional programming to transform the fractional objective function to the subtractive form. Then, by employing Dinkelbach method, we propose an iterative algorithm in which an inner problem is solved in each iteration. Applying majorization-minimization approximation, we make the inner problem convex. Also, by introducing a penalty function to handle integer sub-carrier assignment variables, we propose an iterative algorithm for addressing the inner problem. We show that our proposed algorithm converges to the locally optimal solution which is also demonstrated by our simulation results. In addition, simulation results show that by applying the IBFD capability in OFDMA networks with efficient SI cancellation techniques, our proposed resource allocation algorithm attains a 75% increase in the EE as compared to the half-duplex system.

Launch Pad Method in Multiextremal Multiobjective Optimization Problems

A new method is proposed for approximating the Edgeworth–Pareto hull of a feasible objective set in a multiobjective optimization (MOO) problem with criteria functions having numerous local extrema. The method is based on constructing a launch pad, i.e., a subset of the feasible decision set such that gradient procedures for local optimization of criteria and scalarizing functions of criteria starting at these points yield efficient decisions of the MOO problem. A launch pad is constructed using the optima injection method, which combines the usual multistart approach with a genetic algorithm for Pareto frontier approximation. It is shown that the proposed launch pad method (LPM) can also be used to approximate the effective hull of a nonconvex multidimensional set. A theoretical analysis of LPM is presented, and experimental results are given for the applied problem of constructing control rules for a cascade of reservoirs, which is reduced to a complicated MOO problem with scalarizing functions having numerous local extrema.

Phase transitions of spectral initialization for high-dimensional non-convex estimation

AbstractWe study a spectral initialization method that serves a key role in recent work on estimating signals in non-convex settings. Previous analysis of this method focuses on the phase retrieval problem and provides only performance bounds. In this paper, we consider arbitrary generalized linear sensing models and present a precise asymptotic characterization of the performance of the method in the high-dimensional limit. Our analysis also reveals a phase transition phenomenon that depends on the ratio between the number of samples and the signal dimension. When the ratio is below a minimum threshold, the estimates given by the spectral method are no better than random guesses drawn from a uniform distribution on the hypersphere, thus carrying no information; above a maximum threshold, the estimates become increasingly aligned with the target signal. The computational complexity of the method, as measured by the spectral gap, is also markedly different in the two phases. Worked examples and numerical results are provided to illustrate and verify the analytical predictions. In particular, simulations show that our asymptotic formulas provide accurate predictions for the actual performance of the spectral method even at moderate signal dimensions.

A modular analysis of adaptive (non-)convex optimization: Optimism, composite objectives, variance reduction, and variational bounds

Recently, much work has been done on extending the scope of online learning and incremental stochastic optimization algorithms. In this paper we contribute to this effort in two ways: First, based on a generalization of Bregman divergences and a generic regret decomposition, we provide a self-contained, modular analysis of the two workhorses of online learning: (general) adaptive versions of Mirror Descent (MD) and the Follow-the-Regularized-Leader (FTRL) algorithms. The analysis is done with extra care so as not to introduce assumptions not needed in the proofs and allows to combine, in a straightforward way, different algorithmic ideas (e.g., adaptivity, optimism, implicit updates, variance reduction) and learning settings (e.g., strongly convex or composite objectives). This way we are able to reprove, extend and refine a large body of the literature, while keeping the proofs concise. The second contribution is a by-product of this careful analysis: We present algorithms with improved variational bounds for smooth, composite objectives, including a new family of optimistic MD algorithms with only one projection step per round. Furthermore, we provide a simple extension of adaptive regret bounds to a class of practically relevant non-convex problem settings (namely, star-convex loss functions and their extensions) with essentially no extra effort.

Novel Practical Stability Conditions for Discrete-Time Switched Affine Systems

This technical note proposes novel state feedback stability conditions based on the so called Lyapunov–Metzler inequalities for discrete-time switched affine systems, assuring practical stability of a desired equilibrium point. These conditions are obtained taking into account the volume minimization of an ellipsoidal set containing the nonconvex set of attraction to where the state trajectories are globally attracted. Compared with other recent results, the present ones are based on less conservative conditions for the existence of an attraction set. Moreover, it is not required that the equilibrium point be inside a predetermined set of attainable ones, which enlarges the range of applicability of the present technique. To the best of the authors’ knowledge, it is the first time that the Lyapunov–Metzler inequalities are adopted in the context of switched affine systems. As a second step, the associated nonconvex invariant set is provided. Academical examples illustrate the method and compare the theoretical results with recent ones available in the literature.

Material Interpolation in Multi-Material Topology Optimization for Magnetic Device Design

This paper investigates material interpolation schemes in multi-material topology optimization (MMTO) for magnetic device design. Different interpolations of material properties lead to a non-unique layout and differences in performance, thereby affecting the initial design dependence. However, there is a lack of research on material interpolation in MMTO for the magnetic design problem. In this research, two types of material interpolation schemes are compared: recursive multiphase (RM) and discrete material optimization (DMO). To compare only the effects of material interpolations, the non-convex feasible set of material constraints in RM interpolation schemes were transformed into the convex feasible set by a linear form. The performance profile was employed to obtain the global performance of each interpolation scheme. Examples of the C-core actuator and the permanent magnet actuator (PMA) were applied for performance comparison, and the performance profile indicated that the DMO interpolation scheme is robust to initial material distributions and different material constraints compared to the RM interpolation scheme.

DistributedH∞consensus of second‐order multiagent systems with nonconvex constraints

SummaryIn this paper, the distributedH∞consensus problem for second‐order multiagent systems is studied. We first propose a distributed control algorithm based on local information. In the presence of external disturbances, some sufficient conditions are then derived to guarantee that all agents reach the distributedH∞consensus, and meanwhile, the inputs of agents always stay in the nonconvex sets, making the results in this paper more practical. Finally, a numerical simulation is provided to show the effectiveness of the results.

Market Failures and Equilibria in Banach Lattices: New Tangent and Normal Cones

In this paper, we consider an economy with infinitely many commodities and market failures such as increasing returns to scale and external effects or other regarding preferences. The commodity space is a Banach lattice possibly without interior points in the positive cone in order to include most of the relevant commodity spaces in economics. We propose a new definition of the marginal pricing rule through a new tangent cone to the production set at a point of its (non-smooth) boundary. The major contribution is the unification of many previous works with convex or non-convex production sets, smooth or non-smooth, for the competitive equilibria and for the marginal pricing equilibria, with or without external effects, in finite-dimensional spaces as well as in infinite-dimensional spaces. In order to prove the existence of a marginal pricing equilibria, we also provide a suitable properness condition on non-convex technologies to deal with the emptiness of the interior of the positive cone.

Fixed point problems of nonexpansive mappings for nononvex set in Hilbert spaces

In this paper, we introduce a new concept of W-nonexpansive mappings and obtain fixed point theorems for nonexpansive mappings for non-convex set. Our results resolve fixed pointed problem that nonexpansive mappings be not on closed convex set, and it extends fixed point theorems for nonexpansive mappings.

A proximal point method for difference of convex functions in multi-objective optimization with application to group dynamic problems

We consider the constrained multi-objective optimization problem of finding Pareto critical points of difference of convex functions. The new approach proposed by Bento et al. (SIAM J Optim 28:1104–1120, 2018) to study the convergence of the proximal point method is applied. Our method minimizes at each iteration a convex approximation instead of the (non-convex) objective function constrained to a possibly non-convex set which assures the vector improving process. The motivation comes from the famous Group Dynamic problem in Behavioral Sciences where, at each step, a group of (possible badly informed) agents tries to increase his joint payoff, in order to be able to increase the payoff of each of them. In this way, at each step, this ascent process guarantees the stability of the group. Some encouraging preliminary numerical results are reported.