Polynomial Optimization Problems Research Articles

Evolving scientific discovery by unifying data and background knowledge with AI Hilbert

The discovery of scientific formulae that parsimoniously explain natural phenomena and align with existing background theory is a key goal in science. Historically, scientists have derived natural laws by manipulating equations based on existing knowledge, forming new equations, and verifying them experimentally. However, this does not include experimental data within the discovery process, which may be inefficient. We propose a solution to this problem when all axioms and scientific laws are expressible as polynomials and argue our approach is widely applicable. We model notions of minimal complexity using binary variables and logical constraints, solve polynomial optimization problems via mixed-integer linear or semidefinite optimization, and prove the validity of our scientific discoveries in a principled manner using Positivstellensatz certificates. We demonstrate that some famous scientific laws, including Kepler’s Law of Planetary Motion and the Radiated Gravitational Wave Power equation, can be derived in a principled manner from axioms and experimental data.

On symmetry adapted bases in trigonometric optimization

The problem of computing the global minimum of a trigonometric polynomial is computationally hard. We address this problem for the case, where the polynomial is invariant under the exponential action of a finite group. The strategy is to follow an established relaxation strategy in order to obtain a converging hierarchy of lower bounds. Those bounds are obtained by numerically solving semi-definite programs (SDPs) on the cone of positive semi-definite Hermitian Toeplitz matrices, which is outlined in the book of Dumitrescu Dumitrescu (2007). To exploit the invariance, we show that the group has an induced action on the Toeplitz matrices and prove that the feasible region of the SDP can be restricted to the invariant matrices, whilst retaining the same solution. Then we construct a symmetry adapted basis tailored to this group action, which allows us to block-diagonalize invariant matrices and thus reduce the computational complexity to solve the SDP.The approach is in its generality novel for trigonometric optimization and complements the one that was proposed as a poster at the ISSAC 2022 conference Hubert et al. (2022) and later extended to Hubert et al. (2024). In the previous work, we first used the invariance of the trigonometric polynomial to obtain a classical polynomial optimization problem on the orbit space and subsequently relaxed the problem to an SDP. Now, we first make the relaxation and then exploit invariance.Partial results of this article have been presented as a poster at the ISSAC 2023 conference Metzlaff (2023).

Mutually unbiased bases: polynomial optimization and symmetry

A set of k orthonormal bases of Cd is called mutually unbiased if |⟨e,f⟩|2=1/d whenever e and f are basis vectors in distinct bases. A natural question is for which pairs (d,k) there exist k mutually unbiased bases in dimension d. The (well-known) upper bound k≤d+1 is attained when d is a power of a prime. For all other dimensions it is an open problem whether the bound can be attained. Navascués, Pironio, and Acín showed how to reformulate the existence question in terms of the existence of a certain C∗-algebra. This naturally leads to a noncommutative polynomial optimization problem and an associated hierarchy of semidefinite programs. The problem has a symmetry coming from the wreath product of Sd and Sk.We exploit this symmetry (analytically) to reduce the size of the semidefinite programs making them (numerically) tractable. A key step is a novel explicit decomposition of the Sd≀Sk-module C([d]×[k])t into irreducible modules. We present numerical results for small d,k and low levels of the hierarchy. In particular, we obtain sum-of-squares proofs for the (well-known) fact that there do not exist d+2 mutually unbiased bases in dimensions d=2,3,4,5,6,7,8. Moreover, our numerical results indicate that a sum-of-squares refutation, in the above-mentioned framework, of the existence of more than 3 MUBs in dimension 6 requires polynomials of total degree at least 12.

Scan-Free and Fully Automatic Tracking of Native Knee Anatomy from Dynamic Stereo-Radiography with Statistical Shape and Intensity Models.

Kinematic tracking of native anatomy from stereo-radiography provides a quantitative basis for evaluating human movement. Conventional tracking procedures require significant manual effort and call for acquisition and annotation of subject-specific volumetric medical images. The current work introduces a framework for fully automatic tracking of native knee anatomy from dynamic stereo-radiography which forgoes reliance on volumetric scans. The method consists of three computational steps. First, captured radiographs are annotated with segmentation maps and anatomic landmarks using a convolutional neural network. Next, a non-convex polynomial optimization problem formulated from annotated landmarks is solved to acquire preliminary anatomy and pose estimates. Finally, a global optimization routine is performed for concurrent refinement of anatomy and pose. An objective function is maximized which quantifies similarities between masked radiographs and digitally reconstructed radiographs produced from statistical shape and intensity models. The proposed framework was evaluated against manually tracked trials comprising dynamic activities, and additional frames capturing a static knee phantom. Experiments revealed anatomic surface errors routinely below 1.0 mm in both evaluation cohorts. Median absolute errors of individual bone pose estimates were below 1.0 or mm for 15 out of 18 degrees of freedom in both evaluation cohorts. Results indicate that accurate pose estimation of native anatomy from stereo-radiography may be performed with significantly reduced manual effort, and without reliance on volumetric scans.

Learning of Linear Dynamical Systems as a Noncommutative Polynomial Optimization Problem

Learning of Linear Dynamical Systems as a Noncommutative Polynomial Optimization Problem

PolyARBerNN: A Neural Network Guided Solver and Optimizer for Bounded Polynomial Inequalities

Constraints solvers play a significant role in the analysis, synthesis, and formal verification of complex cyber-physical systems. In this article, we study the problem of designing a scalable constraints solver for an important class of constraints named polynomial constraint inequalities (also known as nonlinear real arithmetic theory). In this article, we introduce a solver named PolyARBerNN that uses convex polynomials as abstractions for highly nonlinears polynomials. Such abstractions were previously shown to be powerful to prune the search space and restrict the usage of sound and complete solvers to small search spaces. Compared with the previous efforts on using convex abstractions, PolyARBerNN provides three main contributions namely (i) a neural network guided abstraction refinement procedure that helps selecting the right abstraction out of a set of pre-defined abstractions, (ii) a Bernstein polynomial-based search space pruning mechanism that can be used to compute tight estimates of the polynomial maximum and minimum values which can be used as an additional abstraction of the polynomials, and (iii) an optimizer that transforms polynomial objective functions into polynomial constraints (on the gradient of the objective function) whose solutions are guaranteed to be close to the global optima. These enhancements together allowed the PolyARBerNN solver to solve complex instances and scales more favorably compared to the state-of-the-art nonlinear real arithmetic solvers while maintaining the soundness and completeness of the resulting solver. In particular, our test benches show that PolyARBerNN achieved 100X speedup compared with Z3 8.9, Yices 2.6, and PVS (a solver that uses Bernstein expansion to solve multivariate polynomial constraints) on a variety of standard test benches. Finally, we implemented an optimizer called PolyAROpt that uses PolyARBerNN to solve constrained polynomial optimization problems. Numerical results show that PolyAROpt is able to solve high-dimensional and high order polynomial optimization problems with higher speed compared to the built-in optimizer in the Z3 8.9 solver.

Tight Sum-of-squares Lower Bounds for Binary Polynomial Optimization Problems

For binary polynomial optimization problems of degree 2d with n variables Sakaue, Takeda, Kim, and Ito [ 33 ] proved that the \(\lceil \frac{n+2d-1}{2}\rceil\) th semidefinite (SDP) relaxation in the SoS/Lasserre hierarchy of SDP relaxations provides the exact optimal value. When n is an odd number, we show that their analysis is tight, i.e., we prove that \(\frac{n+2d-1}{2}\) levels of the SoS/Lasserre hierarchy are also necessary. Laurent [ 24 ] showed that the Sherali-Adams hierarchy requires n levels to detect the empty integer hull of a linear representation of a set with no integral points. She conjectured that the SoS/Lasserre rank for the same problem is n -1. In this article, we disprove this conjecture and derive lower and upper bounds for the rank.

Neural network to solve fuzzy constraint satisfaction problems

<p>It has been proven that solving the constraint satisfaction problem (CSP) is an No Polynomial hard combinatorial optimization problem. This holds true even in cases where the constraints are fuzzy, known as fuzzy constraint satisfaction problems (FCSP). Therefore, the continuous Hopfield neural network model can be utilized to resolve it. The original algorithm was developed by Talaavan in 2005. Many practical problems can be represented as a FCSP. In this paper, we expand on a neural network technique that was initially developed for solving CSP and adapt it to tackle problems that involve at least one fuzzy constraint. To validate the enhanced effectiveness and rapid convergence of our proposed approach, a series of numerical experiments are carried out. The results of these experiments demonstrate the superior performance of the new method. Additionally, the experiments confirm its fast convergence. Specifically, our study focuses on binary instances with ordinary constraints to test the proposed resolution model. The results confirm that both the proposed approaches and the original continuous Hopfield neural network approach exhibit similar performance and robustness in solving ordinary constraint satisfaction problems.</p>

Convex hulls of monomial curves, and a sparse positivstellensatz

AbstractConsider the closed convex hull K of a monomial curve given parametrically as $$(t^{m_1},\ldots ,t^{m_n})$$ ( t m 1 , … , t m n ) , with the parameter t varying in an interval I. We show, using constructive arguments, that K admits a lifted semidefinite description by $$\mathcal {O}(d)$$ O ( d ) linear matrix inequalities (LMIs), each of size $$\left\lfloor \frac{n}{2} \right\rfloor +1$$ n 2 + 1 , where $$d= \max \{m_1,\ldots ,m_n\}$$ d = max { m 1 , … , m n } is the degree of the curve. On the dual side, we show that if a univariate polynomial p(t) of degree d with at most $$2k+1$$ 2 k + 1 monomials is non-negative on $${\mathbb {R}}_+$$ R + , then p admits a representation $$p = t^0 \sigma _0 + \cdots + t^{d-k} \sigma _{d-k}$$ p = t 0 σ 0 + ⋯ + t d - k σ d - k , where the polynomials $$\sigma _0,\ldots ,\sigma _{d-k}$$ σ 0 , … , σ d - k are sums of squares and $$\deg (\sigma _i) \le 2k$$ deg ( σ i ) ≤ 2 k . The latter is a univariate positivstellensatz for sparse polynomials, with non-negativity of p being certified by sos polynomials whose degree only depends on the sparsity of p. Our results fit into the general attempt of formulating polynomial optimization problems as semidefinite problems with LMIs of small size. Such small-size descriptions are much more tractable from a computational viewpoint.

Calculating Global Minimum Points to Binary Polynomial Optimization Problem: Optimizing the Optimal PMU Localization Problem as a Case-Study

State estimation (SE) is an algorithmic function of an energy management system (EMS). SE provides an actual-time monitoring and control of modern electrical power grids. State Estimation can be worked with sufficiency using Phasor Measurement Units optimally placed within a power grid. This paper concerns the implementation of proper algorithms embedded in optimization solvers to the optimal PMU localization problem solving globally. The optimization model is formulated as a 0 - 1 nonlinear minimization problem. The problem is transformed to a polyhedron using linearization methods and B&B tree. In this model, we use a linear cost function under polynomial constraints and binary restrictions on the design variables in a symbolic format. This mathematical model is programmed in the YALMIP environment which is fully compatible with MATLAB. The 0 - 1 Nonlinear Programming (NLP) model is suitable for getting concisely global optimal solutions. The optimal solution is given by a wrapped optimization engine including a local optimizer routine performing together with a mixed-Integer-Linear Programming routine. The solution is achieved within a zero-gap precisely encountered during the iterative process. This tolerance criterion is a necessity for a successful implementation of the B&B tree because it ensures global optimality with an acceptance relative gap. The minimization model is implemented in a YALMIP code fully compatible with MATLAB in two stages. Initially, an objective function with one term is minimized to discover a number of sensors for wide-area monitoring, control and state estimator applications. Then, an extra product is considered in the objective to suffice maximum reliability for observing the network buses. The numerical minimization models are applied to standard power networks in the direction to be solved globally.

On semidefinite programming relaxations for a class of robust SOS-convex polynomial optimization problems

On semidefinite programming relaxations for a class of robust SOS-convex polynomial optimization problems

The Distance to Cubic Symmetry Class as a Polynomial Optimization Problem

The Distance to Cubic Symmetry Class as a Polynomial Optimization Problem

Formal Synthesis of Neural Barrier Certificates for Continuous Systems via Counterexample Guided Learning

This paper presents a novel approach to safety verification based on neural barrier certificates synthesis for continuous dynamical systems. We construct the synthesis framework as an inductive loop between a Learner and a Verifier based on barrier certificate learning and counterexample guidance. Compared with the counterexample-guided verification method based on the SMT solver, we design and learn neural barrier functions with special structure, and use the special form to convert the counterexample generation into a polynomial optimization problem for obtaining the optimal counterexample. In the verification phase, the task of identifying the real barrier certificate can be tackled by solving the Linear Matrix Inequalities (LMI) feasibility problem, which is efficient and makes the proposed method formally sound. The experimental results demonstrate that our approach is more effective and practical than the traditional SOS-based barrier certificates synthesis and the state-of-the-art neural barrier certificates learning approach.

Symmetry Adapted Bases for Trigonometric Optimization

We present an algorithm to compute the global minimum of a trigonometric polynomial, when it is invariant under the exponential action of a Weyl group. This is based on a common relaxation technique that leads to a semi-definite program (SDP). It is then shown how to exploit the invariance in order to reduce the number of variables of the SDP and to simplify its structure significantly. This approach complements the one that was proposed as a poster at the recent ISSAC 2022 conference [HMMR22] and later extended to [HMMR23]. In the previous work, we first used the invariance of the objective function to obtain a classical polynomial optimization problem on the orbit space and subsequently relaxed the problem to an SDP. In the present work, we first apply the relaxation and then exploit symmetry. We show that the Weyl group action is induced by an orthogonal representation and describe its isotypic decomposition.

Nash Equilibrium Problems of Polynomials

This paper studies Nash equilibrium problems that are given by polynomial functions. We formulate efficient polynomial optimization problems for computing Nash equilibria. The Moment-sum-of-squares relaxations are used to solve them. Under generic assumptions, the method can find a Nash equilibrium, if there is one. Moreover, it can find all Nash equilibria if there are finitely many ones of them. The method can also detect nonexistence if there is no Nash equilibrium. Funding: J. Nie was supported by the National Science Foundation [Grant DMS-2110780].

To improve the performance of genetic algorithms by using a novel selection operator

The Genetic Algorithm (GA) was developed as a search engine for difficult non-deterministic polynomial optimization problems. However, it suffers from internal weaknesses, such as premature convergence and low computation efficiency. One critical aspect of the GA is the selection process, which determines new paths and ultimately guides the algorithm towards a solution. This paper details a novel selection procedure that is a perfect blend of the two extremes, namely exploitation and exploration. The proposed technique eliminates the fitness scaling problem by changing the selection pressure continuously during the selection stage. Utilizing traveling salesman problem library (TSPLIB) instances, a performance comparison of the proposed method with a few traditional selection methods was conducted, and the proposed strategy yielded much better outcomes in the form of standard deviations and mean values. A two-sided t-test was also developed, and the results revealed that the proposed strategy enhanced the performance of a GA substantially.

Finding straight line generators through the approximate synthesis of symmetric four-bar coupler curves

The approximate path synthesis of four-bar linkages with symmetric coupler curves is presented. This includes the formulation of a polynomial optimization problem, a characterization of the maximum number of critical points, a complete numerical solution by homotopy continuation, and application to the design of straight line generators. Our approach specifies a desired curve and finds several optimal four-bar linkages with a coupler trace that approximates it. The objective posed simultaneously enforces kinematic accuracy, loop closure, and leads to polynomial first order necessary conditions with a structure that remains the same for any desired trace leading to a generalized result. Ground pivot locations are set as chosen parameters, and it is shown that the objective has a maximum of 73 critical points. The theoretical work is applied to the design of straight line paths. Parameter homotopy runs are executed for 1440 different choices of ground pivots for a thorough exploration. These computations found the expected linkages, namely, Watt, Evans, Roberts, Chebyshev, and other previously unreported linkages which are organized into a 2D atlas using the UMAP algorithm.

Convergences for robust bilevel polynomial programmes with applications

In this paper, we consider a bilevel polynomial optimization problem, where the constraint functions of both the upper-level and lower-level problems involve uncertain parameters. We employ the deterministic robust optimization approach to examine the bilevel polynomial optimization problem under data uncertainties by providing lower bound approximations and convergences of sum-of-squares (SOS) relaxations for the robust bilevel polynomial optimization problem. More precisely, we show that under the convexity of the lower-level problem and either the boundedness of the feasible set or the coercivity of the objective function, the global optimal values of SOS relaxation problems are lower bounds of the global optimal value of the robust bilevel polynomial problem and they converge to this global optimal value when the degrees of SOS polynomials in the relaxation problems tend to infinity. Moreover, an application to an electric vehicle charging scheduling problem with renewable energy sources demonstrates that using the proposed SOS relaxation schemes, we obtain more stable optimal values than applying a direct solution approach as the SOS relaxations are capable of solving these models involving data uncertainties in dynamic charging price and weather conditions.

Learning for Spatial Branching: An Algorithm Selection Approach

The use of machine learning techniques to improve the performance of branch-and-bound optimization algorithms is a very active area in the context of mixed integer linear problems, but little has been done for nonlinear optimization. To bridge this gap, we develop a learning framework for spatial branching and show its efficacy in the context of the Reformulation-Linearization Technique for polynomial optimization problems. The proposed learning is performed offline, based on instance-specific features and with no computational overhead when solving new instances. Novel graph-based features are introduced, which turn out to play an important role for the learning. Experiments on different benchmark instances from the literature show that the learning-based branching rule significantly outperforms the standard rules. History: Accepted by Andrea Lodi, Area Editor/Design & Analysis of Algorithms – Discrete. Funding: This work was supported by Ivey Business School (David G. Burgoyne Faculty Fellowship); FEDER [MTM2014-60191-JIN]; Spanish Ministry of Education [FPU Grant 17/02643, FPU Grant 20/01555]; Conselleria de Cultura, Educacion e Universidade [ED431C 2021/24]; Natural Sciences and Engineering Research Council of Canada [Discovery Grant 2017-04185]; Spanish Ministry of Science and Technology [MTM2017-87197-C3] and Spanish Ministry of Science and Innovation [PID2021-124030NB-C32].

Efficient linear reformulations for binary polynomial optimization problems

We consider unconstrained polynomial minimization problems with binary variables (BPO). These problems can be easily linearized, i.e., reformulated into a MILP in a higher dimensional space. Several linearizations are possible for a given BPO, depending on how each monomial is decomposed and replaced by additional variables and constraints. We focus on finding efficient linearizations that maximize the continuous relaxation bound of the resulting MILP. For this purpose, we introduce the notion of linearization patterns that allow us to model and enumerate the possible decompositions of a degree-d monomial. The assignment of a unique pattern to each monomial of BPO results in a reformulation of BPO into a MILP. Our method, called MaxBound, amounts to searching for an optimal association between monomials and patterns in the sense that it leads to a MILP with the best continuous relaxation bound. We show that this process can be formulated as a MILP which we denote by (MB˜). We further highlight domination properties among the patterns that allow us to discard the dominated patterns and to decrease the size of (MB˜). Another effect of these domination properties is that it now makes sense to search for a reformulation that requires as few additional variables as possible, based only on the non-dominated patterns. We call this reformulation method ND-MinVar and again show that it can be found by solving another MILP. We make an experimental study on degree 4 polynomials that compares the results of both methods and shows the advantages and disadvantages of each.