Nonlinear Equality Constraints Research Articles

A Stochastic Inexact Sequential Quadratic Optimization Algorithm for Nonlinear Equality-Constrained Optimization

A stochastic algorithm is proposed, analyzed, and tested experimentally for solving continuous optimization problems with nonlinear equality constraints. It is assumed that constraint function and derivative values can be computed but that only stochastic approximations are available for the objective function and its derivatives. The algorithm is of the sequential quadratic optimization variety. Distinguishing features of the algorithm are that it only employs stochastic objective gradient estimates that satisfy a relatively weak set of assumptions (while using neither objective function values nor estimates of them) and that it allows inexact subproblem solutions to be employed, the latter of which is particularly useful in large-scale settings when the matrices defining the subproblems are too large to form and/or factorize. Conditions are imposed on the inexact subproblem solutions that account for the fact that only stochastic objective gradient estimates are employed. Convergence results are established for the method. Numerical experiments show that the proposed method vastly outperforms a stochastic subgradient method and can outperform an alternative sequential quadratic programming algorithm that employs highly accurate subproblem solutions in every iteration. Funding: This material is based upon work supported by the National Science Foundation [Awards CCF-1740796 and CCF-2139735] and the Office of Naval Research [Award N00014-21-1-2532].

Constructing order-2 information granules of linguistic expressions with the aid of the principle of justifiable granularity

To capture collective opinions/evaluations in a collection of individual linguistic expressions, this study proposes an approach to construct order-2 information granules by extending the numerical data-based principles of justifiable granularity to a linguistic data-based one. First, the two key criteria of the principle of justifiable granularity, namely coverage and specificity, are formally defined in the context of order-2 information granules. Second, three order-2 information granules construction models by maximizing the product of coverage and specificity are developed for coping with one-dimensional direct linguistic expressions, linguistic preference relations, and multi-dimensional direct linguistic expressions, respectively. Third, considering that the developed order-2 information granules construction models exhibit a non-linear uncertain objective function and equality constraints, the constrained multi-swarm PSO without velocity is improved with the use of the ε constrained handling technique for effectively solving the models. Case studies on the considered three types of linguistic expressions show the applicability of the proposed models and ensuing algorithm. The superiority of the improved algorithm in handling the proposed models is demonstrated by comparison with the original one. The effectiveness of the proposed models in terms of abnormal corrective ability and balance of coverage and specificity is verified by comparing with a family of Top-n methods. The originality of this study lies in the construction of an operational entity, namely order-2 information granule, that reflects the group opinion without specifying the formalism of individual linguistic expressions, which provides an effective and efficient way to aggregate ubiquitous linguistic information for subsequent computation, reasoning and decision-making.

Chance-Constrained Optimization for a Green Multimodal Routing Problem with Soft Time Window under Twofold Uncertainty

This study investigates a green multimodal routing problem with soft time window. The objective of routing is to minimize the total costs of accomplishing the multimodal transportation of a batch of goods. To improve the feasibility of optimization, this study formulates the routing problem in an uncertain environment where the capacities and carbon emission factors of the travel process and the transfer process in the multimodal network are considered fuzzy. Taking triangular fuzzy numbers to describe the uncertainty, this study proposes a fuzzy nonlinear programming model to deal with the specific routing problem. To make the problem solvable, this study adopts the fuzzy chance-constrained programming approach based on the possibility measure to remove the fuzziness of the proposed model. Furthermore, we use linear inequality constraints to reformulate the nonlinear equality constraints represented by the continuous piecewise linear functions and realize the linearization of the nonlinear programming model to improve the computational efficiency of problem solving. After model processing, we can utilize mathematical programming software to run exact solution algorithms to solve the specific routing problem. A numerical experiment is given to show the feasibility of the proposed model. The sensitivity analysis of the numerical experiment further clarifies how improving the confidence level of the chance constraints to enhance the possibility that the multimodal route planned in advance satisfies the real-time capacity constraint in the actual transportation, i.e., the reliability of the routing, increases both the total costs and carbon emissions of the route. The numerical experiment also finds that charging carbon emissions is not absolutely effective in emission reduction. In this condition, bi-objective analysis indicates the conflicting relationship between lowering transportation activity costs and reducing carbon emissions in routing optimization. The sensitivity of the Pareto solutions concerning the confidence level reveals that reliability, economy, and environmental sustainability are in conflict with each other. Based on the findings of this study, the customer and the multimodal transport operator can organize efficient multimodal transportation, balancing the above objectives using the proposed model.

The Regularization Continuation Method for Optimization Problems with Nonlinear Equality Constraints

The Regularization Continuation Method for Optimization Problems with Nonlinear Equality Constraints

A Review Study of Prime Period Perfect Gaussian Integer Sequences

Prime period sequences can serve as the fundamental tool to construct arbitrary composite period sequences. This is a review study of the prime period perfect Gaussian integer sequence (PGIS). When cyclic group {1,2,…,N−1} can be partitioned into k cosets, where N=kf+1 is an odd prime number, the construction of a degree-(k + 1) PGIS can be derived from either matching the flat magnitude spectrum criterion or making the sequence with ideal periodic autocorrelation function (PACF). This is a systematic approach of prime period N=kf+1 PGIS construction, and is applied to construct PGISs with degrees 1, 2, 3 and 5. However, for degrees larger than 3, matching either the flat magnitude spectrum or achieving the ideal PACF encounters a great challenge of solving a system of nonlinear constraint equations. To deal with this problem, the correlation and convolution operations can be applied upon PGISs of lower degrees to generate new PGISs with a degree of 4 and other higher degrees, e.g., 6, 7, 10, 11, 12, 14, 20 and 21 in this paper. In this convolution-based scheme, both degree and pattern of a PGIS vary and can be indeterminate, which is rather nonsystematic compared with the systematic approach. The combination of systematic and nonsystematic schemes contributes great efficiency for constructing abundant PGISs with various degrees and patterns for the associated applications.

Learning bayesian network parameters from limited data by integrating entropy and monotonicity

High-accuracy parameter learning in Bayesian Networks (BNs) is a key challenge in real-time decision support applications, particularly when the available data are limited. Prior/Expert knowledge was introduced to eliminate the drawbacks of insufficient information; however, this method is subjective. In this study, we explored the use of monotonicity constraints to control the causal relationships between the nodes and their parents-nodes in BNs and proposed a new learning algorithm called global domain monotonicity based on maximum information entropy (GDM-MIE), which was designed for parameter learning in uncertain discrete BNs with nonlinear equality constraints when only finite data are available. In the proposed algorithm, a class of monotonicity is encoded as a constraint on information entropy and the parameter learning problem is transformed into constraints among the node parameters based on a known network. Furthermore, we considered the parameters as uncertain entropy information and discussed the monotonicity among parameters in the global spatial domain, proving the accuracy of the logical relationships of the model, system reliability, and time complexity. Finally, the proposed method was validated using standard BNs, and its performance was analyzed by comparing the proposed method with the existing learning algorithms. The results showed that the proposed method is more accurate and has better Kullback–Leibler divergence. To revalidate the rationality of the proposed method, the Alarm and Asia networks were employed as special cases. The GDM-MIE was found to achieve the intended goal of observing the estimated parameters by closely approximating the original real parameters with a small sample size, indicating that the proposed algorithm can served as an efficient and feasible method for learning Bayesian parameter.

Polyhedral Relaxations for Optimal Pump Scheduling of Potable Water Distribution Networks

The classic pump scheduling or optimal water flow (OWF) problem for water distribution networks (WDNs) minimizes the cost of power consumption for a given WDN over a fixed time horizon. In its exact form, the OWF is a computationally challenging mixed-integer nonlinear program (MINLP). It is complicated by nonlinear equality constraints that model network physics, discrete variables that model operational controls, and intertemporal constraints that model changes to storage devices. To address the computational challenges of the OWF, this paper develops tight polyhedral relaxations of the original MINLP, derives novel valid inequalities (or cuts) using duality theory, and implements novel optimization-based bound tightening and cut generation procedures. The efficacy of each new method is rigorously evaluated by measuring empirical improvements in OWF primal and dual bounds over 45 literature instances. The evaluation suggests that our relaxation improvements, model strengthening techniques, and a thoughtfully selected polyhedral relaxation partitioning scheme can substantially improve OWF primal and dual bounds, especially when compared with similar relaxation-based techniques that do not leverage these new methods. History: Accepted by David Alderson, Area Editor for Network Optimization: Algorithms & Applications. Funding: This work was supported by the U.S. Department of Energy (DOE) Advanced Grid Modeling project, Coordinated Planning and Operation of Water and Power Infrastructures for Increased Resilience and Reliability. Incorporation of the PolyhedralRelaxations Julia package was supported by Los Alamos National Laboratory’s Directed Research and Development program under the project Fast, Linear Programming-Based Algorithms with Solution Quality Guarantees for Nonlinear Optimal Control Problems [Grant 20220006ER]. All work at Los Alamos National Laboratory was conducted under the auspices of the National Nuclear Security Administration of the U.S. DOE, Contract No. 89233218CNA000001. This work was also authored in part by the National Renewable Energy Laboratory, operated by the Alliance for Sustainable Energy, LLC, for the U.S. DOE, Contract No. DE-AC36-08GO28308. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0233 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0233 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Set-based state estimation for discrete-time constrained nonlinear systems: An approach based on constrained zonotopes and DC programming

This paper introduces a novel state estimator designed for discrete-time nonlinear dynamical systems that encompass unknown-but-bounded uncertainties, along with state linear inequality and nonlinear equality constraints. Our algorithm is based on constrained zonotopes (CZs) and employs a DC programming approach (where DC stands for difference of convex functions). Recently, techniques such as mean value extension and first-order Taylor extension have been adapted from zonotopes to facilitate the propagation of CZs across nonlinear mappings. While the resulting algorithms, known as CZMV and CZFO, achieve higher precision compared to the original zonotopic versions, they still exhibit sensitivity to the wrapping and dependency effects inherent in interval arithmetic. These interval-related challenges can be mitigated through the use of DC programming, as it allows for the determination of approximation error bounds by solving optimization problems. One direct advantage of this technique is the elimination of the dependency effect. Our set-membership filter, referred to as CZDC, provides an alternative solution to CZMV and CZFO. To showcase the effectiveness of our proposed approach, we conducted experiments using CZDC on two numerical examples.

A Three-element Risley Prism Continuous Scanning System Based on Linear Constraint

In the three-element Risley prism system, non-linear errors, the infinite solution, the blind zone and the singularity are primary factors that affect the scanning accuracy of the system. In this paper, an accurate analytical model of the three-element Risley prism system is proposed. To reduce the errors of single-wedge nonlinear constraint equation, a dimensionality reduction method constrained by a single wedge is adopted. In addition, a reliable accuracy of 5mm is employed to achieve a full-area scanning of the system, and a single wedge linear constraint equation with an angular increment is established to control singularities. Simulation results show that the wedge rotation angle curve is stable and continuous in the process of inverse solution, and the full coverage of the scanning area can be achieved by assigning scanning accuracy to 5mm, which indicates that the proposed method can effectively solve the problems and achieve full-area scanning in the system.

Six-Degree-of-Freedom Rocket Landing Optimization via Augmented Convex–Concave Decomposition

In this paper an augmented convex–concave decomposition (ACCD) method for treating nonlinear equality constraints in an otherwise convex problem is proposed. This augmentation improves greatly the feasibility of the problem when compared to the original convex–concave decomposition approach. The effectiveness of the ACCD is demonstrated by solving a fuel-optimal six-degree-of-freedom rocket landing problem in atmosphere, subject to multiple nonlinear equality constraints. Compared with known approaches such as sequential convex programming, where conventional linearization (with or without slack variables) is employed to treat those nonlinear equality constraints, it is shown that the proposed ACCD leads to more robust convergence of the solution process and a more interpretable behavior of the sequential convex algorithm. The methodology can also be applied effectively in the case where the determination of discrete controls or decision-making variables needs to be made, such as the on–off use of reaction control system thrusters in the rocket landing problem, without the need for a mixed integer solver. Numerical results are shown for a representative, reusable rocket benchmark problem.

Three-Dimensional Unsteady Axisymmetric Viscous Beltrami Vortex Solutions to the Navier–Stokes Equations

This paper is aimed at eliciting consistency conditions for the existence of unsteady incompressible axisymmetric swirling viscous Beltrami vortices and explicitly constructing solutions that obey the conditions as well as the Navier–Stokes equations. By Beltrami flow, it is meant that vorticity, i.e., the curl of velocity, is proportional to velocity at any local point in space and time. The consistency conditions are derived for the proportionality coefficient, the velocity field and external force. The coefficient, whose dimension is of [length−1], is either constant or nonconstant. In the former case, the well-known exact nondivergent three-dimensional unsteady vortex solutions are obtained by solving the evolution equations for the stream function directly. In the latter case, the consistency conditions are given by nonlinear equations of the stream function, one of which corresponds to the Bragg–Hawthorne equation for steady inviscid flow. Solutions of a novel type are found by numerically solving the nonlinear constraint equation at a fixed time. Time dependence is recovered by taking advantage of the linearity of the evolution equation of the stream function. The proportionality coefficient is found to depend on space and time. A phenomenon of partial restoration of the broken scaling invariance is observed at short distances.

A new Lagrangian-based first-order method for nonconvex constrained optimization

In this paper, we introduce a new form of Lagrangian function and propose a simple first-order primal-dual algorithm for solving nonconvex optimization with nonlinear equality constraints. We show that the algorithm generates bounded primal-dual iterates, and establish the convergence to KKT points under standard assumptions. The key features of the proposed method are: (i) it does not require boundedness assumptions on dual iterates generated by the algorithm as well as the set of multipliers; (ii) it is a single-loop algorithm that does not involve any penalty subproblems.

A Nonsmooth Dynamical Systems Perspective on Accelerated Extensions of ADMM

Recently, there has been great interest in connections between continuous-time dynamical systems and optimization methods, notably in the context of accelerated methods for smooth and unconstrained problems. In this paper we extend this perspective to nonsmooth and constrained problems by obtaining differential inclusions associated to novel accelerated variants of the alternating direction method of multipliers (ADMM). Through a Lyapunov analysis, we derive rates of convergence for these dynamical systems in different settings that illustrate an interesting tradeoff between decaying versus constant damping strategies. We also obtain modified equations capturing fine-grained details of these methods, which have improved stability and preserve the leading order convergence rates. An extension to general nonlinear equality and inequality constraints in connection with singular perturbation theory is provided.

Accurate and efficient constrained molecular dynamics of polymers using Newton's method and special purpose code

In molecular dynamics simulations we can often increase the time step by imposing constraints on bond lengths and bond angles. This allows us to extend the length of the time interval and therefore the range of physical phenomena that we can afford to simulate. We examine the existing algorithms and software for solving nonlinear constraint equations in parallel and we explain why it is necessary to advance the state-of-the-art. We present ILVES-PC, a new algorithm for imposing bond constraints on proteins accurately and efficiently. It solves the same system of differential algebraic equations as the celebrated SHAKE algorithm, but ILVES-PC solves the nonlinear constraint equations using Newton's method rather than the nonlinear Gauss-Seidel method. Moreover, ILVES-PC solves the necessary linear systems using a specialized linear solver that exploits the structure of the protein. ILVES-PC can rapidly solve constraint equations as accurately as the hardware will allow. The run-time of ILVES-PC is proportional to the number of constraints. We have integrated ILVES-PC into GROMACS and simulated proteins of different sizes. Compared with SHAKE, we have achieved speedups of up to 4.9× in single-threaded executions and up to 76× in shared-memory multi-threaded executions. Moreover, ILVES-PC is more accurate than P-LINCS algorithm. Our work is a proof-of-concept of the utility of software designed specifically for the simulation of polymers.

Thermo-elasticity displacement formulation for constrained articulated mechanical systems

This paper proposes an approach for thermal analysis of articulated systems subject to boundary and motion constraints (BMC). The solution framework is designed to capture large temperature fluctuations, significant change in geometry due to reference configuration and deformations, and geometric nonlinearity due to articulated mechanical systems (AMS) large displacements and spinning motion. Thermal-expansion displacements, which do not contribute to rigid-body translations, are determined from thermal stretch of position-gradient vectors using a new sweeping matrix technique designed to eliminate dependence on translational rigid-body modes. A new quadratic thermal-energy kinetic form is defined and used to formulate a dynamic force vector that accounts for thermal transient and inertia effects. Nodal thermal displacements are used in formulating AMS differential/algebraic equations (DAEs), and consequently, thermal stresses due to BMC equations are automatically accounted for based on integration of thermal analysis and Lagrange-D’Alembert principle, which is the foundation of computational multibody system (MBS) algorithms. The approach used in this study for large-displacement constrained and unconstrained thermal expansions is based on multiplicative decomposition of position-gradient matrix, instead of strain additive decomposition, for solution of thermo-elasticity problems. Four configurations are used to define continuum geometry and displacements: straight configuration, reference configuration, thermal-expansion configuration, and current configuration. The proposed approach allows applying thermal loads during constrained large displacements, does not impose restrictions on the choice of thermal coefficients, captures reference-configuration geometry and change in inertia forces due to temperature fluctuations, accounts for thermal displacement in formulating AMS nonlinear constraint equations, and allows for integration with MBS algorithms for the study of a wide range of thermo-elasticity problems.

Spherical Search based constrained optimization algorithm for power flow analysis of islanded microgrids

Conventional power flow (PF) algorithms are ineffective in the droop-regulated islanded microgrids as the slack bus voltage and system operating frequency are presumed constant parameters. Such assumptions are not applicable in the operation of the droop-regulated islanded microgrid. We formulate a novel formulation for islanded microgrids to solve the PF problem as a constrained optimization problem. Non-linear and linear constraints are developed to model the power balance and the various modes of Distributed Generation units (DGs). In islanded microgrids, DGs can be operated in PQ, PV, and droop mode. We propose an optimization algorithm named SS-NR (Spherical Search with Newton–Raphson based repair) for solving the formulated problem. We employ Spherical Search (SS) as a base optimizer to minimize the objective function in this algorithm. Moreover, Newton–Raphson based repair operator is also used within the framework of SS to handle non-linear equality constraints of a PF problem. Then, we compare the performance of the proposed algorithm with the state-of-the-art algorithms of global optimization. The experimental results show that the proposed algorithm performs better than the other contenders in convergence and accuracy. Furthermore, to validate the proposed formulation, we compare SS-NR results with the results of a time-domain simulator and other PF tools. This comparative analysis presents the efficacy of the proposed formulation as well as the proposed algorithm.

Multi-pass Extended Kalman Smoother with Partially-known Constraints for Estimation of Vapor Compression Cycles

State and parameter estimation methodologies have the potential to make a significant impact in the development of broad array of capabilities for widely-used vapor compression cycles, including advanced controls, performance monitoring, data-driven modeling, and deployment of digital twin technologies. However, the nonlinearity and numerical stiffness of large physics-based models of these systems pose challenges for the practical implementation of estimators that must also satisfy the physical state constraints. We present a three-pass fixed-interval smoothing method developed in the extended Kalman estimation formalism that incorporates linear inequality and partially-known nonlinear equality constraints defined in terms of unknown parameters of the system. The smoothing method is demonstrated to have high estimation accuracy during joint state and parameter estimation of the cycle model representing a realistic system that is implemented in Julia language leveraging automatic differentiation capabilities.

Optimal Greedy Control in Reinforcement Learning.

We consider the problem of dimensionality reduction of state space in the variational approach to the optimal control problem, in particular, in the reinforcement learning method. The control problem is described by differential algebraic equations consisting of nonlinear differential equations and algebraic constraint equations interconnected with Lagrange multipliers. The proposed method is based on changing the Lagrange multipliers of one subset based on the Lagrange multipliers of another subset. We present examples of the application of the proposed method in robotics and vibration isolation in transport vehicles. The method is implemented in FRUND-a multibody system dynamics software package.

State Estimation Using SCADA and PMU Measurements for Networks Containing Classic HVDC Links

• Hybrid state estimation (HSE) algorithm based on SCADA and PMUs measurements for AC power systems with embedded classic high voltage direct current (HVDC) links. • AC and DC measurements are modeled independently as functions of AC and DC states, respectively. • AC and DC measurements (states) are processed (estimated) simultaneously. • Coupling of AC and DC states is expressed by a set of three nonlinear equality constraints. This paper proposes a hybrid state estimation (HSE) algorithm which employs conventional measurements received from supervisory control and data acquisition (SCADA) system and synchrophasors measured from phasor measurement units (PMUs) for AC power systems with embedded classic high voltage direct current (HVDC) links. DC measurements obtained from the HVDC link are assumed to be sampled at the same rate as AC measurements. Zero injections and AC/DC coupling equations are modeled as equality constraints. By applying the nonlinear weighted least squares (WLS) algorithm with equality constraints, the HSE calculates both AC and DC states simultaneously. The effectiveness of the proposed algorithm is verified by simulations on the IEEE standard 14 and 30 bus test systems. The test results reveal the benefits of the proposed approach in terms of estimation accuracy and convergence speed.

Improved Technique for Autonomous Vehicle Motion Planning Based on Integral Constraints and Sequential Optimization

The study is dedicated to elaborating and analyzing a technique for autonomous vehicle (AV) motion planning based on sequential trajectory and kinematics optimization. The proposed approach combines the finite element method (FEM) basics and nonlinear optimization with nonlinear constraints. There were five main innovative aspects introduced in the study. First, a 7-degree polynomial was used to improve the continuity of piecewise functions representing the motion curves, providing 4 degrees of freedom (DOF) in a node. This approach allows using the irregular grid for roadway segments, increasing spans where the curvature changes slightly, and reducing steps in the vicinity of the significant inflections of motion boundaries. Therefore, the segment length depends on such factors as static and moving obstacles, average road section curvature, camera sight distance, and road conditions (adhesion). Second, since the method implies splitting the optimization stages, a strategy for bypassing the moving obstacles out of direct time dependency was developed. Thus, the permissible area for maneuvering was determined using criteria of safety distance between vehicles and physical limitation of tire–road adhesion. Third, the nodal inequality constraints were replaced by the nonlinear integral equality constraints. In contrast to the generally distributed approach of restricting the planning parameters in nodes, the technique of integral equality constraints ensures the disposition of motion parameters’ curves strictly within the preset boundaries, which is especially important for quite long segments. In this way, the reliability and stability of predicted parameters are improved. Fourth, the seamless continuity of both the sought parameters and their derivatives is ensured in transitional nodes between the planning phases and adjacent global coordinate systems. Finally, the problem of optimization rapidity to match real-time operation requirements was addressed. For this, the quadrature integration approach was implemented to represent and keep all the parameters in numerical form. The study considered cost functions, limitations stipulated by the vehicle kinematics and dynamics, as well as initial and transient conditions between the planning stages. Simulation examples of the predicted trajectories and curves of kinematic parameters are demonstrated. The advantages and limitations of the proposed approach are highlighted.