Differential Constraints Research Articles

A Nested Decomposition Method for the AC Optimal Power Flow of Hierarchical Electrical Power Grids

The interconnected electrical power grids are operated by multilevel control centers with hierarchical architecture. Since large-scale renewables are integrated in different voltage levels, the traditional isolated operation of interconnected power grids is uneconomical and meets operational risk due to lack of coordination. Therefore, the coordination between control centers in different levels and areas are indispensable. However, existing distributed approaches may either encounter numerical problems or converge slowly when the nonlinear AC optimal power flow (ACOPF) models are applied. Moreover, the total iteration number presents an exponential increase with the levels of hierarchical grids. In order to improve computational efficiency, this paper proposes a nested decomposition method for the coordinated operation of multilevel ACOPF problem, which has superlinear convergence and can achieve the optimal solution (KKT point). During each iteration, a projection function, which embodies the optimal objective value of a lower level power grid projected onto its boundary variable space, is computed with second-order exactness. Thus, the proposed method can be applied to nonlinear continuous optimizations with high efficiency. The paper also provides a rigorous proof for its convergence under the condition that the lower level optimization problems are convex with continuously differentiable objectives and constraints. Numerical tests are conducted with three trilevel power grid of different scales, which verify that the computational efficiency and scalability of the proposed algorithm are superior to those of existing methods.

A New Stopping Criterion for Krylov Solvers Applied in Interior Point Methods

When an iterative method is applied to solve the linear equation system in interior point methods (IPMs), the attention is usually placed on accelerating their convergence by designing appropriate preconditioners, but the linear solver is applied as a black box with a standard termination criterion which asks for a sufficient reduction of the residual in the linear system. Such an approach often leads to an unnecessary 'oversolving' of linear equations. In this paper, an IPM that relies on an inner termination criterion not based on the residual of the linear system is introduced and analyzed. Moreover, new indicators for the early termination of the inner iterations are derived from a deep understanding of IPM needs. The new technique has been adapted to the Conjugate Gradient (CG) and to the Minimum Residual method (MINRES) applied in the IPM context. The new criterion has been tested on a set of quadratic optimization problems including compressed sensing, image processing and instances with partial differential equation constraints, and it has been compared to standard residual tests with variable tolerance. Evidence gathered from these computational experiments shows that the new technique delivers significant improvements in terms of inner (linear) iterations and those translate into significant savings of the IPM solution time.

KDF: Kinodynamic Motion Planning via Geometric Sampling-Based Algorithms and Funnel Control

We integrate sampling-based planning techniques with funnel-based feedback control to develop KDF, a new framework for solving the kinodynamic motion-planning problem via funnel control. The considered systems evolve subject to complex, nonlinear, and uncertain dynamics (also known as differential constraints). First, we use a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">geometric</i> planner to obtain a high-level safe path in a user-defined extended free space. Second, we develop a low-level funnel control algorithm that guarantees safe tracking of the path by the system. Neither the planner nor the control algorithm uses information on the underlying dynamics of the system, which makes the proposed scheme easily distributable to a large variety of different systems and scenarios. Intuitively, the funnel control module is able to implicitly accommodate the dynamics of the system, allowing hence the deployment of purely geometrical motion planners. Extensive computer simulations and hardware experiments with a 6-DOF robotic arm validate the proposed approach.

A Data-Driven Approach to Viscous Fluid Mechanics: The Stationary Case

We introduce a data-driven approach to the modelling and analysis of viscous fluid mechanics. Instead of including constitutive laws for the fluid’s viscosity in the mathematical model, we suggest directly using experimental data. Only a set of differential constraints, derived from first principles, and boundary conditions are kept of the classical PDE model and are combined with a data set. The mathematical framework builds on the recently introduced data-driven approach to solid-mechanics (Kirchdoerfer and Ortiz in Comput Methods Appl Mech Eng 304:81–101, 2016; Conti et al. in Arch Ration Mech Anal 229:79–123, 2018). We construct optimal data-driven solutions that are material model free in the sense that no assumptions on the rheological behaviour of the fluid are made or extrapolated from the data. The differential constraints of fluid mechanics are recast in the language of constant rank differential operators. Adapting abstract results on lower-semicontinuity and {mathscr {A}}-quasiconvexity, we show a Gamma -convergence result for the functionals arising in the data-driven fluid mechanical problem. The theory is extended to compact nonlinear perturbations, whence our results apply not only to inertialess fluids but also to fluids with inertia. Data-driven solutions provide a new relaxed solution concept. We prove that the constructed data-driven solutions are consistent with solutions to the classical PDEs of fluid mechanics if the data sets have the form of a monotone constitutive relation.

Variational problem of adaptive optimal control. Theoretical and applied computer analysis

The problem of adaptive optimal control of a dynamical system, which belongs to the class of conditional variational problems with moving boundaries, is considered. A variational and computer study of the controlled adaptive motion of a material point is carried out for the problem of the energy quality functional minimizing with a moving, not predetermined right transboundary and in the case when the mass of the point changes depending on the unfixed final time. The problem is solved using the schemes and procedures of the classical calculus of variations, as well as adaptive estimation techniques, including the derivation of the variation of the auxiliary quality functional, the corresponding Euler equations, and the adaptive estimation algorithm. When solving a general conditional variational problem, the obtained closed system of differential equations was studied for the formation of an adaptive optimal control system for a dynamic plant with a given performance functional. The results of the unconditional formulation of the problem are generalized to the case of additional differential (nonholonomic) and holonomic constraints. In a variational adaptive optimal control problem, the transversality condition is formulated in terms of the local programming condition. The developed variational scheme of adaptive optimal synthesis can be used in the calculation and design of controlled dynamic systems. This optimization scheme is also promising for use in systems where operating time is non-fixed in advance. The results achieved in this paper concern obtaining specific equations, expressions, and formulas relative to the model example under study and finding graphs of the main time functions that determine the nature of the movement of the control object and the quality of the corresponding transients. The proposed adaptive optimal control algorithms for purposeful movement of the studied material point were tested in digital mode and showed their effectiveness which makes them promising for further use in more complex nonlinear adaptive systems of dynamic optimal control.

Quantum kernel methods for solving regression problems and differential equations

We propose several approaches for solving regression problems and differential equations (DEs) with quantum kernel methods. We compose quantum models as weighted sums of kernel functions, where variables are encoded using feature maps and model derivatives are represented using automatic differentiation of quantum circuits. While previously quantum kernel methods primarily targeted classification tasks, here we consider their applicability to regression tasks, based on available data and differential constraints. We use two strategies to approach these problems. First, we devise a mixed model regression with a trial solution represented by kernel-based functions, which is trained to minimize a loss for specific differential constraints or datasets. Second, we use support vector regression that accounts for the structure of differential equations. The developed methods are capable of solving both linear and nonlinear systems. Contrary to prevailing hybrid variational approaches for parametrized quantum circuits, we perform training of the weights of the model classically. Under certain conditions this corresponds to a convex optimization problem, which can be solved with provable convergence to global optimum of the model. The proposed approaches also favor hardware implementations, as optimization only uses evaluated Gram matrices, but require a quadratic number of function evaluations. We highlight trade-offs when comparing our methods to those based on variational quantum circuits such as the recently proposed differentiable quantum circuits approach. The proposed methods offer potential quantum enhancement through the rich kernel representations using the power of quantum feature maps, and start the quest towards provably trainable quantum DE solvers.

Improved Bayesian Optimization Framework for Inverse Thermal Conductivity Based on Transient Plane Source Method

In order to reduce the errors caused by the idealization of the conventional analytical model in the transient planar source (TPS) method, a finite element model that more closely represents the actual heat transfer process was constructed. The average error of the established model was controlled at below 1%, which was a significantly better result than for the analytical model, which had an average error of about 5%. Based on probabilistic optimization and heuristic optimization algorithms, an optimization model of the inverse heat transfer problem with partial thermal conductivity differential equation constraints was constructed. A Bayesian optimization algorithm with an adaptive initial population (BOAAIP) was proposed by analyzing the influencing factors of the Bayesian optimization algorithm upon inversion. The improved Bayesian optimization algorithm is not affected by the range and individuals of the initial population, and thus has better adaptability and stability. To further verify its superiority, the Bayesian optimization algorithm was compared with the genetic algorithm. The results show that the inversion accuracy of the two algorithms is around 3% when the thermal conductivity of the material is below 100 Wm-1K-1, and the calculation speed of the improved Bayesian optimization algorithm is three to four times faster than that of the genetic algorithm.

Mutational Bias and Natural Selection Driving the Synonymous Codon Usage of Single-Exon Genes in Rice (Oryza sativa L.)

The relative abundance of single-exon genes (SEGs) in higher plants is perplexing. Uncovering the synonymous codon usage pattern of SEGs will benefit for further understanding their underlying evolutionary mechanism in plants. Using internal correspondence analysis (ICA), we reveal a significant difference in synonymous codon usage between SEGs and multiple-exon genes (MEGs) in rice. But the effect is weak, accounting for only 2.61% of the total codon usage variability. SEGs and MEGs contain remarkably different base compositions, and are under clearly differential selective constraints, with the former having higher GC content, and evolving relatively faster during evolution. In the group of SEGs, the variability in synonymous codon usage among genes is partially due to the variations in GC content, gene function, and gene expression level, which accounts for 22.03%, 5.99%, and 3.32% of the total codon usage variability, respectively. Therefore, mutational bias and natural selection should work on affecting the synonymous codon usage of SEGs in rice. These findings may deepen our knowledge for the mechanisms of origination, differentiation and regulation of SEGs in plants.

T-norms driven loss functions for machine learning

Injecting prior knowledge into the learning process of a neural architecture is one of the main challenges currently faced by the artificial intelligence community, which also motivated the emergence of neural-symbolic models. One of the main advantages of these approaches is their capacity to learn competitive solutions with a significant reduction of the amount of supervised data. In this regard, a commonly adopted solution consists of representing the prior knowledge via first-order logic formulas, then relaxing the formulas into a set of differentiable constraints by using a t-norm fuzzy logic. This paper shows that this relaxation, together with the choice of the penalty terms enforcing the constraint satisfaction, can be unambiguously determined by the selection of a t-norm generator, providing numerical simplification properties and a tighter integration between the logic knowledge and the learning objective. When restricted to supervised learning, the presented theoretical framework provides a straight derivation of the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vanishing gradient problem in very deep structures. However, the proposed learning formulation extends the advantages of the cross-entropy loss to the general knowledge that can be represented by neural-symbolic methods. In addition, the presented methodology allows the development of novel classes of loss functions, which are shown in the experimental results to lead to faster convergence rates than the approaches previously proposed in the literature.

Riemann problems for generalized gas dynamics

In this paper, a few classes of exact solutions are obtained using the differential constraints method for generalized gas dynamics equations. The solutions to Riemann problems for two different kinds of initial data are determined with a complete characterization of the solutions through shock waves and/or rarefaction waves.

Embedding hard physical constraints in neural network coarse-graining of three-dimensional turbulence

We demonstrate an approach to enforce mass conservation constraints for three-dimensional incompressible turbulence inside the convolutional neural network architecture. Our method shows increased interpretability and adheres to periodic boundary conditions, while showing high accuracy. This approach is generic for differential constraints L of the form L(V) = G, and can be extended to different applications and neural network architectures.

Knowledge-informed generative adversarial network for functional calibration of computer models

Functional model calibration considers the input dependency of model parameters by explicitly considering them as functional calibration parameters. In this study, a novel knowledge-informed generative adversarial network (KI-GAN) is proposed for the functional calibration of computer models under uncertainty. The proposed KI-GAN leverages the superior distribution learning ability of generative models for uncertainty quantification of model parameters and uses prior knowledge to inform generative model training by formulating a knowledge-based loss function that can easily incorporate different types of physical and expert knowledge of the computer model and its parameters. Two experiments are presented to demonstrate the effectiveness of the proposed KI-GAN for calibrating the functional model parameters under various scenarios. The results indicate that the proposed KI-GAN can effectively encode various types of prior knowledge on the computer model and its parameters, such as partial differential equation constraints and derivative constraints, which can significantly improve the parameter estimation accuracy and reduce estimation uncertainty. Furthermore, with appropriate constraints applied, the proposed KI-GAN can estimate model parameters with reasonable accuracy even under sparse and noisy observations and quantify the underlying uncertainties. Finally, the trained generators can accurately infer the probabilistic distributions of unobserved model outputs without running the computer model.

Multivariate density estimation from privatised data: universal consistency and minimax rates

We revisit the classical problem of nonparametric density estimation but impose local differential privacy constraints. Under such constraints, the original multivariate data cannot be directly observed, and all estimators are functions of the randomised output of a suitable privacy mechanism. The statistician is free to choose the form of the privacy mechanism, and in this work we propose to add Laplace distributed noise to a discretisation of the location of an observed vector. Based on these randomised data, we propose a novel estimator of the density function, which can be viewed as a privatised version of the well-studied histogram density estimator. Our theoretical results include universal pointwise consistency and strong universal -consistency. In addition, a convergence rate for Lipschitz continuous functions is derived, which is complemented by a matching minimax lower bound. We illustrate the trade-off between data utility and privacy by means of a small simulation study.

A Reachability-Based Spatio-Temporal Sampling Strategy for Kinodynamic Motion Planning

By limiting the planning domain to “ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> Informed Set”, some sampling-based motion planners (SBMP) (e.g., Informed RRT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> , BIT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> ) can solve the geometric motion planning problems efficiently. However, the construction of informed set (IS) will be very challenging, when further differential constraints are considered. For the time-optimal kinodynamic motion planning problem, this paper defines a modified time informed set (MTIS) to limit the planning domain. Due to drawing inspiration from Hamilton-Jacobi-Bellman (HJB) reachability analysis, MTIS, compared with the original TIS, can not only help save the running time of SBMP, but also extend the applicable scope from linear systems to polynomial nonlinear systems with control constrains. On this basis, a spatio-temporal sampling strategy adapted to MTIS is proposed. Firstly, MTIS is used to estimate the optimal cost and the valid tree structure is reused, so that we do not need to provide a solution trajectory in advance. Secondly, this strategy is generic, allowing it to be combined with common SBMP (such as SST, etc.) to accelerate convergence and reduce the memory requirement. Several simulation experiments also demonstrate the effectiveness of proposed method.

On the Lagrange variational problem

We investigate the stationarity of variational integrals evaluated on solutions of a&nbsp;system of differential equations. First, the fundamental concepts are relieved of accidental structures and of hypothetical assumptions. The differential constraints

Quantitative Information Flow Techniques for Studying Optimality in Differential Privacy

Universal optimality for differential privacy seeks to characterise the "best" mechanisms, which are the ones that provide the most utility to some class of consumers whilst simultaneously satisfying a fixed ε-differential privacy constraint. The study of universal optimality was initiated in 2009 in two important papers: the first by Ghosh et al., who discovered a universally optimal mechanism for a large class of consumers within the differential privacy constraint context of "counting queries" on statistical datasets; the second by Brenner and Nissim who found that outside of this context, no such universally optimal mechanisms exist. Since those remarkable results, there have been few advances in this field. In this article we detail some recent work which sheds new light on these results using Quantitative Information Flow (QIF), a mathematical framework for quantifying information leaks under a Bayesian adversarial model. Using standard QIF reasoning we show how the earlier results can be both generalised and extended, opening up new avenues for exploring optimality in a wide variety of differential privacy contexts.

Time-Variation in Online Nonconvex Optimization Enables Escaping From Spurious Local Minima

A major limitation of online algorithms that track the optimizers of time-varying nonconvex optimization problems is that they focus on a specific local minimum trajectory, which may lead to poor spurious local solutions. In this article, we show that the natural temporal variation may help simple online tracking methods find and track time-varying global minima. To this end, we investigate the properties of a time-varying projected gradient flow system with inertia, which can be regarded as the continuous-time limit of (1) the optimality conditions for a discretized sequential optimization problem with a proximal regularization and (2) the online tracking scheme. We introduce the notion of the dominant trajectory and show that the inherent temporal variation could reshape the landscape of the Lagrange functional and help a proximal algorithm escape the spurious local minimum trajectories if the global minimum trajectory is dominant. For a problem with twice continuously differentiable objective function and constraints, sufficient conditions are derived to guarantee that no matter how a local search method is initialized, it will track a time-varying global solution after some time. The results are illustrated on a benchmark example with many local minima.

Using Legendre Spectral Element Method with Split Bregman Method for Solving Optimal Control Problem with Elliptic Partial Differential Equation Constraint

Using Legendre Spectral Element Method with Split Bregman Method for Solving Optimal Control Problem with Elliptic Partial Differential Equation Constraint

Cooperative Differentially Private LQG Control With Measurement Aggregation

When multiple agents solve cooperatively a joint optimal control problem, it is generally beneficial for them to coordinate their control signals. However, such strategies require that the agents share their local measurements, which may be privacy-sensitive. Motivated by this issue, this letter considers the Linear Quadratic Gaussian (LQG) control problem subject to differential privacy constraints. Differential privacy ensures that the published signals of an algorithm are not too sensitive to the data of any single participating agent. We propose a two-stage architecture for differentially private LQG control and show how to optimize it by leveraging a solution that we previously developed for the Kalman filtering problem. The first stage of this architecture is most easily implemented by a coordinator aggregating and perturbing the agents’ measurements appropriately, but it can also be implemented without a trusted aggregator by using a secure sum protocol. Numerical simulations illustrate the performance improvement of this architecture over simpler alternatives such as directly perturbing the agents’ measurements.

Rotary Inverted Pendulum Control using Robust Generalized Dynamic Inversion with Adaptive Neural Estimation

A novel robust generalized dynamic inversion control system with adaptive neural estimation (RGDI-ANE) is designed for trajectory tracking of the under actuated rotary inverted pendulum (RIP). The RGDI control design begins by prescribing a virtual differential constraint dynamics (VCD) that imitates the desired tracking control objectives. The baseline RGDI control law is derived by inverting the VCD using dynamically scaled Generalized Inversion, and then augmenting a sliding mode control (SMC) element to provide robustness against performance deterioration due to dynamic scaling of the Moore-Penrose generalized inverse. Furthermore, an adaptive estimator that is based on radial basis function neural networks (RBF-NN) is adopted to limit the dependency of RGDI control on the RIP mathematical model. The weighting matrices of the RBF-NN are updated using a Lyapunov control function. The closed loop RGDI-ANE control system is shown to guarantee semi-global asymptotic stability. Computer simulations along with experimental tests have been conducted on Quanser's RIP system to validate the performance of RGDI-ANE control, and the results are compared with the results obtained by using the SMC and the linear quadratic control methodologies.