Smart classify and optimize for portfolio construction - a differentiable quadratic programming approach

Scientific machine learning is a burgeoning discipline which blends scientific computing and machine learning.Traditionally, scientific computing focuses on large-scale mechanistic models, usually differential equations, that are derived from scientific laws that simplified and explained phenomena.On the other hand, machine learning focuses on developing non-mechanistic data-driven models which require minimal knowledge and prior assumptions.The two sides have their pros and cons: differential equation models are great at extrapolating, the terms are explainable, and they can be fit with small data and few parameters.Machine learning models on the other hand require "big data" and lots of parameters but are not biased by the scientists ability to correctly identify valid laws and assumptions.However, the recent trend has been to merge the two disciplines, allowing explainable models that are data-driven, require less data than traditional machine learning, and utilize the knowledge encapsulated in centuries of scientific literature.The promise is to fuse a priori domain knowledge which doesn't fit into a "dataset", allow this knowledge to specify a general structure that prevents overfitting, reduces the number of parameters, and promotes extrapolatability, while still utilizing machine learning techniques to learn specific unknown terms in the model.This has started to be used for outcomes like automated hypothesis generation and accelerated scientific simulation.The purpose of this blog post is to introduce the reader to the tools of scientific machine learning, identify how they come together, and showcase the existing open source tools which can help one get started.We will be focusing on differentiable programming frameworks in the major languages for scientific machine learning: C++, Fortran, Julia, MATLAB, Python, and R.We will be comparing two important aspects: efficiency and composability.Efficiency will be taken in the context of scientific machine learning: by now most tools are well-optimized for the giant neural networks found in traditional machine learning, but, as will be discussed here, that does not necessarily make them efficient when deployed inside of differential equation solvers or when mixed with probabilistic programming tools.Additionally, composability is a key aspect of scientific machine learning since our toolkit is not ML in isolation.Our goal is not to do machine learning as seen in a machine learning conference (classification, NLP, etc.), and it's not to do traditional machine learning as applied to scientific data.Instead, we are putting ML models and techniques into the heart of scientific simulation tools to accelerate and enhance them.Our neural networks need to fully integrate with tools that simulate satellites and robotics simulators.They need to integrate with the packages that we use in our scientific work for verifying numerical accuracy, tracking units, estimating uncertainty, and much more.We need our neural networks to play nicely with existing packages for delay

In this work, we obtain the risk-aware controller for a class of risk-sensitive mean field games. We show that the value function satisfying the Hamilton–Jacobi–Bellman equation associated with the risk-sensitive mean field game can be found by solving a stochastic differential game problem between two players. We combine the stochastic game-theoretic differential dynamic programming approach with a second order expansion of the Fokker–Planck equation to solve the forward-backward partial differential equation associated with the mean field game. The proposed algorithm is applied to numerical examples with linear and nonlinear dynamics to show its effectiveness.

With the increase of the operating mileage, a large amount of energy consumption generated by metro systems needs to be taken seriously. One of the effective ways to reduce the energy consumption is to collaboratively optimize the driving strategy and train timetable by considering the regenerative energy (RE). However, the dimensionality and computational time will increase accordingly in optimization as the number of operating trains rises. With the intention of tackling this problem by efficiently reducing dimensionality, the energy-efficient metro train operation problem is optimized in this paper by applying the discrete differential dynamic programming (DDDP) approach. Firstly, the model calculating the net energy consumption that takes into account the RE is formulated. Then, the optimization model will be transformed to a discrete decision problem by using Space-Time-Speed (STS) network methodology, and the corresponding solution will be obtained through the DDDP based algorithm. Finally, two case studies will be conducted in a metro network to illustrate the effectiveness of the proposed approach.

Computational Guidance Using Sparse Gauss-Hermite Quadrature Differential Dynamic Programming

Sanitary sewer systems are fundamental and expensive facilities for controlling water pollution. Optimizing sewer design is a difficult task due to its associated hydraulic and mathematical complexities. Therefore, a genetic algorithm (GA) based approach has been developed. A set of diameters for all pipe segments in a sewer system is regarded as a chromosome for the proposed GA model. Hydraulic and topographical constraints are adopted in order to eliminate inappropriate chromosomes, thereby improving computational efficiency. To improve the solvability of the proposed model, the nonlinear cost optimization model is approximated and transformed into a quadratic programming (QP) model. The system cost, pipe slopes, and pipe buried depths of each generated chromosome are determined using the QP model. A sewer design problem cited in literature has been solved using the GA-QP model. The solution obtained from the GA model is comparable to that produced by the discrete differential dynamic programming approach. Finally, several near-optimum designs produced using the modeling to generate alternative approach are discussed and compared for improving the final design decision.

A hybrid artificial neural network—differential dynamic programming approach for short-term hydro scheduling

The dynamic programming formulation of the forward principle of optimality in the solution of optimal control problems results in a partial differential equation with initial boundary condition whose solution is independent of terminal cost and terminal constraints. Based on this property, two computational algorithms are described. The first-order algorithm with minimum computer storage requirements uses only integration of a system of differential equations with specified initial conditions and numerical minimization in finite-dimensional space. The second-order algorithm is based on the differential dynamic programming approach. Either of the two algorithms may be used for problems with nondifferentiable terminal cost or terminal constraints, and the solution of problems with complicated terminal conditions (e.g., with free terminal time) is greatly simplified.

In this paper, the notion of differential dynamic programming is used to develop new second-order and first-order successive-approximation methods for determining optimal control.The unconstrained, nonlinear control problem is first considered, and a second-order algorithm is developed which has wider application then existing second-variation and second-order algorithms. A new first-order algorithm emerges as a special case of the second-order one. Control inequality constraints are introduced into the problem and a second-order algorithm is devised which is able to solve this constrained problem. It is believed that control constraints have not been handled previously in this way. Again, a first-order algorithm emerges as a special case. The usefulness of the second-order algorithms is illustrated by the computer solution of three control problems.The methods presented in this paper have been extended by the author to solve problems with terminal constraints and implicitly given final time. Details of these procedures are not given in this paper, but the relevant references are cited.

Based on the differential genetic programming, a new design method is proposed for optimal and/or robust controllers of nonlinear systems. First we introduce a new type of the genetic programming (GP), so-called differential GP (DGP), combining GP with an automatic differentiation scheme, which could solve Hamilton-Jacobi-Bellman (HJB)/ Hamilton-Jacobi-Isaacs(HJI)/ Francis-Byrnes-Isidori (FBI) equations. Lastly, the effectiveness of a DGP based design method is demonstrated through some design examples of nonlinear systems

Differential Dynamic Programming is a successive approximation technique, based on Dynamic Programming rather than the Calculus of Variations, for determining optimal control of non-linear systems. In each iteration, the system equations are integrated in forward time using the current nominal control, and accessory equations which yield the coefficients of a linear or quadratic expansion of the cost function in the neighbourhood of the nominal trajectory are integrated in reverse time, yielding an improved control law. This control law is applied to the system equations, producing a new, improved trajectory. By continued iteration, the procedure produces control functions which successively approximate to the optimal control function.

Based on the differential genetic programming, a new design method is proposed for optimal and/or robust controllers of nonlinear systems. First we introduce a new type of the genetic programming (GP), so-called differential GP (DGP), combining GP with an automatic differentiation scheme, which could solve Hamilton-Jacobi-Bellman(HJB)/Hamilton-Jacobi-Isaacs(HJI)/Francis-Byrnes-Isidori (FBI) equations. Lastly, the effectiveness of a DGP based design method is demonstrated through some design examples of nonlinear systems.

A combination of genetic algorithm and discrete differential dynamic programming approach (called GA-DDDP) is proposed and developed to optimize the operation of the multiple reservoir system. The demonstration is carried out through application to the Mae Klong system in Thailand. The objective of optimization is to obtain the optimal operating policies by minimizing the total irrigation deficits during a critical drought year. The performance of the proposed algorithm is compared with the modified genetic algorithm. The results show that the proposed GA-DDDP provides optimal solutions, converging into the same fitness values within a short time. The GA is able to produce satisfactory results that are very close to those obtained from GA-DDDP but required alot more computation time to obtain the precise results. The difficulties in selecting optimal parameters of GA as well as finding a feasible initial trial trajectory of DDDP are significant problems and time-consuming. The significant advantage obtained from GA-DDDP is saving of computational resource as GA-DDDP requires no need for optimizing parameters and deriving feasible initial trial trajectories. Because DDDP is a part of GA-DDDP, the good performance of GA-DDDP is obtained when applied to a small system where numbers of discretizations and variables have no influence to the dimensionality problem of DDDP.

A facility stage discrete differential dynamic programming approach is presented for determining the optimal operating policy for a multiple purpose multiple reservoir system. The stages in the dynamic programming formulation are defined to represent reservoirs and state variables are defined to represent the amounts of release from reservoirs. Discrete differential dynamic programming, which is an iterative technique, is used to solve the dynamic programming problem to determine the optimal operating policy. The optimal operating policy is the policy which provides the maximum returns for the system. The approach is applied to an example problem which has been solved by gradient projection and conjugate gradient techniques.

Quantum optimal control problems are typically solved by gradient-based algorithms such as GRAPE, which suffer from exponential growth in storage with increasing number of qubits and linear growth in memory requirements with increasing number of time steps. Employing QOC for discrete lattices reveals that these memory requirements are a barrier for simulating large models or long time spans. We employ a non-standard differentiable programming approach that significantly reduces the memory requirements at the cost of a reasonable amount of recomputation. The approach exploits invertibility properties of the unitary matrices to reverse the computation during back-propagation. We utilize QOC software written in the differentiable programming framework JAX that implements this approach, and demonstrate its effectiveness for lattice gauge theory.

Deep transform and metric learning network: Wedding deep dictionary learning and neural network