Finite-time Transition Research Articles

Optimal Control of Underdamped Systems: An Analytic Approach

Optimal control theory deals with finding protocols to steer a system between assigned initial and final states, such that a trajectory-dependent cost function is minimized. The application of optimal control to stochastic systems is an open and challenging research frontier, with a spectrum of applications ranging from stochastic thermodynamics to biophysics and data science. Among these, the design of nanoscale electronic components motivates the study of underdamped dynamics, leading to practical and conceptual difficulties. In this work, we develop analytic techniques to determine protocols steering finite time transitions at a minimum thermodynamic cost for stochastic underdamped dynamics. As cost functions, we consider two paradigmatic thermodynamic indicators. The first is the Kullback–Leibler divergence between the probability measure of the controlled process and that of a reference process. The corresponding optimization problem is the underdamped version of the Schrödinger diffusion problem that has been widely studied in the overdamped regime. The second is the mean entropy production during the transition, corresponding to the second law of modern stochastic thermodynamics. For transitions between Gaussian states, we show that optimal protocols satisfy a Lyapunov equation, a central tool in stability analysis of dynamical systems. For transitions between states described by general Maxwell-Boltzmann distributions, we introduce an infinite-dimensional version of the Poincaré-Lindstedt multiscale perturbation theory around the overdamped limit. This technique fundamentally improves the standard multiscale expansion. Indeed, it enables the explicit computation of momentum cumulants, whose variation in time is a distinctive trait of underdamped dynamics and is directly accessible to experimental observation. Our results allow us to numerically study cost asymmetries in expansion and compression processes and make predictions for inertial corrections to optimal protocols in the Landauer erasure problem at the nanoscale.

Transition between metastable equilibria: Applications to binary-choice games.

Transitions between metastable equilibria in the low-temperature phase of dynamical Ising game with activity spillover are studied in the infinite time limit. It is shown that exponential enhancement due to activity spillover, which takes place in finite-time transitions, is absent in the infinite time limit. In order to demonstrate that, the analytical description for infinite time trajectory is developed. An analytical approach to estimate the probability of transition between metastable equilibria in the infinite time limit is introduced and its results are compared with those of kinetic Monte Carlo simulation. Our study sheds light on the dynamics of the Ising game and has implications for the understanding of transitions between metastable states in complex systems.

Landau theory for finite-time dynamical phase transitions

We study the time evolution of thermodynamic observables that characterise the dissipative nature of thermal relaxation after an instantaneous temperature quench. Combining tools from stochastic thermodynamics and large-deviation theory, we develop a powerful theory for computing the large-deviation statistics of such observables. Our method naturally leads to a description in terms of a dynamical Landau theory, a versatile tool for the analysis of finite-time dynamical phase transitions. The topology of the associated Landau potential allows for an unambiguous identification of the dynamical order parameter and of the phase diagram. As an immediate application of our method, we show that the probability distribution of the heat exchanged between a mean-field spin model and the environment exhibits a singular point, a kink, caused by a finite-time dynamical phase transition. Using our Landau theory, we conduct a detailed study of the phase transition. Although the manifestation of the new transition is similar to that of a previously found finite-time transition in the magnetisation, the properties and the dynamical origins of the two turn out to be very different.

Fault diagnosis for linear heterodirectional hyperbolic ODE–PDE systems using backstepping-based trajectory planning

This paper is concerned with the fault diagnosis problem for general linear heterodirectional hyperbolic ODE–PDE systems. A systematic solution is presented for additive time-varying actuator, process and sensor faults in the presence of disturbances. The faults and disturbances are represented by the solutions of finite-dimensional ODEs, which allow to take a large class of signals into account. For disturbances, that are only bounded, a threshold for secured fault diagnosis is derived. By applying integral transformations to the system an algebraic fault detection equation to detect faults in finite time is obtained. The corresponding integral kernels result from the realization of a finite-time transition between a non-equilibrium initial state and a vanishing final state of a hyperbolic ODE–PDE system. For this new challenging problem, a systematic trajectory planning approach is presented. In particular, this problem is facilitated by mapping the kernel equations into backstepping coordinates and tracing the solution of the transition problem back to a simple trajectory planning. The fault diagnosis for a 4 × 4 heterodirectional hyperbolic system coupled with a second order ODE demonstrates the results of the paper.

Cluster-based network model

Abstract

Generalized Markov stability of network communities

We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different timescales. The specific implementation of the quality function and the resulting optimal community structure thus become dependent both on the type of Markov process and on the specific Markov times considered. For instance, if we use a natural Markov chain dynamics and discount its stationary distribution (that is, we take as reference process the dynamics at infinite time) we obtain the standard formulation of the Markov stability. Notably, the possibility to use finite-time transition probabilities to define the reference process naturally allows detecting communities at different resolutions, without the need to consider a continuous-time Markov chain in the small time limit. The main advantage of our general formulation of Markov stability based on dynamical flows is that we work with lumped Markov chains on network partitions, having the same stationary distribution of the original process. In this way the form of the quality function becomes invariant under partitioning, leading to a self-consistent definition of community structures at different aggregation scales.

Motion planning for a class of boundary controlled 1D port-Hamiltonian systems

A flatness-based approach for motion planning for a class of boundary controlled port-Hamiltonian systems with distributed parameters is presented. The goal is to achieve open-loop output tracking or finite-time transitions between steady states or operating profiles. Introducing new (fictious) boundary conditions in terms of so-called flat outputs, the port-Hamiltonian system is reformulated as a Cauchy problem in the spatial domain. The parametrization of any system variable and input by the flat output is computed using two different solution approaches. By assigning a suitable desired trajectory for the flat output, the input parametrization yields the feed forward control law to solve the motion planning task. The presented theory is applied to the wave equation with spatially varying parameters and is evaluated by numerical calculations and simulations.

Stochastic control and non-equilibrium thermodynamics: fundamental limits.

We consider damped stochastic systems in a controlled (time-varying) potential and study their transition between specified Gibbs-equilibria states in finite time. By the second law of thermodynamics, the minimum amount of work needed to transition from one equilibrium state to another is the difference between the Helmholtz free energy of the two states and can only be achieved by a reversible (infinitely slow) process. The minimal gap between the work needed in a finite-time transition and the work during a reversible one, turns out to equal the square of the optimal mass transport (Wasserstein-2) distance between the two end-point distributions times the inverse of the duration needed for the transition. This result, in fact, relates non-equilibrium optimal control strategies (protocols) to gradient flows of entropy functionals via the Jordan-Kinderlehrer-Otto scheme. The purpose of this paper is to introduce ideas and results from the emerging field of stochastic thermodynamics in the setting of classical regulator theory, and to draw connections and derive such fundamental relations from a control perspective in a multivariable setting.

Flatness-based feedforward control design and two-degrees-of-freedom tracking control for semilinear plug flow reactors

Flatness-based feedforward control design for semilinear plug flow reactors is considered in combination with an easily tunable PI controller to achieve the tracking of desired output trajectories. The approach is based on a subdivision of the relevant time–space region along the characteristic curves of the underlying partial differential equations and a parameterization of the solution in terms of summable formal power series. An approximation for the solution is retrieved by applying an adequate summation method to the power series. Simulations illustrate the applicability of the approach to achieve finite time transitions between operating points along prescribed transient paths and its robustness with respect to uncertainties in the parameters of the reactor.

Computational methods for birth-death processes.

Many important stochastic counting models can be written as general birth-death processes (BDPs). BDPs are continuous-time Markov chains on the non-negative integers in which only jumps to adjacent states are allowed. BDPs can be used to easily parameterize a rich variety of probability distributions on the non-negative integers, and straightforward conditions guarantee that these distributions are proper. BDPs also provide a mechanistic interpretation - birth and death of actual particles or organisms - that has proven useful in evolution, ecology, physics, and chemistry. Although the theoretical properties of general BDPs are well understood, traditionally statistical work on BDPs has been limited to the simple linear (Kendall) process. Aside from a few simple cases, it remains impossible to find analytic expressions for the likelihood of a discretely-observed BDP, and computational difficulties have hindered development of tools for statistical inference. But the gap between BDP theory and practical methods for estimation has narrowed in recent years. There are now robust methods for evaluating likelihoods for realizations of BDPs: finite-time transition, first passage, equilibrium probabilities, and distributions of summary statistics that arise commonly in applications. Recent work has also exploited the connection between continuously- and discretely-observed BDPs to derive EM algorithms for maximum likelihood estimation. Likelihood-based inference for previously intractable BDPs is much easier than previously thought and regression approaches analogous to Poisson regression are straightforward to derive. In this review, we outline the basic mathematical theory for BDPs and demonstrate new tools for statistical inference using data from BDPs.

Birth/birth-death processes and their computable transition probabilities with biological applications.

Birth-death processes track the size of a univariate population, but many biological systems involve interaction between populations, necessitating models for two or more populations simultaneously. A lack of efficient methods for evaluating finite-time transition probabilities of bivariate processes, however, has restricted statistical inference in these models. Researchers rely on computationally expensive methods such as matrix exponentiation or Monte Carlo approximation, restricting likelihood-based inference to small systems, or indirect methods such as approximate Bayesian computation. In this paper, we introduce the birth/birth-death process, a tractable bivariate extension of the birth-death process, where rates are allowed to be nonlinear. We develop an efficient algorithm to calculate its transition probabilities using a continued fraction representation of their Laplace transforms. Next, we identify several exemplary models arising in molecular epidemiology, macro-parasite evolution, and infectious disease modeling that fall within this class, and demonstrate advantages of our proposed method over existing approaches to inference in these models. Notably, the ubiquitous stochastic susceptible-infectious-removed (SIR) model falls within this class, and we emphasize that computable transition probabilities newly enable direct inference of parameters in the SIR model. We also propose a very fast method for approximating the transition probabilities under the SIR model via a novel branching process simplification, and compare it to the continued fraction representation method with application to the 17th century plague in Eyam. Although the two methods produce similar maximum a posteriori estimates, the branching process approximation fails to capture the correlation structure in the joint posterior distribution.

On Trajectory Planning, Backstepping Controller Design and Sliding Modes in Active Front-Ends

A flatness-based online trajectory planning algorithm for two-level voltage source converters is presented allowing for transitions in finite time. For trajectory tracking, two controllers are derived using a backstepping approach. One of them is a sliding mode controller. A method for decoupling the impact of the switching actions on the sliding surfaces is presented. The flatness of the system is used also for the prediction of the switching frequency. The main results are verified experimentally.

Flatness of a class of linear Volterra partial integro–differential equations

Flatness–based trajectory planning is addressed for a parabolic–like linear Volterra partial integro–differential equation (PIDE) with boundary control. For this purpose the PIDE is formally integrated which results in an implicit state and input parametrization. Successive approximation is used to explicitly resolve the implicit result in terms of a recursion scheme. Absolute and uniform convergence is verified by restricting the flat output trajectory to Gevrey class functions of a particular order. Simulation results confirm the applicability and illustrate the tracking behavior for finite time transitions between operating points along predefined transition paths.

Partitioning algorithms for multi-agent systems based on finite-time proximity metrics

We address a generalized Voronoi partitioning problem for a team of mobile agents with nonlinear dynamics with respect to a state-dependent proximity metric. In particular, the proximity (pseudo-) metric corresponds to the reduction of a generalized energy metric that occurs during the transfer of an agent to an arbitrary destination with zero terminal velocity, in finite time. The realization of every finite-time state transition takes place by means of a class of continuous feedback control laws that render the closed loop dynamics of each mobile agent non-Lipschitzian. The arrival time also turns out to be a state-dependent quantity, whose functional description is not prescribed a priori. We show that the partitioning problem studied in this work can admit a decentralized solution, that is, each agent can compute its own cell independently from its teammates provided that is aware of the positions and velocities of its neighboring agents. Numerical simulations that illustrate the theoretical developments are also presented.

Optimal driving protocols for nano-sized devices and their dependence on couplings to reservoirs

The development of efficient artificial nanodevices poses challenges which are of fundamental and technological nature. Recent progress has been made in the context of finite-time thermodynamics. A central question in finite-time thermodynamics is to identify the optimal procedure to extract the greatest amount of work from a system operating under well-defined constraints. For externally controlled small systems, the optimal driving protocol maximizes the mean work spend in a finite-time transition between two given system states under the constraints of given initial and final energy values, and a fixed total operation time. For simplicity we consider an externally controlled single level system, which is embedded in a thermal environment and coupled to a particle reservoir. The optimal protocols are calculated from a master equation approach for different system-reservoir couplings. For open systems, the system-reservoir couplings are shown to have a striking influence on the optimal driving setup. We point out that the optimal protocols have discontinuous jumps at the initial and final times. Finally, this work provides a first attempt to extend these calculations to larger system sizes.

Semi-numerical trajectory planning for coupled systems of linear diffusion-convection-reaction equations

The trajectory planning problem is considered for coupled linear diffusion-convection-reaction equations using a semi-numerical approach. For this, spatial discretization is applied to determine approximate solutions to the eigenvalue problem, which are processed in a spectral approach for flatness-based trajectory planning and feedforward control. The assignment of suitable desired trajectories for the flat output is considered to realize finite time transitions between operating points as well as transient output trajectories. Simulation results for an example with cross-diffusion document the applicability of the proposed method.

Transition probabilities for general birth–death processes with applications in ecology, genetics, and evolution

A birth-death process is a continuous-time Markov chain that counts the number of particles in a system over time. In the general process with n current particles, a new particle is born with instantaneous rate λ(n) and a particle dies with instantaneous rate μ(n). Currently no robust and efficient method exists to evaluate the finite-time transition probabilities in a general birth-death process with arbitrary birth and death rates. In this paper, we first revisit the theory of continued fractions to obtain expressions for the Laplace transforms of these transition probabilities and make explicit an important derivation connecting transition probabilities and continued fractions. We then develop an efficient algorithm for computing these probabilities that analyzes the error associated with approximations in the method. We demonstrate that this error-controlled method agrees with known solutions and outperforms previous approaches to computing these probabilities. Finally, we apply our novel method to several important problems in ecology, evolution, and genetics.

Flatness-based trajectory planning for diffusion–reaction systems in a parallelepipedon—A spectral approach

Spectral analysis is considered for the flatness-based solution of the trajectory planning problem for a boundary controlled diffusion–reaction system defined on a 1 ≤ r -dimensional parallelepipedon. By exploiting the Riesz spectral properties of the system operator, it is shown that a suitable reformulation of the resolvent operator allows a systematic introduction of a basic output, which yields a parametrization of both the system state and the boundary input in terms of differential operators of infinite order. Their convergence is verified for both infinite-dimensional and finite-dimensional actuator configurations by restricting the basic output to certain Gevrey classes involving non-analytic functions. With this, a systematic approach is introduced for basic output trajectory assignment and feedforward tracking control towards the realization of finite-time transitions between stationary profiles.

Optimal protocols for Hamiltonian and Schrödinger dynamics

For systems in an externally controllable time dependent potential, the optimal protocolminimizes the mean work spent in a finite time transition between given initial and finalvalues of a control parameter. For an initially thermalized ensemble, we consider bothHamiltonian evolution for classical systems and Schrödinger evolution for quantum systems.In both cases, we show that for harmonic potentials, the optimal work is givenby the adiabatic work even in the limit of short transition times. This result iscounter-intuitive because the adiabatic work is substantially smaller than the work for aninstantaneous jump. We also perform numerical calculations for the optimal protocol forHamiltonian dynamics in an anharmonic quartic potential. For a two-level spinsystem, we give examples where the adiabatic work can be reached in either afinite or an arbitrarily short transition time depending on the allowed parameterspace.

Optimal protocols for minimal work processes in underdamped stochastic thermodynamics

For systems in an externally controllable time-dependent potential, the optimal protocol minimizes the mean work spent in a finite-time transition between two given equilibrium states. For overdamped dynamics which ignores inertia effects, the optimal protocol has been found to involve jumps of the control parameter at the beginning and end of the process. Including the inertia term, we show that this feature not only persists but that even delta-peak-like changes of the control parameter at both boundaries make the process optimal. These results are obtained by analyzing two simple paradigmatic cases: First, a Brownian particle dragged by a harmonic optical trap through a viscous fluid and, second, a Brownian particle subject to an optical trap with time-dependent stiffness. These insights could be used to improve free energy calculations via either thermodynamic integration or "fast growth" methods using Jarzynski's equality.