Continuous State Space Model Research Articles

Neural Projection Filter: Learning Unknown Dynamics Driven by Noisy Observations.

In this article, we propose the novel neural stochastic differential equations (SDEs) driven by noisy sequential observations called neural projection filter (NPF) under the continuous state-space models (SSMs) framework. The contributions of this work are both theoretical and algorithmic. On the one hand, we investigate the approximation capacity of the NPF, i.e., the universal approximation theorem for NPF. More explicitly, under some natural assumptions, we prove that the solution of the SDE driven by the semimartingale can be well approximated by the solution of the NPF. In particular, the explicit estimation bound is given. On the other hand, as an important application of this result, we develop a novel data-driven filter based on NPF. Also, under certain condition, we prove the algorithm convergence; i.e., the dynamics of NPF converges to the target dynamics. At last, we systematically compare the NPF with the existing filters. We verify the convergence theorem in linear case and experimentally demonstrate that the NPF outperforms existing filters in nonlinear case with robustness and efficiency. Furthermore, NPF could handle high-dimensional systems in real-time manner, even for the 100-D cubic sensor, while the state-of-the-art (SOTA) filter fails to do it.

Meta-learning-based continuous state-space models for rapid monitoring using heterogeneous grade sources with uneven sampling

In an industrial batch, it is common to produce products with different grades, so only a limited number of data are available for each grade of product. The data are also easily contaminated by external disturbances and noise. The sampling interval would be messy and irregular if contaminated samples were removed. Thus, it is difficult to build models for monitoring the corresponding process. This paper proposes a meta-learning-based functional continuous state space model for fast modeling and monitoring of multi-grade batch processes. It takes advantage of wavelet function properties to effectively handle the irregular sampling issue. In addition, the meta-learning method uses diverse historical multi-grade datasets for modeling. It rapidly establishes an efficient and reliable monitoring model for new running batches with a small number of batch data. The effectiveness of the proposed method is shown through a numerical case and an industrial case in the chemical batch process.

A weak semantic approach to bisimulation metrics in models with nondeterminism and continuous state spaces

Bisimulation metrics are a successful instrument used to estimate the behavioural distance between probabilistic concurrent systems. They have been defined in both discrete and continuous state space models. However, the weak semantics approach, where non-observable actions are abstracted away, has been adopted only in the discrete case. In this paper we fill this gap and provide a weak bisimulation metric for models with continuous state spaces. A technical difficulty is to provide a suitable notion of weak transition, which requires to lift transitions leaving from states to transitions leaving from a continuous distribution over states. We prove that our weak bisimulation metric is non-expansive, thus allowing for compositional reasoning. We prove that systems at distance zero are equated by a suitable notion of probabilistic weak bisimulation. We apply our theory in a case study where continuous distributions derive from the evolution of the physical environment.

Smoothing for continuous dynamical state space models with sampled system coefficients based on sparse kernel learning

A new smoother for a continuous dynamical state space model with sampled system coefficients is proposed. This is completely different from conventional approaches, such as Rauch–Tung–Striebel smoother. In the proposed method, the state vector as a continuous function of time is represented by kernel models. The state process model, namely the differential equation, is treated as part of the measurement model at the sampling instants of the system coefficients. Sparse solution of the kernel weights is obtained through a special regularization strategy called the Lasso estimator. The optimization problem appearing in the Lasso estimation is solved by the fast iterative shrinkage threshold algorithm. The hyperparameters involved, namely the kernel widths and the regularization coefficients, are selected objectively through generalized cross-validation or corrected Akaike information criterion tailored to the Lasso estimator. A simple two-dimension example is employed in the simulation to demonstrate the application and also the performance of the proposed method. It is shown that the proposed method could provide state vector estimates with satisfactory accuracy not only at the sampling instants of the observations but also at any other instants. The sparsity of the solution could also be clearly seen in the experiment. The proposed method can be viewed as an alternative smoothing method, rather than a replacement for conventional smoothers, due to the difficult model tuning and increased computation load.

Robot-assisted flexible needle insertion using universal distributional deep reinforcement learning.

Flexible needle insertion is an important minimally invasive surgery approach for biopsy and radio-frequency ablation. This approach can minimize intraoperative trauma and improve postoperative recovery. We propose a new path planning framework using multi-goal deep reinforcement learning to overcome the difficulties in uncertain needle-tissue interactions and enhance the robustness of robot-assisted insertion process. This framework utilizes a new algorithm called universal distributional Q-learning (UDQL) to learn a stable steering policy and perform risk management by visualizing the learned Q-value distribution. To further improve the robustness, universal value function approximation is leveraged in the training process of UDQL to maximize generalization and connect to diagnosis by adapting fast re-planning and transfer learning. Computer simulation and phantom experimental results show our proposed framework can securely steer flexible needles with high insertion accuracy and robustness. The framework also improves robustness by providing distribution information to clinicians for diagnosis and decision making during surgery. Compared with previous methods, the proposed framework can perform multi-target needle insertion through single insertion point qunder continuous state space model with higher accuracy and robustness.

Continuous state space model of the ITER pulsed power electrical network for stability analysis

The ITER large power supplies of the superconducting magnets and heating and current drive systems, fed by the Pulsed Power Electrical Network, can absorb from the grid active and reactive power up to 500 MW and 950 Mvar respectively. In this paper, a new analytical approach based on the state space formulation is proposed to investigate the dynamic stability of such complex system, including the main power components and their control, with the aim of identifying possible instability phenomena due to interactions among the ac/dc converters, reactive power compensation systems and related controllers. This model describes the dynamics of the main components of the ITER pulsed power supply systems and it is based on small signal approach to tackle the non linearity of the ac/dc conversion and reactive power compensation systems; the discrete phenomena have been approximated by continuous transfer functions, so the linear control theory can be applied for the stability analysis, making this method very effective and fast. Moreover a modular approach is adopted; thus this analytical model can be also easily adapted to other similar electrical power systems.

Discrete Time Approximation of Continuous Time Nonlinear State Space Models

In this paper we discuss the problem of approximating a continuous time nonlinear state space model with an explicit discrete time state space model. Two steps are made to do so. The first one is to replace the classical zero-order-hold (ZOH) excitation signal assumption by a more general concept of low pass (LP) signals. It will be shown that opposed to the ZOH-property, the LP-property is maintained for a wide class of (non)linear systems, including cascaded and closed loop systems. The second step is based on a result from the signal processing theory that states that a BL-signal can be exactly predicted from its past samples by a fixed causal linear filter. This result is generalized to the discrete time integration of LP-signals by introducing an error bound that can be made arbitrarily small by increasing the sample frequency. Both ideas are then combined to bound the approximation errors of an explicit discrete time nonlinear state space approximation for a continuous time nonlinear state space model. The order of the decay of the approximation error as a function of the sample frequency is given. It is shown that eventually the aliasing error is the dominating error, at a cost of a (slightly) increased model complexity. These results are directly applicable to nonlinear system identification, and will be experimentally verified on the identification of a closed loop nonlinear system (the silverbox) using a discrete time nonlinear state space model.

Computation of information rates by means of discrete states density recursion

Starting from the existing works on the computation of information rates of channels with memory, we present a modification of these methods that can be used for continuous-state space models. The principle used for developing this alternative method is based on the partitioning of the continuous-state space into sub-sets, each one representing a state of a trellis, and the association between the states and the probability density functions built and updated on such sub-sets. The paper discusses the cases in which the method provides a computational advantage and it presents numerical results regarding the relevant example of the Wiener phase noise model.

On the Stability of Kalman--Bucy Diffusion Processes

The Kalman-Bucy filter is the optimal state estimator for an Ornstein-Uhlenbeck diffusion given that the system is partially observed via a linear diffusion-type (noisy) sensor. Under Gaussian assumptions, it provides a finite-dimensional exact implementation of the optimal Bayes filter. It is generally the only such finite-dimensional exact instance of the Bayes filter for continuous state-space models. Consequently, this filter has been studied extensively in the literature since the seminal 1961 paper of Kalman and Bucy. The purpose of this work is to review, re-prove and refine existing results concerning the dynamical properties of the Kalman-Bucy filter so far as they pertain to filter stability and convergence. The associated differential matrix Riccati equation is a focal point of this study with a number of bounds, convergence, and eigenvalue inequalities rigorously proven. New results are also given in the form of exponential and comparison inequalities for both the filter and the Riccati flow.

Continuous model in dq frame of Thyristor Controlled Reactors for stability analysis of high power electrical systems

This paper illustrates a formulation of a continuous state space model in dq frame of the Thyristor Controlled Reactor (TCR), which can be used for stability studies by eigenvalue analysis. Differently from the previous approaches, this model is just addressed to reproduce the operation of the non-linear part of the TCR system, without considering shunted capacitor or control system, thus obtaining a characterization of the TCR independent from the application. A typical problem of the continuous models in dq frame of the TCR system is the presence of an un-damped homogenous current component in the solution, which is inconsistent with the real operation; the proposed model solves this problem improving the accuracy. The model performance has been also evaluated in the frequency domain comparing the obtained results with a numerical simulation of the TCR implemented by PSIM program.

Adaptive Model Predictive Vibration Control of a Cantilever Beam with Real-Time Parameter Estimation

This paper presents an adaptive-predictive vibration control system using extended Kalman filtering for the joint estimation of system states and model parameters. A fixed-free cantilever beam equipped with piezoceramic actuators serves as a test platform to validate the proposed control strategy. Deflection readings taken at the end of the beam have been used to reconstruct the position and velocity information for a second-order state-space model. In addition to the states, the dynamic system has been augmented by the unknown model parameters: stiffness, damping constant, and a voltage/force conversion constant, characterizing the actuating effect of the piezoceramic transducers. The states and parameters of this augmented system have been estimated in real time, using the hybrid extended Kalman filter. The estimated model parameters have been applied to define the continuous state-space model of the vibrating system, which in turn is discretized for the predictive controller. The model predictive control algorithm generates state predictions and dual-mode quadratic cost prediction matrices based on the updated discrete state-space models. The resulting cost function is then minimized using quadratic programming to find the sequence of optimal but constrained control inputs. The proposed active vibration control system is implemented and evaluated experimentally to investigate the viability of the control method.

Design of a Novel Fuzzy PI Controller Based on Artificial Bee Colony Algorithm for VSC-HVDC Transmission Link

This paper presents a continuous time state space model for a VSC-HVDC transmission link in the d-q reference frame. An Artificial Bee Colony (ABC) algorithm based Fuzzy proportional plus integral (FPI) controller is proposed for a decoupled back to back VSC-HVDC transmission link single machine power system. Here, the ABC approach is used for selecting the nonlinear variable gain parameters of scaling factors of Fuzzy controller to enhance its performance. Simulation result has been presented to show and validate the efficiency of new approach for damping oscillations over a wide range of operating conditions and compared with the conventional PI controller tuned by Ziggler-Nichol technique, and Fuzzy PI controller without optimization. It has been found that the proposed controller is quite effective in enhancing the power system dynamic stability significantly.

Difference in modes of firing rate modulation between cortical areas

Neurons in vivo exhibit irregular firing, and some aspect of their firing pattern such as the degree of firing irregularity does not fluctuate greatly in time for individual neurons, while it differs among neurons [1]. By analyzing spike trains recorded from a variety of cortical areas, it has been revealed that neurons in motor-related cortical areas fire regularly while neurons in visual and prefrontal cortex fire randomly [2]. Here, the firing irregularity is defined as a variability of consecutive interspike intervals, independent of the firing rate. In those studies, the firing rate has been ignored, because it may fluctuate greatly in time depending on external stimuli or behavioral contexts of an animal. Though the neuronal firing rate may fluctuate with external stimuli, this does not necessarily mean that the rate is extrinsic and perfectly controllable; there could be some uncontrollable intrinsic aspect in the firing rate fluctuation. In this contribution, we go contrary to the above-mentioned studies, and attempt to characterize neuronal spike trains solely in terms of the firing rate, ignoring the firing irregularity. The key feature of neuronal spike trains we now pay attention to is dynamic mode of rate fluctuation, such that the rate either fluctuates smoothly in time or changes abruptly between several stationary states. We compare two different models of a fluctuating rate process for their goodness-of-fit with respect to spike trains recorded from single neurons in several cortical areas of rats. The first model is the state space model constructed under the Empirical Bayes method [3,4]. The generative inhomogeneous Poisson process is defined as the conditional distribution of the spike times ti≡t1,t2,...,tn given the rateλ(t), ptiλ(t)= ∏i=1nλ(ti)exp-∫ 0Tλ(t)dt. We give the prior distribution of λ(t) by incorporating the tendency of the rate to be flat, pλ(t)∝exp-β∫ 0Tdλ/dt2dt. The hyperparameter β is selected by the principle of maximizing the marginal likelihood, and then we obtain the maximum a posteriori (MAP) estimate for the rate λ⌢(t). The second alternative model is the Hidden Markov model (HMM) whose hidden states represent different rate states [5-7]. The number of states is selected with a variation Bayes method. In this way, two different estimations of the firing rate are obtained using the continuous state space model and the discrete HMM. We compare them for each spike train obtained from a neuron, and examine whether the continuous or discrete representation of the state transition is more plausible. Then, we examine how the different preferences are distributed in different cortical areas.

Efficient Gibbs Sampling for Markov Switching GARCH Models

We develop efficient simulation techniques for Bayesian inference on switching GARCH models. Our contribution to existing literature is manifold. First, we discuss different multi-move sampling techniques for Markov Switching (MS) state space models with particular attention to MS-GARCH models. Our multi-move sampling strategy is based on the Forward Filtering Backward Sampling (FFBS) applied to an approximation of MS-GARCH. Another important contribution is the use of multi-point samplers, such as the Multiple-Try Metropolis (MTM) and the Multiple Trial Metropolize Independent Sampler, in combination with FFBS for the MS-GARCH process. In this sense we extend to the MS state space models the work of So (2006) on efficient MTM sampler for continuous state space models. Finally, we suggest to further improve the sampler efficiency by introducing the antithetic sampling of Craiu and Meng (2005) and Craiu and Lemieux (2007) within the FFBS. Our simulation experiments on MS-GARCH model show that our multi-point and multi-move strategies allow the sampler to gain efficiency when compared with single-move Gibbs sampling.

Decentralized PI-D Controller Applied to an Aerothermic Process

Abstract The aerothermic process is a pilot scale heating and ventilation system. It is equipped with a heater grid and a centrifugal blower, fully connected through the Humusoft MF624 data acquisition system for real time control. The interaction between its main variables is considered as challenging for mono-variable controllers. An abrupt change in the ventilator speed might cause an undesirable disturbance in the air temperature representing a factor that must be managed to conserve energy. To annul the effect of this interaction, this paper presents an experimental comparison between three forms of the PID controller: the conventional PID controller, the PI-D controller and its decentralized version. A multi-variable continuous state space model is obtained from on-line experimental data. The outcome of the experimental results is that the main control objectives, such as set-point tracking and interactions rejection, are well achieved for the temperature and the air flow simultaneously.

New Method for Parameter Estimation in Probabilistic Models: Minimum Probability Flow

Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function. We propose a new parameter fitting method, minimum probability flow (MPF), which is applicable to any parametric model. We demonstrate parameter estimation using MPF in two cases: a continuous state space model, and an Ising spin glass. In the latter case, MPF outperforms current techniques by at least an order of magnitude in convergence time with lower error in the recovered coupling parameters.

Identification of a continuous time nonlinear state space model for the external power system dynamic equivalent by neural networks

Based on the concept of the external power system dynamic equivalent obtained for the study system, in this paper a reduced-order artificial neural network is proposed, to construct a model for the external part. The mastermind behind the proposed method is to identify the external part as a dynamic–algebraic ANN, and this separation between dynamic equations in the state space and algebraic equations is useful to solve the prediction problem. Moreover, using similarity transformations, the state space model can be simplified, such that all the nonlinearities are embedded in the algebraic part. Since usually the study system equations are available in the continuous time domain, the external part is converted to the continuous time domain by a novel method. To obtain this model, the system should be excited first by a sort of random disturbances, and then data measured on the boundary nodes is used to identify the model. Identification process is accomplished by training the proposed network which can be used to predict behavior of the external system with a high degree of accuracy. Such an equivalent has wide applications for dynamic stability studies.

Automata control systems

A framework is developed for the control design and stability analysis of state-feedback systems made out of automaton–controller pairs, here referred to as automata control systems. A single theorem, based on the Bellman–Ford algorithm, provides the conditions for the design of the controllers that make a given automaton optimal and stable. The automata approximation of continuous state-space models is also developed for the design of state-feedback controllers that can drive continuous plants. The approximation of continuous plants through automata makes the design of state-feedback controllers independent of the state-space description. No distinction is made in the treatment of linear and nonlinear plants. Controller synthesis and specification of the domains of attraction for the resulting plant–controller pair are systematically obtained for continuous time-invariant state-space models. The application of this framework for the stability analysis of the exact model of a digital filter is presented. The automata approximation is applied to design a single controller that stabilises a forced pendulum around two equilibria. The design of switching controllers using automata approximation is also developed and applied to the longitudinal motion control of an aircraft.

Continuous Time Regime Switching Models and Applications in Estimating Processes with Stochastic Volatility and Jumps

A regime switching model in continuous time is introduced where a variety of jumps are allowed in addition to the diffusive component. The characteristic function of the process is derived in closed form, and is subsequently employed to create the likelihood function. In addition, standard results of the option pricing literature can be employed in order to compute derivative prices. To this end, the relationship between the physical and the risk adjusted probability measure is explored. The generic relationship between Markov chains and [jump] diffusions is also investigated, and it is shown that virtually any stochastic volatility model model can be approximated arbitrarily well by a carefully chosen continuous time Markov chain. Therefore, the approach presented here can be utilized in order to estimate, filter and carry out option pricing for such continuous state-space models, without the need for simulation based approximations. An empirical example illustrates these contributions of the paper, estimating a stochastic volatility jump diffusion model.

Modeling and smoothing unequally spaced sequence data

The application of the continuous state space model to unequally spaced sequence data is discussed and illustrated. The continuous model implies a discrete model for the observed data. Practical expressions for relevant discrete model quantities are given. These quantities are required for the digital processing of the data and in particular for the application of the Kalman and smoothing filter and related calculations. Applications illustrate the procedures.