Stable Spline Research Articles

Numerical investigation of unconditionally stable spline function for three-dimensional time-fractional telegraph equations

Telegraph equations are hyperbolic partial differential equations that may be used to represent reaction–diffusion processes in a variety of engineering and biological disciplines. The development of numerical techniques for telegraph-type equations has received a lot of interest in the literature in recent years. The main objective of this article is to introduce and evaluate a new approach for approximating the time-fractional telegraph equation utilizing spline functions. Initially, an operational matrix method based on the consolidation of block pulse functions and Fibonacci wavelets is presented to derive the solutions to Time-Fractional Telegraph Equations (TFTs). The suggested approach converts the fractional model into an algebraic equation system that can be solved utilizing the Newton iteration method. The Crank Nicolson approach is also offered for the solution of three-dimensional time-fractional telegraph equations using the Trigonometric Quintic B-spline (TQBS). The rationale behind using the collocation method is to select specific collocation spots where the differential equation is fulfilled exactly. The suggested technique combats nonlinearity by employing a quasi-linearization procedure. The time-fractional derivative’s discretization is done using the Caputo fractional derivative formula. The calculated solutions are obtained using a combination of a trigonometric Quintic B-spline and the Caputo fractional derivative. The primary goal is to verify the well-posedness and produce a numerical solution for an initial–boundary value issue for a hyperbolic equation using finite-difference methods. Accordingly, the research developed the exponentially fitted approach for solving initial boundary value problems utilizing finite difference formulae and temporal frequencies. The scheme’s convergence is demonstrated using normal analytical approaches, demonstrating that the method is unconditionally stable and has an order of convergence. MATLAB software is used to run the numerical simulations. Two model examples with boundary layer behaviour are investigated to support the theoretical conclusion. Furthermore, the graphs show that exact and numerical solutions are near together, demonstrating the method’s precision.

Sparse estimation in linear dynamic networks using the stable spline horseshoe prior

Identification of the so-called dynamic networks is one of the most challenging problems appeared recently in control literature. Such systems consist of large-scale interconnected systems, also called modules. To recover full networks dynamics the two crucial steps are topology detection, where one has to infer from data which connections are active, and modules estimation. Since a small percentage of connections are effective in many real systems, the problem finds also fundamental connections with group-sparse estimation. In particular, in the linear setting modules correspond to unknown impulse responses expected to have null norm but in a small fraction of samples. This paper introduces a new Bayesian approach for linear dynamic networks identification where impulse responses are described through the combination of two particular prior distributions. The first one is a block version of the horseshoe prior, a model possessing important global–local shrinkage features. The second one is the stable spline prior, that encodes information on smooth-exponential decay of the modules. The resulting model is called stable spline horseshoe (SSH) prior. It implements aggressive shrinkage of small impulse responses while larger impulse responses are conveniently subject to stable spline regularization. Inference is performed by a Markov Chain Monte Carlo scheme, tailored to the dynamic context and able to efficiently return the posterior of the modules in sampled form. Numerical studies show that the new approach can accurately reconstruct “line by line” networks dynamics without assuming any knowledge on the topology also when thousands of unknown impulse response coefficients must be inferred from data sets of relatively small size.

Linear system identification using the sequential stabilizing spline algorithm

Paradoxically, even if stability (with its many facets) is the key concept in control, including system stability constraints in an identification procedure is still an unresolved puzzle under many aspects. In fact, also in the linear setting there is no procedure proposed in the literature able to estimate the dominant pole of a dynamic system ensuring stability. In this paper, we design a new algorithm, namely the sequential stabilizing spline (SSS) algorithm, which fills this gap. Our approach determines the system’s stability radius by combining Gaussian regression and stochastic simulation, using stable spline kernels to model the one-step ahead predictor impulse responses. Then, a stable model is recovered through a novel sequential convex optimization procedure whose computational time is orders of magnitude faster than approaches based on linear matrix inequalities (LMI). Numerical experiments are included and show that SSS can return models with high long-term predictive capability. SSS performance is comparable to that achieved by classical prediction error methods equipped with an oracle that has access to the test set to tune the best model order for any prediction horizon.

FRF estimation using multiple kernel-based regularisation

Nonparametric kernel-based modelling of dynamical systems offers important advantages over other nonparametric techniques; the estimate is a continuous function, the model complexity is continuously tuneable, and stability, causality and smoothness are imposed on the impulse response estimate. However, for lightly damped systems, most of the existing kernel-based approaches for estimating the impulse- or frequency response function fail because classical kernels are not appropriate for describing lowly damped resonances. Smoothness is imposed on the entire frequency axis with the diagonal correlated or stable spline kernel, with as a result that resonances cannot be captured well. By introducing the superposition of different kernels, carrying prior knowledge about the resonant poles of the system, we make the kernel-based modelling of lightly-damped systems possible with high-accuracy. We use a frequency domain local rational modelling technique as preprocessing step to determine the most dominant poles, and include these as prior knowledge in the kernels. The performance of the new kernel is demonstrated on a highly resonating simulated system and compared to the state of the art nonparametric frequency domain approaches.

Sample Complexity and Minimax Properties of Exponentially Stable Regularized Estimators

Recent studies have shown how regularization may play an important role in linear system identification. An effective approach consists of searching for the impulse response in a high-dimensional space, e.g., a reproducing kernel Hilbert space (RKHS). Complexity is then controlled using a regularizer, e.g., the RKHS norm, able to encode smoothness and stability information. Examples are RKHSs induced by the so-called stable spline or tuned-correlated kernels, which contain a parameter that regulates impulse response exponential decay. In this article, we derive nonasymptotic upper bounds on the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _2$</tex-math></inline-formula> error of these regularized schemes and study their optimality in order (in the minimax sense). Under white noise inputs and Gaussian measurement noises, we obtain conditions which ensure the optimal convergence rate for all the class of stable spline estimators and several generalizations. Theoretical findings are then illustrated via a numerical experiment.

Learning linear modules in a dynamic network using regularized kernel-based methods

In order to identify one system (module) in an interconnected dynamic network, one typically has to solve a Multi-Input-Single-Output (MISO) identification problem that requires identification of all modules in the MISO setup. For application of a parametric identification method this would require estimating a large number of parameters, as well as an appropriate model order selection step for a possibly large scale MISO problem, thereby increasing the computational complexity of the identification algorithm to levels that are beyond feasibility. An alternative identification approach is presented employing regularized kernel-based methods. Keeping a parametric model for the module of interest, we model the impulse response of the remaining modules in the MISO structure as zero mean Gaussian processes (GP) with a covariance matrix (kernel) given by the first-order stable spline kernel, accounting for the noise model affecting the output of the target module and also for possible instability of systems in the MISO setup. Using an Empirical Bayes (EB) approach the target module parameters are estimated through an Expectation–Maximization (EM) algorithm with a substantially reduced computational complexity, while avoiding extensive model structure selection. Numerical simulations illustrate the potentials of the introduced method in comparison with the state-of-the-art techniques for local module identification.

Image Morphing in Deep Feature Spaces: Theory and Applications

This paper combines image metamorphosis with deep features. To this end, images are considered as maps into a high-dimensional feature space and a structure-sensitive, anisotropic flow regularization is incorporated in the metamorphosis model proposed by Miller and Younes (Int J Comput Vis 41(1):61–84, 2001) and Trouvé and Younes (Found Comput Math 5(2):173–198, 2005). For this model, a variational time discretization of the Riemannian path energy is presented and the existence of discrete geodesic paths minimizing this energy is demonstrated. Furthermore, convergence of discrete geodesic paths to geodesic paths in the time continuous model is investigated. The spatial discretization is based on a finite difference approximation in image space and a stable spline approximation in deformation space; the fully discrete model is optimized using the iPALM algorithm. Numerical experiments indicate that the incorporation of semantic deep features is superior to intensity-based approaches.

Kernel-based identification of non-causal systems with application to inverse model control

Models of inverse systems are commonly encountered in control, e.g., feedforward. The aim of this paper is to address several aspects in identification of inverse models, including model order selection and dealing with unstable inverse systems that originate from inverting non-minimum phase dynamics. A kernel-based regularization framework is developed for identification of non-causal systems. It is shown that ‘unstable’ models can be viewed as bounded, but non-causal, operators. As the main contribution, a range of the required kernels for non-causal systems is developed, including non-causal stable spline kernels. Benefits of the approach are confirmed in an example, including non-causal feedforward control for non-minimum phase systems.

Stable spline identification of linear systems under missing data

A different route to identification of time-invariant linear systems has been recently proposed which does not require committing to a specific parametric model structure. Impulse responses are described in a nonparametric Bayesian framework as zero-mean Gaussian processes. Their covariances are given by the so-called stable spline kernels encoding information on regularity and BIBO stability. In this paper, we demonstrate that these kernels also lead to a new family of radial basis functions kernels suitable to model system components subject to disturbances given by filtered white noise. This novel class, in cooperation with the stable spline kernels, paves the way to a new approach to solve missing data problems in both discrete and continuous-time settings. Numerical experiments show that the new technique may return models more predictive than those obtained by standard parametric Prediction Error Methods, also when these latter exploit the full data set.

Continuous-Time DC Kernel—A Stable Generalized First-Order Spline Kernel

The stable spline (SS) kernel and the diagonal correlated (DC) kernel are two kernels that have been applied and studied extensively for kernel-based regularized LTI system identification. In this note, we show that similar to the derivation of the SS kernel, the continuous-time DC kernel can be derived by applying the same “stable” coordinate change to a “generalized” first-order spline kernel, and thus, can be interpreted as a stable generalized first-order spline kernel. This interpretation provides new facets to understand the properties of the DC kernel. In particular, we derive a new orthonormal basis expansion of the DC kernel and the explicit expression of the norm of the reproducing kernel Hilbert space associated with the DC kernel. Moreover, for the nonuniformly sampled DC kernel, we derive its maximum entropy property and show that its kernel matrix has tridiagonal inverse.

Dynamic characteristics of oxygen consumption

BackgroundPrevious studies have indicated that oxygen uptake (VO_2) is one of the most accurate indices for assessing the cardiorespiratory response to exercise. In most existing studies, the response of VO_2 is often roughly modelled as a first-order system due to the inadequate stimulation and low signal to noise ratio. To overcome this difficulty, this paper proposes a novel nonparametric kernel-based method for the dynamic modelling of VO_2 response to provide a more robust estimation.MethodsTwenty healthy non-athlete participants conducted treadmill exercises with monotonous stimulation (e.g., single step function as input). During the exercise, VO_2 was measured and recorded by a popular portable gas analyser (K4b^2, COSMED). Based on the recorded data, a kernel-based estimation method was proposed to perform the nonparametric modelling of VO_2. For the proposed method, a properly selected kernel can represent the prior modelling information to reduce the dependence of comprehensive stimulations. Furthermore, due to the special elastic net formed by mathcal {L}_1 norm and kernelised mathcal {L}_2 norm, the estimations are smooth and concise. Additionally, the finite impulse response based nonparametric model which estimated by the proposed method can optimally select the order and fit better in terms of goodness-of-fit comparing to classical methods.ResultsSeveral kernels were introduced for the kernel-based VO_2 modelling method. The results clearly indicated that the stable spline (SS) kernel has the best performance for VO_2 modelling. Particularly, based on the experimental data from 20 participants, the estimated response from the proposed method with SS kernel was significantly better than the results from the benchmark method [i.e., prediction error method (PEM)] (76.0pm 5.72 vs 71.4pm 7.24%).ConclusionsThe proposed nonparametric modelling method is an effective method for the estimation of the impulse response of VO2—Speed system. Furthermore, the identified average nonparametric model method can dynamically predict VO_2 response with acceptable accuracy during treadmill exercise.

An empirical Bayes approach to identification of modules in dynamic networks

We present a new method of identifying a specific module in a dynamic network, possibly with feedback loops. Assuming known topology, we express the dynamics by an acyclic network composed of two blocks where the first block accounts for the relation between the known reference signals and the input to the target module, while the second block contains the target module. Using an empirical Bayes approach, we model the first block as a Gaussian vector with covariance matrix (kernel) given by the recently introduced stable spline kernel. The parameters of the target module are estimated by solving a marginal likelihood problem with a novel iterative scheme based on the Expectation–Maximization algorithm. Additionally, we extend the method to include additional measurements downstream of the target module. Using Markov Chain Monte Carlo techniques, it is shown that the same iterative scheme can solve also this formulation. Numerical experiments illustrate the effectiveness of the proposed methods.

Kernel-based SPS

One of the central issues in system identification consists not only in obtaining a good model of the process under study but also an informative confidence interval around it. This problem is often referred to as robust identification in the literature. Following the classical paradigm, one first obtains a model through prediction error minimization. Asymptotic theory is then invoked to extract quality tags from the normal approximation of the estimates’ distribution. This paper proposes an alternative route for robust linear system identification. Our procedure relies on the use of kernel-based regularization for both impulse response estimation and confidence intervals computation. The main novelty is that the kernel is not used to define a Gaussian density for the impulse response but just a prior satisfying some symmetry properties forming the basis of the recently developed sign-perturbed sums (SPS) framework. For system identification, SPS is then combined with the stable spline (SS) kernel to account for impulse response regularity and exponential stability. Numerical experiments show that SS+SPS can provide more accurate confidence intervals than those commonly achieved in the Gaussian regression framework (which, in turn, were already shown to outperform those based on the classical paradigm).

Identification of stable linear systems via the sequential stabilizing spline algorithm

Stability of identified models can be achieved, in the framework of parametric Prediction Error Methods (PEM) by imposing nonlinear constraints on the parameter space. When using regularisation techniques with, for instance, stable spline kernels, stability of the predictor model is guaranteed; however this does not imply stability of the model, nor there is a trivial way to guarantee the identified model is stable. An important approach available in the literature to recover stability relies on linear matrix inequalities (LMI). In particular, by means of overparametrization, a convex optimization problem is formulated. Its solution has to balance adherence to a preliminarily obtained (unconstrained) estimate and system stability according to a design parameter δ that determines dominant poles location. In this paper we propose a new stabilization algorithm. Our approach is non convex but does not require overparametrization and embeds regularization through the use of Gaussian regression and stable spline kernels. It is implemented by a very efficient sequential convex optimization procedure, namely the sequential stabilizing spline (SSS) algorithm. In comparison with LMI, numerical experiments show that SSS can provide more predictive models with a computational time orders of magnitude faster.

Generalized System Identification with Stable Spline Kernels

Regularized least-squares approaches have been successfully applied to linear system identification. Recent approaches use quadratic penalty terms on the unknown impulse response defined by stable ...

A new kernel-based approach to system identification with quantized output data

In this paper we introduce a novel method for linear system identification with quantized output data. We model the impulse response as a zero-mean Gaussian process whose covariance (kernel) is given by the recently proposed stable spline kernel, which encodes information on regularity and exponential stability. This serves as a starting point to cast our system identification problem into a Bayesian framework. We employ Markov Chain Monte Carlo methods to provide an estimate of the system. In particular, we design two methods based on the so-called Gibbs sampler that allow also to estimate the kernel hyperparameters by marginal likelihood maximization via the expectation–maximization method. Numerical simulations show the effectiveness of the proposed scheme, as compared to the state-of-the-art kernel-based methods when these are employed in system identification with quantized data.

Extension of First-Order Stable Spline Kernel to Encode Relative Degree

This paper focuses on the kernel-based system identification methods, which estimate the impulse response of the target system in the Bayesian estimation framework. This paper discusses about continuous-time systems, and proposes a new kernel based on a prior that the relative degree of the target system is higher than or equal to two. Such a prior is identical to a prior on the continuity of the impulse response at time zero. The proposed kernel is an extension of the first-order Stable Spline kernel, which is one of the most famous kernels. Numerical examples are shown to demonstrate the effectiveness of the proposed kernel.

Nonparametric modelling of VO2 response to exercise.

This paper investigates the modelling of oxygen consumption (VO2) response to jogging exercise on treadmill. Unlike most of the previous methods, which often use simple parametric models, e.g., first order linear time invariant model, this study applied a nonparametric kernel based regularised method to estimate VO2 to address the ill-conditioned modelling problem and achieve accurate estimation. In particular, it is worthy to be noted that the selection of kernels will affect the results for different modelling scenarios. Therefore, in this research, both radial basis kernel and stable spline kernel were selected for testing. In order to select the favourable kernel for this system, a simulation related to VO2-jogging speed was carried out. The results of simulation indicated that spline kernel can achieve higher accuracy comparing to radial basis function kernel. Experimentally, the kernel based estimation method and spline kernel were tested using six participants. From the results, an average impulse response is obtained. It showed the VO2 estimation, based on the average finite impulse response, is fitted well to the six observations collected from the participants.

Stable spline deconvolution for dynamic susceptibility contrast MRI.

To present the stable spline (SS) deconvolution method for the quantification of the cerebral blood flow (CBF) from dynamic susceptibility contrast MRI. The SS method was compared with both the block-circulant singular value decomposition (oSVD) and nonlinear stochastic regularization (NSR) methods. oSVD is one of the most popular deconvolution methods in dynamic susceptibility contrast MRI (DSC-MRI). NSR is an alternative approach that we proposed previously. The three methods were compared using simulated data and two clinical data sets. The SS method correctly reconstructed the dispersed residue function and its peak in presence of dispersion, regardless of the delay. In absence of dispersion, SS performs similarly to oSVD and does not correctly reconstruct the residue function and its peak. SS and NSR better differentiate healthy and pathologic CBF values compared with oSVD in all simulated conditions. Using acquired data, SS and NSR provide more clinically plausible and physiological estimates of the residue function and CBF maps compared with oSVD. The SS method overcomes some of the limitations of oSVD, such as unphysiological estimates of the residue function and NSR, the latter of which is too computationally expensive to be applied to large data sets. Thus, the SS method is a valuable alternative for CBF quantification using DSC-MRI data. Magn Reson Med 78:1801-1811, 2017. © 2017 International Society for Magnetic Resonance in Medicine.

Sparse plus low rank network identification: A nonparametric approach

Modeling and identification of high-dimensional stochastic processes is ubiquitous in many fields. In particular, there is a growing interest in modeling stochastic processes with simple and interpretable structures. In many applications, such as econometrics and biomedical sciences, it seems natural to describe each component of that stochastic process in terms of few factor variables, which are not accessible for observation, and possibly of few other components of the stochastic process. These relations can be encoded in graphical way via a structured dynamic network, referred to as “sparse plus low-rank (S + L) network” hereafter. The problem of finding the S + L network as well as the dynamic model can be posed as a system identification problem. In this paper, we introduce two new nonparametric methods to identify dynamic models for stochastic processes described by a S + L network. These methods take inspiration from regularized estimators based on recently introduced kernels (e.g. “stable spline”, “tuned-correlated” etc.). Numerical examples show the benefit to introduce the S + L structure in the identification procedure.