Dynamic Network Identification Research Articles

SYSDYNET - A MATLAB App and Toolbox for Dynamic Network Identification ,

Identification in interconnected systems requires the handling of phenomena that go beyond the classical open-loop and closed-loop type of identification problems. Over the last decade a comprehensive theory has been developed for addressing identification problems in linear dynamic networks, formulated in a module framework, where the network structure is characterized by a directed graph in which nodes are signals and links are transfer functions. The resulting methods and approaches have been collected in a MATLAB App and Toolbox, supported by an attractive graphical user interface that provides an interactive workflow for manipulating the structural properties of dynamic networks, applying basic network operations like immersion and module invariance testing, and for investigating network/module generic identifiability and selecting appropriate predictor model inputs and outputs. The workflow supports the allocation of external excitation signals (actuation) and measured node signals (sensing) so as to achieve generic identifiability and provide consistent estimation of target modules. The Toolbox includes algorithms for actual network simulation and identification.

Integrating data-informativity conditions in predictor models for single module identification in dynamic networks

For consistent identification of a target module in a dynamic network with the local direct method, basically two prime conditions need to be satisfied: (a) a set of structural conditions on the choice of the predictor model, i.e. a set of input and output node variables, and (b) conditions on data-informativity. While for conditions (a) constructive algorithms for node selection have been presented that appropriately guarantee that the identified object can indeed reveal the target module, the requirements for satisfying (b) have not yet been integrated fully. In this paper, we will present simplified path-based results for generic data-informativity, and show how they can be integrated in constructive algorithms for predictor model selection that provide consistent target module estimates. It is shown that data-informativity not only requires a sufficient number of external excitation signals to be present in the network, but also puts restrictions on the structure of the predictor model, i.e. the selection of input and output node variables. Some examples are presented to illustrate the new results.

A frequency domain approach for local module identification in dynamic networks

In classical approaches of dynamic network identification, in order to identify a system (module) embedded in a dynamic network, one has to formulate a Multi-input-Single-output (MISO) identification problem that requires identification of a parametric model for all the modules constituting the MISO setup including (possibly) the noise model, and determine their model order. This requirement leads to model order selection steps for modules that are of no interest to the experimenter which increases the computational complexity for large-sized networks. Also, identification using a parametric noise model (like BJ method) can suffer from local minima, however neglecting the noise model has its impact on the variance of the estimates. In this paper, we provide a two-step identification approach to avoid these issues such that the complexity of the problem is independent of the complexity of the network. The first step involves performing a non-parametric indirect approach for a MISO identification problem to get the non-parametric frequency response function (FRF) estimates and its variance as a function of frequency. In the second step, the estimated non-parametric FRF of the target module is smoothed using a parametric frequency domain estimator with the estimated variance from the previous step as the non-parametric noise model. The developed approach is practical with weak assumptions on noise, uses the available toolbox, requires a parametric model only for the target module of interest, and uses a non-parametric noise model to reduce the variance of the estimates. Numerical simulations illustrate the potentials of the introduced method in comparison with the classical identification methods.

Allocation of Excitation Signals for Generic Identifiability of Linear Dynamic Networks

A recent research direction in data-driven modeling is the identification of dynamic networks, in which measured vertex signals are interconnected by dynamic edges represented by causal linear transfer functions. The major question addressed in this article is where to allocate external excitation signals such that a network model set becomes generically identifiable when measuring all vertex signals. To tackle this synthesis problem, a novel graph structure, referred to as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">directed pseudotree</i> , is introduced, and the generic identifiability of a network model set can be featured by a set of disjoint directed pseudotrees that cover all the parameterized edges of an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">extended graph</i> , which includes the correlation structure of the process noises. Thereby, an algorithmic procedure is devised, aiming to decompose the extended graph into a minimal number of disjoint pseudotrees, whose roots then provide the appropriate locations for excitation signals. Furthermore, the proposed approach can be adapted using the notion of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">antipseudotrees</i> to solve a dual problem, which is to select a minimal number of measurement signals for generic identifiability of the overall network, under the assumption that all the vertices are excited.

Optimal Allocation of Excitation and Measurement for Identification of Dynamic Networks

In this paper, the problem of choosing the best allocation of excitations and measurements for the identification of a dynamic network is formally stated and analyzed. The best choice will be one that achieves the most accurate identification with the least costly experiment. Accuracy is assessed by the trace of the asymptotic covariance matrix of the parameters estimates, whereas the cost criterion is the number of excitations and measurements. Analytical and numerical results are presented for two classes of dynamic networks in state space form: branches and cycles. From these results, a number of guidelines for the choice emerge, which are based either on the topology of the network or on the relative magnitude of the modules being identified. An example is given to illustrate that these guidelines can to some extent be applied to networks of more generic topology.

Handling unmeasured disturbances in data-driven distributed control with virtual reference feedback tuning

The data-driven synthesis of a distributed controller in the presence of noise is considered, via the distributed virtual reference feedback tuning (DVRFT) framework. The analysis is performed for a linear interconnected system on an arbitrary graph that is subject to unmeasured exogenous inputs. By solving a dynamic network identification problem with prediction-error filtering and a tailor-made noise model, we show that the distributed model-reference control problem can be solved directly from data. Sufficient conditions are provided for which the local controller estimates are consistent. Moreover, it is shown how the method can be applied in the single-input-single-output case, leading to consistent estimates with standard virtual reference feedback tuning as well. The effectiveness of the method is demonstrated via a small network example with two interconnected systems.

Consistent identification of dynamic networks subject to white noise using Weighted Null-Space Fitting

Identification of dynamic networks has been a flourishing area in recent years. However, there are few contributions addressing the problem of simultaneously identifying all modules in a network of given structure. In principle the prediction error method can handle such problems but this methods suffers from well known issues with local minima and how to find initial parameter values. Weighted Null-Space Fitting is a multi-step least-squares method and in this contribution we extend this method to rational linear dynamic networks of arbitrary topology with modules subject to white noise disturbances. We show that WNSF reaches the performance of PEM initialized at the true parameter values for a fairly complex network, suggesting consistency and asymptotic efficiency of the proposed method.

A Local Direct Method for Module Identification in Dynamic Networks With Correlated Noise

The identification of local modules in dynamic networks with known topology has recently been addressed by formulating conditions for arriving at consistent estimates of the module dynamics, under the assumption of having disturbances that are uncorrelated over the different nodes. The conditions typically reflect the selection of a set of node signals that are taken as predictor inputs in an multiple-input-single-output (MISO) identification setup. In this paper an extension is made to arrive at an identification setup for the situation that process noises on the different node signals can be correlated with each other. In this situation the local module may need to be embedded in an multiple-input--multiple-output (MIMO) identification setup for arriving at a consistent estimate with maximum likelihood properties. This requires the proper treatment of confounding variables. The result is a set of algorithms that, based on the given network topology and disturbance correlation structure, selects an appropriate set of node signals as predictor inputs and outputs in an MISO or MIMO identification setup. Three algorithms are presented that differ in their approach of selecting measured node signals. Either a maximum or a minimum number of measured node signals can be considered, as well as a preselected set of measured nodes.

Combining Experiments for Linear Dynamic Network Identification in the Presence of Nonlinearities

In many practical applications it might be desirable to excite only point at a time in an interconnection of multiple dynamic subsystems (e.g. large-scale system). Therefore multiple experiments need to be combined to successfully identify one or more subsystems in the network of subsystems. This papers illustrates how the identification of a linear subsystem of a dynamical network containing one or more nonlinear subsystems can result in biased estimates when multiple experiments are combined using the Best Linear Approximation (BLA) based approach.

A sequential least squares algorithm for ARMAX dynamic network identification

Identification of dynamic networks in prediction error setting often requires the solution of a non-convex optimization problem, which can be difficult to solve especially for large-scale systems. Focusing on ARMAX models of dynamic networks, we instead employ a method based on a sequence of least-squares steps. For single-input single-output models, we show that the method is equivalent to the recently developed Weighted Null Space Fitting, and, drawing from the analysis of that method, we conjecture that the proposed method is both consistent as well as asymptotically efficient under suitable assumptions. Simulations indicate that the sequential least squares estimates can be of high quality even for short data sets.

Identification of a Class of Nonlinear Dynamical Networks

Identification of dynamic networks has attracted considerable interest recently. So far the main focus has been on linear time-invariant networks. Meanwhile, most real-life systems exhibit nonlinear behaviors; consider, for example, two stochastic linear time-invariant systems connected in series, each of which has a nonlinearity at its output. The estimation problem in this case is recognized to be challenging, due to the analytical intractability of both the likelihood function and the optimal one-step ahead predictors of the measured nodes. In this contribution, we introduce a relatively simple prediction error method that may be used for the estimation of nonlinear dynamical networks. The estimator is defined using a deterministic predictor that is nonlinear in the known signals. The estimation problem can be defined using closed-form analytical expressions in several non-trivial cases, and Monte Carlo approximations are not necessarily required. We show, that this is the case for some block-oriented networks with no feedback loops and where all the nonlinear modules are polynomials. Consequently, the proposed method can be applied in situations considered challenging by current approaches. The performance of the estimation method is illustrated on a numerical simulation example.

Weighted Null-Space Fitting for Identification of Cascade Networks

For identification of systems embedded in dynamic networks, the prediction error method (PEM) with a correct parametrization of the complete network provides asymptotically efficient estimates. However, the network complexity often hinders a successful application of PEM, which requires minimizing a non-convex cost function that can become more intricate for more complex networks. For this reason, identification in dynamic networks often focuses in obtaining consistent estimates of modules of interest. A downside of these approaches is that splitting the network in several modules for identification often costs asymptotic efficiency. In this paper, we consider dynamic networks with the modules connected in serial cascade, with measurements affected by sensor noise. We propose an algorithm that estimates all the modules in the network simultaneously without requiring the minimization of a non-convex cost function. This algorithm is an extension of Weighted Null-Space Fitting (WNSF), a weighted least-squares method that provides asymptotically efficient estimates for single-input single-output systems. We illustrate the performance of the algorithm with simulation studies, which suggest that a network WNSF method may also be asymptotically efficient when applied to cascade structures. Finally, we discuss the possibility of extension to more general networks affected by sensor noise.

Identification in dynamic networks

System identification is a common tool for estimating (linear) plant models as a basis for model-based predictive control and optimization. The current challenges in process industry, however, ask for data-driven modelling techniques that go beyond the single unit/plant models. While optimization and control problems become more and more structured in the form of decentralized and/or distributed solutions, the related modelling problems will need to address structured and interconnected systems. An introduction will be given to the current state of the art and related developments in the identification of linear dynamic networks. Starting from classical prediction error methods for open-loop and closed-loop systems, several consequences for the handling of network situations will be presented and new research questions will be highlighted.

Identification and Prediction in Dynamic Networks with Unobservable Nodes

The interest for system identification in dynamic networks has increased recently with a wide variety of applications. In many cases, it is intractable or undesirable to observe all nodes in a network and thus, to estimate the complete dynamics. Furthermore, it might even be challenging to estimate a subset of the network if key nodes are unobservable due to correlation between the nodes. In this contribution, we will discuss an approach to treat this problem. The approach relies on additional measurements that are dependent on the unobservable nodes and thus indirectly contain information about them. These measurements are used to form an alternative indirect model that is only dependent on observed nodes. The purpose of estimating this indirect model can be either to recover information about modules in the original network or to make accurate predictions of variables in the network. Examples are provided for both recovery of the original modules and prediction of nodes.

Identification of dynamic networks with rank-reduced process noise

In dynamic network identification usually the assumption is made that there is a full rank process noise affecting the network. For large scale networks with many variables this assumption is not realistic as the noise could be generated by a limited number of sources. We extend prediction error identification methods by allowing rank-reduced process noise in the network. The developed method is based on a modification of the typical predictor expression and an appropriate modification of the identification criterion. It is shown that this method leads to consistent estimates, and we provide a method to reduce the variance of the estimates, which is confirmed by simulations.

Evaluation of artificial time series microarray data for dynamic gene regulatory network inference

High-throughput technology like microarrays is widely used in the inference of gene regulatory networks (GRNs). We focused on time series data since we are interested in the dynamics of GRNs and the identification of dynamic networks. We evaluated the amount of information that exists in artificial time series microarray data and the ability of an inference process to produce accurate models based on them. We used dynamic artificial gene regulatory networks in order to create artificial microarray data. Key features that characterize microarray data such as the time separation of directly triggered genes, the percentage of directly triggered genes and the triggering function type were altered in order to reveal the limits that are imposed by the nature of microarray data on the inference process. We examined the effect of various factors on the inference performance such as the network size, the presence of noise in microarray data, and the network sparseness. We used a system theory approach and examined the relationship between the pole placement of the inferred system and the inference performance. We examined the relationship between the inference performance in the time domain and the true system parameter identification. Simulation results indicated that time separation and the percentage of directly triggered genes are crucial factors. Also, network sparseness, the triggering function type and noise in input data affect the inference performance. When two factors were simultaneously varied, it was found that variation of one parameter significantly affects the dynamic response of the other. Crucial factors were also examined using a real GRN and acquired results confirmed simulation findings with artificial data. Different initial conditions were also used as an alternative triggering approach. Relevant results confirmed that the number of datasets constitutes the most significant parameter with regard to the inference performance.

Identifiability of dynamic networks with part of the nodes noise-free

In dynamic network identification a major goal is to uniquely identify the topology and dynamic links between the measured node variables. It is common practice to assume that process noises affect every output in multivariable system identification, and every node in dynamic networks with a full rank noise process. For many practical situations this assumption might be overly strong. This leads to the question of how to handle situations where the process noise is not full rank, i.e. when the number of white noise processes driving the network is strictly smaller than the number of nodes. In this paper a first step towards answering this question is taken by addressing the case of a dynamic network where some nodes are noise-free, and others are disturbed with a (correlated) process noise. In this situation the predictor filters that generate the one-step-ahead prediction of the node signals are non-unique, and the appropriate identification criterion leads to a constrained optimization problem. It is assessed when it is possible to distinguish between models on the basis of this criterion, leading to new notions of network identifiability. It appears that a sufficient condition for network identifiability is that every node signal in the network is excited by an external excitation signal or a process noise signal that is uncorrelated with other node excitations.

Errors-in-variables identification in dynamic networks — Consistency results for an instrumental variable approach

In this paper we consider the identification of a linear module that is embedded in a dynamic network using noisy measurements of the internal variables of the network. This is an extension of the errors-in-variables (EIV) identification framework to the case of dynamic networks. The consequence of measuring the variables with sensor noise is that some prediction error identification methods no longer result in consistent estimates. The method developed in this paper is based on a combination of the instrumental variable philosophy and closed-loop prediction error identification methods, and leads to consistent estimates of modules in a dynamic network. We consider a flexible choice of which internal variables need to be measured in order to identify the module of interest. This allows for a flexible sensor placement scheme. We also present a method that can be used to validate the identified model.

Identifiability in dynamic network identification

Dynamic networks are structured interconnections of dynamical systems driven by external excitation and disturbance signals. We develop the notion of network identifiability, a property of a parameterized model set that ensures that module dynamics are uniquely related to the filters that specify the one-step-ahead predictors of all node signals in the network. It can be used to specify which presence of excitation signals will result in a unique representation of the network dynamics in a particular network model parametrization. This uniqueness is necessary for detecting the topology of the network from measured data, and for consistently estimating the network dynamics. We combine aspects of the classical notion of system identifiability with a uniqueness-oriented parametrization concept, and extend this to the situation of highly structured model sets. All node signals in the network are treated in a symmetric way. The presented concept and theory allow for the incorporation of particular structural prior knowledge of the network structure

System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques

Model estimation and structure detection with short data records are two issues that receive increasing interests in System Identification. In this paper, a multiple kernel-based regularization method is proposed to handle those issues. Multiple kernels are conic combinations of fixed kernels suitable for impulse response estimation, and equip the kernel-based regularization method with three features. First, multiple kernels can better capture complicated dynamics than single kernels. Second, the estimation of their weights by maximizing the marginal likelihood favors sparse optimal weights, which enables this method to tackle various structure detection problems, e.g., the sparse dynamic network identification and the segmentation of linear systems. Third, the marginal likelihood maximization problem is a difference of convex programming problem. It is thus possible to find a locally optimal solution efficiently by using a majorization minimization algorithm and an interior point method where the cost of a single interior-point iteration grows linearly in the number of fixed kernels. Monte Carlo simulations show that the locally optimal solutions lead to good performance for randomly generated starting points.