Observable State Space Research Articles

Self-supervised multi-transformation learning for time series anomaly detection

Time series anomaly detection aims to find specific patterns in time series that do not conform to general rules, which is one of the important research directions in machine learning. However, most existing time series anomaly detection methods solely consider the original observed state space. In this paper, we investigate the different views of the original signals by reconstructing them from the different transformations to learn better comprehensive representations of normal patterns. Specifically, we propose a time series anomaly detection model based on self-supervised multi-transformation learning while jointly learning the noise and filter transformation of the normal time series and capturing the anomaly simultaneously in both transformation patterns. Firstly, the model randomly adds noise to the original signal to construct the noisy signal, and the Kalman filter is applied to the original signal to construct the filtered signal. The noisy and filtered signals are subsequently used to reconstruct the original signal, and the randomly added noise is classified. Finally, the model captures the two transformation patterns of normal time series signals and transforms the anomaly detection problem into the reconstruction metric problem of time series signals under noise and filter transformations. The effectiveness of the proposed method is verified on seven time series datasets. The source code is available at https://github.com/hh2668790143/MTAE.

Parameter identification for Hammerstein nonlinear system with polynomial and state space model

This study investigates a two-stage parameter identification algorithm for the Hammerstein nonlinear system based on special test signals. The studied Hammerstein nonlinear system has a static nonlinear subsystem represented by polynomial basis function and a dynamic linear subsystem described by canonical observable state space model, and special test signals composed of binary signals and random signals are applied to parameter identification separation of the nonlinear subsystem and linear subsystem. The detailed identification procedures consist of two main steps. Firstly, using the characteristics that binary signals do not excite the static nonlinear subsystem, the dynamic linear subsystem parameters are identified through recursive least squares algorithm based on input-output data of binary signals. Secondly, unmeasurable state variables of the identified system are replaced with estimated values, thus the nonlinear subsystem parameters are obtained using recursive least squares algorithm with the help of input-output data of random signals. The efficiency and accuracy of proposed identification scheme are confirmed on experiment results of a numerical simulation and a practical nonlinear process, and experimental simulation results show that the developed two-stage identification algorithm has excellent predictive performance for identifying the Hammerstein nonlinear state space systems.

Edge intelligence enabled optimal scheduling with distributed price-responsive load for regenerative electric boilers

Heat supply accounts for a substantial amount of terminal energy usage. However, along with price rises in primary energy, there is an urgent need to reduce the average cost of energy consumption during the purchasing of thermal services. Electric heating, an electricity-fed heating production and delivery technology, has been suggested as a promising method for improving heating efficiency, due to the ease of scheduling. However, the traditional centralized operating methods of electricity purchasing rely on explicit physical modeling of every detail, and accurate future predictions, the implementation of which are rarely practical in reality. To facilitate model-free decisions in the field of electricity purchasing, heat storage, and supply management, aimed at cost saving in a real-time price environment, this study proposes a scheduling framework based on deep reinforcement learning (DRL) and the existence of responsive users. First, the structure of a distributed heating system fed by regenerative electric boilers (REBs), which facilitate shiftable heat-load control, is introduced. A terminal heat demand response model based on thermal sensation vote (TSV), characterizing the consumption flexibility of responsive users, is also proposed. Second, due to thermal system inertia, the sequential decision problem of electric heating load scheduling is transformed into a specific Markov decision process (MDP). Finally, the edge intelligence (EI) deployed on the demand side uses a twin delayed deterministic policy gradient (TD-3) algorithm to address the action space continuity of electric heating devices. The combination of a DRL strategy and the computing power of EI enables real-time optimal scheduling. Unlike the traditional method, the trained intelligent agent makes adaptive control strategies according to the currently observed state space, thus avoiding prediction uncertainty. The simulation results validate that the intelligent agent responds positively to changes in electricity prices and weather conditions, reducing electricity consumption costs while maintaining user comfort. The adaptability and generalization of the proposed approach to different conditions is also demonstrated.

Resilient Operation of Distribution Grids Using Deep Reinforcement Learning

This article utilizes deep reinforcement learning to develop an intelligent resilience controller (IRC) that devises fast real-time operation decisions to strategically dispatch distributed generation and energy storage units for restoring power to customers after sudden outages. The proposed IRC learns the failure development pattern of uncertain high-impact events and is able to explore a large action space in the partially observable state space of distribution grids under widespread outages. A spatiotemporal hurricane impact analysis model is presented as an example of uncertain high-impact events, and its parameters are used in training the IRC model and preparing it for similar events. In the proposed model, the distribution grid operation under uncertainty is modeled as a Markov decision process (MDP), and actions taken by the operator are rewarded based on operation costs. Since the number of distributed energy resources can be significant, the scalability issue of the method is addressed by reformulating the problem as a sequential MDP. The proposed model is implemented on a test distribution grid undergoing a hurricane, and its performance is compared with common operation strategies, indicating the superiority of the proposed model in terms of reduced operation cost and close to zero running time. Further analysis shows the adaptability of the proposed model to hurricanes of various intensities.

Comparison of vector and matrix format tangential interpolation for FDNE

Vector format tangential interpolation (VFTI) framework for deriving controllable and observable descriptor state space model using Loewner matrix pencil from the transfer matrix data has been recently proposed for VLSI circuits. It uses data sampled directionally i.e. two columns or two rows of the transfer matrix, called tangential interpolation data. In VFTI the information contained in the frequency samples is not fully utilized and the accuracy gets affected with large number of samples. Matrix format tangential interpolation (MFTI) has been proposed as an enhancement to VFTI in the literature which uses the entire transfer matrix so that all the information contained in the samples is fully utilized. This paper investigates the applicability of MFTI for state space modeling of power system FDNEs and proposes a novel MATLAB® based implementation for Loewner matrix construction. Properties of MFTI such as convergence speed, accuracy, stability and passivity are compared with vector fitting (VF) and VFTI. Impact of number of samples is also studied.

Macro Action Selection with Deep Reinforcement Learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also widely accepted as a challenging testbed for AI research because of its enormous state space, partially observed information, multi-agent collaboration, and so on. With the help of annual AIIDE and CIG competitions, a growing number of SC bots are proposed and continuously improved. However, a large gap remains between the top-level bot and the professional human player. One vital reason is that current SC bots mainly rely on predefined rules to select macro actions during their games. These rules are not scalable and efficient enough to cope with the enormous yet partially observed state space in the game. In this paper, we propose a deep reinforcement learning (DRL) framework to improve the selection of macro actions. Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as LastOrder. Our evaluation, based on training against all bots from the AIIDE 2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning rate, outperforming 26 bots in total 28 entrants.

Realization of time-delay systems

The paper addresses the problem of transforming a single-input single-output nonlinear retarded time-delay system, described by an input–output equation, in the traditional observable state space form. The solution is generalized from the delay-free case and depends on integrability of certain submodule of differential 1-forms. The integrability conditions are improved to make them constructive. Finally, it is explained why one may obtain two realizations, which are not connected by bi-causal change of state coordinates.

Combined state and parameter estimation for Hammerstein systems with time delay using the Kalman filtering

SummaryThis paper discusses the state and parameter estimation problem for a class of Hammerstein state space systems with time delay. Both the process and the measurement noises are considered in the system. On the basis of the observable canonical state space form and the key term separation, a pseudolinear regressive identification model is obtained. For the unknown states in the information vector, the Kalman filter is used to search for the optimal state estimates. A Kalman filter–based least squares iterative and a recursive least squares algorithms are proposed. Extending the information vector to include the latest information terms, which are missed for the time delay, the Kalman filter–based recursive extended least squares algorithm is derived to obtain the estimates of the unknown time delay, parameters, and states. The numerical simulation results are given to illustrate the effectiveness of the proposed algorithms.

Accessibility conditions of MIMO nonlinear control systems on homogeneous time scales

A necessary and sufficient accessibility condition for the set of nonlinear higher order input-output (i/o) delta differential equations is presented. The accessibility definition is based on the concept of an autonomous element that is specified to the multi-input multi-output systems. The condition is presented in terms of the greatest common left divisor of two left differential polynomial matrices associated with the system of the i/o delta-differential equations defined on a homogenous time scale which serves as a model of time and unifies the continuous and discrete time. We associate the subspace $\mathcal{H}_{\infty}$ of the vector space of differential one-forms with the considered system.This subspace is invariant with respect to taking delta derivatives.The relation between $\mathcal{H}_\infty$ and the element of a left free module over the ring of left differential polynomials is presented.The presented accessibility condition provides a basis for system reduction, i.e. for finding the transfer equivalent minimal accessible representation of the set of the i/o equations which is a suitable starting point for constructing an observable and accessible state space realization. Moreover, the condition allows to check the transfer equivalence of nonlinear systems, defined on homogeneous time scales.

Leader-Following Output Consensus in a Network of Linear Agents with Communication Noises

The leader-following output consensus problem of multi-agent systems (MAS) is studied in this paper. Each agent is modeled by a single-input single-output (SISO) system which can be further described by a controllable and observable linear state space model. An observer is constructed to estimate the agent's state, and the estimated state is shared with neighbor agents via the noisy communication channels. Similar to the previous work, in the proposed protocol a time-varying gain is employed to attenuate the noise's effect. However, in this paper, each agent is allowed to have its own time-varying gain. Some sufficient conditions on the time-varying gain are given for ensuring the consensus in the mean square sense. Finally, a simulation example is presented to verify the theoretical results.

Polynomial accessibility condition for the multi-input multi-output nonlinear control system

The paper presents a computation-oriented necessary and sufficient accessibility condition for the set of nonlinear higher- order input-output differential equations. The condition is presented in terms of the greatest common left divisor of two polynomial matrices, associated with the system of input-output equations. The basic difference from the linear case is that the elements of the polynomial matrices belong to a non-commutative polynomial ring. The condition found provides a basis for finding the accessible representation of the set of input-output equations, which is a suitable starting point for the construction of an observable and accessible state space realization. Moreover, the condition allows us to check the transfer equivalence of two nonlinear systems.

Observability and Controllability of MIMO Control Systems via Difference Equations

The paper applies the difference equations to unify the study of observability and controllability conditions of discrete-time multi-input multi-output (MIMO) nonlinear control systems. The necessary and sufficient condition for irreducibility of the set of nonlinear multi input- multi output (MIMO) is presented in terms of the greatest common left divisor of two polynomial matrices describing the behavior of the system which is suitable for constructing an observable and accessible state space realization. We show that the concepts of controllability and observability are related to systems of difference equations. It is well known that a solvable system of linear algebraic equations has a solution if and only if the rank of the system matrix is full .This method is more clear, straight-forward and therefore better suited for implementation in different computer packages such as Matlab.

Angular Motion Estimation Using Dynamic Models in a Gyro-Free Inertial Measurement Unit

In this paper, we summarize the results of using dynamic models borrowed from tracking theory in describing the time evolution of the state vector to have an estimate of the angular motion in a gyro-free inertial measurement unit (GF-IMU). The GF-IMU is a special type inertial measurement unit (IMU) that uses only a set of accelerometers in inferring the angular motion. Using distributed accelerometers, we get an angular information vector (AIV) composed of angular acceleration and quadratic angular velocity terms. We use a Kalman filter approach to estimate the angular velocity vector since it is not expressed explicitly within the AIV. The bias parameters inherent in the accelerometers measurements' produce a biased AIV and hence the AIV bias parameters are estimated within an augmented state vector. Using dynamic models, the appended bias parameters of the AIV become observable and hence we can have unbiased angular motion estimate. Moreover, a good model is required to extract the maximum amount of information from the observation. Observability analysis is done to determine the conditions for having an observable state space model. For higher grades of accelerometers and under relatively higher sampling frequency, the error of accelerometer measurements is dominated by the noise error. Consequently, simulations are conducted on two models, one has bias parameters appended in the state space model and the other is a reduced model without bias parameters.

Observable state space realizations for multivariable systems

This paper derives two canonical state space forms (i.e., the observer canonical form and the observability canonical form) from multiple-input multiple-output systems described by difference equations. The state space model is expressed by the first-order difference equation and is equivalent to the input–output representation. More specifically, by setting the different state variables, the difference equations or the input–output representations can be transformed into two observable canonical forms and the canonical state space model can be also transformed into the difference equations. Finally, two examples are given.

Observable State Space Realization of Networked MIMO System for a Novel Virtual Observer-Based Delay Compensation

With the ever increasing complexity of cyber-physical system, the coupling and interconnection between distributed nodes lead to a complicated description in mathematics. For example, time delays cannot be ignored in the cyber-physical system modeling. And delay compensation often requires the delayed Multi-Input-Multi-Output (MIMO) system to be presented in state space model and all time delays to be combined to sole side of input or output. The aim of this paper is to model the MIMO system with multiple delays in state space form and simultaneously combine both input and output delays into output channel only. In this paper, following up with the modeling of Networked MIMO System, an observable state-space realization of delayed transfer function matrix is developed, including the case with repeated poles. Meanwhile, multiple time delays are combined and allocated to the output side only. Based on the observable state space model, an observer can be designed and contributed to the delay compensation. Derivation and simulation results prove that the resulting model of Networked Control System (NCS) through the proposed modeling method is equivalent with the original networked MIMO system with both input and output delays in terms of input-output relationship, and thus it can be used in the prospective observer-based delay.

Semiparametric estimation of Markov decision processes with continuous state space

We propose a general two-step estimator for a popular Markov discrete choice model that includes a class of Markovian games with continuous observable state space. Our estimation procedure generalizes the computationally attractive methodology of Pesendorfer and Schmidt-Dengler (2008) that assumed finite observable states. This extension is non-trivial as the policy value functions are solutions to some type II integral equations. We show that the inverse problem is well-posed. We provide a set of primitive conditions to ensure root- T consistent estimation for the finite dimensional structural parameters and the distribution theory for the value functions in a time series framework.

Semiparametric Estimation of Markov Decision Processes with Continuous State Space

We propose a general two-step estimation method for the structural parameters of popular semiparametric Markovian discrete choice models that include a class of Markovian Games and allow for continuous observable state space. The estimation procedure is simple as it directly generalizes the computationally attractive methodology of Pesendorfer and Schmidt-Dengler (2008) that assumed finite observable states. This extension is non-trivial as the value functions, to be estimated nonparametrically in the first stage, are defined recursively in a non-linear functional equation. Utilizing structural assumptions, we show how to consistently estimate the infinite dimensional parameters as the solution to some type II integral equations, the solving of which is a well-posed problem. We provide sufficient set of primitives to obtain root-T consistent estimators for the finite dimensional structural parameters and the distribution theory for the value functions in a time series framework.

An Almost-Deadbeat Hybrid Observer for a Class of Linear and Nonlinear Systems

In this paper a hybrid state observer with deadbeat-like properties for a class of planar linear and nonlinear continuous time systems is presented. Finite-time convergence This is achieved either by performing a reset exactly when the estimation error has a suitable value, or by properly initializing the observer reset. In the first case the reset time can be determined using a bisection method provided the estimation error has an oscillatory behavior. In the second case the bisection algorithm is applied in the observer state space and does not require oscillatory behavior of the error dynamics. Both approaches can be used to improve the performance of a given local observer. We also discuss a robustified version in the presence of measurement noise.

Extension of the transfer function approach to the realization problem of nonlinear systems to discrete-time case

In this paper the polynomial approach to the realization problem of nonlinear control systems, i.e. the problem of finding an observable state space representation of a SISO nonlinear system described by an input-output equation, is extended to the discrete-time case. To find the solution the so-called adjoint polynomials and adjoint transfer functions are employed establishing direct connection to the solution known from linear systems. In addition, in case a system is not realizable it is shown that, unlike in continuous-time case, there always exists a postcompensator which, when combined with the system in series connection, makes compensated system realizable. Due to the possibility to use the transfer function algebra when combining systems this problem can be solved in natural way.

Using Assertions to Enhance the Correctness of Kmelia Components and their Assemblies

The Kmelia component model is an abstract formal component model based on services. It is dedicated to the specification and development of correct components. This work enriches the Kmelia language to allow the description of data, expressions and assertions when specifying components and services. The objective is to enable the use of assertions in Kmelia in order to support expressive service descriptions, to support client/supplier contracts with pre/post-conditions, and to enhance formal analysis of component-based systems. Assertions are used to perform analysis of services, component assemblies and service compositions. Additionally we enable the definition of virtual contexts for required services and the corresponding observable state space for the components which provide the services. We illustrate the work with the verification of consistency properties involving data at component and assembly levels.