Input Reconstruction Research Articles

A robust supervised subspace learning approach for output-relevant prediction and detection against outliers

This paper proposes a novel robust supervised subspace learning (RSSL) method for output-relevant prediction and detection against outliers. RSSL learns the robust subspaces by optimizing a joint problem over both the prediction of output and the reconstruction of input. To this end, the learned subspaces/data representations are informative, i.e., they are encapsulated with the critic information related to both the input and output, and thus can benefit the following tasks of output-related modeling and detection. Besides, we separate sparse items from the raw measurements to suppress the effects of outliers. An efficient optimization algorithm is designed to solve the optimization problem of RSSL. We further conduct post orthogonal decomposition upon the subspaces provided by RSSL so that the trimmed subspaces are more suitable for output-related detection. The efficacy of the proposed method is extensively verified by synthesis data and benchmark data.

Deception attack monitoring in vulnerable hydroelectric generator system

AbstractThis article deals with unknown deception attacks detection of hydroelectric generator. Such cyber attacks are switching disturbances that affect the input information transmitted from the power system stabilizer to the plant through a constrained communication network. The generator dynamics can be modeled as an augmented state model, whereas the unknown input reconstruction is obtained from a modified Kalman filter recursions under communication constraints. The obtained design ensures the minimization of disturbances with accurate cyberattack monitoring.

Expansion of Information in the Binary Autoencoder With Random Binary Weights.

This letter studies the expansion and preservation of information in a binary autoencoder where the hidden layer is larger than the input. Such expansion is widespread in biological neural networks, as in the olfactory system of a fruit fly or the projection of thalamic inputs to the neocortex. We analyze the threshold model, the kWTA model, and the binary matching pursuit model to find how the sparsity and the dimension of the encoding influence the input reconstruction, similarity preservation, and mutual information across layers. It is shown that the sparser activation of the hidden layer is preferable for preserving information between the input and the output layers. All three models show optimal similarity preservation at dense, not sparse, hidden layer activation. Furthermore, with a large enough hidden layer, it is possible to get zero reconstruction error for any input just by varying the thresholds of neurons. However, we show that the preference for sparsity is due to the noise in the weight matrix between layers. A fixed number of nonzero connections to every neuron achieves better information preservation and input reconstruction for the dense hidden layer activation. The theoretical results give useful insight into models of neural computation based on sparse binary representation and association memory.

Observer Based Multi-Level Fault Reconstruction for Interconnected Systems.

The problem of local fault (unknown input) reconstruction for interconnected systems is addressed in this paper. This contribution consists of a geometric method which solves the fault reconstruction (FR) problem via observer based and a differential algebraic concept. The fault diagnosis (FD) problem is tackled using the concept of the differential transcendence degree of a differential field extension and the algebraic observability. The goal is to examine whether the fault occurring in the low-level subsystem can be reconstructed correctly by the output at the high-level subsystem under given initial states. By introducing the fault as an additional state of the low subsystem, an observer based approached is proposed to estimate this new state. Particularly, the output of the lower subsystem is assumed unknown, and is considered as auxiliary outputs. Then, the auxiliary outputs are estimated by a sliding mode observer which is generated by using global outputs and inverse techniques. After this, the estimated auxiliary outputs are employed as virtual sensors of the system to generate a reduced-order observer, which is caplable of estimating the fault variable asymptotically. Thus, the purpose of multi-level fault reconstruction is achieved. Numerical simulations on an intensified heat exchanger are presented to illustrate the effectiveness of the proposed approach.

Interval observer‐based fault tolerant control strategy with fault estimation and compensation

AbstractIn this paper, a new strategy of fault tolerant control (FTC) is proposed for linear system with unknown inputs and actuator faults. First, a reduced‐order observer which can estimate asymptotically the system states is designed. Secondly, an interval observer which is robust to the unknown inputs but sensitive to the actuator faults in a sense of interval estimations of the states is developed for the output variables. So the interval observer is then used as actuator detector. Third, another interval observer which is different from the first one in that it is robust to not only the unknown inputs but also the actuator faults is constructed. Moreover, based on the interval observer, an asymptotical unknown input reconstruction method is proposed. Third, an observer‐based state feedback controller with a fault compensation controller is designed based on H∞ index performance. In our control strategy, the fault detector and robust H∞ state feedback controller are always in active, while the fault estimator and the fault compensation controller is activated only when the fault detector issues an alarm. Finally, a simulation example is given to illustrate the effectiveness of this new control strategy.

On dynamical input reconstruction in a distributed second order equation

Abstract A second order nonlinear differential equation is considered. An algorithm for reconstructing an input from inaccurate measurements of the solution at discrete times is designed. The algorithm based on the constructions of feedback control theory and theory of ill-posed problems is stable with respect to informational noises and computational errors.

Comparison of Novelty Detection Methods for Detection of Various Rotary Machinery Faults.

Condition monitoring is an indispensable element related to the operation of rotating machinery. In this article, the monitoring system for the parallel gearbox was proposed. The novelty detection approach is used to develop the condition assessment support system, which requires data collection for a healthy structure. The measured signals were processed to extract quantitative indicators sensitive to the type of damage occurring in this type of structure. The indicator’s values were used for the development of four different novelty detection algorithms. Presented novelty detection models operate on three principles: feature space distance, probability distribution, and input reconstruction. One of the distance-based models is adaptive, adjusting to new data flowing in the form of a stream. The authors test the developed algorithms on experimental and simulation data with a similar distribution, using the training set consisting mainly of samples generated by the simulator. Presented in the article results demonstrate the effectiveness of the trained models on both data sets.

Localization of Invariable Sparse Errors in Dynamic Systems

Understanding the dynamics of complex systems is a central task in many different areas ranging from biology via epidemics to economics and engineering. Unexpected behaviour of dynamic systems or even system failure is sometimes difficult to comprehend. Such a data-mismatch can be caused by endogenous model errors including misspecified interactions and inaccurate parameter values. These are often difficult to distinguish from unmodelled process influencing the real system like unknown inputs or faults. Localizing the root cause of these errors or faults and reconstructing their dynamics is only possible if the measured outputs of the system are sufficiently informative. Here, we present criteria for the measurements required to localize the position of error sources in large dynamic networks. We assume that faults or errors occur at a limited number of positions in the network. This invariable sparsity differs from previous sparsity definitions for inputs to dynamic systems. We provide an exact criterion for the recovery of invariable sparse inputs to nonlinear systems and formulate an optimization criterion for invariable sparse input reconstruction. For linear systems we can provide exact error bounds for this reconstruction method.

Autoencoder networks extract latent variables and encode these variables in their connectomes

Advances in electron microscopy and data processing techniques are leading to increasingly large and complete microscale connectomes. At the same time, advances in artificial neural networks have produced model systems that perform comparably rich computations with perfectly specified connectivity. This raises an exciting scientific opportunity for the study of both biological and artificial neural networks: to infer the underlying circuit function from the structure of its connectivity. A potential roadblock, however, is that – even with well constrained neural dynamics – there are in principle many different connectomes that could support a given computation. Here, we define a tractable setting in which the problem of inferring circuit function from circuit connectivity can be analyzed in detail: the function of input compression and reconstruction, in an autoencoder network with a single hidden layer. Here, in general there is substantial ambiguity in the weights that can produce the same circuit function, because largely arbitrary changes to input weights can be undone by applying the inverse modifications to the output weights. However, we use mathematical arguments and simulations to show that adding simple, biologically motivated regularization of connectivity resolves this ambiguity in an interesting way: weights are constrained such that the latent variable structure underlying the inputs can be extracted from the weights by using nonlinear dimensionality reduction methods.

Improving the Accuracy of Dam Inflow Predictions Using a Long Short-Term Memory Network Coupled with Wavelet Transform and Predictor Selection

Accurate and reliable dam inflow prediction models are essential for effective reservoir operation and management. This study presents a data-driven model that couples a long short-term memory (LSTM) network with robust input predictor selection, input reconstruction by wavelet transformation, and efficient hyper-parameter optimization by K-fold cross-validation and the random search. First, a robust analysis using a “correlation threshold” for partial autocorrelation and cross-correlation functions is proposed, and only variables greater than this threshold are selected as input predictors and their time lags. This analysis indicates that a model trained on a threshold of 0.4 returns the highest Nash–Sutcliffe efficiency value; as a result, six principal inputs are selected. Second, using additional subseries reconstructed by the wavelet transform improves predictability, particularly for flow peak. The peak error values of LSTM with the transform are approximately one-half to one-quarter the size of those without the transform. Third, for a K of 5 as determined by the Silhouette coefficients and the distortion score, the wavelet-transformed LSTMs require a larger number of hidden units, epochs, dropout, and batch size. This complex configuration is needed because the amount of inputs used by these LSTMs is five times greater than that of other models. Last, an evaluation of accuracy performance reveals that the model proposed in this study, called SWLSTM, provides superior predictions of the daily inflow of the Hwacheon dam in South Korea compared with three other LSTM models by 84%, 78%, and 65%. These results strengthen the potential of data-driven models for efficient and effective reservoir inflow predictions, and should help policy-makers and operators better manage their reservoir operations.

SurvNet: A Novel Deep Neural Network for Lung Cancer Survival Analysis With Missing Values.

Survival analysis is important for guiding further treatment and improving lung cancer prognosis. It is a challenging task because of the poor distinguishability of features and the missing values in practice. A novel multi-task based neural network, SurvNet, is proposed in this paper. The proposed SurvNet model is trained in a multi-task learning framework to jointly learn across three related tasks: input reconstruction, survival classification, and Cox regression. It uses an input reconstruction mechanism cooperating with incomplete-aware reconstruction loss for latent feature learning of incomplete data with missing values. Besides, the SurvNet model introduces a context gating mechanism to bridge the gap between survival classification and Cox regression. A new real-world dataset of 1,137 patients with IB-IIA stage non-small cell lung cancer is collected to evaluate the performance of the SurvNet model. The proposed SurvNet achieves a higher concordance index than the traditional Cox model and Cox-Net. The difference between high-risk and low-risk groups obtained by SurvNet is more significant than that of high-risk and low-risk groups obtained by the other models. Moreover, the SurvNet outperforms the other models even though the input data is randomly cropped and it achieves better generalization performance on the Surveillance, Epidemiology, and End Results Program (SEER) dataset.

Strong structural input and state observability of linear time-invariant systems: Graphical conditions and algorithms

The paper studies input and state observability (ISO) of discrete-time linear time-invariant network systems whose dynamics are affected by unknown inputs. More precisely, we aim at reconstructing the initial state and the sequence of unknown inputs from the system outputs, and we will use the term ISO when the input reconstruction is possible with delay one, namely the inputs up to time k−1 and the states up to time k can be obtained from the outputs up to time k, while the term unconstrained ISO will refer to the case where there is some arbitrary delay in the input reconstruction. We focus on the problem of s-structural ISO (resp. s-structural unconstrained ISO) wherein the objective is to find conditions such that for all system matrices that carry the same network structure, the resulting system is ISO (resp. unconstrained ISO). We provide first a graphical characterization for s-structural unconstrained ISO, and subsequently, sufficient conditions and necessary conditions for s-structural ISO. For the latter, under the assumption of zero feedthrough, these conditions coincide and characterise ISO. The conditions presented are in terms of existence of suitable uniquely restricted matchings in bipartite graphs associated with the structured system. In order to test these conditions, we present polynomial-time algorithms. Finally, we discuss an equivalent reformulation of the main conditions in terms of coloring algorithms as in the literature of zero forcing sets.

Model-based analysis and forecast of sleep-wake regulatory dynamics: Tools and applications to data.

Extensive clinical and experimental evidence links sleep-wake regulation and state of vigilance (SOV) to neurological disorders including schizophrenia and epilepsy. To understand the bidirectional coupling between disease severity and sleep disturbances, we need to investigate the underlying neurophysiological interactions of the sleep-wake regulatory system (SWRS) in normal and pathological brains. We utilized unscented Kalman filter based data assimilation (DA) and physiologically based mathematical models of a sleep-wake regulatory network synchronized with experimental measurements to reconstruct and predict the state of SWRS in chronically implanted animals. Critical to applying this technique to real biological systems is the need to estimate the underlying model parameters. We have developed an estimation method capable of simultaneously fitting and tracking multiple model parameters to optimize the reconstructed system state. We add to this fixed-lag smoothing to improve reconstruction of random input to the system and those that have a delayed effect on the observed dynamics. To demonstrate application of our DA framework, we have experimentally recorded brain activity from freely behaving rodents and classified discrete SOV continuously for many-day long recordings. These discretized observations were then used as the "noisy observables" in the implemented framework to estimate time-dependent model parameters and then to forecast future state and state transitions from out-of-sample recordings.

Spatial readout of visual looming in the central brain of Drosophila.

Visual systems can exploit spatial correlations in the visual scene by using retinotopy, the organizing principle by which neighboring cells encode neighboring spatial locations. However, retinotopy is often lost, such as when visual pathways are integrated with other sensory modalities. How is spatial information processed outside of strictly visual brain areas? Here, we focused on visual looming responsive LC6 cells in Drosophila, a population whose dendrites collectively cover the visual field, but whose axons form a single glomerulus-a structure without obvious retinotopic organization-in the central brain. We identified multiple cell types downstream of LC6 in the glomerulus and found that they more strongly respond to looming in different portions of the visual field, unexpectedly preserving spatial information. Through EM reconstruction of all LC6 synaptic inputs to the glomerulus, we found that LC6 and downstream cell types form circuits within the glomerulus that enable spatial readout of visual features and contralateral suppression-mechanisms that transform visual information for behavioral control.

Reduced order framework for convection dominant and pure diffusive problems based on combination of deep long short‐term memory and proper orthogonal decomposition/dynamic mode decomposition methods

AbstractIn many real‐world applications, mathematical models are highly complex, and numerical simulations in high‐dimensional systems are challenging. Model order reduction is a useful method to obtain a reasonable approximation by significantly reducing the computational cost of such problems. Deep learning technology is a recent improvement in artificial neural networks that can find more hidden information from the data. Deep learning has the advantage of processing data in its raw form and trains the nonlinear system with different levels of representation and predicts the data. In this article, a reduced order model framework based on a combination of deep learning [long short‐term memory (LSTM)] and proper orthogonal decomposition/dynamic mode decomposition (POD/DMD) modes is presented. Due to the robustness and stability of the LSTM recurrent neural network in predicting chaotic dynamical systems, we consider LSTM architecture to develop our data‐driven reduced order modeling (ROM). We investigate the proposed method performance by solving two well‐known canonical cases: a steady shear flow exhibiting the Kelvin‐Helmholtz instability, and two‐dimensional and unsteady mass diffusion equation. The focus of this article is to use LSTM deep recursive neural network to learn the time dynamics and POD/DMD to generate the order reduction model. The results show that the proposed method is very accurate in predicting time dynamics and input reconstruction.

Unknown Input Reconstruction via Interval Observer and State and Unknown Input Compensation Feedback Controller Designs

In this paper, we investigate the state estimation, unknown input and measurement noise reconstruction problems and the feedback controller design issues for a linear discrete-time system with both unknown inputs and measurement noises. First, an augmented system is constructed and the state vector of the augmented system consists of the original system state and the measurement noise, and the preconditions between the original system and the augmented system is discussed in detail. Second, for the augmented system, a reduced-order observer is designed so that the original system state estimates and the measurement noise reconstruction can be obtained. Third, in order to get the asymptotical unknown input reconstruction, an interval observer for part of the measurable output is proposed and an unknown input reconstruction method based on the interval observer is developed. Finally, an observer-based state feedback and unknown input controller is designed and the closed-loop system stability is analyzed. We point out that the closed-loop system satisfies the so-called separation property. At last, two simulation examples are given to verify the effectiveness of the proposed methods.

State estimation and reconstruction of unknown inputs with arbitrary relative degree via a predefined-time algebraic solver

This paper proposes a sliding mode observer for the estimation of the state variables and the reconstruction of the unknown inputs of nonlinear dynamical systems regardless of whether these systems satisfy relative-degree-related conditions or not. This observer is based on a predefined-time sliding mode algebraic solver and does not require the explicit use of state transformations. The use of the proposed method is illustrated through its application to an abstract system and the model of a chemical process, achieving an accurate estimation of both the state vector and the unknown inputs in each case, while an extended Kalman filter applied to the chemical process for comparison maintains significant estimation error.

Unsupervised Word Translation with Adversarial Autoencoder

Crosslingual word embeddings learned from monolingual embeddings have a crucial role in many downstream tasks, ranging from machine translation to transfer learning. Adversarial training has shown impressive success in learning crosslingual embeddings and the associated word translation task without any parallel data by mapping monolingual embeddings to a shared space. However, recent work has shown superior performance for non-adversarial methods in more challenging language pairs. In this article, we investigate adversarial autoencoder for unsupervised word translation and propose two novel extensions to it that yield more stable training and improved results. Our method includes regularization terms to enforce cycle consistency and input reconstruction, and puts the target encoders as an adversary against the corresponding discriminator. We use two types of refinement procedures sequentially after obtaining the trained encoders and mappings from the adversarial training, namely, refinement with Procrustes solution and refinement with symmetric re-weighting. Extensive experimentations with high- and low-resource languages from two different data sets show that our method achieves better performance than existing adversarial and non-adversarial approaches and is also competitive with the supervised system. Along with performing comprehensive ablation studies to understand the contribution of different components of our adversarial model, we also conduct a thorough analysis of the refinement procedures to understand their effects.

Dynamic Input Reconstruction Algorithm for a Nonlinear Equation with Distributed Parameters

We consider a differential equation of the Klein–Gordon type with an unknown input. The solution of this equation is assumed to be measured with some errors. We indicate a reconstruction algorithm whose input is formed by the feedback principle and serves as approximations to the unknown input of the original equation.

Reconstruction of Nonlinear Contact Forces Beyond Limited Measurement Locations Using an SVD Modal Filtering Approach

Assessment of operating loads is often approached as an inverse process using vibration response data and some form of system model to reconstruct external forces. Any mismatch in dynamic behavior between the model and the real system may distort the estimated loads. However, if the changes in system behavior are of interest, then those changes may be characterized through force reconstruction as equivalent loads acting in conjunction with the original external inputs. While these system changes may be linear in nature, the concept may be extended to nonlinear behavior, including intermittent contact between components. A modal based methodology for localization and reconstruction of inputs at potentially unmeasured locations is applied to the characterization of local system changes through their equivalent loading. An analytical study of a simple M-C-K system subject to contact nonlinearity is presented for validation. An experimental study is then presented using a multi-plate structure subject to impact loading and an unmodeled contact condition.