Sufficient Conditions For Identifiability Research Articles

VARMA Models with Single- or Mixed-Frequency Data: New Conditions for Extended Yule–Walker Identification

This paper deals with the identifiability of VARMA models with VAR order greater than or equal to the MA order, in the context of mixed-frequency data (MFD) using extended Yule–Walker equations. The main contribution is that necessary and sufficient conditions for identifiability in the single-frequency data case are expressed in an original way and yield new results in the MFD case. We also provide two counterexamples that answer an open question in this topic about whether certain sufficient conditions are necessary for identifiability. Therefore, this paper expands the set of models that can be identified with MFD using extended Yule–Walker equations. The main idea is that with MFD, some autocovariance blocks are not available from observed variables and, in some cases, the new conditions in this paper can be used to reconstruct all the non-available covariance blocks from available covariance blocks.

A Necessary Condition for Network Identifiability With Partial Excitation and Measurement

This paper considers dynamic networks where vertices and edges represent manifest signals and causal dependencies among the signals, respectively. We address the problem of how to determine if the dynamics of a network can be identified when only partial vertices are measured and excited. A necessary condition for network identifiability is presented, where the analysis is performed based on identifying the dependency of a set of rational functions from excited vertices to measured ones. This condition is further characterised by using an edge-removal procedure on the associated bipartite graph. Moreover, on the basis of necessity analysis, we provide a necessary and sufficient condition for identifiability in circular networks.

Identifiability in the Behavioral Setting

Identifiability, i.e., uniqueness of a solution of the identification problem, is a fundamental issue in system identification and data-driven control. Necessary and sufficient identifiability conditions for deterministic linear time-invariant systems that do not require a priori given input/output partitioning of the variables nor controllability of the true system are derived in the article. The prior knowledge needed for identifiability is the number of inputs, lag, and order of the true system. Our results are based on a modification of the notion of a most powerful unfalsified model for finite data and a novel algorithm for its computation. We provide a generalization of a result that became known as the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fundamental lemma</i> and a novel nonparametric data-driven representation of the system behavior based on general data matrix structures. The results assume exact data, however, low-rank approximation allows their application in the case of noisy data. We compare empirically low-rank approximation of the Hankel, Page, and trajectory matrices in the errors-in-variables setting. Although the Page and trajectory matrices are unstructured, the parameter estimates obtained are less accurate than the one obtained from the Hankel matrix.

Causal discovery of 1-factor measurement models in linear latent variable models with arbitrary noise distributions

The problem of causal discovery is especially challenging when the variables of interest cannot be directly measured. In measurement models, the measured variables were generated by latent causal variables that are causally related to each other, and by estimating the measurement model from measured data, one is able to recover the latent variables and their causal relations. In this paper, we provide precise sufficient identifiability conditions for the linear pure measurement model, and show what information of the causal structure can be recovered from observed data without prior knowledge of data distributions. In particular, we first show that, based on second-order statistics, although the pure measurement model is in general not fully identifiable under the assumption of two pure measurement variables, we can identify the set of all candidate measurement models. We then further prove that the pure measurement model can be identified uniquely based on higher-order statistics. Next, to address more general situations, we offer the identifiability conditions of linear measurement models with arbitrary noise distributions. We finally develop a unified method to learn pure measurement models from data. Experimental results on both synthetic and real-world data demonstrate the usefulness of our theory and the effectiveness of our approach.

Conditions of local parameter identifiability for systems of differential equations with an infinite-dimensional parameter

The problem of parameter identification (determining parameters of a system by observing solutions or functions of them) is one of the main problems of the applied theory of differential equations. The property of local identifiability plays the most important role in solving this problem. The presence of this property means that one can uniquely determine values of parameters of a system in a neighborhood of a particular parameter by observing solutions. Earlier the case of a finite-dimensional parameter was mostly studied in this issue. The problem of local parameter identifiability in the case of an infinite-dimensional parameter is less studied. In this paper we propose a new method for obtaining sufficient conditions for local parameter identifiability in the case of an infinite-dimensional parameter. Under these conditions an infinite-dimensional parameter belonging to certain classes is locally identifiable by observing a solution at a finite set of points. For system with linear dependence on a parameter we establish the genericity of the mentioned conditions.

Generic Identifiability of the DINA Model and Blessing of Latent Dependence.

Cognitive diagnostic models are a powerful family of fine-grained discrete latent variable models in psychometrics. Within this family, the DINA model is a fundamental and parsimonious one that has received significant attention. Similar to other complex latent variable models, identifiability is an important issue for CDMs, including the DINA model. Gu and Xu (Psychometrika 84(2):468-483, 2019) established the necessary and sufficient conditions for strict identifiability of the DINA model. Despite being the strongest possible notion of identifiability, strict identifiability may impose overly stringent requirements on designing the cognitive diagnostic tests. This work studies a slightly weaker yet very useful notion, generic identifiability, which means parameters are identifiable almost everywhere in the parameter space, excluding only a negligible subset of measure zero. We propose transparent generic identifiability conditions for the DINA model, relaxing existing conditions in nontrivial ways. Under generic identifiability, we also explicitly characterize the forms of the measure-zero sets where identifiability breaks down. In addition, we reveal an interesting blessing-of-latent-dependence phenomenon under DINA-that is, dependence between the latent attributes can restore identifiability under some otherwise unidentifiable [Formula: see text]-matrix designs. The blessing of latent dependence provides useful practical implications and reassurance for real-world designs of cognitive diagnostic assessments.

Reducing bias due to misclassified exposures using instrumental variables

AbstractExposures are often misclassified in observational studies. Any analysis that does not make proper adjustments for misclassification may result in biased estimates of model parameters, resulting in distorted inference. Settings where a multicategory exposure variable has more than two nominal categories or where no validation data are available to assess misclassification probabilities are common in practice but seldom considered in the literature. This article presents a novel method of analyzing cohort data with a misclassified, multicategory exposure variable and a binary response variable that uses instrumental variables in lieu of a validation dataset. First, a sufficient condition is obtained for model identifiability. Then, methods for model estimation and inference are proposed after adopting a sufficient condition for identifiability. We consider a variational Bayesian inference procedure aided by automatic differentiation along with Markov chain Monte Carlo‐based computation. Operating characteristics of the proposed methods are assessed through simulation studies. For the purpose of illustration, the proposed Bayesian methods are applied to the U.S. breast cancer mortality data sampled from the Surveillance Epidemiology and End Results database, where reported treatment therapy is the misclassified multicategory exposure variable.

Offline Pricing and Demand Learning with Censored Data

We study a single product pricing problem with demand censoring in an offline data-driven setting. In this problem, a retailer has a finite amount of inventory and faces a random demand that is price sensitive in a linear fashion with unknown price sensitivity and base demand distribution. Any unsatisfied demand that exceeds the inventory level is lost and unobservable. We assume that the retailer has access to an offline data set consisting of triples of historical price, inventory level, and potentially censored sales quantity. The retailer’s objective is to use the offline data set to find an optimal price, maximizing his or her expected revenue with finite inventories. Because of demand censoring in the offline data, we show that the existence of near-optimal algorithms in a data-driven problem—which we call problem identifiability—is not always guaranteed. We develop a necessary and sufficient condition for problem identifiability by comparing the solutions to two distributionally robust optimization problems. We propose a novel data-driven algorithm that hedges against the distributional uncertainty arising from censored data, with provable finite-sample performance guarantees regardless of problem identifiability and offline data quality. Specifically, we prove that, for identifiable problems, the proposed algorithm is near-optimal and, for unidentifiable problems, its worst-case revenue loss approaches the best-achievable minimax revenue loss that any data-driven algorithm must incur. Numerical experiments demonstrate that our proposed algorithm is highly effective and significantly improves both the expected and worst-case revenues compared with three regression-based algorithms. This paper was accepted by J. George Shanthikumar, big data analytics. Funding: This work was supported by the MIT Data Science Laboratory. J. Bu was partially supported by a Hong Kong Polytechnic University Start-up Fund for New Recruits [Project ID P0039585]. Supplemental Material: Data and the online appendices are available at https://doi.org/10.1287/mnsc.2022.4382 .

Moment‐based estimation for the multivariate COGARCH(1,1) process

AbstractFor the multivariate COGARCH process, we obtain explicit expressions for the second‐order structure of the “squared returns” process observed on an equidistant grid. Based on this, we present a generalized method of moments estimator for its parameters. Under appropriate moment and strong mixing conditions, we show that the resulting estimator is consistent and asymptotically normal. Sufficient conditions for strong mixing, stationarity and identifiability of the model parameters are discussed in detail. We investigate the finite sample behavior of the estimator in a simulation study.

System identification approach for inverse optimal control of finite-horizon linear quadratic regulators

The inverse optimal control for finite-horizon discrete-time linear quadratic regulators is investigated in this paper, which is to estimate the parameters in the objective function using noisy measurements of partial optimal states only. By the Pontryagin’s minimum principle, the concerned inverse optimal control problem is recast as the identification of a parameterized causal-and-anticausal mixed system excited by boundary conditions. Sufficient identifiability conditions for the unknown parameters are provided in terms of the system model itself, rather than relying on the exact values of optimal states or control inputs. In addition, an elegant algebraic solution is provided for the concerned identification problem that is inherently a challenging optimization problem with trilinear equality constraints, and it can recover the true parameters (up to a scalar ambiguity) in the absence of measurement noise or can consistently identify the parameters (up to a scalar ambiguity) in the presence of white measurement noise. The presented algebraic solution relies on recursive matrix calculations so that its computational burden is much less than directly solving a high-dimensional non-convex optimization problem as done in many existing works. The effectiveness of the proposed method as well as its noise sensitivity issue is illustrated by simulation examples.

Identifiability of Dynamic Networks from Structure

Abstract In this paper we determine under which additional requirements a dynamic network is generically identifiable when some structures within it are known to be generically identifiable. We derive a set of necessary or sufficient conditions to determine generic identifiability of the whole network. The conditions take form as rank conditions of these specific structures and also in the topology of the network. Necessary and sufficient conditions for generic identifiability are given for classes of networks with parallel paths among the nodes. For the quite general case of networks whose graph is acyclic, we present necessary and sufficient conditions to determine whether a particular node needs to be excited and/or measured.

Nonlinear mixed effects modeling and warping for functional data using B-splines

In functional data the interest is to find a global mean pattern, but also to capture the individual curve differences in phase and amplitude. This can be done conveniently by building in random effects on two levels: in the warping functions to account for individual phase variations; and in the linear structure to deal with individual amplitude variations. Via an appropriate choice of the warping function and B-spline approximations, estimation in the nonlinear mixed effects functional model is feasible, and does not require any prior knowledge on landmarks for the functional data. Sufficient and necessary conditions for identifiability of the flexible model are provided. A theoretical study is conducted: we establish asymptotic normality and consistency of the estimators of the registration and amplitude models, convergence of the iterative process, and consistency of the final estimator provided by the iterative process. The finite-sample performance of the proposed estimation procedure is investigated in a simulation study, which includes comparisons with existing methods. The added value of the developed method is further illustrated via the analysis of a real data example.

Polytopic Matrix Factorization: Determinant Maximization Based Criterion and Identifiability

We introduce Polytopic Matrix Factorization (PMF) as a novel data decomposition approach. In this new framework, we model input data as unknown linear transformations of some latent vectors drawn from a polytope. In this sense, the article considers a semi-structured data model, in which the input matrix is modeled as the product of a full column rank matrix and a matrix containing samples from a polytope as its column vectors. The choice of polytope reflects the presumed features of the latent components and their mutual relationships. As the factorization criterion, we propose the determinant maximization (Det-Max) for the sample autocorrelation matrix of the latent vectors. We introduce a sufficient condition for identifiability, which requires that the convex hull of the latent vectors contains the maximum volume inscribed ellipsoid of the polytope with a particular tightness constraint. Based on the Det-Max criterion and the proposed identifiability condition, we show that all polytopes that satisfy a particular symmetry restriction qualify for the PMF framework. Having infinitely many polytope choices provides a form of flexibility in characterizing latent vectors. In particular, it is possible to define latent vectors with heterogeneous features, enabling the assignment of attributes such as nonnegativity and sparsity at the subvector level. The article offers examples illustrating the connection between polytope choices and the corresponding feature representations.

Multi-Pass Sequential Mini-Batch Stochastic Gradient Descent Algorithms for Noise Covariance Estimation in Adaptive Kalman Filtering

Estimation of unknown noise covariances in a Kalman filter is a problem of significant practical interest in a wide array of applications. Although this problem has a long history, reliable algorithms for their estimation were scant, and necessary and sufficient conditions for identifiability of the covariances were in dispute until recently. Necessary and sufficient conditions for covariance estimation and a batch estimation algorithm were presented in our previous study. This paper presents stochastic gradient descent algorithms for noise covariance estimation in adaptive Kalman filters that are an order of magnitude faster than the batch method for similar or better root mean square error. More significantly, these algorithms are applicable to non-stationary systems where the noise covariances can occasionally jump up or down by an unknown magnitude. The computational efficiency of the new algorithms stems from adaptive thresholds for convergence, recursive fading memory estimation of the sample cross-correlations of the innovations, and accelerated stochastic gradient descent algorithms. The comparative evaluation of the proposed methods on a number of test cases demonstrates their computational efficiency and accuracy.

The Set of Linear Time-Invariant Unfalsified Models With Bounded Complexity is Affine

We consider exact system identification in the behavioral setting: Given an exact (noise-free) finite time series, find the set of bounded complexity linear time-invariant systems that fit the data exactly. First, we modify the notion of the most powerful unfalsified model for the case of finite data by fixing the number of inputs and minimizing the order. Then, we give necessary and sufficient identifiability conditions, i.e., conditions under which the true data generating system coincides with the most powerful unfalsified model. Finally, we show that the set of bounded complexity exact models is affine: Every exact model is a sum of the most powerful unfalsified model and an autonomous model with bounded complexity.

New trends in modeling and control of hybrid systems

New trends in modeling and control of hybrid systems

Identification of affine dynamical systems from a single trajectory

For dynamical systems of the form ẋ = Ax + c, henceforth called affine systems, we study the problem of determining system parameters A, c from data derived from a single trajectory. We establish necessary and sufficient conditions for identifiability of parameters from either a full trajectory or a set of discrete data points sampled from a single trajectory, expressed as geometrical conditions on the trajectory. We describe conditions under which the system has an equilibrium point and examine the problem of identifiability of the equilibrium point from data. We briefly examine the analytical upper and lower bounds for the maximal permissible uncertainty that will guarantee an inverse with specified qualitative properties of the resulting system (e.g., stability, spirality, and so on). Finally, we illustrate an application of the theory to parameter estimation for a class of nonlinear systems consisting of linear-in-parameters systems and demonstrate that affine approximations can yield more accurate estimates of parameter values than those based on finite differences.

Enhancing Explainability of Neural Networks Through Architecture Constraints.

Prediction accuracy and model explainability are the two most important objectives when developing machine learning algorithms to solve real-world problems. Neural networks are known to possess good prediction performance but suffer from a lack of model interpretability. In this article, we propose to enhance the explainability of neural networks through the following architecture constraints: 1) sparse additive subnetworks; 2) projection pursuit with orthogonality constraint; and 3) smooth function approximation. It leads to an enhanced explainable neural network (ExNN) with a superior balance between prediction performance and model interpretability. We derive sufficient identifiability conditions for the proposed ExNN model. The multiple parameters are simultaneously estimated by a modified minibatch gradient descent method based on the backpropagation algorithm for calculating the derivatives and the Cayley transform for preserving the projection orthogonality. Through simulation study under six different scenarios, we compare the proposed method to several benchmarks, including least absolute shrinkage and selection operator, support vector machine, random forest, extreme learning machine, and multilayer perceptron. It is shown that the proposed ExNN model keeps the flexibility of pursuing high prediction accuracy while attaining improved interpretability. Finally, a real data example is employed as a showcase application.

Sufficient and Necessary Conditions for the Identifiability of the Q-matrix

Restricted latent class models (RLCMs) have recently gained prominence in educational assessment, psychiatric evaluation, and medical diagnosis. Different from conventional latent class models, restrictions on the RLCM model parameters are imposed by a design matrix to respect practitioners' scientific assumptions. The design matrix, called $Q$-matrix in the cognitive diagnosis literature, is usually constructed by practitioners and domain experts, yet it is subjective and could be misspecified. To address this problem, researchers have proposed to estimate the $Q$-matrix from data. On the other hand, the fundamental learnability issue of the $Q$-matrix and model parameters remains underexplored and existing studies often impose stronger than needed or even impractical conditions. This paper proposes sufficient and necessary conditions for joint identifiability of the $Q$-matrix and the RLCM model parameters under different types of RLCMs. The developed identifiability conditions only depend on the design matrix and are easy to verify in practice.

Offline Pricing and Demand Learning with Censored Data

We study a single product pricing problem with demand censoring in an offline data-driven setting. In this problem, a retailer is given a finite amount of inventory, and faces a random demand that is price sensitive in a linear fashion with unknown price sensitivity and base demand distribution. Any unsatisfied demand that exceeds the inventory level is lost and unobservable. We assume that the retailer has access to an offline data set consisting of triples of historical price, inventory level and potentially censored sales quantity. The retailer's objective is to use the offline data set to find an optimal price, maximizing her expected revenue with finite inventories. Due to demand censoring in the offline data, we show that the existence of near-optimal algorithms in a data-driven problem -- which we call problem identifiability -- is not always guaranteed. We develop a necessary and sufficient condition for problem identifiability, and show that it is directly related to some quality metrics of the censored data. We propose a novel data-driven algorithm that hedges against the distribution uncertainty arising from censored data, and admits provable finite-sample performance guarantees regardless of problem identifiability and offline data quality. Specifically, we prove that, for identifiable problems, the proposed algorithm is near-optimal, and, for unidentifiable problems, its worst-case revenue loss approaches the best-achievable performance guarantee among all data-driven algorithms. The numerical experiments demonstrate that our algorithm is highly effective and significantly improves both the expected revenue and worst-case revenue compared with three baseline algorithms.