Model Order Estimation Research Articles

A parametric classification rule based on the exponentially embedded family.

In this paper, we extend the exponentially embedded family (EEF), a new approach to model order estimation and probability density function construction originally proposed by Kay in 2005, to multivariate pattern recognition. Specifically, a parametric classifier rule based on the EEF is developed, in which we construct a distribution for each class based on a reference distribution. The proposed method can address different types of classification problems in either a data-driven manner or a model-driven manner. In this paper, we demonstrate its effectiveness with examples of synthetic data classification and real-life data classification in a data-driven manner and the example of power quality disturbance classification in a model-driven manner. To evaluate the classification performance of our approach, the Monte-Carlo method is used in our experiments. The promising experimental results indicate many potential applications of the proposed method.

Subspace Alignment and Separation for Multiple Frequency Estimation

In this letter, a new subspace based estimator that can effectively provide the order and frequencies of multiple sinusoids in noise is proposed. The estimator, referred to as SAS-Est (Subspace Aligning and Separating Estimator), simultaneously seeks to separate the steering vectors from the noise subspace and align them to the signal subspace. The angles between subspaces and the generalized Kullback-Leibler divergence are used in characterizing the alignment and separation. Minimizing the divergence leads to maximal subspace separation and best alignment, thus allowing improved performance. Simulations in additive white Gaussian noise show that the new estimator offers an improvement for both model order and frequency estimation. When compared with other methods, the improvement is more pronounced for high model orders and low signal-to-noise ratio values.

Non-informative hierarchical Bayesian inference for non-negative matrix factorization

Non-negative matrix factorization (NMF) is an intuitive, non-negative, and interpretable approximation method. Canonical NMF approach could derive some basic components to represent original data, while probabilistic NMF approaches try to introduce some reasonable constraints to optimize the canonical NMF model. However, both of them cannot handle ground-truth bases discovering and model order determination problems. In general, the model order of basis matrix needs to be pre-defined. The model order determines the capability and accuracy of data structure discovering. However, how to accurately infer the model order of basis matrix has not been well investigated. In this paper, we propose a method called non-informative hierarchical Bayesian non-negative matrix factorization (NHBNMF) to automatically determine the model order and discover the data structure. They are achieved through hierarchical Bayesian inference model, maximum a posteriori (MAP) criterion, and non-informative parameters. In NHBNMF method, we first introduce a structure with two-level parameters to enable the entire model to approach the distributions of ground-truth bases. Then we use non-informative parameter scheme to eliminate the hyper-parameter to enable automatic searching. Finally, the model order and ground-truth bases are discovered by using MAP criterion and L2-norm selection. The experiments are conducted based on both synthetic and real-world datasets to show the effectiveness of our algorithm. The results demonstrate that our algorithm can accurately estimate the model order and discover the ground-truth bases. Even for the complicated FERET facial dataset, our algorithm still obtained interpretable bases and achieved satisfactory accuracy of the model order estimation.

IMPROVED MODEL ORDER ESTIMATION FOR NONLINEAR DYNAMIC SYSTEMS

In system modeling the choice of proper model structure is an essential task. Model structure is defined if both the model class and the size of the model within this class are determined. In dynamic system modeling model size is mainly determined by model order. The paper deals with the question of model order estimation when neural networks are used for modeling nonlinear dynamic systems. One of the possible ways of estimating the order of a neural model is the application of Lipschitz quotient. Although it is easy to use this method, its main drawback is the high sensitivity to noisy data. The paper proposes a new way to reduce the effect of noise. The idea of the proposed method is to combine the original Lipschitz method and the Errors In Variables (EIV) approach. The paper presents the details of the proposed combined method and gives the results of an extensive experimental study.

Robust data driven model order estimation for independent component analysis of FMRI data with low contrast to noise.

Independent component analysis (ICA) has been successfully utilized for analysis of functional MRI (fMRI) data for task related as well as resting state studies. Although it holds the promise of becoming an unbiased data-driven analysis technique, a few choices have to be made prior to performing ICA, selection of a method for determining the number of independent components (nIC) being one of them. Choice of nIC has been shown to influence the ICA maps, and various approaches (mostly relying on information theoretic criteria) have been proposed and implemented in commonly used ICA analysis packages, such as MELODIC and GIFT. However, there has been no consensus on the optimal method for nIC selection, and many studies utilize arbitrarily chosen values for nIC. Accurate and reliable determination of true nIC is especially important in the setting where the signals of interest contribute only a small fraction of the total variance, i.e. very low contrast-to-noise ratio (CNR), and/or very focal response. In this study, we evaluate the performance of different model order selection criteria and demonstrate that the model order selected based upon bootstrap stability of principal components yields more reliable and accurate estimates of model order. We then demonstrate the utility of this fully data-driven approach to detect weak and focal stimulus-driven responses in real data. Finally, we compare the performance of different multi-run ICA approaches using pseudo-real data.

Filtered-Covariance Function-Based Subspace Identification with Bound Effects Integration

A novel subspace covariance-based identification method that achieves consistent system estimates in the presence of highly-colored noise without requiring the use of weighting matrices has been derived recently. However, the use of Fourier transform in the computation of correlation functions requires the need to include the estimation of initial and final conditions. In this article we propose a solution to get unbiased estimates regardless of initial or end effects. Moreover, to further improve the model order estimation a filter-based solution is considered. This results in a new covariance-based algorithm that uses frequency weights computed with limit conditions in mind. A simulation example illustrates the algorithm performance.

Time Variant RFI Suppression for SAR Using Iterative Adaptive Approach

Under the condition of time variant RFI, the limitation of training sample size causes great performance degradation of the conventional Radio Frequency Interference (RFI) suppression algorithm based on eigen-subspace projection (ESP) method. A novel RFI suppression method using iterative adaptive approach (IAA) and orthogonal subspace projection (OSP) method is proposed for synthetic aperture radar (SAR). Dispensing with parametric search and model order estimation, the proposed method estimates the RFI power spectrum adaptively and iteratively, utilizing few training samples and filtering the RFI based on the OSP method. Both the simulation and experimental results are provided to illustrate the performance of the proposed method.

Dynamic Dictionary Algorithms for Model Order and Parameter Estimation

In this paper, we present and evaluate dynamic dictionary-based estimation methods for joint model order and parameter estimation. In dictionary-based estimation, a continuous parameter space is discretized, and vector-valued dictionary elements are formed for specific parameter values. A linear combination of a subset of dictionary elements is used to represent the model, where the number of elements used is the estimated model order, and the parameters corresponding to the selected elements are the parameter estimates. In static-based methods, the dictionary is fixed; while in the dynamic methods proposed here, the parameter sampling, and hence the dictionary, adapt to the data. We propose two dynamic dictionary-based estimation algorithms in which the dictionary elements are dynamically adjusted to improve parameter estimation performance. We examine the performance of both static and dynamic algorithms in terms of probability of correct model order selection and the root mean-squared error of parameter estimates. We show that dynamic dictionary methods overcome the problem of estimation bias induced by quantization effects in static dictionary-based estimation, and we demonstrate that dictionary-based estimation methods are capable of parameter estimation performance comparable to the Cramer-Rao lower bound and to traditional ML-based model estimation over a wide range of signal-to-noise ratios.

On the use of dispersion analysis for model assessment in structural identification

One of the most important issues faced in parametric time-domain identification and subsequent experimental/operational modal analysis is the correct estimation of model order, which in turn determines the number of structural vibration modes. The aim of this study is to provide a quantitative and physically meaningful framework for model order assessment that is characterized by global applicability, in the sense of implementation in both state-space and transfer function model representations. To this end and under the assumption of stationary wideband excitation, a novel dispersion analysis scheme is proposed for the quantification of every mode’s relative importance to the total stochastic response, which is based on a modal decomposition of the covariance matrix. Subsequently, after defining the modal dispersion matrix, a corresponding metric is introduced and used either as a stand alone tool for model order assessment, or as an extension of existing tools, such as stabilization diagrams. The method is validated through both simulated (NASA Mini-Mast truss) and experimental (suspended steel subframe flexible structure) identification problems, for which a subspace and a prediction-error estimation method are utilized and compared under the proposed quantitative indices. Moreover, performance comparisons with other energy-based metrics are also reported. The results indicate that the proposed method can be effectively used in parametric time-domain structural identification, for both order assessment and comparison of diverse model-based estimation methods.

On fractional models for human motion tracking

In this work we present the construction and evaluation of recursive Bayesian estimation algorithms for human motion tracking using fractional order models. The presented schemes include both fixed fractional order as well as variable order models with implicit and explicit model order estimation schemes. The performance of the designed techniques is compared to a set of conventional integral order methods using several measurements of human hand and head motions.

Model order estimation via penalizing adaptively the likelihood (PAL)

This paper introduces a novel rule for model order estimation based on penalizing adatively the likelihood (PAL). The penalty term of PAL, which is data adaptive (as the name suggests), has several unique features: it is “small” (e.g. comparable to AIC penalty) for model orders, let us say n, less than or equal to the true order, denoted by n0, and it is “large” (e.g. of the same order as BIC penalty) for n>n0; furthermore this is true not only as the data sample length increases (which is the case most often considered in the literature) but also as the signal-to-noise ratio (SNR) increases (the harder case for AIC, BIC and the like); and this “oracle-like” behavior of PAL's penalty is achieved without any knowledge about n0. The paper presents a number of simulation examples to show that PAL has an excellent performance also in non-asymptotic regimes and compare this performance with that of AIC and BIC.

Default Bayesian Estimation of the Fundamental Frequency

Joint fundamental frequency and model order estimation is an important problem in several applications. In this paper, a default estimation algorithm based on a minimum of prior information is presented. The algorithm is developed in a Bayesian framework, and it can be applied to both real- and complex-valued discrete-time signals which may have missing samples or may have been sampled at a non-uniform sampling frequency. The observation model and prior distributions corresponding to the prior information are derived in a consistent fashion using maximum entropy and invariance arguments. Moreover, several approximations of the posterior distributions on the fundamental frequency and the model order are derived, and one of the state-of-the-art joint fundamental frequency and model order estimators is demonstrated to be a special case of one of these approximations. The performance of the approximations are evaluated in a small-scale simulation study on both synthetic and real world signals. The simulations indicate that the proposed algorithm yields more accurate results than previous algorithms. The simulation code is available online.

Identification of Modal Parameters by Frequency Domain Subspace Method (Consideration of Residual Terms and Estimation of Model Order)

Identification of Modal Parameters by Frequency Domain Subspace Method (Consideration of Residual Terms and Estimation of Model Order)

Entropy-based subspace separation for multiple frequency estimation

A subspace-based algorithm for estimating the order and the frequencies of multiple sinusoids embedded in noise is proposed. The new estimator (referred to as E-MUSIC) uses the entropy of a random variable related to the angles between the signal and noise subspaces as its objective function. Maximizing the entropy tends to achieve uniform angle distribution and thus leads to maximal subspace separation. The entropy-based objective function and the performance of the E-MUSIC algorithm is compared with some reference algorithms in the literature. Simulations which are performed in additive white and colored Gaussian noise show that the E-MUSIC offers an improvement for both model order and multiple frequency estimation. The improvement is more pronounced for high model orders and large SNR values. ► We propose a method to estimate model order and the frequencies of a mixture of sinusoids. ► The method employs a new cost function which is based on the entropy of the angles between subspaces. ► The results show considerable improvement over the existing methods of multiple frequency estimators.

Efficient Foster-Type Macromodels for Rectangular Planar Interconnections

A novel macromodeling technique for passive linear multiport interconnecting structures is presented. It is based on an inherently stable Foster-type equivalent-circuit realization of the Greens function, given in the form of an infinite eigenfunction expansion. In order to obtain a finite number of equivalent-circuit elements, a rapidly convergent double sum representation of the corresponding port transfer-impedance function is cast into a newly modified Foster-type equivalent circuit with additional inductive elements. For a time-domain analysis, a system-theory based model-order estimation is presented and evaluated for trapezoidal excitation functions. Numerical computations are shown for a rectangular parallel-plate example structure, and validated by the results of full-wave field simulations, based on the finite-integration theory and method of moments. The suggested macromodeling technique provides a computationally very efficient and versatile approach for system analysis in the frequency- and time-domain with linear/nonlinear passive and active components.

Model identification for infinite variance autoregressive processes

We consider model identification for infinite variance autoregressive time series processes. It is shown that a consistent estimate of autoregressive model order can be obtained by minimizing Akaike’s information criterion, and we use all-pass models to identify noncausal autoregressive processes and estimate the order of noncausality (the number of roots of the autoregressive polynomial inside the unit circle in the complex plane). We examine the performance of the order selection procedures for finite samples via simulation, and use the techniques to fit a noncausal autoregressive model to stock market trading volume data.

System Identification with an Extended Threshold M-Estimator for Pseudo-Linear Models Structure

This paper deals with the problem of the estimation/validation of pseudo-linear models structure when outliers are present in the residuals. These corrupted data lead to unbounded modeling errors, involve long tail of the probability density function, and damage the model order estimation. Thus, from a ω-corrupted distribution model of these data, a parameterized robust estimation criterion (PREC) with an extended scaling factor in the Huber's norm and a robust final prediction error criterion (RFPE) based on M-estimates are proposed. Simulation results are given and the estimated orders of the robust models are derived from the new model validation criterion.

ARMA model order and parameter estimation using genetic algorithms

A new method for simultaneously determining the order and the parameters of autoregressive moving average (ARMA) models is presented in this article. Given an ARMA (p, q) model in the absence of any information for the order, the correct order of the model (p, q) as well as the correct parameters will be simultaneously determined using genetic algorithms (GAs). These algorithms simply search the order and the parameter spaces to detect their correct values using the GA operators. The proposed method works on the principle of maximizing the GA fitness value relying on the deviation between the actual plant output, with or without an additive noise, and the estimated plant output. Simulation results show in detail the efficiency of the proposed approach. In addition to that, a practical model identification and parameter estimation is conducted in this article with results obtained as desired. The new method is compared with other well-known methods for ARMA model order and parameter estimation.

The Consistency of MDL for Linear Regression Models With Increasing Signal-to-Noise Ratio

Recent work by Ding and Kay has demonstrated that the Bayesian information criterion (BIC) is an inconsistent estimator of model order in nested model selection as the noise variance τ*→ 0. Unfortunately, Ding and Kay have erroneously concluded that the minimum description length (MDL) principle also leads to inconsistent estimates of model order in this setting by equating BIC with MDL. This correspondence shows that only the earlier MDL criterion based on asymptotic assumptions has this problem, and proves that the new MDL linear regression criteria based on normalized maximum likelihood and Bayesian mixture codes satisfy the notion of consistency as τ*→ 0. The main result may be used as a basis to easily establish similar consistency results for other closely related information theoretic regression criteria.

DOA estimation of wideband sources without estimating the number of sources

In this paper, we propose a new technique to estimate wideband source directions from the sensor snapshots without requiring to know the number of sources present in the scenario. This work is motivated by the fact that the existing model order estimation (number of sources) techniques for wideband source scenario are either inaccurate or computationally expensive. Direction-of-arrival (DOA) estimation is realized using a beamformer framework which imposes nulls in the spatial spectrum along the source directions. The null width along the frequency axis is widened by introducing a new data dependent term into the optimization problem, thus achieving wideband capability. Furthermore, the temporal processing of the data snapshots drastically reduces the number of snapshots required for wideband DOA estimation. The effectiveness of the proposed formulation is studied with simulated experiments.