Description Length Criterion Research Articles

Minimal Polynomial Method for Estimating Parameters of Signals Received by an Antenna Array

The effectiveness of the projection minimal polynomial method for solving the problem of determining the number of sources of signals acting on an antenna array (AA) with an arbitrary configuration and their angular directions has been studied. The method proposes estimating the degree of the minimal polynomial of the correlation matrix (CM) of the input process in the AA on the basis of a statistically validated root-mean-square criterion. Special attention is paid to the case of the ultrashort sample of the input process when the number of samples is considerably smaller than the number of AA elements, which is important for multielement AAs. It is shown that the proposed method is more effective in this case than methods based on the AIC (Akaike’s Information Criterion) or minimum description length (MDL) criterion.

Information Theoretic Criteria for Induction Motor Fault Identification

The objective of our work is to identify the inter-turn incipient short fault that occurs in the induction motor at no load condition. The method used in fault identification is Information Theoretic Criteria, which uses Frequency Signal Dimension Order (FSDO) estimator and fault decision module. The FSDO estimator estimates the number of frequencies in stator current signatures using Minimum Description Length (MDL) criterion and Akaike Information Criteria (AIC). Fault decision module uses the number of frequencies as fault index in detecting the fault and identifying the fault severity. The proposed method is able to identify the fault from the data buried in noise. MDL yields consistent estimate, the fault index obtained using MDL criterion is considered for diagnosing the faults. The CDF plot of MDL and AIC helps in proving the fault severity results obtained from fault index value. With very lesser values of sampled data, the proposed method is able to distinguish between healthy and faulty conditions. This novel approach diagnoses the inter-turn incipient fault with very simple calculation and it needs only very few number of measured data for estimating the number of fault frequencies associated with the faulty conditions.

An MDL approach to efficiently discover communities in bipartite network

An minimum description length (MDL) criterion is proposed to choose a good partition for a bipartite network. A heuristic algorithm based on combination theory is presented to approach the optimal partition. As the heuristic algorithm automatically searches for the number of partitions, no user intervention is required. Finally, experiments are conducted on various datasets, and the results show that our method generates higher quality results than the state-of-art methods, cross-association and bipartite, recursively induced modules. Experiment results also show the good scalability of the proposed algorithm. The method is applied to traditional Chinese medicine (TCM) formula and Chinese herbal network whose community structure is not well known, and found that it detects significant and it is informative community division.

A Wavelet Spatial Correlation Algorithm to Partial Discharge Denoising Based on MDL Criterion

The partial discharge (PD) detection systems are often vulnerable to strong external interferences, and sometimes the PD signals are submerged in noises (white noise for example) completely. So the signals acquired must be preprocessed to obtain the reliable PD information. While there are many methods for white noise denoising, mostly are not very suitable for partial discharge. The wavelet transform (WT) coefficient of PD and white noises have different spread characteristics in different WT scales. Based on the Information Theory, The Minimum Information Description Length (MDL) criterion is a optimization strategy, a small amount of signal parameter is requried to the PD signals representation, the paper proposes a wavelet spatial correlation algorithm to partial discharge denoising based on MDL criterion: optimal wavelet function is selected based on MDL, then have the white noise reduced in WT, the algorithm has wonderful virtues such as free from any parameters estimation about noise, free from presetting threshhold and threshold chooseing behavior, so the algorithm is highly adaptive. Large amount of experimental results illustrate that the method presented in this paper are efficient and feasible and outperforms other general method of PD noise reduction.

Channel Prediction Using IAA-Based Spectral Estimation in Precoded TDD-MIMO Systems

This letter proposes a channel prediction scheme for precoded time division duplex multiple-input multiple-output (TDD-MIMO) systems. Based on the sum-of-sinusoidal (SoS) channel model, the proposed scheme first uses iterative adaptive approach (IAA)-based spectral estimation algorithm for sinusoidal components estimation according to the channel state information (CSI) detected in the discontinuous uplink periods. Then the minimum description length (MDL) criterion is used to choose the proper number of sinusoidal components for the channel model. Finally, time extrapolation is performed to the estimated channel model to predict the future downlink CSI that is needed for transmit precoding. The scheme is verified by simulations.

Improved Channel Estimation for OFDM Systems by Exploiting Ordered Space

In this paper, we propose an improved channel estimation method for orthogonal frequency division multiplexing (OFDM) systems in sparse multipath fading channels. The proposed method transforms the channel estimates into an ordered space and uses the minimum description length (MDL) criterion to estimate the number of paths and the path positions. Simulation results demonstrated that an improved channel estimation performance is obtained due to the reduction of signal space.

Research on the Speech Enhancement Method based on PCA/KLT Algorithms

In this paper, we propose a speech enhancement technique in terms of subspace methods to reduce the white or colored noise in strong background noise environment. This subspace approach based on Karhunen-Loève transform (KLT) and implemented via Principal Component Analysis (PCA). The subspace selection provided by the minimum description length (MDL) criterion. An offset factor generated from the white noise was used to modify the variance to adapt to the specified colored noise. The objective speech quality measures SegSNR have been introduced to evaluate the performance of the proposed method in time domain. A large amount of data and figures testify that our algorithm provides high performance for a large scale of input signal-to-noise ratio (-5~10dB). The performance of our algorithm is assessed in white and colored noise.

Dynamic Modeling for a Non-Linear System Based on the MOESP Algorithm

In order to measure accurately air fuel ratio in the engine exhaust gas, the Hammerstein model of the exhaust gas oxygen (EGO) sensor was identified using the multivariable output error state space (MOESP) algorithm. Firstly, a static model of the EGO sensor was identified based on engine experiment data as a static nonlinear part of the Hammerstein model. Then, the MOESP algorithm was used to build a state space model (SSM) of dynamic linear part of the Hammerstein model. To estimate model order, the Akaike Information Criterion (AIC) and the Minimum Description Length (MDL) criterion were computed, and validity of structure identification was verified by residual analysis. Then, the auto-regressive model with exogenous input (ARX) was compared with a subspace model. The result shows that a two-order SSM using MOESP algorithm is suitable to dynamic part of the Hammerstein model for EGO sensors.

Dimension estimation-based spectrum sensing for cognitive radio

In this article, we will derive closed-form expressions of false alarm probabilities for a given threshold for the dimension estimation-based detector (DED) using Akaike information criterion (AIC) and the minimum description length (MDL) criterion. Specifically, the DED algorithm will be formulated as a binary hypothesis test using AIC and MDL curves. Based on the proposed statistic test, we will express the probability of false alarm of the DED algorithm for a fixed threshold using the cumulative density function for the distribution of Tracy-Widom of order two. The derived analytical decision thresholds are verified with Monte-Carlo simulations and a comparison between simulation and analytical results to confirm the theoretical results are presented. These results confirm the very good match between simulation and theoretic results.

On Testing the Extent of Noncircularity

In this correspondence, we provide a multiple hypothesis test to detect the number of latent noncircular signals in a complex Gaussian random vector. Our method sequentially tests the results of individual generalized likelihood ratio test (GLRT) statistics with known asymptotic distributions to form the multiple hypothesis detector. Specifically, we are able to set a threshold yielding a precise probability of error. This test can be used to statistically determine if a given complex observation is circular Gaussian, and if not, how many latent signals in the observation are noncircular. Simulations are used to quantify the performance of the detector as compared to a detector based on the minimum description length (MDL) criterion. The utility of the detector is shown by applying it to a beamforming application using independent component analysis (ICA).

Description length and dimensionality reduction in functional data analysis

The use of description length principles to select an appropriate number of basis functions for functional data is investigated. A flexible definition of the dimension of a random function that is constructed directly from the Karhunen–Loève expansion of the observed process or data generating mechanism is provided. The results obtained show that although the classical, principle component variance decomposition technique will behave in a coherent manner, in general, the dimension chosen by this technique will not be consistent in the conventional sense. Two description length criteria are described. Both of these criteria are proved to be consistent and it is shown that in low noise settings they will identify the true finite dimension of a signal that is embedded in noise. Two examples, one from mass spectroscopy and the other from climatology, are used to illustrate the basic ideas. The application of different forms of the bootstrap for functional data is also explored and used to demonstrate the workings of the theoretical results.

A Blind Collision Resolution Protocol Based on Cooperative Transmission

Network-assisted diversity multiple access (NDMA) was recently proposed to resolve collision. The blind NDMA method using independent component analysis (ICA) ICA-NDMA, has been developed to overcome the difficulties of synchronization and orthogonal identification codes required by the training-based NDMA protocols. In this paper, we propose a novel blind collision resolution protocol, named as ICA-CoopNDMA, by employing cooperative transmission in ICA-NDMA. When K-node collision happened, a cooperative transmission mechanism is adopted to collect different mixtures of the K original packets in the following slots. After that, the destination can recover the original packets using ICA. This protocol offers spatial diversity by using relay nodes and is robust to the wireless channel. In order to determine the collision multiplicity K, a cooperative blind detection algorithm is also proposed based on minimum description length (MDL) criterion. This algorithm saves one detection slot compared with conventional MDL-based method used by ICA-NDMA. Hence, the delay of ICA-CoopNDMA is reduced. Simulation and theoretical analysis results show that, the proposed protocol can work effectively under both fast and slow fading channels. Compared with ICA-NDMA, the maximum stable throughput and average packet delay are obviously better, and the energy efficiency is higher under heavy traffic load.

On Semi-Supervised Learning Genetic-Based and Deterministic Annealing EM Algorithm for Dirichlet Mixture Models

We propose a genetic-based and deterministic annealing expectation-maximization (GA&DA-EM) algorithm for learning Dirichlet mixture models from multivariate data. This algorithm is capable of selecting the number of components of the model using the minimum description length (MDL) criterion. Our approach benefits from the properties of Genetic algorithms and deterministic annealing algorithm by combination of both into a single procedure. The population-based stochastic search of the GA&DA explores the search space more thoroughly than the EM method. Therefore, our algorithm enables escaping from local optimal solutions since the algorithm becomes less sensitive to its initialization. The GA&DA-EM algorithm is elitist which maintains the monotonic convergence property of the EM algorithm. We conducted experiments on the WebKB and 20NEWSGROUPS. The results show that show that 1) the GA&DA-EM outperforms the EM method since: Our approach identifies the number of components which were used to generate the underlying data more often than the EM algorithm. 2) the algorithm alternatives to EM that overcoming the challenges of local maxima.

A New Distance Measure for a Variable-Sized Acoustic Model Based on MDL Technique

Embedding a large vocabulary speech recognition system in mobile devices requires a reduced acoustic model obtained by eliminating redundant model parameters. In conventional optimization methods based on the minimum description length (MDL) criterion, a binary Gaussian tree is built at each state of a hidden Markov model by iteratively finding and merging similar mixture components. An optimal subset of the tree nodes is then selected to generate a downsized acoustic model. To obtain a better binary Gaussian tree by improving the process of finding the most similar Gaussian components, this paper proposes a new distance measure that exploits the difference in likelihood values for cases before and after two components are combined. The mixture weight of Gaussian components is also introduced in the component merging step. Experimental results show that the proposed method outperforms MDL-based optimization using either a Kullback-Leibler (KL) divergence or weighted KL divergence measure. The proposed method could also reduce the acoustic model size by 50% with less than a 1.5% increase in error rate compared to a baseline system.

Technology extraction from time series data reflecting expert operator skills and knowledge

Continuation processes in chemical and/or biotechnical plants always generate a large amount of time series data. However, since conventional process models are described as a set of control models, it is difficult to explain the complicated and active plant behaviours. Based on the background, this paper proposes a novel method to develop a process response model from continuous time-series data. The method consists of the following phases: (1) reciprocal correlation analysis, (2) process response model, (3) extraction of control rules, (4) extraction of a workflow and (5) detecting outliers. The main contribution of the research is to establish a method to mine a set of meaningful control rules from Learning Classifier System (LCS) using the Minimum Description Length (MDL) criteria and Tabu search method. The proposed method has been applied to an actual process of a biochemical plant and has shown the validity and the effectiveness.

Using neural networks and statistical tests for detecting changes in the process dynamics

In this paper, a real-time system for detecting changes in dynamic systems is designed. This study involves the determination and identification of black-box models, the design of a model-based residual generator and finally the evaluation of the residuals via two statistical information tests; the Page-Hinkley test and the Bayes classifier. It is intended to reveal any drift from the normal behaviour of the process. The process behaviour under its normal operating conditions is established by a reliable model. In order to obtain this reliable model for the process dynamics, the black-box identification by means of a NARX (Non-linear Auto-Regressive (AR) with exogenous input) model has been chosen in this study. It is based on the neural network approach. This paper shows also the choice and the performance of the neural network in the training and test phases. An analysis of the inputs number, hidden neurons and their influence on the behaviour of the neural predictor is carried out. Three statistical criteria, Aikeke's Information Criterion (AIC), Minimum Description Length (MDL) and Bayesian Information Criterion (BIC), are used for the validation of the experimental data. After describing the system architecture and the proposed methodology of the fault detection, we present a realistic application in order to show the technique's potential. The purpose is to detect the occurrence of change, and pinpoint the moment it occurred.

未知な変動パラメータを含むウィナーシステムの同定

Wiener systems which consist of a dynamic linear block followed by a static nonlinear element have been used in numerous applications. In many cases, the system parameters are affected by changes in the environmental conditions. This paper describes a new approach to the on-line identification of time-varying Wiener systems. The time-varying linear parameters and the static nonlinear characteristics are estimated by neural networks which can represent various nonlinear characteristics. The initial states of the linear model in each estimation window are not available in the Wiener systems. Thus, the initial states and the other system parameters are estimated simultaneously by nonlinear optimization techniques. Furthermore, the optimal numbers of hidden units in the neural networks are determined by the minimum description length (MDL) criterion. As a result of simulation by this method, more accurate parameters can be obtained than without estimation of the initial states and MDL. © 2010 Wiley Periodicals, Inc. Electron Comm Jpn, 93(4): 1–9, 2010; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecj.10247

Hidden Markov Modeling of Human Pivoting

In modeling human movement using hidden Markov models (HMM), the “optimal” HMM with an appropriate number of states is determined based on the minimum description length (MDL) criterion. Human pivoting, typifying graspless manipulation, is modeled using Gaussian mixture HMMs. Analyzing the obtained HMMs using metric multidimensional scaling (MDS) showed the features of individual movement. Such dissimilarity analysis can be used to validate models of tacit skills in human manipulation.

Application of the Evidence Procedure to the Estimation of Wireless Channels

We address the application of the Bayesian evidence procedure to the estimation of wireless channels. The proposed scheme is based on relevance vector machines (RVM) originally proposed by M. Tipping. RVMs allow to estimate channel parameters as well as to assess the number of multipath components constituting the channel within the Bayesian framework by locally maximizing the evidence integral. We show that, in the case of channel sounding using pulse-compression techniques, it is possible to cast the channel model as a general linear model, thus allowing RVM methods to be applied. We extend the original RVM algorithm to the multiple-observation/multiple-sensor scenario by proposing a new graphical model to represent multipath components. Through the analysis of the evidence procedure we develop a thresholding algorithm that is used in estimating the number of components. We also discuss the relationship of the evidence procedure to the standard minimum description length (MDL) criterion. We show that the maximum of the evidence corresponds to the minimum of the MDL criterion. The applicability of the proposed scheme is demonstrated with synthetic as well as real-world channel measurements, and a performance increase over the conventional MDL criterion applied to maximum-likelihood estimates of the channel parameters is observed.

Multi-component image segmentation in homogeneous regions based on description length minimization: Application to speckle, Poisson and Bernoulli noise

In this article, a minimum description length (MDL) criterion adapted to independent multi-component image segmentation into homogeneous regions is proposed. This approach, based on a deformable polygonal grid, allows us to segment noisy multi-component images perturbed with spatially independent speckle, Poisson or Bernoulli noise. The advantages of using such a multi-component approach rather than a mono-component one is demonstrated on synthetic and real images. This segmentation method is also applicable to multi-component images whose components do not follow the same noise statistics or have not been previously registered.