Higher Order Statistical Information Research Articles

Complex Correntropy Applied to a Compressive Sensing Problem in an Impulsive Noise Environment

Correntropy is a similarity function capable of extracting high-order statistical information from data. It has been used in different kinds of applications as a cost function to overcome traditional methods in non-Gaussian noise environments. One of the recent applications of correntropy was in the theory of compressive sensing, which takes advantage of sparsity in a transformed domain to reconstruct the signal from a few measurements. Recently, an algorithm called l 0 -MCC was introduced. It applies the Maximum Correntropy Criterion (MCC) in order to deal with a non-Gaussian noise environment in a compressive sensing problem. However, because correntropy was only defined for real-valued data, it was not possible to apply the l 0 -MCC algorithm in a straightforward way to compressive sensing problems dealing with complex-valued measurements. This paper presents a generalization of the l 0 -MCC algorithm to complex-valued measurements. Simulations show that the proposed algorithm can outperform traditional minimization algorithms such as Nesterov's algorithm (NESTA) and the l 0 -Least Mean Square (l 0 -LMS) in the presence of non-Gaussian noise.

The Attribute Optimization Method Based on the Probability Kernel Principal Component Analysis and Its Application

The principal component analysis (PCA) is the most common attribute optimization analysis techniques, but it is a linear method and exists the problem of lack of probability model and the absence of higher-order statistics information. It has poor comprehensive ability to complex non-linear attributes. Therefore, in order to overcome two shortcomings of the principal component analysis (PCA) and improve the effect of attribute optimization, this paper studies the probability kernel principal component analysis (PKPCA) method which is based on Bayesian theory and kernel principal component analysis (KPCA). First, the sample data are mapped to the high dimensional feature space, then define probability model of the data in high-dimensional space, and finally, expectation maximization (EM) estimated is used to get the best results. This method has both the advantage of probability analysis and kernel principal component analysis (KPCA). It is able to effectively adapt to more complex reservoir conditions and can realize the non-linear probability analysis. The probability kernel principal component analysis (PKPCA) method is applied to reservoir prediction of the Southern oil fields in China. The predicted results show that the method can improve the precision of the attribute optimization, while improving the accuracy of the forecasts of reservoir.

Complex correntropy function: Properties, and application to a channel equalization problem

The use of correntropy as a similarity measure has been increasing in different scenarios due to the well-known ability to extract high-order statistic information from data. Recently, a new similarity measure between complex random variables was defined and called complex correntropy. Based on a Gaussian kernel, it extends the benefits of correntropy to complex-valued data. However, its properties have not yet been formalized. This paper studies the properties of this new similarity measure and extends this definition to positive-definite kernels. Complex correntropy is applied to a channel equalization problem as good results are achieved when compared with other algorithms such as the complex least mean square (CLMS), complex recursive least squares (CRLS), and least absolute deviation (LAD).

Automatic ocular artifacts removal in EEG using deep learning

Abstract Ocular artifacts (OAs) are one the most important form of interferences in the analysis of electroencephalogram (EEG) research. OAs removal/reduction is a key analysis before the processing of EEG signals. For classic OAs removal methods, either an additional electrooculogram (EOG) recording or multi-channel EEG is required. To address these limitations of existing methods, this paper investigates the use of deep learning network (DLN) to remove OAs in EEG signals. The proposed method consists of offline stage and online stage. In the offline stage, training samples without OAs are intercepted and used to train an DLN to reconstruct the EEG signals. The high-order statistical moments information of EEG is therefore learned. In the online stage, the trained DLN is used as a filter to automatically remove OAs from the contaminated EEG signals. Compared with the exiting methods, the proposed method has the following advantages: (i) nonuse of additional EOG reference signals, (ii) any few number of EEG channels can be analyzed, (iii) time saving, and (iv) the strong generalization ability, etc. In this paper, both public database and lab individual data for EEG analysis are used, we compared the proposed method with the classic independent component analysis (ICA), kurtosis-ICA (K-ICA), Second-order blind identification (SOBI) and a shallow network method. Experimental results show that the proposed method performs better even for very noisy EEG.

Independent Vector Analysis Applied to Remove Muscle Artifacts in EEG Data

Electroencephalogram (EEG) data are often contaminated by various electrophysiological artifacts. Among all these artifacts, the muscle activity is particularly difficult to remove. In the literature, independent component analysis (ICA) and canonical correlation analysis (CCA), as blind source separation techniques, are the most popular methods. In this paper, we introduce a novel method for removing muscle artifacts in EEG data based on independent vector analysis. This method exploits both the second-order and higher order statistical information and thus takes advantage of both ICA and CCA. The proposed method is evaluated on realistic simulated data and is shown to significantly outperform ICA and CCA. In addition, the proposed method is applied on real ictal EEG data seriously contaminated with muscle artifacts. The proposed method is able to largely suppress muscle artifacts without altering the underlying EEG activity.

High-Order Local Pooling and Encoding Gaussians Over a Dictionary of Gaussians.

Local pooling (LP) in configuration (feature) space proposed by Boureau et al. explicitly restricts similar features to be aggregated, which can preserve as much discriminative information as possible. At the time it appeared, this method combined with sparse coding achieved competitive classification results with only a small dictionary. However, its performance lags far behind the state-of-the-art results as only the zero-order information is exploited. Inspired by the success of high-order statistical information in existing advanced feature coding or pooling methods, we make an attempt to address the limitation of LP. To this end, we present a novel method called high-order LP (HO-LP) to leverage the information higher than the zero-order one. Our idea is intuitively simple: we compute the first- and second-order statistics per configuration bin and model them as a Gaussian. Accordingly, we employ a collection of Gaussians as visual words to represent the universal probability distribution of features from all classes. Our problem is naturally formulated as encoding Gaussians over a dictionary of Gaussians as visual words. This problem, however, is challenging since the space of Gaussians is not a Euclidean space but forms a Riemannian manifold. We address this challenge by mapping Gaussians into the Euclidean space, which enables us to perform coding with common Euclidean operations rather than complex and often expensive Riemannian operations. Our HO-LP preserves the advantages of the original LP: pooling only similar features and using a small dictionary. Meanwhile, it achieves very promising performance on standard benchmarks, with either conventional, hand-engineered features or deep learning-based features.

Analysis Model of Physical Leakage Flow Based on Blind Source Separation Theory

A new physical leakage flow analysis model is established based on blind source separation theory (BSS). The model simultaneously considers the correlation and uncertainty between water consumption and physical leakage flow, takes constraints independent component analysis (CICA) algorithm in BSS, respectively establishes reference vector of water consumption and physical leakage flow to obtain the waveform information of physical leakage flow. The standard deviation and mean of the two flows are respectively calculated based on water balance analysis report and minimum night flow, to achieve the amplitude reduction of physical leakage flow. This analysis model is the application of the high-order statistical information of water distribution network operation data with high simulation accuracy.

Learning Linear Representation of Space Partitioning Trees Based on Unsupervised Kernel Dimension Reduction.

Space partitioning trees, which sequentially divide and subdivide a space into disjoint subsets using splitting hyperplanes, play a key role in accelerating the query of samples in the cybernetics and computer vision domains. Associated methods, however, suffer from the curse of dimensionality or stringent assumptions on the data distribution. This paper presents a new concept, termed kernel dimension reduction-tree (KDR-tree), that relies on linear projections computed based on an unsupervised kernel dimension reduction approach. The proposed concept does not rely on any assumption on the data distribution and can capture higher-order statistical information encapsulated within the data. This paper then develops two variants of the KDR-tree concept: 1) to handle residual data [i.e., the residual-based KDR-tree (rKDR-tree) algorithm] and 2) to cope with larger datasets, [i.e., the sampling-based KDR-tree (sKDR-tree) algorithm]. By directly comparing the KDR-tree concept to competitive techniques, involving several benchmark datasets, this paper shows that the sKDR-tree yields a better performance for non-Gaussian distributed datasets. Based on the analysis of three datasets, this paper highlights, experimentally, that the rKDR-tree has the potential to discover the intrinsic dimension. This paper also provides a theoretical analysis about the KDR-tree concept to outline why it outperforms existing techniques if the data distribution is non-Gaussian.

Cyclostationary correntropy: Definition and applications

Information extraction is a frequent and relevant problem in digital signal processing. In the past few years, different methods have been utilized for the parameterization of signals and the achievement of efficient descriptors. When the signals possess statistical cyclostationary properties, the Cyclic Autocorrelation Function (CAF) and the Spectral Cyclic Density (SCD) can be used to extract second-order cyclostationary information. However,second-order statistics tightly depends on the assumption of gaussianity, as the cyclostationary analysis in this case should comprise higher-order statistical information. This paper proposes a new mathematical formulation for the higher-order cyclostationary analysis based on the correntropy function. The cyclostationary analysis is revisited focusing on the information theory, while the Cyclic Correntropy Function (CCF) and Cyclic Correntropy Spectral Density (CCSD) are also presented. The CCF has different properties compared with CAF that can be very useful in non-gaussian signal processing, especially in the impulsive noise environment which implies in the expansion of the class of problems addressed by the second-order cyclostationary analysis. In particular, we prove that the CCF contains information regarding second- and higher-order cyclostationary moments, being a generalization of the CAF. The performance of the aforementioned functions in the extraction of higher-order cyclostationary characteristics is analyzed in a wireless communication system in which non-gaussian noise is present. The results demonstrate the advantages of the proposed method over the second-order cyclostationary.

Removal of EOG artifacts from EEG using a cascade of sparse autoencoder and recursive least squares adaptive filter

Electrooculogram (EOG) artifacts are the most important form of interferences in electroencephalogram (EEG) based brain computer interfaces (BCIs). In traditional methods for EOG artifacts removal, either an additional EOG recording in real time or multi-channel (more than three channels) EEG recording is required. To address these limitations of existing methods, a method using a cascade of sparse autoencoder (SAE) and recursive least squares (RLS) adaptive filter is proposed to remove the EOG artifacts from EEG. The proposed approach consists of offline stage and online stage. The high-order statistical moments information in the EOG artifacts can be learned automatically by using only EOG signals during offline stage and so an SAE model is obtained. In the online stage, the learned SAE model is firstly used to identify and extract preliminary EOG artifacts from a given raw EEG signal. Then an RLS adaptive filter uses the identified EOG artifacts as reference signal to remove interference without parallel EOG recordings. Compared with the exiting methods, the proposed method has the following advantages: (i) nonuse of an additional EOG recording in removal process, (ii) few number of EEG channels being used in removal process, and (iii) time-saving. The performance of the proposed method is evaluated by EEG classification accuracy and time consumption. Compared with traditional methods, the proposed method is proven to be more effective and faster. Moreover, experiment results also show good generalization ability in cross-subject testing scenarios.

Two-level independent component regression model for multivariate spectroscopic calibration

In this paper, a two-level independent component regression (ICR) model is developed for multivariate spectroscopic calibration. Compared to the traditionally used principal component regression and partial least squared regression model, the ICR model is more efficient to extract high order statistical information from the spectra data. To improve the calibration performance, an ensemble form of the ICR model is proposed. In the first level of the method, various subspaces are constructed based on the independent component decomposition of the original data space. Meanwhile, by defining a related index, the most important variables in each subspace are selected for ICR modeling, which form the second level of the proposed method. A Bayesian inference strategy is further developed for probabilistic combination of calibration results obtained from different subspaces. For performance evaluation, two case studies are carried out on a benchmark spectra dataset.

SAR Ground Moving Target Imaging Algorithm Based on Parametric and Dynamic Sparse Bayesian Learning

In this paper, a novel synthetic aperture radar (SAR) ground moving target imaging (GMTIm) algorithm is presented within a parametric and dynamic sparse Bayesian learning (SBL) framework. A new time–frequency representation, which is known as Lv's distribution (LVD), is employed on the moving targets to determine the parametric dictionary used in the SBL framework. To combat the inherent accuracy limitations of the LVD and extrinsic perturbation errors, a dynamical refinement process is further developed and incorporated into the SBL framework to obtain highly focused SAR image of multiple moving targets. An emerging inference technique, which is known as variational Bayesian expectation–maximization, is applied to achieve an efficient Bayesian inference for the focused SAR moving target image. A remarkable advantage of the proposed algorithm is to provide a fully posterior distribution (Bayesian inference) for the SAR moving target image, rather than a poor point estimate used in conventional methods. Because of utilizing high-order statistical information, the error propagation problem is desirably ameliorated in an iterative manner. The perturbations, known as the multiplicative phase error and additive clutter and noise, are both well adjusted for further improving the image quality. Experimental results by using simulated spotlight-SAR data and real Gotcha data have demonstrated the superiority of the proposed algorithm over other reported ones.

A Promising Technique for Blind Identification: The Generic Statistics

The generic statistics, related to lower-order derivatives of the log characteristic function (CAF) evaluating at the processing points located away from the origin, are frequently used in multivariate statistical signal processing. In comparison with the conventional statistics, e.g., cumulants, the generic statistics have the following advantages: (a) they can offer the structural simplicity and controllable statistical stability of lower-order statistics, and retain higher-order statistical information; (b) if the derivatives of the log CAF were evaluated at all (infinitely many) possible processing-points, a complete description of the joint CAF would be obtained. Furthermore, we show in this paper that even if a random process is symmetrically distributed, the odd-order generic statistics are not equal to zero, while in such a case the odd-order cumulants are equal to zero. For these reasons, a family of blind identification (BI) methods, in which the mixing matrix is obtained by decomposing the tensor constructed by the higher order derivatives of the log CAF of the observations, is proposed to achieve BI of underdetermined mixtures. Simulation results show that the BI methods based on generic statistics have superior performance to the existing cumulant-based method, such as FOOBI method, especially, when the SNR of the observations is high and/or the data block is short.

Monitoring multi-mode plant-wide processes by using mutual information-based multi-block PCA, joint probability, and Bayesian inference

A multi-mode plant-wide process monitoring scheme that integrates mutual information (MI)-based multi-block principal component analysis (PCA), joint probability, and Bayesian inference is developed. Given that the prior process is not always available, an MI-based block division method is proposed to divide blocks automatically by considering both cross-relations and high-order statistical information among variables. The PCA monitoring model is established in each sub-block and each mode, and a joint probability based on T2 statistics is defined to identify running-on mode. Then, the statistics in different sub-blocks are combined by using Bayesian inference to provide an intuitive indication. Finally, an improved contribution plot method is proposed to identify the root cause of faults. The feasibility and efficiency of the proposed method are evaluated by case studies on a numerical process and the Tennessee Eastman benchmark process. Monitoring results and comparisons with conventional PCA methods indicate the superiority of the proposed method.

Adaptive Conditioning of Multiple-Point Statistical Facies Simulation to Flow Data with Probability Maps

Multiple-point statistics (MPS) provides a flexible grid-based approach for simulating complex geologic patterns that contain high-order statistical information represented by a conceptual prior geologic model known as a training image (TI). While MPS is quite powerful for describing complex geologic facies connectivity, conditioning the simulation results on flow measurements that have a nonlinear and complex relation with the facies distribution is quite challenging. Here, an adaptive flow-conditioning method is proposed that uses a flow-data feedback mechanism to simulate facies models from a prior TI. The adaptive conditioning is implemented as a stochastic optimization algorithm that involves an initial exploration stage to find the promising regions of the search space, followed by a more focused search of the identified regions in the second stage. To guide the search strategy, a facies probability map that summarizes the common features of the accepted models in previous iterations is constructed to provide conditioning information about facies occurrence in each grid block. The constructed facies probability map is then incorporated as soft data into the single normal equation simulation (snesim) algorithm to generate a new candidate solution for the next iteration. As the optimization iterations progress, the initial facies probability map is gradually updated using the most recently accepted iterate. This conditioning process can be interpreted as a stochastic optimization algorithm with memory where the new models are proposed based on the history of the successful past iterations. The application of this adaptive conditioning approach is extended to the case where multiple training images are proposed as alternative geologic scenarios. The advantages and limitations of the proposed adaptive conditioning scheme are discussed and numerical experiments from fluvial channel formations are used to compare its performance with non-adaptive conditioning techniques.

Online Monitoring of Multivariate Processes Using Higher-Order Cumulants Analysis

In this study, a novel approach is developed for online state monitoring based on higher-order cumulants analysis (HCA). This approach applies higher-order cumulants to monitor a multivariate process, and while conventional approaches such as independent components analysis (ICA) uses variance to monitor process. Variance is lower-order statistics and is only sensitive to amplitude. In contrast, higher-order cumulants, the typical higher-order statistics, carry important information and are sensitive to both amplitude and phase, particularly for non-Gaussian distributions. The main idea of this novel approach is to monitor the cumulants of dominant independent components and residuals of the ICA model. Therefore, higher-order statistical information of multivariate processes can be monitored online. Furthermore, a variable contribution analysis scheme is developed for HCA to diagnose faults. The proposed approach is applied to the Tennessee Eastman (TE) process to exhibit its effectiveness. The results de...

Digit Recognition Based on Distance Metric by Gabor Transformation

Distance metric is important for image transform, it reflects the similarity of manifold between pre and post transformation. Tangent distance method is usually used to approximate nonlinear manifolds by its tangent hyperplane in digit recognition; however, it sometimes neglects the high-order statistic information of the image. A new image distance metric is proposed in this paper which is based on Gabor transformation to obtain the Gabor feature vector, then the feature manifold can be approximated by curve surfaces based on second-order Taylor expansion with respect to intrinsic variables, and the distance metric by Gabor transformation is defined as the minimum distance between the approximated curved surfaces in feature observation space. Experiments show that the distance metric based on Gabor transformation and manifold works well on digit recognition, it excelled tangent distance method.

Is statistical learning constrained by lower level perceptual organization?

In order for statistical information to aid in complex developmental processes such as language acquisition, learning from higher-order statistics (e.g. across successive syllables in a speech stream to support segmentation) must be possible while perceptual abilities (e.g. speech categorization) are still developing. The current study examines how perceptual organization interacts with statistical learning. Adult participants were presented with multiple exemplars from novel, complex sound categories designed to reflect some of the spectral complexity and variability of speech. These categories were organized into sequential pairs and presented such that higher-order statistics, defined based on sound categories, could support stream segmentation. Perceptual similarity judgments and multi-dimensional scaling revealed that participants only perceived three perceptual clusters of sounds and thus did not distinguish the four experimenter-defined categories, creating a tension between lower level perceptual organization and higher-order statistical information. We examined whether the resulting pattern of learning is more consistent with statistical learning being “bottom-up,” constrained by the lower levels of organization, or “top-down,” such that higher-order statistical information of the stimulus stream takes priority over perceptual organization and perhaps influences perceptual organization. We consistently find evidence that learning is constrained by perceptual organization. Moreover, participants generalize their learning to novel sounds that occupy a similar perceptual space, suggesting that statistical learning occurs based on regions of or clusters in perceptual space. Overall, these results reveal a constraint on learning of sound sequences such that statistical information is determined based on lower level organization. These findings have important implications for the role of statistical learning in language acquisition.

Charrelation and Charm: Generic Statistics Incorporating Higher-Order Information

Classical Higher-Order Statistics (HOS, in the form of high-order cumulants) are a powerful tool in the context of multivariate statistical analysis, often entailing valuable statistical information beyond Second-Order statistics (SOS), albeit at the expense of increased computational and notational complexity and compromised statistical stability (in the sense that longer observation intervals might be required in order to fully realize the advantages of HOS over SOS). In this paper, we consider new generic tools, offering the structural simplicity and controllable statistical stability of SOS on the one hand, yet retaining higher-order statistical information on the other hand. While cumulants are related to high-order derivatives of the log characteristic function at the origin, our new tools are related to lower-order (first and second) derivatives away from the origin, at locations called processing-points, and are termed charmean (or charm, in short) and charrelation. The charm and charrelation coincide with the classical mean and covariance (resp.) when the processing-point approaches the origin, but can offer continuously tunable tradeoffs between statistical stability and information contents as the processing-point is dragged away from the origin. Our goal in this paper is to introduce the underlying mathematical-statistical concepts for the development and analysis of charm- and charrelation-based estimation. We derive explicit expressions for the asymptotic bias and variance of their sample-estimates, which in turn enable data-driven selection of the processing-points, so as to minimize the predicted mean square estimation error in a given problem-as we demonstrate in several simulation examples.

Separation of global time-variable gravity signals into maximally independent components

The Gravity Recovery and Climate Experiment (GRACE) products provide valuable information about total water storage variations over the whole globe. Since GRACE detects mass variations integrated over vertical columns, it is desirable to separate its total water storage anomalies into their original sources. Among the statistical approaches, the principal component analysis (PCA) method and its extensions have been frequently proposed to decompose the GRACE products into space and time components. However, these methods only search for decorrelated components that on the one hand are not always interpretable and on the other hand often contain a superposition of independent source signals. In contrast, independent component analysis (ICA) represents a technique that separates components based on assumed statistical independence using higher-order statistical information. If one assumes that independent physical processes generate statistically independent signal components added up in the GRACE observations, separating them by ICA is a reliable strategy to identify these processes. In this paper, the performance of the conventional PCA, its rotated extension and ICA are investigated when applied to the GRACE-derived total water storage variations. These analyses have been tested on both a synthetic example and on the real GRACE level-2 monthly solutions derived from GeoForschungsZentrum Potsdam (GFZ RL04) and Bonn University (ITG2010). Within the synthetic example, we can show how imposing statistical independence in the framework of ICA improves the extraction of the ‘original’ signals from a GRACE-type super-position. We are therefore confident that also for the real case the ICA algorithm, without making prior assumptions about the long-term behaviour or on the frequencies contained in the signal, improves over the performance of PCA and its rotated extension in the separation of periodical and long-term components.