Fourth Order Blind Identification Research Articles

Asymptotic and bootstrap tests for subspace dimension

Many linear dimension reduction methods proposed in the literature can be formulated using an appropriate pair of scatter matrices. The eigen-decomposition of one scatter matrix with respect to another is then often used to determine the dimension of the signal subspace and to separate signal and noise parts of the data. Three popular dimension reduction methods, namely principal component analysis (PCA), fourth order blind identification (FOBI) and sliced inverse regression (SIR) are considered in detail and the first two moments of subsets of the eigenvalues are used to test for the dimension of the signal space. The limiting null distributions of the test statistics are discussed and novel bootstrap strategies are suggested for the small sample cases. In all three cases, consistent test-based estimates of the signal subspace dimension are introduced as well. The asymptotic and bootstrap tests are illustrated in real data examples.

Independent component analysis for multivariate functional data

We extend two methods of independent component analysis, fourth order blind identification and joint approximate diagonalization of eigen-matrices, to vector-valued functional data. Multivariate functional data occur naturally and frequently in modern applications, and extending independent component analysis to this setting allows us to distill important information from this type of data, going a step further than the functional principal component analysis. To allow the inversion of the covariance operator we make the assumption that the dependency between the component functions lies in a finite-dimensional subspace. In this subspace we define fourth cross-cumulant operators and use them to construct the two novel, Fisher consistent methods for solving the independent component problem for vector-valued functions. Both simulations and an application on a hand gesture data set show the usefulness and advantages of the proposed methods over functional principal component analysis.

An overview of properties and extensions of FOBI

Recently, Spurek, Tabor, Struksi and Smieja (2018) suggested an independent component analysis method that is obtained via a generalized eigenvalue decomposition and stated that their estimator enjoys several useful properties. The estimator is however already known since 1989 as FOBI (Fourth order blind identification), and it indeed has many nice properties, even outside the independent component model. As there seems to be interest in the readers of Knowledge-Based Systems in the estimator, we shortly review it and various of its properties not mentioned by Spurek et al. (2018), along with some recent extensions of FOBI.

Independent component analysis: A statistical perspective

Independent component analysis (ICA) is a data analysis tool that can be seen as a refinement of principal component analysis or factor analysis. ICA recovers the structures in the data which stay hidden if only the covariance matrix is used in the analysis. The ICA problem is formulated as a latent variable model where the observed variables are linear combinations of unobserved mutually independent non‐Gaussian variables. The goal is to recover linear transformations back to these latent independent components (ICs). As a statistical tool, the unmixing procedure is expressed as a functional in a relevant semiparametric model which further allows a careful formulation of the inference problem and the comparison of competing estimation procedures. For most approaches, the ICs are found in two steps, (a) by standardizing the random vector and then (b) by rotating the standardized vector to the ICs. In the projection pursuit, the ICs can be found either one‐by‐one or simultaneously and this is discussed in detail when the convex combination of the squared third and fourth cumulants is used as a projection index. Alternative projection indices and their use are also explained. The classical fourth‐order blind identification (FOBI) and joint approximate diagonalization of eigenmatrices (JADE) are described as well. The statistical tools for the comparison of consistent and asymptotically multivariate normal unmixing matrix estimates are discussed. Finally, recent extensions for times series, matrix‐ and tensor‐valued and functional data are reviewed.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms Statistical Models > Multivariate Models Statistical and Graphical Methods of Data Analysis > Dimension Reduction Statistical and Graphical Methods of Data Analysis > Information Theoretic Methods

JADE for Tensor-Valued Observations

ABSTRACTIndependent component analysis is a standard tool in modern data analysis and numerous different techniques for applying it exist. The standard methods however quickly lose their effectiveness when the data are made up of structures of higher order than vectors, namely, matrices or tensors (e.g., images or videos), being unable to handle the high amounts of noise. Recently, an extension of the classic fourth-order blind identification (FOBI) specially suited for tensor-valued observations was proposed and showed to outperform its vector version for tensor data. In this article, we extend another popular independent component analysis method, the joint approximate diagonalization of eigen-matrices (JADE), for tensor observations. In addition to the theoretical background, we also provide the asymptotic properties of the proposed estimator and use both simulations and real data to show its usefulness and superiority over its competitors. Supplementary material including the proofs of the theorems and the codes for running the simulations and the real data example are available online.

Independent component analysis for tensor-valued data

In preprocessing tensor-valued data, e.g., images and videos, a common procedure is to vectorize the observations and subject the resulting vectors to one of the many methods used for independent component analysis (ICA). However, the tensor structure of the original data is lost in the vectorization and, as a more suitable alternative, we propose the matrix- and tensor fourth order blind identification (MFOBI and TFOBI). In these tensorial extensions of the classic fourth order blind identification (FOBI) we assume a Kronecker structure for the mixing and perform FOBI simultaneously on each direction of the observed tensors. We discuss the theory and assumptions behind MFOBI and TFOBI and provide two different algorithms and related estimates of the unmixing matrices along with their asymptotic properties. Finally, simulations are used to compare the method’s performance with that of classical FOBI for vectorized data and we end with a real data clustering example.

Asymptotic and Bootstrap Tests for the Dimension of the Non-Gaussian Subspace

Dimension reduction is often a preliminary step in the analysis of large data sets. The so-called non-Gaussian component analysis searches for a projection onto the non-Gaussian part of the data, and it is then important to know the correct dimension of the non-Gaussian signal subspace. In this letter, we develop asymptotic as well as bootstrap tests for the dimension based on the popular fourth-order blind identification method.

Fourth Moments and Independent Component Analysis

In independent component analysis it is assumed that the components of the observed random vector are linear combinations of latent independent random variables, and the aim is then to find an estimate for a transformation matrix back to these independent components. In the engineering literature, there are several traditional estimation procedures based on the use of fourth moments, such as FOBI (fourth order blind identification), JADE (joint approximate diagonalization of eigenmatrices), and FastICA, but the statistical properties of these estimates are not well known. In this paper various independent component functionals based on the fourth moments are discussed in detail, starting with the corresponding optimization problems, deriving the estimating equations and estimation algorithms, and finding asymptotic statistical properties of the estimates. Comparisons of the asymptotic variances of the estimates in wide independent component models show that in most cases JADE and the symmetric version of FastICA perform better than their competitors.

Study on the Use of Blind ICA in Diesel Engine Vibration Processing

A method which can be used to analyze and separate the vibration signals of diesel engine is proposed. The vibration signals contain a great deal of information about the engine’s fault state, and it is hard to obtain the fault characteristic parameters because of the complex mechanical movement and operating conditions. Study on vibration by fourth order blind identification is carried out in this paper. And FOBI model that estimate the separation matrix by independent component analysis is established and applied to diesel engine vibration to separate the different signals. The results show that signals of different characteristics can be separated perfectly. This method can be used as the pre-processing step to obtain the fault characteristic parameters.

Estimating source kurtosis directly from observation data for ICA

Based on the fourth order blind identification (FOBI) in ICA methods, this paper shows that kurtosis of each independent source component can be estimated with eigenvalues of two relative matrices directly from observation data. With the new approach, the total number of super-Gaussian components in noiseless observation data can be directly calculated before any de-mixing algorithm is performed. As an application, a new switching criterion for the extended infomax algorithm is presented. Experimental results demonstrate the effectiveness of the kurtosis estimation algorithm and the new alternative switching criterion.

Fourth-order blind identification of underdetermined mixtures of sources (FOBIUM)

For about two decades, numerous methods have been developed to blindly identify overdetermined (P/spl les/N) mixtures of P statistically independent narrowband (NB) sources received by an array of N sensors. These methods exploit the information contained in the second-order (SO), the fourth-order (FO) or both the SO and FO statistics of the data. However, in practical situations, the probability of receiving more sources than sensors increases with the reception bandwidth and the use of blind identification (BI) methods able to process underdetermined mixtures of sources, for which P>N may be required. Although such methods have been developed over the past few years, they all present serious limitations in practical situations related to the radiocommunications context. For this reason, the purpose of this paper is to propose a new attractive BI method, exploiting the information contained in the FO data statistics only, that is able to process underdetermined mixtures of sources without the main limitations of the existing methods, provided that the sources have different trispectrum and nonzero kurtosis with the same sign. A new performance criterion that is able to quantify the identification quality of a given source and allowing the quantitative comparison of two BI methods for each source, is also proposed in the paper. Finally, an application of the proposed method is presented through the introduction of a powerful direction-finding method built from the blindly identified mixture matrix.

A new algorithm on multiple unknown source signals estimation and separation in a reverberant space

In many voice-related applications, the presence of echoes and overlapping speech signals can degrade the quality or intelligibility of a desired speech signal to be processed. It is, therefore, important to cancel the echoes and to separate overlapping speech signals from a mixture of these components, so that a specific function of the system, for instance, transmission, speech identification, or recognition, can be accomplished with better performance. However, in many cases we do not know the properties of the communication channel, and sometimes even the number of speech sources is unknown. In this paper, we propose to use a reference signal to determine the channel characteristics. When the estimated channel parameter matrices are obtained, a recurrence formula can then be used to separate various speech signals including their reverberant counterparts. As a finite impulse response (FIR) model is used to describe the observation model of the sources in the reverberant environment, it is not necessary for the processing speech signals to be uncorrelated. Because it involves only simple computation, our approach can be used in online applications. In this paper, we will investigate the validity of our algorithm and compare it with extended fourth-order blind identification (EFOBI). It is found that our method preserves both signal waveforms and their amplitudes even in a noisy environment, whereas EFOBI has not been able to achieve similar performance.

Indeterminacy and identifiability of blind identification

Blind identification of source signals is studied from both theoretical and algorithmic aspects. A mathematical structure is formulated from which the acceptable indeterminacy is represented by an equivalence relation. The concept of identifiability is then defined. Two identifiable cases are shown along with blind identification algorithms. The performance of FOBI (fourth-order blind identification), EFOBI (extended FOBI), and AMUSE algorithms is evaluated by some heuristic arguments and simulation results. It is shown that EFOBI outperforms the FOBI algorithm, and the AMUSE algorithm performs better than EFOBI in the case of nonwhite source signals. AMUSE is applied to a speech extraction problem and shown to have promising results. >