Independent Subspace Analysis Research Articles

Audio-based cough counting using independent subspace analysis.

In this paper, an algorithm designed to detect characteristic cough events in audio recordings is presented, significantly reducing the time required for manual counting. Using time-frequency representations and independent subspace analysis (ISA), sound events that exhibit characteristics of coughs are automatically detected, producing a summary of the events detected without the need for a pre-trained model. Using a dataset created from publicly available audio recordings, this algorithm has been tested on a variety of synthesized audio scenarios representative of those likely to be encountered by subjects undergoing an ambulatory cough recording, achieving a true positive rate of 76% with an average of 2.85 false positives per minute.

딥러닝 및 독립 부분 공간 분석을 기반으로 한 당뇨성 망막변증 진단

당뇨성 망막변증 진단 시스템을 개발하기 위해서 안저 영상과 인공지능 기법을 사용하는 것은 산업계에서 유망한 분야중의 하나이다. 당뇨성 망막변증의 검출은 영상의 색깔, 해상도, 밝기의 큰 변동과 클래스간의 불균형 분포를 갖는 데이터, 영상내의 병변이 아주 작아서 힘든 작업이다. 이 논문의 목적은 당뇨성 망막변증의 검출을 위한 더 좋은 방법을 찾는 것이다. 합성곱 신경회로망에 기반한 딥러닝과 독립 부분 공간 알고리즘을 표현을 학습하고 안저 데이터로부터 특징들을 추출하기 위해서 구현하였다. 이렇게 추출된 특징들은 당뇨성 망막변증을 검출하기 위해 분류기에 입력된다. 감독학습을 사용하여 당뇨성 망막변증을 검출하는데 최상의 결과를 얻었다.

Subspace clustering via stacked independent subspace analysis networks with sparse prior information

Sparse subspace clustering (SSC) method has gained considerable attention in recent decades owing to its advantages in the fields of clustering. In essence, SSC is to learn a sparse affinity matrix followed by striving for a low-dimensional representation of data. However, the SSC and its variants mainly focus on building high-quality affinity matrix while ignoring the importance of low-dimensional feature derived from the affinity matrix. Moreover, due to their intrinsic linearity of models, they cannot efficiently handle data with the nonlinear distribution. In this paper, we propose a stacked independent subspace analysis (ISA) with sparse prior information called stacked-ISASP to deal with these two issues. Powered by handling data with nonlinear structure, our method aims at seeking a low-dimensional feature from the image data. Concretely, the model can stack the modified independent subspace analysis networks by incorporating the prior subspace information from the original data. To validate the efficiency of the proposed method, we compare our proposed stacked-ISASP method with the state-of-the-art methods on real datasets. Experimental results show that our approach can not only learn a better low-dimensional structure from the data but also achieve better performance for the classification task.

Multidataset Independent Subspace Analysis With Application to Multimodal Fusion.

Unsupervised latent variable models-blind source separation (BSS) especially-enjoy a strong reputation for their interpretability. But they seldom combine the rich diversity of information available in multiple datasets, even though multidatasets yield insightful joint solutions otherwise unavailable in isolation. We present a direct, principled approach to multidataset combination that takes advantage of multidimensional subspace structures. In turn, we extend BSS models to capture the underlying modes of shared and unique variability across and within datasets. Our approach leverages joint information from heterogeneous datasets in a flexible and synergistic fashion. We call this method multidataset independent subspace analysis (MISA). Methodological innovations exploiting the Kotz distribution for subspace modeling, in conjunction with a novel combinatorial optimization for evasion of local minima, enable MISA to produce a robust generalization of independent component analysis (ICA), independent vector analysis (IVA), and independent subspace analysis (ISA) in a single unified model. We highlight the utility of MISA for multimodal information fusion, including sample-poor regimes ( N = 600 ) and low signal-to-noise ratio, promoting novel applications in both unimodal and multimodal brain imaging data.

Video Human Behavior Recognition Based on ISA Deep Network Model

Vision-based behavior recognition is the analysis and recognition of human behavior in video. It has been widely used in many aspects such as multimedia information retrieval, behavior monitoring, and robot perception. This paper uses the Independent Subspace Analysis (ISA) deep network model feature extraction method, which is based on the ISA model and neural network theory, and combines data preprocessing methods, [Formula: see text]-means clustering methods, and Support Vector Machine (SVM) classifiers to achieve video classification and identification of human behavior. The ISA-based deep network model feature extraction method is an unsupervised learning method that can obtain behavior characteristics with good invariance and characterization capabilities in video human behavior. The experiment was conducted on the basis of the Hollywood2 human behavior data set. This experiment was compared with other commonly used human behavior feature extraction and recognition methods. The experimental results validated the effectiveness and advantages of this method in the classification and recognition of human behavior.

Lie Group Methods in Blind Signal Processing.

This paper deals with the use of Lie group methods to solve optimization problems in blind signal processing (BSP), including Independent Component Analysis (ICA) and Independent Subspace Analysis (ISA). The paper presents the theoretical fundamentals of Lie groups and Lie algebra, the geometry of problems in BSP as well as the basic ideas of optimization techniques based on Lie groups. Optimization algorithms based on the properties of Lie groups are characterized by the fact that during optimization motion, they ensure permanent bonding with a search space. This property is extremely significant in terms of the stability and dynamics of optimization algorithms. The specific geometry of problems such as ICA and ISA along with the search space homogeneity enable the use of optimization techniques based on the properties of the Lie groups and . An interesting idea is that of optimization motion in one-parameter commutative subalgebras and toral subalgebras that ensure low computational complexity and high-speed algorithms.

Spread spectrum data embedding in audio with UISA based cooperative detection

In the context of data embedding in audio for communications, Spread Spectrum (SS)-based techniques, combined with auditory models, are efficient in terms of robustness and perceptual quality of the modified host audio. However, their main drawback is the limited embedding capacity due to the strong interference caused by the host audio. In this work, we combine under-determined Blind Source Separation with a generic SS receiver -made of a Wiener equalizer and a correlation-based demodulator- to efficiently reduce the effect of the host interference. Two blocks are added to the generic receiver: an Under-determined Independent Subspace Analysis (UISA) block is placed after the Wiener equalizer in order to separate the components of the equalized output, and a cooperative detection block is applied to the UISA outputs in order to extract the relevant information from all available components. The UISA block uses Empirical Mode Decomposition to obtain multiple observations of the modified host signal. The performance of the proposed system in terms of bit error rate vs. bit-rate is significantly improved: average error rate is null for bit-rates below 500 bps and smaller than 0.1% for bit-rates reaching 1 kbps. The objective evaluation of the perceptual quality of the modified audio confirms the imperceptibility of the embedded data. The proposed system also exhibits robustness against common signal processing operations such as gain modification, noise, MPEG compression and re-quantization.

Multiple Independent Subspace Clusterings

Multiple clustering aims at discovering diverse ways of organizing data into clusters. Despite the progress made, it’s still a challenge for users to analyze and understand the distinctive structure of each output clustering. To ease this process, we consider diverse clusterings embedded in different subspaces, and analyze the embedding subspaces to shed light into the structure of each clustering. To this end, we provide a two-stage approach called MISC (Multiple Independent Subspace Clusterings). In the first stage, MISC uses independent subspace analysis to seek multiple and statistical independent (i.e. non-redundant) subspaces, and determines the number of subspaces via the minimum description length principle. In the second stage, to account for the intrinsic geometric structure of samples embedded in each subspace, MISC performs graph regularized semi-nonnegative matrix factorization to explore clusters. It additionally integrates the kernel trick into matrix factorization to handle non-linearly separable clusters. Experimental results on synthetic datasets show that MISC can find different interesting clusterings from the sought independent subspaces, and it also outperforms other related and competitive approaches on real-world datasets.

The Phase and Shift-Invariant Feature by Adaptive Independent Subspace Analysis for Cortical Complex Cells

It has previously been suggested that complex cells in primary visual cortex (V1) selectly respond to bars and edges at a particular location and orientation emerged from the natural image rather than random stimuli. In this paper, we proposed an adaptive independent subspace analysis (AISA) algorithm based on the maximum likelihood of super-gaussian distribution of the norm of the partially independent components involving difference to efficiently study the characteristics of complex cells. The multiple parameters are updated with different restrictions by contrast divergence (CD) method based on the super-Gaussian (sparse) distribution. In terms of the AISA results, the inside and outside subspace of energy correlation illustrate the properties of independence. Furthermore, AISA features, similar to the characteristics of complex cells in V1, have phase and shift-invariant, as well as frequency and orientation selectivity via the energy function of receptive fields (RFs) from tuning curve test with sinusoidal function. Compared with the principal component analysis (PCA) filters and other ICA filters, AISA filters are analogous to Gabor filters for edge detection and human visual system via varying the frequency and orientation representation.

Adaptive Independent Subspace Analysis of Brain Magnetic Resonance Imaging Data

Methods for image registration, segmentation, and visualization of magnetic resonance imaging (MRI) data are used widely to help medical doctors in supporting diagnostics. The large amount and complexity of MRI data require looking for new methods that allow for efficient processing of this data. Here, we propose using the adaptive independent subspace analysis (AISA) method to discover meaningful electroencephalogram activity in the MRI scan data. The results of AISA (image subspaces) are analyzed using image texture analysis methods to calculate first order, gray-level co-occurrence matrix, gray-level size-zone matrix, gray-level run-length matrix, and neighboring gray-tone difference matrix features. The obtained feature space is mapped to the 2D space using the t-distributed stochastic neighbor embedding method. The classification results achieved using the k-nearest neighbor classifier with 10-fold cross-validation have achieved 94.7% of accuracy (and f-score of 0.9356) from the real autism spectrum disorder dataset.

Joint Independent Subspace Analysis: Uniqueness and Identifiability

This paper deals with the identifiability of joint independent subspace analysis (JISA). JISA is a recently-proposed framework that subsumes independent vector analysis (IVA) and independent subspace analysis (ISA). Each underlying mixture can be regarded as a dataset; therefore, JISA can be used for data fusion. In this paper, we assume that each dataset is an overdetermined mixture of several multivariate Gaussian processes, each of which has independent and identically distributed samples. This setup is not identifiable when each mixture is considered individually. Given these assumptions, JISA can be restated as coupled block diagonalization (CBD) of its correlation matrices. Hence, JISA identifiability is tantamount to CBD uniqueness. In this work, we provide necessary and sufficient conditions for uniqueness and identifiability of JISA and CBD. Our analysis is based on characterizing all the cases in which the Fisher information matrix is singular. We prove that non-identifiability may occur only due to pairs of underlying random processes with the same dimension. Our results provide further evidence that irreducibility has a central role in the uniqueness analysis of block-based decompositions. Our contribution extends previous results on the uniqueness and identifiability of ISA, IVA, coupled matrix and tensor decompositions. We provide examples to illustrate our results.

Low Contrast Surface Inspection under Uneven Illumination Using Independent Subspace Analysis

When using the machine vision technique to detect the low contrast surface image, the difference between the defect and the background of the low contrast surface is not obvious. The existing method is difficult to detect these defects, and the influence of uneven illumination makes the problem more difficult, the purpose of this paper is to detect defects in low-contrast surface images such as LCDs that are unevenly illuminated. The proposed method in this paper is based on the independent subspace analysis (ISA), which includes the learning and detection stage. In the training stage, a batch of defective low contrast images is used to find a set of independent basis images. During the test stage, the low contrast image to be measured is reconstructed by the nonlinear combination of a set of basis images obtained by learning, and the error between the reconstructed image and the sample to be measured is used to determine whether exist a defect. The experimental results show that the reconstruction image method based on the basis image through ISA has achieved good results.

Independent Subspace Analysis of the Sea Surface Temperature Variability: Non-Gaussian Sources and Sensitivity to Sampling and Dimensionality

We propose an expansion of multivariate time-series data into maximally independent source subspaces. The search is made among rotations of prewhitened data which maximize non-Gaussianity of candidate sources. We use a tensorial invariant approximation of the multivariate negentropy in terms of a linear combination of squared coskewness and cokurtosis. By solving a high-order singular value decomposition problem, we extract the axes associated with most non-Gaussianity. Moreover, an estimate of the Gaussian subspace is provided by the trailing singular vectors. The independent subspaces are obtained through the search of “quasi-independent” components within the estimated non-Gaussian subspace, followed by the identification of groups with significant joint negentropies. Sources result essentially from the coherency of extremes of the data components. The method is then applied to the global sea surface temperature anomalies, equatorward of 65°, after being tested with non-Gaussian surrogates consistent with the data anomalies. The main emerging independent components and subspaces, supposedly generated by independent forcing, include different variability modes, namely, The East-Pacific, the Central Pacific, and the Atlantic Niños, the Atlantic Multidecadal Oscillation, along with the subtropical dipoles in the Indian, South Pacific, and South-Atlantic oceans. Benefits and usefulness of independent subspaces are then discussed.

An Improvement in Representation of Audio Signal in Time-Frequency Plane using EMD-2TEMD Based Approach

This study proposed an enhanced time-frequency representation of audio signal using EMD-2TEMD based approach. To analyze non-stationary signal like audio, timefrequency representation is an important aspect. In case of representing or analyzing such kind of signal in time-frequency-energy distribution, hilbert spectrum is a recent approach and popular way which has several advantages over other methods like STFT, WT etc. Hilbert-Huang Transform (HHT) is a prominent method consists of Empirical Mode Decomposition (EMD) and Hilbert Spectral Analysis (HSA). An enhanced method called Turning Tangent empirical mode decomposition (2T-EMD) has recently developed to overcome some limitations of classical EMD like cubic spline problems, sifting stopping condition etc. 2T-EMD based hilbert spectrum of audio signal encountered some issues due to the generation of too many IMFs in the process where EMD produces less. To mitigate those problems, a mutual implementation of 2T-EMD & classical EMD is proposed in this paper which enhances the representation of hilbert spectrum along with significant improvements in source separation result using Independent Subspace Analysis (ISA) based clustering in case of audio signals. This refinement of hilbert spectrum not only contributes to the future work of source separation problem but also many other applications in audio signal processing.

Joint Independent Subspace Analysis Using Second-Order Statistics

This paper deals with a novel generalization of classical blind source separation (BSS) in two directions. First, relaxing the constraint that the latent sources must be statistically independent. This generalization is well-known and sometimes termed independent subspace analysis (ISA). Second, jointly analyzing several ISA problems, where the link is due to statistical dependence among corresponding sources in different mixtures. When the data are one-dimensional, i.e., multiple classical BSS problems, this model, known as independent vector analysis (IVA), has already been studied. In this paper, we combine IVA with ISA and term this new model joint independent subspace analysis (JISA). We provide full performance analysis of JISA, including closed-form expressions for minimal mean square error (MSE), Fisher information and Cramer–Rao lower bound, in the separation of Gaussian data. The derived MSE applies also for non-Gaussian data, when only second-order statistics are used. We generalize previously known results on IVA, including its ability to uniquely resolve instantaneous mixtures of real Gaussian stationary data, and having the same arbitrary permutation at all mixtures. Numerical experiments validate our theoretical results and show the gain with respect to two competing approaches that either use a finer block partition or a different norm.

Learning Visual Spatial Pooling by Strong PCA Dimension Reduction.

In visual modeling, invariance properties of visual cells are often explained by a pooling mechanism, in which outputs of neurons with similar selectivities to some stimulus parameters are integrated so as to gain some extent of invariance to other parameters. For example, the classical energy model of phase-invariant V1 complex cells pools model simple cells preferring similar orientation but different phases. Prior studies, such as independent subspace analysis, have shown that phase-invariance properties of V1 complex cells can be learned from spatial statistics of natural inputs. However, those previous approaches assumed a squaring nonlinearity on the neural outputs to capture energy correlation; such nonlinearity is arguably unnatural from a neurobiological viewpoint but hard to change due to its tight integration into their formalisms. Moreover, they used somewhat complicated objective functions requiring expensive computations for optimization. In this study, we show that visual spatial pooling can be learned in a much simpler way using strong dimension reduction based on principal component analysis. This approach learns to ignore a large part of detailed spatial structure of the input and thereby estimates a linear pooling matrix. Using this framework, we demonstrate that pooling of model V1 simple cells learned in this way, even with nonlinearities other than squaring, can reproduce standard tuning properties of V1 complex cells. For further understanding, we analyze several variants of the pooling model and argue that a reasonable pooling can generally be obtained from any kind of linear transformation that retains several of the first principal components and suppresses the remaining ones. In particular, we show how the classic Wiener filtering theory leads to one such variant.

Separation of the atmospheric variability into non-Gaussian multidimensional sources by projection pursuit techniques

We develop an expansion of space-distributed time series into statistically independent uncorrelated subspaces (statistical sources) of low-dimension and exhibiting enhanced non-Gaussian probability distributions with geometrically simple chosen shapes (projection pursuit rationale). The method relies upon a generalization of the principal component analysis that is optimal for Gaussian mixed signals and of the independent component analysis (ICA), optimized to split non-Gaussian scalar sources. The proposed method, supported by information theory concepts and methods, is the independent subspace analysis (ISA) that looks for multi-dimensional, intrinsically synergetic subspaces such as dyads (2D) and triads (3D), not separable by ICA. Basically, we optimize rotated variables maximizing certain nonlinear correlations (contrast functions) coming from the non-Gaussianity of the joint distribution. As a by-product, it provides nonlinear variable changes ‘unfolding’ the subspaces into nearly Gaussian scalars of easier post-processing. Moreover, the new variables still work as nonlinear data exploratory indices of the non-Gaussian variability of the analysed climatic and geophysical fields. The method (ISA, followed by nonlinear unfolding) is tested into three datasets. The first one comes from the Lorenz’63 three-dimensional chaotic model, showing a clear separation into a non-Gaussian dyad plus an independent scalar. The second one is a mixture of propagating waves of random correlated phases in which the emergence of triadic wave resonances imprints a statistical signature in terms of a non-Gaussian non-separable triad. Finally the method is applied to the monthly variability of a high-dimensional quasi-geostrophic (QG) atmospheric model, applied to the Northern Hemispheric winter. We find that quite enhanced non-Gaussian dyads of parabolic shape, perform much better than the unrotated variables in which concerns the separation of the four model’s centroid regimes (positive and negative phases of the Arctic Oscillation and of the North Atlantic Oscillation). Triads are also likely in the QG model but of weaker expression than dyads due to the imposed shape and dimension. The study emphasizes the existence of nonlinear dyadic and triadic nonlinear teleconnections.

Ideal Binocular Disparity Detectors Learned Using Independent Subspace Analysis on Binocular Natural Image Pairs.

An influential theory of mammalian vision, known as the efficient coding hypothesis, holds that early stages in the visual cortex attempts to form an efficient coding of ecologically valid stimuli. Although numerous authors have successfully modelled some aspects of early vision mathematically, closer inspection has found substantial discrepancies between the predictions of some of these models and observations of neurons in the visual cortex. In particular analysis of linear-non-linear models of simple-cells using Independent Component Analysis has found a strong bias towards features on the horoptor. In order to investigate the link between the information content of binocular images, mathematical models of complex cells and physiological recordings, we applied Independent Subspace Analysis to binocular image patches in order to learn a set of complex-cell-like models. We found that these complex-cell-like models exhibited a wide range of binocular disparity-discriminability, although only a minority exhibited high binocular discrimination scores. However, in common with the linear-non-linear model case we found that feature detection was limited to the horoptor suggesting that current mathematical models are limited in their ability to explain the functionality of the visual cortex.

Multiple audio watermarking system based on CDMA

Multiple watermarking is the technique that permits to embed multiple data into the same media content. In this paper, a new multiple audio watermarking system is discussed. Our main objective is to increase the capacity of watermark information, under the inaudibility constraint, while maintaining a high degree of robustness against several disturbances. At the emitter, code division multiple access (CDMA) is used to embed multiple watermarks in the time domain of the host signal. At the receiver, we consider the detection process as a matter of blind source separation (BSS) between the audio signal and the embedded data. Two BSS techniques have been tested in the new multi-watermarking system: under-determined independent subspace analysis (UISA) and Gaussian mixture models (GMM). Experimental results exhibit that inaudibility and high robustness of hidden data can be achieved simultaneously with a substantial increase of the amount of watermark information (to a multiplicative factor of 4).

Multiple audio watermarking system based on CDMA

Multiple watermarking is the technique that permits to embed multiple data into the same media content. In this paper, a new multiple audio watermarking system is discussed. Our main objective is to increase the capacity of watermark information, under the inaudibility constraint, while maintaining a high degree of robustness against several disturbances. At the emitter, code division multiple access CDMA is used to embed multiple watermarks in the time domain of the host signal. At the receiver, we consider the detection process as a matter of blind source separation BSS between the audio signal and the embedded data. Two BSS techniques have been tested in the new multi-watermarking system: under-determined independent subspace analysis UISA and Gaussian mixture models GMM. Experimental results exhibit that inaudibility and high robustness of hidden data can be achieved simultaneously with a substantial increase of the amount of watermark information to a multiplicative factor of 4.