High-dimensional Noise Research Articles

Erratum: Testing for high-dimensional white noise using maximum cross-correlations

Erratum: Testing for high-dimensional white noise using maximum cross-correlations

Default Bayesian analysis with global-local shrinkage priors

We provide a framework for assessing the default nature of a prior distribution using the property of regular variation, which we study for global-local shrinkage priors. In particular, we show that the horseshoe priors, originally designed to handle sparsity, are regularly varying and thus are appropriate for default Bayesian analysis. To illustrate our methodology, we discuss four problems of noninformative priors that have been shown to be highly informative for nonlinear functions. In each case, we show that global-local horseshoe priors perform as required. Global-local shrinkage priors can separate a low-dimensional signal from high-dimensional noise even for nonlinear functions.

Dimensionality reduction and dynamical filtering: Stimulated Brillouin scattering in optical fibers.

Stimulated Brillouin scattering (SBS) is a noise-driven nonlinear interaction between acoustical and optical waves. In optical fibers, SBS can be observed at relatively low optical powers and can severely limit signal transmission. Although SBS is initiated by high dimensional noise, it also exhibits many of the hallmarks of a complex nonlinear dynamical system. We report here a comprehensive experimental and numerical study of the fluctuations in the reflected Stokes wave produced by SBS in optical fibers. Using time series analysis, we demonstrate a reduction of dimensionality and dynamical filtering of the Stokes wave. We begin with a careful comparison of the measured average transmitted and reflected intensities from below the SBS threshold to saturation of the transmitted power. Initially the power spectra and correlation functions of the time series of the reflected wave fluctuations at the SBS threshold and above are measured and simulated. Much greater dynamical insight is provided when we study the scaling behavior of the intensity fluctuations using Hurst exponents and detrended fluctuation analysis for time scales extending over six orders of magnitude. At the highest input powers, we notice the emergence of three distinct dynamical scaling regimes: persistent, Brownian, and antipersistent. Next, we explore the Hilbert phase fluctuations of the intensity time series and amplitude-phase coupling. Finally, time-delay embedding techniques reveal a gradual reduction in dimensionality of the spatiotemporal dynamics as the laser input is increased toward saturation of the transmitted power. Through all of these techniques, we find a transition from noisier to smoother dynamics with increasing input power. We find excellent agreement between our experimental measurements and simulations.

PCA consistency for the power spiked model in high-dimensional settings

In this paper, we propose a general spiked model called the power spiked model in high-dimensional settings. We derive relations among the data dimension, the sample size and the high-dimensional noise structure. We first consider asymptotic properties of the conventional estimator of eigenvalues. We show that the estimator is affected by the high-dimensional noise structure directly, so that it becomes inconsistent. In order to overcome such difficulties in a high-dimensional situation, we develop new principal component analysis (PCA) methods called the noise-reduction methodology and the cross-data-matrix methodology under the power spiked model. We show that the new PCA methods can enjoy consistency properties not only for eigenvalues but also for PC directions and PC scores in high-dimensional settings.

Very High Dimensional Semiparametric Models

Very high dimensional semiparametric models play a major role in many areas, in particular in signal detection problems when sparse signals or sparse events are hidden among high dimensional noise. Concrete examples are genomic studies in biostatistics or imaging problems. In a broad context all kind of statistical inference and model selection problems were discussed for high dimensional data.

Comparative Evaluation of Classifiers in the Presence of Statistical Interactions between Features in High Dimensional Data Settings

A central challenge in high dimensional data settings in biomedical investigations involves the estimation of an optimal prediction algorithm to distinguish between different disease phenotypes. A significant complicating aspect in these analyses can be attributed to the presence of features that exhibit statistical interactions. Indeed, in several clinical investigations such as genetic studies of complex diseases, it is of interest to specifically identify such features. In this paper, we compare the performance of four commonly used classifiers (K-Nearest Neighbors, Prediction Analysis for Microarrays, Random Forests and Support Vector Machines) in settings involving high dimensional datasets including statistically interacting feature subsets. We evaluate the performance of these classifiers under conditions of varying sample size, levels of signal-to-noise ratio and strength of statistical interactions among features. We summarize two datasets from studies in diabetes and cardiovascular disease involving gene expression, metabolomics and proteomics measurements and compare results obtained using the four classifiers. Simulation studies revealed that the classifier Prediction Analysis of Microarrays had the highest classification accuracy in the absence of noise, statistical interactions and when feature distributions were multivariate Gaussian within each class. In the presence of statistical interactions, modest effect sizes and the absence of noise, Support Vector Machines achieved the best performance followed closely by Random Forests. Random Forests was optimal in settings that included both significant levels of high dimensional noise features and statistical interactions between biomarker pairs. The data applications revealed similar trends in the relative performances of each classifier. Random Forests had the highest classification accuracy among the four classifiers and was successful in incorporating interaction effects between features in the presence of noise in high dimensional datasets.

A Novel Framework for Gene Selection

Microarray data are highly redundant and noisy, and most genes are believed to be uninformative with respect to studied classes, as only a fraction of genes may present distinct profiles for different classes of samples. This paper proposed a novel hybrid framework (NHF) for the classification of high dimensional microarray data, which combined information gain(IG), F-score, genetic algorithm(GA), particle swarm optimization(PSO) and support vector machines(SVM). In order to identify a subset of informative genes embedded out of a large dataset which is contaminated with high dimensional noise, the proposed method is divided into three stages. In the first stage, IG is used to construct a ranking list of features, and only 10% features of the ranking list are provided for the second stage. In the second stage, PSO performs the feature selection task combining SVM. F-score is considered as a part of the objective function of PSO. The feature subsets are filtered according to the ranking list from the first stage, and then the results of it are supplied to the initialization of GA. Both the SVM parameter optimization and the feature selection are dynamically executed by PSO. In the third stage, GA initializes the individual of population from the results of the second stage, and an optimal result of feature selection is gained using GA integrating SVM. Both the SVM parameter optimization and the feature selection are dynamically performed by GA. The performance of the proposed method was compared with that of the PSO based, GA based, Ant colony optimization (ACO) based and simulated annealing (SA) based methods on five benchmark data sets, leukemia, colon, breast cancer, lung carcinoma and brain cancer. The numerical results and statistical analysis show that the proposed approach is capable of selecting a subset of predictive genes from a large noisy data set, and can capture the correlated structure in the data. In addition, NHF performs significantly better than the other methods in terms of prediction accuracy with smaller subset of features.

On information plus noise kernel random matrices

Kernel random matrices have attracted a lot of interest in recent years, from both practical and theoretical standpoints. Most of the theoretical work so far has focused on the case were the data is sampled from a low-dimensional structure. Very recently, the first results concerning kernel random matrices with high-dimensional input data were obtained, in a setting where the data was sampled from a genuinely high-dimensional structure---similar to standard assumptions in random matrix theory. In this paper, we consider the case where the data is of the type "information${}+{}$noise." In other words, each observation is the sum of two independent elements: one sampled from a "low-dimensional" structure, the signal part of the data, the other being high-dimensional noise, normalized to not overwhelm but still affect the signal. We consider two types of noise, spherical and elliptical. In the spherical setting, we show that the spectral properties of kernel random matrices can be understood from a new kernel matrix, computed only from the signal part of the data, but using (in general) a slightly different kernel. The Gaussian kernel has some special properties in this setting. The elliptical setting, which is important from a robustness standpoint, is less prone to easy interpretation.

Intrinsic Dimensionality Estimation of High-Dimension, Low Sample Size Data with D-Asymptotics

High-dimension, low sample size (HDLSS) data are becoming common in various fields such as genetic microarrays, medical imaging, text recognition, finance, chemometrics, and so on. Such data have surprising and often counterintuitive geometric structures because of the high-dimensional noise that dominates and corrupts the local neighborhoods. In this article, we estimate the intrinsic dimension (ID) that allows one to distinguish between deterministic chaos and random noise of HDLSS data. A new ID estimating methodology is given and its properties are studied by using a d-asymptotic approach.

An axiomatic approach to intrinsic dimension of a dataset

We perform a deeper analysis of an axiomatic approach to the concept of intrinsic dimension of a dataset proposed by us in the IJCNN'07 paper. The main features of our approach are that a high intrinsic dimension of a dataset reflects the presence of the curse of dimensionality (in a certain mathematically precise sense), and that dimension of a discrete i.i.d. sample of a low-dimensional manifold is, with high probability, close to that of the manifold. At the same time, the intrinsic dimension of a sample is easily corrupted by moderate high-dimensional noise (of the same amplitude as the size of the manifold) and suffers from prohibitively high computational complexity (computing it is an NP-complete problem). We outline a possible way to overcome these difficulties.

Detection of weak chaos in infant respiration

This paper concerns the application of newly developed methods for decomposition of an infant respiratory signal into locally stable nonsinusoidal periodic components. Each estimated component has dynamical variation in its three periodicity attributes, i.e., periodicity, scaling factors, and the waveform or pattern associated with the successive segments. Earlier, it has been reported with the application of conventional surrogate analysis and with the cylindrical basis function modeling that the underlying system is distinctly different from linearly filtered Gaussian process, and most probably the human respiratory system behaves as a nonlinear periodic oscillator with two or three degrees of freedom being driven by a high-dimensional noise source. Here, the surrogate analysis is extended and four new types of nonlinear surrogates have been proposed, which are produced by randomizing one or multiple periodicity attributes while preserving certain individual relationships. In this way, a new type of dissection of dynamics is possible, which can lead to a proper understanding of couplings between different controlling parameters.

Advantages of Rhythmic Movements at Resonance: Minimal Active Degrees of Freedom, Minimal Noise, and Maximal Predictability

Using time delay embedding, the authors applied phase space reconstruction to the time series of rhythmic movements of a hand-held pendulum. Subjects (N = 6) produced the manual oscillations about the wrist at the pendulum's resonant frequency and at a higher and a lower frequency. The number of active degrees of freedom required to capture the dynamics of the rhythmic behavior was 3 for the resonant frequency and 4 for each of the nonresonant frequencies. The residual high-dimensional noise was similarly lowest for the resonant frequency. Whereas 33% and 20%, respectively, of the vectors in the phase spaces of the dynamics higher and lower than resonance were unpredictable, only 12% were unpredictable at resonance. Finally, the predictability of the evolving dynamics extended farther into the future for oscillations at the resonant frequency. At resonance, the prediction horizon was 5 times farther than the prediction horizon for the higher than resonance behavior and 2.5 times farther than that for the lower than resonance behavior. The results suggest that, in pendular oscillations of a limb or limb segment, attunement of the central nervous system to the resonant frequency minimizes the variables to be controlled and maximizes the predictability of the rhythmic movement's chaotic dynamics.

Detecting Nonlinearity in Experimental Data

The technique of surrogate data has been used as a method to test for membership of particular classes of linear systems. We suggest an obvious extension of this to classes of nonlinear parametric models and demonstrate our methods with respiratory data from sleeping human infants. Although our data are clearly distinct from the different classes of linear systems we are unable to distinguish between our data and surrogates generated by nonlinear models. Hence we conclude that human respiration is likely to be a nonlinear system with more than two degrees of freedom with a limit cycle that is driven by high dimensional dynamics or noise.

The point correlation dimension: performance with nonstationary surrogate data and noise.

The dynamics of many biological systems have recently been attributed to low-dimensional chaos instead of high-dimensional noise, as previously though. Because biological data are invariably nonstationary, especially when recorded over a long interval, the conventional measures of low-dimensional chaos (e.g., the correlation dimension algorithms) cannot be applied. A new algorithm, the point correction dimension (PD2i) was developed to deal with this fundamental problem. In this article we describe the details of the algorithm and show that the local mean PD2i will accurately track dimension in nonstationary surrogate data.

Effect of the proton phototransfer reaction on the laser radiation of heteroaryloxazoles

the SA by high-dimensional noise for values of r comparable with the variance of this noise, "breaks" should be observed on the curves in Figs. 4a and b. For r less than the coordinates of these breaks, the slope of in C(r)(m) equals the dimension of the imbedded phase space, i.e., m. Of course, if the intensity of the technical noise is so high that the fractal structure of the SA is "smeared" completely, then the curves of in C(r)(m). versus In do r not have a section with equal slope, and therefore such noise dominates in the laser radiation.

EEG and clinical follow-up of epileptic patients after temporal lobectomy : L. Karaguiosov, S. Avramov, K. Karaguisov (Sofia, Bulgaria)

Objective: Recently it has been shown that the adult sleep EEG is mostly determined by high-dimensional, linear dynamics with the exception of the A phase of the cyclic alternating pattern which displays more synchronized nonlinear dynamics. It is not known how these two different types of brain dynamics develop in early life; for this reason the aim of this study was that to extend the nonlinear analysis to the EEG during sleep recorded in premature and full-term newborns.Methods: EEG epochs were chosen from a total of 24 polygraphic recordings from 14 babies (9 males and 5 females) aged between 33 weeks 3 days and 4 months conceptional age. All subjects were neurologically normal and showed normal psychomotor development at follow-up. A total of 243 artifact-free epochs was chosen during active sleep (AS, 74 total epochs), quiet sleep (QS, 76 total epochs) and indeterminate sleep (IS, 93 total epochs). The dynamic properties of the EEG were assessed by means of the nonlinear cross prediction test which uses 3 different ‘model’ time series in order to predict nonlinearly the original data set (Pred, Ama, and Tir). Pred is a measure of the predictability of the time series, and Ama and Tir are measures of asymmetry, indicating nonlinear structure.Results: Our results show that the structure of sleep EEG in newborns is significantly different from that of adults, it cannot be distinguished from that of high-dimensional noise in the majority of epochs, and shows a tendency to become nonlinear in nature, mostly during QS, in a small percentage of the epochs analyzed.Conclusions: These findings can be interpreted as the effect of immature synaptic interconnections between neurons in the newborn brain.

Convergence of Distributions Generated by Stationary Stochastic Processes

Convergence of Distributions Generated by Stationary Stochastic Processes