High-dimensional Time Series Data Research Articles

An Exponential Inequality for U-Statistics Under Mixing Conditions

The family of U-statistics plays a fundamental role in statistics. This paper proves a novel exponential inequality for U-statistics under the time series setting. Explicit mixing conditions are given for guaranteeing fast convergence, the bound proves to be analogous to the one under independence, and extension to non-stationary time series is straightforward. The proof relies on a novel decomposition of U-statistics via exploiting the temporal correlatedness structure. Such results are of interest in many fields where high-dimensional time series data are present. In particular, applications to high-dimensional time series inference are discussed.

필터링된 잔차를 이용한 희박벡터자기회귀모형에서의 변수 선택 측도

Vector autoregressive (VAR) models in high dimension suffer from noisy estimates, unstable predictions and hard interpretation. Consequently, the sparse vector autoregressive (sVAR) model, which forces many small coefficients in VAR to exactly zero, has been suggested and proven effective for the modeling of high dimensional time series data. This paper studies coupling measures to select non-zero coefficients in sVAR. The basic idea based on the simulation study reveals that removing the effect of other variables greatly improves the performance of coupling measures. sVAR model coefficients are asymmetric; therefore, asymmetric coupling measures such as Granger causality improve computational costs. We propose two asymmetric coupling measures, filtered-cross-correlation and filtered-Granger-causality, based on the filtered residuals series. Our proposed coupling measures are proven adequate for heavy-tailed and high order sVAR models in the simulation study.

A lazy associative classifier for time series

Association rule mining that mainly focuses on symbolic items presented in transactions has attracted considerable interest since a rule provides a concise and intuitive description of knowledge. However, a time series is a sequence of data that is typically recorded in temporal order at fixed intervals of time. In order to mining rules in the context of time series data, a symbolic aggregate approximation (SAX) representation that could discretize the real-valued and high-dimensional time series data into segments and convert each segment to a symbol is applied in this paper. On this basis, a modified CBA algorithm is proposed to discover Class Sequential Rules (CSRs) and make the final prediction at first. Then we propose a new lazy associative classification method, in which the computation is performed on a demand driven basis. This is in contrast to rule-based classification methods like CBA which generate excessive number of rules, but is still unable to cover some test data with the discovered rules. Various experimental results show that our lazy associative classification for time series can be interpretable and competitive with the current state-of-the-art algorithm. In addition, four different methods that select the mined CSR(s) are proposed for carrying out associative classification.

Extracting Low-Dimensional Latent Structure from Time Series in the Presence of Delays.

Noisy, high-dimensional time series observations can often be described by a set of low-dimensional latent variables. Commonly used methods to extract these latent variables typically assume instantaneous relationships between the latent and observed variables. In many physical systems, changes in the latent variables manifest as changes in the observed variables after time delays. Techniques that do not account for these delays can recover a larger number of latent variables than are present in the system, thereby making the latent representation more difficult to interpret. In this work, we introduce a novel probabilistic technique, time-delay gaussian-process factor analysis (TD-GPFA), that performs dimensionality reduction in the presence of a different time delay between each pair of latent and observed variables. We demonstrate how using a gaussian process to model the evolution of each latent variable allows us to tractably learn these delays over a continuous domain. Additionally, we show how TD-GPFA combines temporal smoothing and dimensionality reduction into a common probabilistic framework. We present an expectation/conditional maximization either (ECME) algorithm to learn the model parameters. Our simulations demonstrate that when time delays are present, TD-GPFA is able to correctly identify these delays and recover the latent space. We then applied TD-GPFA to the activity of tens of neurons recorded simultaneously in the macaque motor cortex during a reaching task. TD-GPFA is able to better describe the neural activity using a more parsimonious latent space than GPFA, a method that has been used to interpret motor cortex data but does not account for time delays. More broadly, TD-GPFA can help to unravel the mechanisms underlying high-dimensional time series data by taking into account physical delays in the system.

Fast time-series prediction using high-dimensional data: evaluating confidence interval credibility.

I propose an index for evaluating the credibility of confidence intervals for future observables predicted from high-dimensional time-series data. The index evaluates the distance from the current state to the data manifold. I demonstrate the index with artificial datasets generated from the Lorenz'96 II model [Lorenz, in Proceedings of the Seminar on Predictability, Vol. 1 (ECMWF, Reading, UK, 1996), p. 1], the Lorenz'96 I model [Hansen and Smith, 2859:TROOCI>2.0.CO;2">J. Atmos. Sci. 57, 2859 (2000).

Risk patterns and correlated brain activities. Multidimensional statistical analysis of FMRI data in economic decision making study.

Decision making usually involves uncertainty and risk. Understanding which parts of the human brain are activated during decisions under risk and which neural processes underly (risky) investment decisions are important goals in neuroeconomics. Here, we analyze functional magnetic resonance imaging (fMRI) data on 17 subjects who were exposed to an investment decision task from Mohr, Biele, Krugel, Li, and Heekeren (in NeuroImage 49, 2556-2563, 2010b). We obtain a time series of three-dimensional images of the blood-oxygen-level dependent (BOLD) fMRI signals. We apply a panel version of the dynamic semiparametric factor model (DSFM) presented in Park, Mammen, Wolfgang, and Borak (in Journal of the American Statistical Association 104(485), 284-298, 2009) and identify task-related activations in space and dynamics in time. With the panel DSFM (PDSFM) we can capture the dynamic behavior of the specific brain regions common for all subjects and represent the high-dimensional time-series data in easily interpretable low-dimensional dynamic factors without large loss of variability. Further, we classify the risk attitudes of all subjects based on the estimated low-dimensional time series. Our classification analysis successfully confirms the estimated risk attitudes derived directly from subjects' decision behavior.

Piecewise Trend Approximation: A Ratio-Based Time Series Representation

A time series representation, piecewise trend approximation (PTA), is proposed to improve efficiency of time series data mining in high dimensional large databases. PTA represents time series in concise form while retaining main trends in original time series; the dimensionality of original data is therefore reduced, and the key features are maintained. Different from the representations that based on original data space, PTA transforms original data space into the feature space of ratio between any two consecutive data points in original time series, of which sign and magnitude indicate changing direction and degree of local trend, respectively. Based on the ratio-based feature space, segmentation is performed such that each two conjoint segments have different trends, and then the piecewise segments are approximated by the ratios between the first and last points within the segments. To validate the proposed PTA, it is compared with classical time series representations PAA and APCA on two classical datasets by applying the commonly used K-NN classification algorithm. For ControlChart dataset, PTA outperforms them by 3.55% and 2.33% higher classification accuracy and 8.94% and 7.07% higher for Mixed-BagShapes dataset, respectively. It is indicated that the proposed PTA is effective for high dimensional time series data mining.

Quantitative Evaluation of Two-Factor Analysis Applied to Hepatic Perfusion Study Using Contrast-enhanced Ultrasound

Focal liver lesions (FLLs) are usually quantitatively assessed by time-intensity curves (TICs) extracted from contrast-enhanced ultrasound (CEUS) image sequences. To overcome the subjectivity of manual region of interest (ROI) selection and automatically extract TICs, a novel factor analysis method called replace approximation (RA) was proposed. Assuming that the two factors are the arterial and portal vein phases, respectively, the high-dimensional time-series data are mapped into 1-D space, where the TIC at each pixel in the image becomes a point along a one-dimensional axis. The RA method aims to seek two apexes corresponding to the factor curves (the targeted TICs) in the subspace. This method was tested on 18 free-breathing datasets with respiratory motion correction. The experimental results showed that the RA method extracted physiological factor curves and the corresponding factor images efficiently. The mean correlation coefficient between the factor curves and the corresponding ROI measurements was 0.95 ± 0.02. Furthermore, the wash-in time ratio indexes of FLLs derived from the factor curves were used to perform parametric imaging, which could represent the characteristics of different types of FLLs. These results indicate that two-factor analysis has the potential to perform quantitative analysis of hepatic perfusion, which would be helpful to the differential diagnosis of FLLs.

A parallel dimensionality reduction for time-series data and some of its applications

The subsequence matching in a large time-series database has been an interesting problem. Many methods have been proposed that cope with this problem in an adequate extent. One of the good ideas is reducing properly the dimensionality of time-series data. In this paper, we propose a new method to reduce the dimensionality of high-dimensional time-series data. The method is simpler than existing ones based on the discrete Fourier transform and the discrete cosine transform. Furthermore, our dimensionality reduction may be executed in parallel. The method is used to time-series data matching problem and it decreases drastically the complexity of the corresponding algorithm. The method preserves planar geometric blocks and it is also applied to minimum bounding rectangles as well.

Oracally efficient spline smoothing of nonlinear additive autoregression models with simultaneous confidence band

Under weak conditions of smoothness and mixing, we propose spline-backfitted spline (SBS) estimators of the component functions for a nonlinear additive autoregression model that is both computationally expedient for analyzing high dimensional large time series data, and theoretically reliable as the estimator is oracally efficient and comes with asymptotically simultaneous confidence band. Simulation evidence strongly corroborates with the asymptotic theory.

Perception as Modeling: A neural network that extracts and models predictable elements from input.

Event Abstract Back to Event Perception as Modeling: A neural network that extracts and models predictable elements from input. Perception is not simply the passive analysis of input data, it involves modeling the external world, making inferences about predictable events and noting when something unexpected happens. In most models of sensory processing, activity generated by external stimuli in peripheral sensors is used to drive networks that perform computations such as object recognition. We consider a different scheme in which stimulus driven activity acts as a training signal for a spontaneously active network. This 'predictor' network continually attempts to model the training data in the sense of being able to generate it spontaneously, even if the sensory input is cut off. Of course, only certain aspects of the sensory data may be predictable. To address this, an 'extractor' circuit that is guided by feedback from the predictor produces the training signal for the predictor network. The extractor pulls out from the sensory stream aspects of the data that the predictor network can reproduce. This automatically divides the input data into three categories: 1) noise, which is the part of the input stream ignored by the extractor, 2) a predictable signal that is isolated by the extractor circuit and internally reproduced by the predictor, and 3) surprising events, which are elements of the input stream generated by the extractor that do not match the output of the predictor. The predictor is a recurrent network of firing-rate model neurons with a linear readout that provides both the output of and feedback to the network. As in the work of Jaeger and Haas (Science, 2004), the learning algorithm only modifies the weights connecting the network to the output unit, but unlike their approach, we leave all feedback intact during learning. The key element is a novel learning rule we call FORCE learning that restricts errors in the output to small values throughout training. This was originally a supervised learning scheme, but in the extractor-predictor approach, the extractor circuit acts as the supervisor for the predictor network, so the scheme is unsupervised. Due to the nature of FORCE learning, the modeling network always generates a close match to the target and thus does not generate 'hallucinations' unrelated to external reality. The rate of change of the readout weights provides a measure of whether the target function can be generated autonomously by the predictor. We use this measure as a supervisory signal for the extractor, which is modified if the target signal it is extracting from the input data cannot be autonomously modeled. The combined recurrent predictor network and linear-filter extractor is successful at finding and modeling predictable structure (if it exists) in high-dimensional time series data even when polluted by extremely complex noise. This network can be viewed as a general method for extracting patterns from complicated time series or, from a systems neuroscience perspective, as a model of perception where the way to understand the world is through internal prediction, with further processing used only when sensory input fails to match expectations. Conference: Computational and systems neuroscience 2009, Salt Lake City, UT, United States, 26 Feb - 3 Mar, 2009. Presentation Type: Poster Presentation Topic: Poster Presentations Citation: (2009). Perception as Modeling: A neural network that extracts and models predictable elements from input.. Front. Syst. Neurosci. Conference Abstract: Computational and systems neuroscience 2009. doi: 10.3389/conf.neuro.06.2009.03.253 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 03 Feb 2009; Published Online: 03 Feb 2009. Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract Supplemental Data The Authors in Frontiers Google Google Scholar PubMed Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Spline-backfitted kernel smoothing of nonlinear additive autoregression model

Application of nonparametric and semiparametric regression techniques to high-dimensional time series data has been hampered due to the lack of effective tools to address the ``curse of dimensionality.'' Under rather weak conditions, we propose spline-backfitted kernel estimators of the component functions for the nonlinear additive time series data that are both computationally expedient so they are usable for analyzing very high-dimensional time series, and theoretically reliable so inference can be made on the component functions with confidence. Simulation experiments have provided strong evidence that corroborates the asymptotic theory.

Information Mining Over Heterogeneous and High-Dimensional Time-Series Data in Clinical Trials Databases

An effective analysis of clinical trials data involves analyzing different types of data such as heterogeneous and high dimensional time series data. The current time series analysis methods generally assume that the series at hand have sufficient length to apply statistical techniques to them. Other ideal case assumptions are that data are collected in equal length intervals, and while comparing time series, the lengths are usually expected to be equal to each other. However, these assumptions are not valid for many real data sets, especially for the clinical trials data sets. An addition, the data sources are different from each other, the data are heterogeneous, and the sensitivity of the experiments varies by the source. Approaches for mining time series data need to be revisited, keeping the wide range of requirements in mind. In this paper, we propose a novel approach for information mining that involves two major steps: applying a data mining algorithm over homogeneous subsets of data, and identifying common or distinct patterns over the information gathered in the first step. Our approach is implemented specifically for heterogeneous and high dimensional time series clinical trials data. Using this framework, we propose a new way of utilizing frequent itemset mining, as well as clustering and declustering techniques with novel distance metrics for measuring similarity between time series data. By clustering the data, we find groups of analytes (substances in blood) that are most strongly correlated. Most of these relationships already known are verified by the clinical panels, and, in addition, we identify novel groups that need further biomedical analysis. A slight modification to our algorithm results an effective declustering of high dimensional time series data, which is then used for "feature selection." Using industry-sponsored clinical trials data sets, we are able to identify a small set of analytes that effectively models the state of normal health.

Hierarchical visualization of time-series data using switching linear dynamical systems

We propose a novel visualization algorithm for high-dimensional time-series data. In contrast to most visualization techniques, we do not assume consecutive data points to be independent. The basic model is a linear dynamical system which can be seen as a dynamic extension of a probabilistic principal component model. A further extension to a particular switching linear dynamical system allows a representation of complex data onto multiple and even a hierarchy of plots. Using sensible approximations based on expectation propagation, the projections can be performed in essentially the same order of complexity as their static counterpart. We apply our method on a real-world data set with sensor readings from a paper machine.

Neural maps for faithful data modelling in medicine — state-of-the-art and exemplary applications

In the present contribution, the application of neural networks and, especially, maps in medical applications is considered. Thereby, we shortly highlight some problems in medical applications and refer to recently developed example solutions. We review extensions and modifications of the basic self-organizing map for faithful data representation. We demonstrate these approaches in two typical medical applications: high-dimensional time series analysis and data visualization.

New embedding technique for estimating the delay

It has been reported that the chaos attractor generated from a differential equation containing a delay often attains a high dimension. When the high-dimensional time series data generated from a system containing a delay are analyzed, it is expected that the orbital reconstruction by the Takens embedding theorem is practically difficult. An approximate low-dimensional orbital reconstruction taking into account the delay is needed. However, when time series data containing a delay are treated, it is important to first estimate the delay accurately. Several methods have been proposed for estimating the delay. The previous methods are either those that cannot accurately estimate the delay while the existence of the delay can be recognized or those that can be applied only under the assumption that all state variables are observable. In this paper, a new embedding method and an algorithm are proposed for estimating the delay from an observed time series data set. By simulation, accurate estimation of the delay is confirmed. © 2000 Scripta Technica, Electron Comm Jpn Pt 3, 83(10): 10–16, 2000

Nonparametric Regression with Singular Design

Theories of nonparametric regression are usually based on the assumption that the design density exists. However, in some applications such as those involving high-dimensional or chaotic time series data, the design measure may be singular and may be likely to have a fractal (nonintegral) dimension. In this paper, the popular Nadaraya–Watson estimator is studied under the general setup that the continuity of the design measure is governed by the local or pointwise dimension. It will be shown in the iid setup that the nonparametric regression estimator achieves a convergence rate which is dependent only on the pointwise dimension. The case of time series data is also studied. For the latter case, a new mixing condition is introduced, and an assumption of marginal or joint density is completely avoided. Three examples, a fractal regression and two applications for predicting chaotic time series, are used to illustrate the implications of the obtained results.