Continuous-time Regression Research Articles

Word Frequency and Predictability Dissociate in Naturalistic Reading.

Many studies of human language processing have shown that readers slow down at less frequent or less predictable words, but there is debate about whether frequency and predictability effects reflect separable cognitive phenomena: are cognitive operations that retrieve words from the mental lexicon based on sensory cues distinct from those that predict upcoming words based on context? Previous evidence for a frequency-predictability dissociation is mostly based on small samples (both for estimating predictability and frequency and for testing their effects on human behavior), artificial materials (e.g., isolated constructed sentences), and implausible modeling assumptions (discrete-time dynamics, linearity, additivity, constant variance, and invariance over time), which raises the question: do frequency and predictability dissociate in ordinary language comprehension, such as story reading? This study leverages recent progress in open data and computational modeling to address this question at scale. A large collection of naturalistic reading data (six datasets, >2.2 M datapoints) is analyzed using nonlinear continuous-time regression, and frequency and predictability are estimated using statistical language models trained on more data than is currently typical in psycholinguistics. Despite the use of naturalistic data, strong predictability estimates, and flexible regression models, results converge with earlier experimental studies in supporting dissociable and additive frequency and predictability effects.

Kernel methods for regression in continuous time over subsets and manifolds

Kernel methods for regression in continuous time over subsets and manifolds

Umibato: estimation of time-varying microbial interaction using continuous-time regression hidden Markov model.

MotivationAccumulating evidence has highlighted the importance of microbial interaction networks. Methods have been developed for estimating microbial interaction networks, of which the generalized Lotka–Volterra equation (gLVE)-based method can estimate a directed interaction network. The previous gLVE-based method for estimating microbial interaction networks did not consider time-varying interactions.ResultsIn this study, we developed unsupervised learning-based microbial interaction inference method using Bayesian estimation (Umibato), a method for estimating time-varying microbial interactions. The Umibato algorithm comprises Gaussian process regression (GPR) and a new Bayesian probabilistic model, the continuous-time regression hidden Markov model (CTRHMM). Growth rates are estimated by GPR, and interaction networks are estimated by CTRHMM. CTRHMM can estimate time-varying interaction networks using interaction states, which are defined as hidden variables. Umibato outperformed the existing methods on synthetic datasets. In addition, it yielded reasonable estimations in experiments on a mouse gut microbiota dataset, thus providing novel insights into the relationship between consumed diets and the gut microbiota.Availability and implementationThe C++ and python source codes of the Umibato software are available at https://github.com/shion-h/Umibato.Supplementary information Supplementary data are available at Bioinformatics online.

Efficient estimation methods for non-Gaussian regression models in continuous time

In this paper, we develop an efficient nonparametric estimation theory for continuous time regression models with non-Gaussian Levy noises in the case when the unknown functions belong to Sobolev ellipses. Using the Pinsker’s approach, we provide a sharp lower bound for the normalized asymptotic mean square accuracy. However, the main result obtained by Pinsker for the Gaussian white noise model is not correct without additional conditions for the ellipse coefficients. We find such constructive sufficient conditions under which we develop efficient estimation methods. We show that the obtained conditions hold for the ellipse coefficients of an exponential form. For exponential coefficients, the sharp lower bound is calculated in explicit form. Finally, we apply this result to signals number detection problems in multi-pass connection channels and we obtain an almost parametric convergence rate that is natural for this case, which significantly improves the rate with respect to power-form coefficients.

The BLUE in continuous-time regression models with correlated errors

In this paper, the problem of best linear unbiased estimation is investigated for continuous-time regression models. We prove several general statements concerning the explicit form of the best linear unbiased estimator (BLUE), in particular when the error process is a smooth process with one or several derivatives of the response process available for construction of the estimators. We derive the explicit form of the BLUE for many specific models including the cases of continuous autoregressive errors of order two and integrated error processes (such as integrated Brownian motion). The results are illustrated on many examples.

Continuous-Time Volatility Regression in Large Panels

This paper studies a semiparametric inference procedure for a finite-dimensional parameter in a continuous-time regression model involving high-frequency data in a large cross-section. The model concerns the relationship between a noisy dependent process and a possibly nonlinear transform of stochastic volatility over a fixed time span, with its coefficients allowed to depend on a set of firm-specific characteristics. The construction of the estimator involves two steps: the nonparametric recovery of stochastic volatility processes, followed by a parametric second stage that uses the volatility estimates. I show that the estimator admits a central limit theorem and provide a consistent estimator of the asymptotic variance based on a factor-analytic method. The finite sample performance of the inference procedure is satisfactory in a realistically calibrated Monte Carlo setting. In a novel empirical application, I study the relationship between bid-ask spread and the spot standard deviation of asset price. The slope coefficient estimate, which measures the heterogeneous level of information asymmetry, is closely related to firm characteristics such as measures of valuation uncertainty and institutional ownership.

Improved robust model selection methods for a Lévy nonparametric regression in continuous time

ABSTRACT In this paper, we develop the James–Stein improved method for the estimation problem of a nonparametric periodic function observed with Lévy noises in continuous time. An adaptive model selection procedure based on the weighted improved least squares estimates is constructed. The improvement effect for nonparametric models is studied. It turns out that in non-asymptotic setting the accuracy improvement for nonparametric models is more important than for parametric ones. Moreover, sharp oracle inequalities for the robust risks have been shown and the adaptive efficiency property for the proposed procedures has been established. The numerical simulations are given.

IMPROVED MODEL SELECTION METHOD FOR AN ADAPTIVE ESTIMATION IN SEMIMARTINGALE REGRESSION MODELS

This paper considers the problem of robust adaptive efficient estimating of a periodic function in a continuous time regression model with the dependent noises given by a general square integrable semimartingale with a conditionally Gaussian distribution. An example of such noise is the non-Gaussian Ornstein-Uhlenbeck-Levy processes. An adaptive model selection procedure, based on the improved weighted least square estimates, is proposed. Under some conditions on the noise distribution, sharp oracle inequality for the robust risk has been proved and the robust efficiency of the model selection procedure has been established. The numerical analysis results are given.

Improving free-viewing fixation-related EEG potentials with continuous-time regression

BackgroundIn the analysis of combined ET-EEG data, there are several issues with estimating FRPs by averaging. Neural responses associated with fixations will likely overlap with one another in the EEG recording and neural responses change as a function of eye movement characteristics. Especially in tasks that do not constrain eye movements in any way, these issues can become confounds. New methodHere, we propose the use of regression based estimates as an alternative to averaging. Multiple regression can disentangle different influences on the EEG and correct for overlap. It thereby accounts for potential confounds in a way that averaging cannot. Specifically, we test the applicability of the rERP framework, as proposed by Smith and Kutas (2015b), (2017), or Sassenhagen (2018) to combined eye tracking and EEG data from a visual search and a scene memorization task. ResultsResults show that the method successfully estimates eye movement related confounds in real experimental data, so that these potential confounds can be accounted for when estimating experimental effects. Comparison with existing methodsThe rERP method successfully corrects for overlapping neural responses in instances where averaging does not. As a consequence, baselining can be applied without risking distortions. By estimating a known experimental effect, we show that rERPs provide an estimate with less variance and more accuracy than averaged FRPs. The method therefore provides a practically feasible and favorable alternative to averaging. ConclusionsWe conclude that regression based ERPs provide novel opportunities for estimating fixation related EEG in free-viewing experiments.

Free-viewing fixation related EEG-potentials with continuous-time regression

Free-viewing fixation related EEG-potentials with continuous-time regression

On the analysis of discrete time competing risks data.

Regression methodology has been well developed for competing risks data with continuous event times, both for the cause-specific hazard and cumulative incidence functions. However, in many applications, including those from the Surveillance, Epidemiology, and End Results (SEER) program of the National Cancer Institute, the event times may be observed discretely. Naive application of continuous time regression methods to such data is not appropriate. We propose maximum likelihood inferences for estimation of model parameters for the discrete time cause-specific hazard functions, develop predictions for the associated cumulative incidence functions, and derive consistent variance estimators for the predicted cumulative incidence functions. The methods are readily implemented using standard software for generalized estimating equations, where models for different causes may be fitted separately. For the SEER data, it may be desirable to model different event types on different time scales and the methods are generalized to accommodate such scenarios, extending earlier work on continuous time data. Simulation studies demonstrate that the methods perform well in realistic set-ups. The methodology is illustrated with stage III colon cancer data from SEER.

Oracle inequalities for the stochastic differential equations

This paper is a survey of recent results on the adaptive robust non parametric methods for the continuous time regression model with the semi - martingale noises with jumps. The noises are modeled by the L\'evy processes, the Ornstein -- Uhlenbeck processes and semi-Markov processes. We represent the general model selection method and the sharp oracle inequalities methods which provide the robust efficient estimation in the adaptive setting. Moreover, we present the recent results on the improved model selection methods for the nonparametric estimation problems.

Large deviations of regression parameter estimator in continuous-time models with sub-Gaussian noise

A continuous-time regression model with a jointly strictly sub-Gaussian random noise is considered in the paper. Upper exponential bounds for probabilities of large deviations of the least squares estimator for the regression parameter are obtained.

Consistent estimator of nonparametric structural spurious regression model for high frequency data

We propose a new nonparametric estimator for continuous-time regression models with nonstationary error terms. While other conventional nonparametric estimators such as the Nadaraya–Watson and local linear estimators are not consistent, our estimator achieves consistency and asymptotic normality.

Adaptive estimation of continuous-time regression models using high-frequency data

We derive the asymptotic efficiency bound for regular estimates of the slope coefficient in a linear continuous-time regression model for the continuous martingale parts of two Itô semimartingales observed on a fixed time interval with asymptotically shrinking mesh of the observation grid. We further construct an estimator from high-frequency data that achieves this efficiency bound and, indeed, is adaptive to the presence of infinite-dimensional nuisance components. The estimator is formed by taking optimal weighted average of local nonparametric volatility estimates that are constructed over blocks of high-frequency observations. The asymptotic efficiency bound is derived under a Markov assumption for the bivariate process while the high-frequency estimator and its asymptotic properties are derived in a general Itô semimartingale setting. To study the asymptotic behavior of the proposed estimator, we introduce a general spatial localization procedure which extends known results on the estimation of integrated volatility functionals to more general classes of functions of volatility. Empirically relevant numerical examples illustrate that the proposed efficient estimator provides nontrivial improvement over alternatives in the extant literature.

Robust model selection for a semimartingale continuous time regression from discrete data

The paper considers the problem of estimating a periodic function in a continuous time regression model observed under a general semimartingale noise with an unknown distribution in the case when continuous observation cannot be provided and only discrete time measurements are available. Two specific types of noises are studied in detail: a non-Gaussian Ornstein–Uhlenbeck process and a time-varying linear combination of a Brownian motion and compound Poisson process. We develop new analytical tools to treat the adaptive estimation problems from discrete data. A lower bound for the frequency sampling, needed for the efficiency of the procedure constructed by discrete observations, has been found. Sharp non-asymptotic oracle inequalities for the robust quadratic risk have been derived. New convergence rates for the efficient procedures have been obtained. An example of the regression with a martingale noise exhibits that the minimax robust convergence rate may be both higher or lower as compared with the minimax rate for the “white noise” model. The results of Monte-Carlo simulations are given.

Estimation of a Regression with the Pulse Type Noise from Discrete Data

This paper considers the problem of estimating parameters in a periodic regression in continuous time with a semimartingale noise by discrete time observations. Improved estimates for the regression parameters are proposed. It is established that under some general conditions these estimates have an advantage in the mean square accuracy over the least squares estimates. The asymptotic minimaxity of the improved estimates has been proved in the robust risk sense. The properties of the proposed procedure for the models with non-Gaussian noises of pulse type have been studied. The pulse disturbances have random intensity and occur at random times which form a Poisson process.

Improved estimation in a non-Gaussian parametric regression

The paper considers the problem of estimating the parameters in a continuous time regression model with a non-Gaussian noise of pulse type. The vector of unknown parameters is assumed to belong to a compact set. The noise is specified by the Ornstein–Uhlenbeck process driven by the mixture of a Brownian motion and a compound Poisson process. Improved estimates for the unknown regression parameters, based on a special modification of the James–Stein procedure with smaller quadratic risk than the usual least squares estimates, are proposed. The developed estimation scheme is applied for the improved parameter estimation in the discrete time regression with the autoregressive noise depending on unknown nuisance parameters.

Efficient robust nonparametric estimation in a semimartingale regression model

Dans cette article nous considérons le problème d’estimation robuste d’une fonction périodique dans un modèle de régression en temps continu avec un bruit dépendant décrit par une semi martingale carrée intégrable de distribution inconnue. Un exemple de ce bruit est un processus d’Ornstein–Uhlenbeck non gaussien avec sauts (voir (J. R. Stat. Soc. Ser. B Stat. Methodol. 63 (2001) 167–241), (Ann. Appl. Probab. 18 (2008) 879–908)). Nous proposons une procédure adaptative de sélection de modèle basée sur les estimateurs des moindres carrés pondérés. Sous des conditions générales sur les deux premiers moments de la distribution du bruit, des inégalités d’Oracle non asymptotiques pointues pour des risques quadratiques robustes sont obtenues et l’efficacité robuste est établie. Nous avons établi aussi que dans le cas du processus d’Ornstein–Uhlenbeck non Gaussian, la borne inférieure pour le risque quadratique robuste est donnée par la limite de l’intensité du bruit quand la fréquence tend vers l’infini. Nous donnons un exemple d’un modèle de régression avec un bruit martingale où la vitesse de convergence du risque quadratique devient plus lente si l’intensité du bruit tend vers l’infini.

Asymptotic Properties of Koenker–Bassett Estimator in Regression Model with Long-Range Dependence

This article deals with asymptotic distribution of Koenker-Bassett estimator in continuous time regression model with long-range dependent noise. It is proved that the asymptotic distribution of the normed estimator coincides with asymptotic distribution of the integral of an indicator random process generated by the random noise weighted by regression function gradient. A theorem is formulated also on asymptotic normality of the last integral.