Understanding generalization of functional linear regression from non-i.i.d. sample viewpoint

?Two complaints against the linear functional regression model have been that the estimated regression lines are not symmetrical in x and y and that they depend on the scale of the observed data in unpredictable ways. If one takes into account the ratio of measurement X, then the estimates from linear functional regression are symmetrical (i.e., the regression of y on x and x on y are the same line). Using nonlinear least squares estimation, functional regression is easily applied to fit both nonlinear and linear models. Again taking into account X, linear and nonlinear models can be naturally rescaled (as with ordinary regression estimates) according to the units of the observed variables. The log-transformed functional regression model and the relationships between principal methods and the linear functional regression model are considered. Controversies surrounding the major axis are discussed; sim? ulation results are presented, and the validity of the method is questioned. Treatment of the linear, allometry, polynomial, and von Bertalanffy models is discussed, and a numerical example is presented. [Functional regression; in variables; major axis; reduced major axis; standard major axis; nonlinear regression; allometry.] The critical work of systematic biologists often deals with measuring ontogenetic changes and differentiating entities using regression. These problems have been made more interesting and difficult due to the realization that has grown since the days of Kermack and Haldane (1950) that ordinary least squares (OLS) is often not a suitable regression method; in biological contexts the estimated relationship be? tween any two can be biased by the existence of substantial measurement error and/or biological variation (Ricker, 1973). Alternatives to OLS usually fall under the heading of functional or structural re? gression. Kendall and Stuart (1973) pro? vided a clear introduction to these some? times confusing methods. In this paper I deal with only the func? tional relationship regression model be? cause it seems the simplest and most gen? erally applicable of the errors in variables models. However, there is a clear distinc? tion between the functional relationships posited between the mathematical vari? ables (e.g., linear, quadratic, allometric, etc.) and the functional regression method, which estimates parameters assuming that mathematical x and y are both observed with error. Least Squares Estimates Let two mathematical {xiry^,i = 1, . . ., n, be related through some func? tional relationship yi = i(xif fi), where xi and y{ are scalar but fi may be a vector of p variables. We are not able to observe xi and l/i directly but instead observe the ran? dom Xi = xi + 8{ and Yi = f(x{, fi) + ?i, where 5{ and ei are independent, nor? mal random with mean zero and variances a25 and a\, respectively. The mi? nus log likelihood for estimating the n + p unknowns (xlf..., x?, fiu ..., fip) is then (Fuller, 1987; Seber and Wild, 1989) -L = S {[Vi ffe ??/(2a2f) + (X, x{)2/(2a28)}

Functional linear regression analysis aims to model regression relations which include a functional predictor. The analog of the regression parameter vector or matrix in conventional multivariate or multiple-response linear regression models is a regression parameter function in one or two arguments. If, in addition, one has scalar predictors, as is often the case in applications to longitudinal studies, the question arises how to incorporate these into a functional regression model. We study a varying-coefficient approach where the scalar covariates are modeled as additional arguments of the regression parameter function. This extension of the functional linear regression model is analogous to the extension of conventional linear regression models to varying-coefficient models and shares its advantages, such as increased flexibility; however, the details of this extension are more challenging in the functional case. Our methodology combines smoothing methods with regularization by truncation at a finite number of functional principal components. A practical version is developed and is shown to perform better than functional linear regression for longitudinal data. We investigate the asymptotic properties of varying-coefficient functional linear regression and establish consistency properties.

Functional linear regression is one of the main modeling tools for working with functional data. Since functional data are usually stream data essentially and there are some noises in functional data. Many numerical research studies of machine learning indicate that the noise samples not only increase the amount of storage space, but also affect the performance of algorithm. Therefore, in this paper we consider a new learning strategy by introducing incremental learning, Markov sampling for functional linear regression and propose a novel functional incremental linear square regression algorithm based on Markov sampling (FILSR-MS). To have a better understanding of the proposed FILSR-MS, we not only estimate the generalization bound of the proposed algorithm and establish the fast learning rate, but also present some useful discussions. The performance of the proposed algorithm is validated by the numerical experiments for benchmark repository.

We study regression models for the situation where both dependent and independent variables are square-integrable stochastic processes. Questions concerning the definition and existence of the corresponding functional linear regression models and some basic properties are explored for this situation. We derive a representation of the regression parameter function in terms of the canonical components of the processes involved. This representation establishes a connection between functional regression and functional canonical analysis and suggests alternative approaches for the implementation of functional linear regression analysis. A specific procedure for the estimation of the regression parameter function using canonical expansions is proposed and compared with an established functional principal component regression approach. As an example of an application, we present an analysis of mortality data for cohorts of medflies, obtained in experimental studies of aging and longevity.

SummaryWe propose a novel method for variable selection in functional linear concurrent regression. Our research is motivated by a fisheries footprint study where the goal is to identify important time-varying sociostructural drivers influencing patterns of seafood consumption, and hence the fisheries footprint, over time, as well as estimating their dynamic effects. We develop a variable-selection method in functional linear concurrent regression extending the classically used scalar-on-scalar variable-selection methods like the lasso, smoothly clipped absolute deviation (SCAD) and minimax concave penalty (MCP). We show that in functional linear concurrent regression the variable-selection problem can be addressed as a group lasso, and their natural extension: the group SCAD or a group MCP problem. Through simulations, we illustrate that our method, particularly with the group SCAD or group MCP, can pick out the relevant variables with high accuracy and has minuscule false positive and false negative rate even when data are observed sparsely, are contaminated with noise and the error process is highly non-stationary. We also demonstrate two real data applications of our method in studies of dietary calcium absorption and fisheries footprint in the selection of influential time-varying covariates.

We propose nonparametric methods for functional linear regression which are designed for sparse longitudinal data, where both the predictor and response are functions of a covariate such as time. Predictor and response processes have smooth random trajectories, and the data consist of a small number of noisy repeated measurements made at irregular times for a sample of subjects. In longitudinal studies, the number of repeated measurements per subject is often small and may be modeled as a discrete random number and, accordingly, only a finite and asymptotically nonincreasing number of measurements are available for each subject or experimental unit. We propose a functional regression approach for this situation, using functional principal component analysis, where we estimate the functional principal component scores through conditional expectations. This allows the prediction of an unobserved response trajectory from sparse measurements of a predictor trajectory. The resulting technique is flexible and allows for different patterns regarding the timing of the measurements obtained for predictor and response trajectories. Asymptotic properties for a sample of n subjects are investigated under mild conditions, as n→∞, and we obtain consistent estimation for the regression function. Besides convergence results for the components of functional linear regression, such as the regression parameter function, we construct asymptotic pointwise confidence bands for the predicted trajectories. A functional coefficient of determination as a measure of the variance explained by the functional regression model is introduced, extending the standard R2 to the functional case. The proposed methods are illustrated with a simulation study, longitudinal primary biliary liver cirrhosis data and an analysis of the longitudinal relationship between blood pressure and body mass index.

This article studies a general regression model for a scalar quality response with mixed types of process predictors including process images, functional sensing signals, and scalar process setup attributes. To represent a set of time-dependent process images, a third-order tensor is employed for preserving not only the spatial correlation of pixels within one image but also the temporal dependency among a sequence of images. Although there exist some papers dealing with either tensorial or functional regression, there is little research to thoroughly study a regression model consisting of both tensorial and functional predictors. For simplicity, the presented regression model is called functional linear regression with tensorial and functional predictor (FLR-TFP). The advantage of the presented FLR-TFP model, which is compared to the classical stack-up strategy, is that FLR-TFP can handle both tensorial and functional predictors without destroying the data correlation structure. To estimate an FLR-TFP model, this article presents a new alternating Elastic Net (AEN) estimation algorithm, in which the problem is reformed as three sub-problems by iteratively estimating each group of tensorial, functional, and scalar parameters. To execute the proposed AEN algorithm, a systematic approach is developed to effectively determine the initial running sequence among three sub-problems. The performance of the FLR-TFP model is evaluated using simulations and a real-world case study of friction stir blind riveting process.

ABSTRACTCarbon dioxide is one of the major contributors to Global Warming. In the present study, we develop a differential equation to model the carbon dioxide emission data in the atmosphere using functional linear regression approach. In the proposed method, a differential operator is defined as data smoother and we use the penalized least square fitting criteria to smooth the data. The profile error sum of squares is optimized to estimate the differential operators using functional regression. The solution of the developed differential equation estimates and predicts the rate of change of carbon dioxide in the atmosphere at a particular time. We apply the proposed model to fit the emission of carbon dioxide data in the continental United States. Numerical simulations of a number of test cases depict a satisfactory agreement with real data.

We extend the common linear functional regression model to the case where the dependency of a scalar response on a functional predictor is of polynomial rather than linear nature. Focusing on the quadratic case, we demonstrate the usefulness of the polynomial functional regression model, which encompasses linear functional regression as a special case. Our approach works under mild conditions for the case of densely spaced observations and also can be extended to the important practical situation where the functional predictors are derived from sparse and irregular measurements, as is the case in many longitudinal studies. A key observation is the equivalence of the functional polynomial model with a regression model that is a polynomial of the same order in the functional principal component scores of the predictor processes. Theoretical analysis as well as practical implementations are based on this equivalence and on basis representations of predictor processes. We also obtain an explicit representation of the regression surface that defines quadratic functional regression and provide functional asymptotic results for an increasing number of model components as the number of subjects in the study increases. The improvements that can be gained by adopting quadratic as compared to linear functional regression are illustrated with a case study that includes absorption spectra as functional predictors.

Varying-coefficient partially functional linear quantile regression models

Functional linear regression after spline transformation

Element enrichment factor calculation using grain-size distribution and functional data regression

This article provides an overview of recent nonparametric and semiparametric advances in kernel regression estimation for functional data. In particular, it considers the various statistical techniques based on kernel smoothing ideas that have recently been developed for functional regression estimation problems. The article first examines nonparametric functional regression modelling before discussing three popular functional regression estimates constructed by means of kernel ideas, namely: the Nadaraya-Watson convolution kernel estimate, the kNN functional estimate, and the local linear functional estimate. Uniform asymptotic results are then presented. The article proceeds by reviewing kernel methods in semiparametric functional regression such as single functional index regression and partial linear functional regression. It also looks at the use of kernels for additive functional regression and concludes by assessing the impact of kernel methods on practical real-data analysis involving functional (curves) datasets.

The aim of this paper is to develop a Bayesian functional linear Cox regression model (BFLCRM) with both functional and scalar covariates. This new development is motivated by establishing the likelihood of conversion to Alzheimer's disease (AD) in 346 patients with mild cognitive impairment (MCI) enrolled in the Alzheimer's Disease Neuroimaging Initiative 1 (ADNI-1) and the early markers of conversion. These 346 MCI patients were followed over 48 months, with 161 MCI participants progressing to AD at 48 months. The functional linear Cox regression model was used to establish that functional covariates including hippocampus surface morphology and scalar covariates including brain MRI volumes, cognitive performance (ADAS-Cog), and APOE status can accurately predict time to onset of AD. Posterior computation proceeds via an efficient Markov chain Monte Carlo algorithm. A simulation study is performed to evaluate the finite sample performance of BFLCRM.

Checking the adequacy of functional linear quantile regression model