Functional Regression Methods Research Articles

Dimension reduction for functional regression with a binary response

We propose here a novel functional inverse regression method (i.e., functional surrogate assisted slicing) for functional data with binary responses. Previously developed method (e.g., functional sliced inverse regression) can detect no more than one direction in the functional sufficient dimension reduction subspace. In contrast, the proposed new method can detect multiple directions. The population properties of the proposed method is established. Furthermore, we propose a new method to estimate the functional central space which do not need the inverse of the covariance operator. To practically determine the structure dimension of the functional sufficient dimension reduction subspace, a modified Bayesian information criterion method is proposed. Numerical studies based on both simulated and real data sets are presented.

LQD-RKHS-based distribution-to-distribution regression methodology for restoring the probability distributions of missing SHM data

Data loss is a critical problem in structural health monitoring (SHM). Probability distributions play a highly important role in many applications. Improving the quality of distribution estimations made using incomplete samples is highly important. Missing samples can be compensated for by applying conventional missing data restoration methods; however, ensuring that restored samples roughly follow underlying distributions of true missing data remains a challenge. Another strategy involves directly restoring the probability density function (PDF) for a sensor when samples are missing by leveraging distribution information from another sensor with complete data using distribution regression techniques; existing methods include the conventional distribution-to-distribution regression (DDR) and distribution-to-warping function regression (DWR) methods. Due to constraints on PDFs and warping functions, the regression functions of both methods are estimated from the Nadaraya-Watson kernel estimator with relatively low degrees of precision. This article proposes a new indirect distribution-to-distribution regression method in the context of functional data analysis for restoring distributions of missing SHM data. PDFs are transformed to ordinary functions residing in a Hilbert space via the newly proposed log-quantile-density (LQD) transformation; the regression for distributions is realized in the transformed space via a functional regression model constructed based on the theory of Reproducing Kernel Hilbert Space (RKHS), corresponding result is subsequently mapped back to the density space through the inverse LQD transformation. Test results using field monitoring data indicate that the new method significantly outperforms conventional methods in general cases; however, in extrapolation cases, the new method is inferior to the distribution-to-warping function regression method.

Foreign Workers and the Wage Distribution: What Does the Influence Function Reveal?

This paper draws upon influence function regression methods to determine where foreign workers stand in the distribution of private sector wages in Luxembourg, and assess whether and how much their wages contribute to wage inequality. This is quantified by measuring the effect that a marginal increase in the proportion of foreign workers—foreign residents or cross-border workers—would have on selected quantiles and measures of inequality. Analysis of the 2006 Structure of Earnings Survey reveals that foreign workers have generally lower wages than natives and therefore tend to haul the overall wage distribution downwards. Yet, their influence on wage inequality reveals small and negative. All impacts are further muted when accounting for human capital and, especially, job characteristics. Not observing any large positive inequality contribution on the Luxembourg labour market is a striking result given the sheer size of the foreign workforce and its polarization at both ends of the skill distribution.

On the Use of Functional Additive Models for Electricity Demand and Price Prediction

This paper presents an application of functional additive models in the context of electricity demand and price prediction. Data from the Spanish electricity market are used to obtain the pointwise predictions. Also prediction intervals, based on a bootstrap procedure, are computed. This approach is compared with the use of other functional regression methods applied to the same data set by Aneiros et al. (2016).

Functional regression method for whole genome eQTL epistasis analysis with sequencing data

BackgroundEpistasis plays an essential rule in understanding the regulation mechanisms and is an essential component of the genetic architecture of the gene expressions. However, interaction analysis of gene expressions remains fundamentally unexplored due to great computational challenges and data availability. Due to variation in splicing, transcription start sites, polyadenylation sites, post-transcriptional RNA editing across the entire gene, and transcription rates of the cells, RNA-seq measurements generate large expression variability and collectively create the observed position level read count curves. A single number for measuring gene expression which is widely used for microarray measured gene expression analysis is highly unlikely to sufficiently account for large expression variation across the gene. Simultaneously analyzing epistatic architecture using the RNA-seq and whole genome sequencing (WGS) data poses enormous challenges.MethodsWe develop a nonlinear functional regression model (FRGM) with functional responses where the position-level read counts within a gene are taken as a function of genomic position, and functional predictors where genotype profiles are viewed as a function of genomic position, for epistasis analysis with RNA-seq data. Instead of testing the interaction of all possible pair-wises SNPs, the FRGM takes a gene as a basic unit for epistasis analysis, which tests for the interaction of all possible pairs of genes and use all the information that can be accessed to collectively test interaction between all possible pairs of SNPs within two genome regions.ResultsBy large-scale simulations, we demonstrate that the proposed FRGM for epistasis analysis can achieve the correct type 1 error and has higher power to detect the interactions between genes than the existing methods. The proposed methods are applied to the RNA-seq and WGS data from the 1000 Genome Project. The numbers of pairs of significantly interacting genes after Bonferroni correction identified using FRGM, RPKM and DESeq were 16,2361, 260 and 51, respectively, from the 350 European samples.ConclusionsThe proposed FRGM for epistasis analysis of RNA-seq can capture isoform and position-level information and will have a broad application. Both simulations and real data analysis highlight the potential for the FRGM to be a good choice of the epistatic analysis with sequencing data.

Dimensionality Reduction of High-Dimensional Highly Correlated Multivariate Grapevine Dataset

Viticulturists traditionally have a keen interest in studying the relationship between the biochemistry of grapevines’ leaves/petioles and their associated spectral reflectance in order to understand the fruit ripening rate, water status, nutrient levels, and disease risk. In this paper, we implement imaging spectroscopy (hyperspectral) reflectance data, for the reflective 330 - 2510 nm wavelength region (986 total spectral bands), to assess vineyard nutrient status; this constitutes a high dimensional dataset with a covariance matrix that is ill-conditioned. The identification of the variables (wavelength bands) that contribute useful information for nutrient assessment and prediction, plays a pivotal role in multivariate statistical modeling. In recent years, researchers have successfully developed many continuous, nearly unbiased, sparse and accurate variable selection methods to overcome this problem. This paper compares four regularized and one functional regression methods: Elastic Net, Multi-Step Adaptive Elastic Net, Minimax Concave Penalty, iterative Sure Independence Screening, and Functional Data Analysis for wavelength variable selection. Thereafter, the predictive performance of these regularized sparse models is enhanced using the stepwise regression. This comparative study of regression methods using a high-dimensional and highly correlated grapevine hyperspectral dataset revealed that the performance of Elastic Net for variable selection yields the best predictive ability.

Dynamic single-index model for functional data

We propose a new functional single index model, which called dynamic single-index model for functional data, or DSIM, to efficiently perform non-linear and dynamic relationships between functional predictor and functional response. The proposed model naturally allows for some curvature not captured by the ordinary functional linear model. By using the proposed two-step estimating algorithm, we develop the estimates for both the link function and the regression coefficient function, and then provide predictions of new response trajectories. Besides the asymptotic properties for the estimates of the unknown functions, we also establish the consistency of the predictions of new response trajectories under mild conditions. Finally, we show through extensive simulation studies and a real data example that the proposed DSIM can highly outperform existed functional regression methods in most settings.

Flexible functional regression methods for estimating individualized treatment regimes.

A major focus of personalized medicine is on the development of individualized treatment rules. Good decision rules have the potential to significantly advance patient care and reduce the burden of a host of diseases. Statistical methods for developing such rules are progressing rapidly, but few methods have considered the use of pre-treatment functional data to guide in decision-making. Furthermore, those methods that do allow for the incorporation of functional pre-treatment covariates typically make strong assumptions about the relationships between the functional covariates and the response of interest. We propose two approaches for using functional data to select an optimal treatment that address some of the shortcomings of previously developed methods. Specifically, we combine the flexibility of functional additive regression models with Q-learning or A-learning in order to obtain treatment decision rules. Properties of the corresponding estimators are discussed. Our approaches are evaluated in several realistic settings using synthetic data and are applied to real data arising from a clinical trial comparing two treatments for major depressive disorder in which baseline imaging data are available for subjects who are subsequently treated.

Functional Regression for State Prediction Using Linear PDE Models and Observations

Partial differential equations (PDEs) are commonly used to model a wide variety of physical phenomena. A PDE model of a physical problem is typically described by conservation laws, constitutive laws, material properties, boundary conditions, boundary data, and geometry. In most practical applications, however, the PDE model is only an approximation to the real physical problem due to both (i) the deliberate mathematical simplification of the model to keep it tractable and (ii) the inherent uncertainty of the physical parameters. In such cases, the PDE model may not produce a good prediction of the true state of the underlying physical problem. In this paper, we introduce a functional regression method that incorporates observations into a deterministic linear PDE model to improve its prediction of the true state. Our method is devised as follows. First, we augment the PDE model with a random Gaussian functional which serves to represent various sources of uncertainty in the model. We next derive a linear regression model for the Gaussian functional by utilizing observations and adjoint states. This allows us to determine the posterior distribution of the Gaussian functional and the posterior distribution for our estimate of the true state. Furthermore, we consider the problem of experimental design in this setting, wherein we develop an algorithm for designing experiments to efficiently reduce the variance of our state estimate. We provide several examples from the heat conduction, the convection-diffusion equation, and the reduced wave equation, all of which demonstrate the performance of the proposed methodology.

Foreign Workers and the Wage Distribution: Where Do They Fit in?

The presence of foreign workers is commonly deemed as driving wage inequality upwards. By 2006, seven in ten private sector workers in Luxembourg were foreign. This note builds on recentered influence function regression methods to identify where these foreign workers stand in the distribution of private sector wages, and assess whether and how much their wages contribute to overall wage inequality. Our analysis of the 2006 Structure of Earnings Survey reveals that foreign workers have generally lower wages than natives and therefore tend to haul the overall wage distribution downwards. Yet, surprizingly, their influence on wage inequality reveals small and negative. All impacts are further muted when accounting for human capital and, especially, job characteristics. Not observing any large positive inequality contribution on the Luxembourg labour market is a striking result given the sheer size of the foreign workforce and its polarization at both ends of the skill distribution.

Functional Regression

Functional data analysis (FDA) involves the analysis of data whose ideal units of observation are functions defined on some continuous domain, and the observed data consist of a sample of functions taken from some population, sampled on a discrete grid. Ramsay & Silverman's (1997) textbook sparked the development of this field, which has accelerated in the past 10 years to become one of the fastest growing areas of statistics, fueled by the growing number of applications yielding this type of data. One unique characteristic of FDA is the need to combine information both across and within functions, which Ramsay and Silverman called replication and regularization, respectively. This article focuses on functional regression, the area of FDA that has received the most attention in applications and methodological development. First, there is an introduction to basis functions, key building blocks for regularization in functional regression methods, followed by an overview of functional regression methods, split into three types: (a) functional predictor regression (scalar-on-function), (b) functional response regression (function-on-scalar), and (c) function-on-function regression. For each, the role of replication and regularization is discussed and the methodological development described in a roughly chronological manner, at times deviating from the historical timeline to group together similar methods. The primary focus is on modeling and methodology, highlighting the modeling structures that have been developed and the various regularization approaches employed. The review concludes with a brief discussion describing potential areas of future development in this field.

Index Models for Sparsely Sampled Functional Data

The regression problem involving functional predictors has many important applications and a number of functional regression methods have been developed. However, a common complication in functional data analysis is one of sparsely observed curves, that is predictors that are observed, with error, on a small subset of the possible time points. Such sparsely observed data induce an errors-in-variables model, where one must account for measurement error in the functional predictors. Faced with sparsely observed data, most current functional regression methods simply estimate the unobserved predictors and treat them as fully observed; thus failing to account for the extra uncertainty from the measurement error. We propose a new functional errors-in-variables approach, sparse index model functional estimation (SIMFE), which uses a functional index model formulation to deal with sparsely observed predictors. SIMFE has several advantages over more traditional methods. First, the index model implements a nonlinear regression and uses an accurate supervised method to estimate the lower dimensional space into which the predictors should be projected. Second, SIMFE can be applied to both scalar and functional responses and multiple predictors. Finally, SIMFE uses a mixed effects model to effectively deal with very sparsely observed functional predictors and to correctly model the measurement error. Supplementary materials for this article are available online.

Gaussian functional regression for linear partial differential equations

In this paper, we present a new statistical approach to the problem of incorporating experimental observations into a mathematical model described by linear partial differential equations (PDEs) to improve the prediction of the state of a physical system. We augment the linear PDE with a functional that accounts for the uncertainty in the mathematical model and is modeled as a Gaussian process. This gives rise to a stochastic PDE which is characterized by the Gaussian functional. We develop a Gaussian functional regression method to determine the posterior mean and covariance of the Gaussian functional, thereby solving the stochastic PDE to obtain the posterior distribution for our prediction of the physical state. Our method has the following features which distinguish itself from other regression methods. First, it incorporates both the mathematical model and the observations into the regression procedure. Second, it can handle the observations given in the form of linear functionals of the field variable. Third, the method is non-parametric in the sense that it provides a systematic way to optimally determine the prior covariance operator of the Gaussian functional based on the observations. Fourth, it provides the posterior distribution quantifying the magnitude of uncertainty in our prediction of the physical state. We present numerical results to illustrate these features of the method and compare its performance to that of the standard Gaussian process regression.

Dynamic Retrospective Regression for Functional Data

Samples of curves, or functional data, usually present phase variability in addition to amplitude variability. Existing functional regression methods do not handle phase variability in an efficient way. In this article we propose a functional regression method that incorporates phase synchronization as an intrinsic part of the model, and then attains better predictive power than ordinary linear regression in a simple and parsimonious way. The finite-sample properties of the estimators are studied by simulation. As an example of application, we analyze neuromotor data arising from a study of human lip movement. This article has supplementary materials online.

Warped functional regression

A characteristic feature of functional data is the presence of phase variability in addition to amplitude variability. Existing functional regression methods do not handle time variability in an explicit and efficient way. In this paper we introduce a functional regression method that incorporates time warping as an intrinsic part of the model. The method achieves good predictive power in a parsimonious way and allows unified statistical inference about phase and amplitude components. The asymptotic distribution of the estimators is derived and their finite-sample properties are studied by simulation. An application involving ground-level ozone trajectories is presented.

Sparse wavelet regression with multiple predictive curves

With the advance of techniques, more and more complicated data are extracted and recorded. In this paper, functional regression models with a scalar response and multiple predictive curves are considered. We transform the functional regression models to multiple linear regression models by using the discrete wavelet transformation. When the number of predictive curves is big, the multiple linear regression model usually has much bigger number of features than the sample size. We apply our correlation-based sparse regression method to the resulted high dimensional regression model. The novel feature of our sparse method is that we impose sparsity penalty on the direction of the estimate of the coefficient vector instead of the estimate itself, and only the direction of the estimate is determined by an optimization problem. The estimation consistency of the coefficient curve for the functional regression model is obtained when both the sample size and the number of curves go to infinity. The effects of the discrete observations are discussed. We compare our method with both functional regression methods and other wavelet based sparse regression methods on both simulated data and four real data sets, including the cases of single and multiple predictive curves. The results indicate that sparse wavelet regression methods are better in extracting local features and our method has good predictive performances in all scenarios.

Functional Availability Modelling of CNC Machine Tools

A new method for calculating the instantaneous availability was proposed based on Functional Data Analysis method. It introduced the Quadratic Bernstein Polynomial into the smoothing method firstly for the reliability which estimated by median rank method and estimated the fitting parameters by least square method. Then, under the assumption that the maintenance difficult of the CNCs was decreasing over the work time, chosen the appropriate smoothing basis function based on the trend after the time section adjustment for the estimated maintainability value. The Fourier basis system and the non-linear least squares were selected for the maintainability function smoothing method and the fitting parameters. Finally, the instantaneous availability model of CNCs was built based on the functional linear regression method, and a case example of 15 CNCs was given.

Estimator selection and combination in scalar-on-function regression

Scalar-on-function regression problems with continuous outcomes arise naturally in many settings, and a wealth of estimation methods now exist. Despite the clear differences in regression model assumptions, tuning parameter selection, and the incorporation of functional structure, it remains common to apply a single method to any dataset of interest. In this paper we develop tools for estimator selection and combination in the context of continuous scalar-on-function regression based on minimizing the cross-validated prediction error of the final estimator. A broad collection of functional and high-dimensional regression methods is used as a library of candidate estimators. We find that the performance of any single method relative to others can vary dramatically across datasets, but that the proposed cross-validation procedure is consistently among the top performers. Four real-data analyses using publicly available benchmark datasets are presented; code implementing these analyses and facilitating the application of proposed methods on future datasets is available in a web supplement.

Using Hyperspectral Spectrometry and Functional Models to Characterize Vine-Leaf Composition

Concentrations of certain chemical elements in plants need to be controlled to ensure good crop quality and yield. However, laboratory analyses are usually time-consuming and expensive. Although indirect methods based on leaf reflectance are both faster and less expensive, most are based on indexes that only take into account certain wavelengths and fail to take full advantage of the entire reflectance curve depicted by modern hyperspectral sensors. This paper applies two functional prediction models, i.e., functional linear regression and functional nonparametric methods, to the prediction of the chemical characteristics (moisture, dry mass, and concentrations of nitrogen, phosphorus, potassium, calcium, iron, and magnesium) of vine leaves, using electromagnetic reflectance between 350 and 2500 nm as the input. Cross-validation was used to obtain optimal parameters for the models, which were tested using samples reflecting 5% and 10% of the sample size. The results obtained showed different levels of correlation between reflectance and the predicted data. The nonparametric methods yielded better results as they produced smaller prediction errors than functional regression. Moisture $(R^{2} = 0.96)$ and nitrogen $(R^{2} = 0.95)$ were the best predicted components, whereas magnesium content was the worst predicted component $(R^{2} = 0.77)$ .

Permanent disability classification by combining evolutionary Generalized Radial Basis Function and logistic regression methods

Recently, a novelty multinomial logistic regression method where the initial covariate space is increased by adding the nonlinear transformations of the input variables given by Gaussian Radial Basis Functions (RBFs) obtained by an evolutionary algorithm was proposed. However, there still exist some problems with the standard Gaussian RBF, for example, the approximation of constant valued functions or the approximation of high dimensionality associated to some real problems. In order to face these problems, we propose the use of the generalized Gaussian RBF (GRBF) instead of the standard Gaussian RBF. Our approach has been validated with a real problem of disability classification, to evaluate its effectiveness. Experimental results show that this approach is able to achieve good generalization performance.