Scalar Predictors Research Articles

A method for variable selection in a multivariate functional linear regression model with multiple scalar responses

In this paper, we propose a new functional variable selection procedure for a functional linear model with multiple scalar responses and multiple functional predictors. This procedure takes into account the loss incurred by selecting a subset of variables, measured by a criterion depending on this subset and involving covariance matrices of variables obtained from basis expansions of the initial functional predictors. By sorting in decreasing order the values of this criterion on the subsets obtained by removing each predictor, the set of relevant predictors is expressed as depending on two parameters which then need to be estimated. Estimates of these parameters based on appropriate penalizations of estimates of the criterion are then introduced, thereby leading to our proposal for variable selection. Simulation studies are carried out to evaluate the finite sample performance of the proposed method with comparison to that of several existing methods, and also a weather data set is analyzed.

Bayesian covariance regression in functional data analysis with applications to functional brain imaging.

Function on scalar regression models relate functional outcomes to scalar predictors through the conditional mean function. With few and limited exceptions, many functional regression frameworks operate under the assumption that covariate information does not affect patterns of covariation. In this manuscript, we address this disparity by developing a Bayesian functional regression model, providing joint inference for both the conditional mean and covariance functions. Our work hinges on basis expansions of both the functional evaluation domain and covariate space, to define flexible non-parametric forms of dependence. To aid interpretation, we develop novel low-dimensional summaries, which indicate the degree of covariate-dependent heteroskedasticity. The proposed modeling framework is motivated and applied to a case study in functional brain imaging through electroencephalography, aiming to elucidate potential differentiation in the neural development of children with autism spectrum disorder.

High-dimensional partially linear functional Cox models.

As a commonly employed method for analyzing time-to-event data involving functional predictors, the functional Cox model assumes a linear relationship between the functional principal component (FPC) scores of the functional predictors and the hazard rates. However, in practical scenarios, such as our study on the survival time of kidney transplant recipients, this assumption often fails to hold. To address this limitation, we introduce a class of high-dimensional partially linear functional Cox models, which accommodates the non-linear effects of functional predictors on the response and allows for diverging numbers of scalar predictors and FPCs as the sample size increases. We employ the group smoothly clipped absolute deviation method to select relevant scalar predictors and FPCs, and use B-splines to obtain a smoothed estimate of the non-linear effect. The finite sample performance of the estimates is evaluated through simulation studies. The model is also applied to a kidney transplant dataset, allowing us to make inferences about the non-linear effects of functional predictors on patients' hazard rates, as well as to identify significant scalar predictors for long-term survival time.

Ultra-Efficient MCMC for Bayesian Longitudinal Functional Data Analysis

Functional mixed models are widely useful for regression analysis with dependent functional data, including longitudinal functional data with scalar predictors. However, existing algorithms for Bayesian inference with these models only provide either scalable computing or accurate approximations to the posterior distribution, but not both. We introduce a new MCMC sampling strategy for highly efficient and fully Bayesian regression with longitudinal functional data. Using a novel blocking structure paired with an orthogonalized basis reparameterization, our algorithm jointly samples the fixed effects regression functions together with all subject- and replicate-specific random effects functions. Crucially, the joint sampler optimizes sampling efficiency for these key parameters while preserving computational scalability. Perhaps surprisingly, our new MCMC sampling algorithm even surpasses state-of-the-art algorithms for frequentist estimation and variational Bayes approximations for functional mixed models—while also providing accurate posterior uncertainty quantification—and is orders of magnitude faster than existing Gibbs samplers. Simulation studies show improved point estimation and interval coverage in nearly all simulation settings over competing approaches. We apply our method to a large physical activity dataset to study how various demographic and health factors associate with intraday activity. Supplementary materials for this article are available online.

Covariate-Dependent Clustering of Undirected Networks with Brain-Imaging Data

This article focuses on model-based clustering of subjects based on the shared relationships of subject-specific networks and covariates in scenarios when there are differences in the relationship between networks and covariates for different groups of subjects. It is also of interest to identify the network nodes significantly associated with each covariate in each cluster of subjects. To address these methodological questions, we propose a novel nonparametric Bayesian mixture modeling framework with an undirected network response and scalar predictors. The symmetric matrix coefficients corresponding to the scalar predictors of interest in each mixture component involve low-rankness and group sparsity within the low-rank structure. While the low-rank structure in the network coefficients adds parsimony and computational efficiency, the group sparsity within the low-rank structure enables drawing inference on network nodes and cells significantly associated with each scalar predictor. Being a principled Bayesian mixture modeling framework, our approach allows model-based identification of the number of clusters, offers clustering uncertainty in terms of the co-clustering matrix and presents precise characterization of uncertainty in identifying network nodes significantly related to a predictor in each cluster. Empirical results in various simulation scenarios illustrate substantial inferential gains of the proposed framework in comparison with competitors. Analysis of a real brain connectome dataset using the proposed method provides interesting insights into the brain regions of interest (ROIs) significantly related to creative achievement in each cluster of subjects. Supplementary material shows the convergence rate for the posterior predictive density of the proposed model, additional simulation examples with model misspecification, full conditional distributions to run the Markov chain Monte Carlo (MCMC) algorithm and also presents traceplots for various model parameters to demonstrate convergence of the MCMC algorithm.

Functional support vector machine.

Linear and generalized linear scalar-on-function modeling have been commonly used to understand the relationship between a scalar response variable (e.g. continuous, binary outcomes) and functional predictors. Such techniques are sensitive to model misspecification when the relationship between the response variable and the functional predictors is complex. On the other hand, support vector machines (SVMs) are among the most robust prediction models but do not take account of the high correlations between repeated measurements and cannot be used for irregular data. In this work, we propose a novel method to integrate functional principal component analysis with SVM techniques for classification and regression to account for the continuous nature of functional data and the nonlinear relationship between the scalar response variable and the functional predictors. We demonstrate the performance of our method through extensive simulation experiments and two real data applications: the classification of alcoholics using electroencephalography signals and the prediction of glucobrassicin concentration using near-infrared reflectance spectroscopy. Our methods especially have more advantages when the measurement errors in functional predictors are relatively large.

Sure independent screening for functional regression model

Due to rapid advancements in computer technology, high-dimensional, big, and complex data, such as functional data, where observations are considered as curves, have emerged from many applications in various disciplines of sciences, such as biomedicine, chemometrics, engineering, and social sciences. Variable selection has consequently become one of the most important problems in statistical research in recent years. As the functional data are inherently infinite dimensional, variable selection problem in multiple functional regression model is, therefore, challenging and difficult. In this study, we extend Sure Independence Screening (SIS) method to a multiple functional regression model with a scalar response and functional predictors. We show that the SIS-based procedure for multiple functional regression model retaining important functional predictors while screening out unimportant functional predictors, and reduce the dimension efficiently. Our methodologies are validated by simulation studies as well as some applications to data.

Semiparametric estimation for the functional additive hazards model

AbstractWe propose a new functional additive hazards model to investigate the potential effects of functional and scalar predictors on mortality risks, and develop a penalized least squares estimation method for model parameters based on a pseudoscore estimating equation. A reproducing kernel Hilbert space approach is used to establish the consistency, convergence rate, and joint asymptotic distribution of the resulting estimators for finite‐dimensional and infinite‐dimensional parameters. Our simulation studies demonstrate that the proposed estimation procedure performs well. For illustration, we apply the proposed method to the Medical Information Mart for Intensive Care III dataset.

Predicting Short-Term Intensity Change in Tropical Cyclones Using a Convolutional Neural Network

Abstract This study details a two-method, machine learning approach to predict current and short-term intensity change in global tropical cyclones (TCs), “D-MINT” and “D-PRINT.” D-MINT and D-PRINT use infrared imagery and environmental scalar predictors, while D-MINT also employs microwave imagery. Results show that current TC intensity estimates from D-MINT and D-PRINT are more skillful than three established intensity estimation methods routinely used by operational forecasters for North Atlantic and eastern and western North Pacific TCs. Short-term intensity predictions are validated against five operational deterministic guidances at 6-, 12-, 18-, and 24-h lead times. D-MINT and D-PRINT are less skillful than NHC and consensus TC intensity predictions in North Atlantic and eastern North Pacific TCs, but are more skillful than the other guidances for at least half of the lead times. In western North Pacific, north Indian Ocean, and Southern Hemisphere TCs, D-MINT is more skillful than the JTWC and other individual TC intensity forecasts for over half of the lead times. When probabilistically predicting TC rapid intensification (RI), D-MINT is more skillful in North Atlantic and western North Pacific TCs than three operationally used RI guidances, but less skillful for yes–no RI forecasts. In addition, this work demonstrates the importance of microwave imagery, as D-MINT is more skillful than D-PRINT. Since D-MINT and D-PRINT are convolutional neural network models interrogating two-dimensional structures within TC satellite imagery, this study also demonstrates that those features can yield better short-term predictions than existing scalar statistics of satellite imagery in operational models. Finally, a diagnostics tool is revealed to aid the attribution of the D-MINT/D-PRINT intensity predictions. Significance Statement This study develops a method to predict current and short-term forecasts of tropical cyclone (TC) intensity using artificial intelligence. The resultant models use a convolutional neural network (CNN) that can identify two-dimensional features in satellite imagery that are indicative of TC intensity and future intensity change. The performance results indicate that in several TC basins, the CNN approach is generally more skillful than alternative satellite-based estimates of TC intensity as well as operational short-term forecasts of deterministic intensity change and of similar skill to probabilistic rapid intensification forecasts.

Generalized Partially Functional Linear Model with Unknown Link Function

In existing models with an unknown link function, the issue of predictors containing both multiple functional data and multiple scalar data has not been studied. To fill this gap, we propose a generalized partially functional linear model, which not only models the relationship between multiple scalar and functional predictors and responses, but also automatically estimates the link function. Specifically, we use the functional principal component analysis method to reduce the dimensionality of functional predictors, estimate the regression coefficients using the maximum likelihood estimation method, estimate the link function using the method of local linear regression, iteratively obtain the final estimator, and establish the asymptotic normality of the estimator. The asymptotic normality is illustrated through simulation experiments. Finally, the proposed model is applied to study the influence of environmental, economic, and medical levels on life expectancy in China. In the study, functional predictors are the daily air quality index, temperature, and humidity of 58 cities in 2020, and scalar predictors are GDP and the number of beds in hospitals. The experimental results indicate that the unknown link function model has a smaller prediction error and better performance than both the model with the known link function and the model without a link function.

Gaussian copula function-on-scalar regression in reproducing kernel Hilbert space

Gaussian copula function-on-scalar regression in reproducing kernel Hilbert space

Smoothed quantile regression for partially functional linear models in high dimensions.

Practitioners of current data analysis are regularly confronted with the situation where the heavy-tailed skewed response is related to both multiple functional predictors and high-dimensional scalar covariates. We propose a new class of partially functional penalized convolution-type smoothed quantile regression to characterize the conditional quantile level between a scalar response and predictors of both functional and scalar types. The new approach overcomes the lack of smoothness and severe convexity of the standard quantile empirical loss, considerably improving the computing efficiency of partially functional quantile regression. We investigate a folded concave penalized estimator for simultaneous variable selection and estimation by the modified local adaptive majorize-minimization (LAMM) algorithm. The functional predictors can be dense or sparse and are approximated by the principal component basis. Under mild conditions, the consistency and oracle properties of the resulting estimators are established. Simulation studies demonstrate a competitive performance against the partially functional standard penalized quantile regression. A real application using Alzheimer's Disease Neuroimaging Initiative data is utilized to illustrate the practicality of the proposedmodel.

Goodness-of-fit test for partial functional linear model with errors in scalar covariates

Goodness-of-fit test for partial functional linear model with errors in scalar covariates

Nonlinear function-on-scalar regression via functional universal approximation.

We consider general nonlinear function-on-scalar (FOS) regression models, where the functional response depends on multiple scalar predictors in a general unknown nonlinear form. Existing methods either assume specific model forms (e.g., additive models) or directly estimate the nonlinear function in a space with dimension equal to the number of scalar predictors, which can only be applied to models with a few scalar predictors. To overcome these shortcomings, motivated by the classic universal approximation theorem used in neural networks, we develop a functional universal approximation theorem which can be used to approximate general nonlinear FOS maps and can be easily adopted into the framework of functional data analysis. With this theorem and utilizing smoothness regularity, we develop a novel method to fit the general nonlinear FOS regression model and make predictions. Our new method does not make any specific assumption on the model forms, and it avoids the direct estimation of nonlinear functions in a space with dimension equal to the number of scalar predictors. By estimating a sequence of bivariate functions, our method can be applied to models with a relatively large number of scalar predictors. The good performance of the proposed method is demonstrated by empirical studies on various simulated and realdatasets.

New product life cycle curve modeling and forecasting with product attributes and promotion: A Bayesian functional approach

New products are highly valued by manufacturers and retailers due to their vital role in revenue generation. Product life cycle (PLC) curves often vary by their shapes and are complicated by promotional activities that induce spiky and irregular behaviors. We collaborate with JD.com to develop a flexible PLC curve forecasting framework based on Bayesian functional regression that accounts for useful covariate information, including product attributes and promotion. The functional model treats PLC curves as target variables and includes both scalar and functional predictors, capturing time‐varying promotional activities. Harnessing the power of basis function transformation, the developed model can effectively characterize the local features and temporal evolution of sales curves. Our Bayesian framework can generate initial curve forecasts before the product launch and update the forecasts dynamically as new sales data are collected. We validate the superior performance of our method through extensive numerical experiments using three real‐world data sets. Our forecasting framework reduces the forecasting error by 5.35%–30.76% over JD.com's current model and outperforms alternative models significantly. Furthermore, the estimated promotion effect function provides useful insights into how promotional activities interact with sales curves.

Additive models for symmetric positive-definite matrices and Lie groups

Summary We propose and investigate an additive regression model for symmetric positive-definite matrix-valued responses and multiple scalar predictors. The model exploits the Abelian group structure inherited from either of the log-Cholesky and log-Euclidean frameworks for symmetric positive-definite matrices and naturally extends to general Abelian Lie groups. The proposed additive model is shown to connect to an additive model on a tangent space. This connection not only entails an efficient algorithm to estimate the component functions, but also allows one to generalize the proposed additive model to general Riemannian manifolds. Optimal asymptotic convergence rates and normality of the estimated component functions are established, and numerical studies show that the proposed model enjoys good numerical performance, and is not subject to the curse of dimensionality when there are multiple predictors. The practical merits of the proposed model are demonstrated through an analysis of brain diffusion tensor imaging data.

Estimation for partial functional partially linear additive model

Estimation for partial functional partially linear additive model

SPF: A spatial and functional data analytic approach to cell imaging data.

The tumor microenvironment (TME), which characterizes the tumor and its surroundings, plays a critical role in understanding cancer development and progression. Recent advances in imaging techniques enable researchers to study spatial structure of the TME at a single-cell level. Investigating spatial patterns and interactions of cell subtypes within the TME provides useful insights into how cells with different biological purposes behave, which may consequentially impact a subject's clinical outcomes. We utilize a class of well-known spatial summary statistics, the K-function and its variants, to explore inter-cell dependence as a function of distances between cells. Using techniques from functional data analysis, we introduce an approach to model the association between these summary spatial functions and subject-level outcomes, while controlling for other clinical scalar predictors such as age and disease stage. In particular, we leverage the additive functional Cox regression model (AFCM) to study the nonlinear impact of spatial interaction between tumor and stromal cells on overall survival in patients with non-small cell lung cancer, using multiplex immunohistochemistry (mIHC) data. The applicability of our approach is further validated using a publicly available multiplexed ion beam imaging (MIBI) triple-negative breast cancer dataset.

A robust functional partial least squares for scalar‐on‐multiple‐function regression

AbstractThe scalar‐on‐function regression model has become a popular analysis tool to explore the relationship between a scalar response and multiple functional predictors. Most of the existing approaches to estimate this model are based on the least‐squares estimator, which can be seriously affected by outliers in empirical datasets. When outliers are present in the data, it is known that the least‐squares‐based estimates may not be reliable. This paper proposes a robust functional partial least squares method, allowing a robust estimate of the regression coefficients in a scalar‐on‐multiple‐function regression model. In our method, the functional partial least squares components are computed via the partial robust M‐regression. The predictive performance of the proposed method is evaluated using several Monte Carlo experiments and two chemometric datasets: glucose concentration spectrometric data and sugar process data. The results produced by the proposed method are compared favorably with some of the classical functional or multivariate partial least squares and functional principal component analysis methods.

Modeling Functional Time Series and Mixed-Type Predictors With Partially Functional Autoregressions

In many business and economics studies, researchers have sought to measure the dynamic dependence of curves with high-dimensional mixed-type predictors. We propose a partially functional autoregressive model (pFAR) where the serial dependence of curves is controlled by coefficient operators that are defined on a two-dimensional surface, and the individual and group effects of mixed-type predictors are estimated with a two-layer regularization. We develop an efficient estimation with the proven asymptotic properties of consistency and sparsity. We show how to choose the sieve and tuning parameters in regularization based on a forward-looking criterion. In addition to the asymptotic properties, numerical validation suggests that the dependence structure is accurately detected. The implementation of the pFAR within a real-world analysis of dependence in German daily natural gas flow curves, with seven lagged curves and 85 scalar predictors, produces superior forecast accuracy and an insightful understanding of the dynamics of natural gas supply and demand for the municipal, industry, and border nodes, respectively.