Functional Data Analysis Research Articles

Research Trends on Functional Data Analysis Using Scopus Database: A Bibliometric Analysis

Functional data analysis (FDA) has received significant attention from researchers due to its flexibility and diverse applications in various fields. FDA provides a comprehensive framework for analysing and extracting information from complex and high-dimensional datasets, enabling researchers to obtain insights into the underlying processes, improve modelling, and make accurate predictions. Therefore, understanding the FDA topic and its features and tools, as well as identifying the collaborative networks, are crucial for the development of its research areas. The objective of the present bibliometric study is to analyse the global research trend in FDA areas based on publication outputs, authorships, co-authorships, affiliated countries, and the co-occurrence of author keywords, which will enable researchers to assess the existing knowledge environment, future trends, potential research gaps, and collaboration opportunities. The publications from the year 1989 to 2021 were retrieved from the Scopus database, resulting in 1712 articles in journals after screening. Results have shown that articles published in the Journal of the American Statistical Association received the highest citations. Nearly 43% of the published articles were contributed by the leading authors from the USA, followed by China (11.5%) and Spain (9.4%). According to the QS World University Ranking 2021, eight of the top 20 productive institutions were ranked among the top 100 best universities. The findings indicated that researchers had intensively developed and applied FDA tools and features, such as smoothing, principal component analysis, regression, and clustering, in various domains. In addition, the expansion of FDA tools could be seen based on the recent progress in author keywords. New keywords, including function-on-function regression, function-on-scalar regression, scalar-on-function regression, outlier detection, structural health monitoring, and COVID-19, have arisen recently. Due to public concern about emerging diseases, future FDA work is expected to rise, particularly in the health sciences and biomedical fields.

Reproducibility of continuous glucose monitoring results under real-life conditions in an adult population: a functional data analysis

Continuous glucose monitoring systems (CGM) are a very useful tool to understand the behaviour of glucose in different situations and populations. Despite the widespread use of CGM systems in both clinical practice and research, our understanding of the reproducibility of CGM data remains limited. The present work examines the reproducibility of the results provided by a CGM system in a random sample of a free-living adult population, from a functional data analysis approach. Functional intraclass correlation coefficients (ICCs) and their 95% confidence intervals (CI) were calculated to assess the reproducibility of CGM results in 581 individuals. 62% were females 581 participants (62% women) mean age 48 years (range 18–87) were included, 12% had previously been diagnosed with diabetes. The inter-day reproducibility of the CGM results was greater for subjects with diabetes (ICC 0.46 [CI 0.39–0.55]) than for normoglycaemic subjects (ICC 0.30 [CI 0.27–0.33]); the value for prediabetic subjects was intermediate (ICC 0.37 [CI 0.31–0.42]). For normoglycaemic subjects, inter-day reproducibility was poorer among the younger (ICC 0.26 [CI 0.21–0.30]) than the older subjects (ICC 0.39 [CI 0.32–0.45]). Inter-day reproducibility was poorest among normoglycaemic subjects, especially younger normoglycaemic subjects, suggesting the need to monitor some patient groups more often than others.

Penalized model-based clustering of complex functional data

High dimensional data, large-scale data, imaging and manifold data are all fostering new frontiers of statistics. These type of data are commonly considered in Functional Data Analysis where they are viewed as infinite-dimensional random vectors in a functional space. The rapid development of new technologies has generated a flow of complex data that have led to the development of new modeling strategies by scientists. In this paper, we basically deal with the problem of clustering a set of complex functional data into homogeneous groups. Working in a mixture model-based framework, we develop a flexible clustering technique achieving dimensionality reduction schemes through an L_1 penalization. The proposed procedure results in an integrated modelling approach where shrinkage techniques are applied to enable sparse solutions in both the means and the covariance matrices of the mixture components, while preserving the underlying clustering structure. This leads to an entirely data-driven methodology suitable for simultaneous dimensionality reduction and clustering. The proposed methodology is evaluated through a Monte Carlo simulation study and an empirical analysis of real-world datasets showing different degrees of complexity.

An Outlier Detection Study of Ozone in Kolkata India by the Classical Statistics, Statistical Process Control and Functional Data Analysis

Air pollution is prevalent throughout the entire world due to the release of various gases such as NOx, PM, SO2, tropospheric ozone (O3), etc. Ground-stage ozone is the predominant issue in smog and is the product of the interplay between sunlight and emissions. The destructive impact on the health of the populace might also still occur in cities with noticeably clean air and where ozone levels hardly ever exceed safe limits. Therefore, the findings of small variations in air quality and the technique of regulating air contamination are thought-provoking. The study employs various techniques to effectively observe and assess strategies for detecting and eliminating outliers in ozone emissions from pollution episodes. This technique helps to describe the sources and exceedance values and enhance the value of monitoring the data. In this study, the data have some missing observations. The method of imputation, the classical statistical technique, the statistical process control (SPC) technique, functional data analysis (FDA), and functional process control help to fill in the data and detect outliers, trend deviations, and changes in ozone concentration at ground level. A comparison study is carried out using these three techniques: classical analysis, SPC, and FDA, and the results show how the statistical process control and functional data methods performed better than the classical technique for the detection of outliers and also in what way this methodology can enable an additional, comprehensive method of defining air pollution control measures and water pollution control measures.

Functional data analysis approach in population studies: an application to the gender gap in life expectancy

AbstractThis work analyses the contribution of ages and causes of death to gender gap in life expectancy in 20 European and non-European countries between 1959 and 2015, using Functional Data Analysis. Data were retrieved from the WHO Mortality Database and from the Human Mortality Database. We propose a Functional Principal Component Analysis of the age profiles of cause-specific contributions, to identify the main components of the distribution of the age-specific contributions according to causes of death, and to summarize them with few components. Our findings show that the narrowing gender gap in life expectancy was mainly driven by decreasing differences in cardiovascular diseases. Additionally, the study reveals that the age cause contributions act almost entirely on only two dimensions: level (extent of the cause-specific contribution to the overall mortality gender gap) and age pattern (location of the curves across ages). Notably, in the last period, it is not the "quantum" of the cause-specific contributions that matters, but the "timing", i.e. location across the age spectrum. Moreover, our results show that in the most recent period the gender gap in life expectancy is affected by composition of the causes of death more than it was in previous periods. We emphasise that Functional Data Analysis could prove useful to deepen our understanding of complex demographic phenomena.

Shape-based functional data analysis

Functional data analysis (FDA) is a fast-growing area of research and development in statistics. While most FDA literature imposes the classical L2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {L}^2$$\\end{document} Hilbert structure on function spaces, there is an emergent need for a different, shape-based approach for analyzing functional data. This paper reviews and develops fundamental geometrical concepts that help connect traditionally diverse fields of shape and functional analyses. It showcases that focusing on shapes is often more appropriate when structural features (number of peaks and valleys and their heights) carry salient information in data. It recaps recent mathematical representations and associated procedures for comparing, summarizing, and testing the shapes of functions. Specifically, it discusses three tasks: shape fitting, shape fPCA, and shape regression models. The latter refers to the models that separate the shapes of functions from their phases and use them individually in regression analysis. The ensuing results provide better interpretations and tend to preserve geometric structures. The paper also discusses an extension where the functions are not real-valued but manifold-valued. The article presents several examples of this shape-centric functional data analysis using simulated and real data.

Estimation of a clustering model for non Gaussian functional data

Model-based clustering analysis of functional data often has normality assumption. This article considers clustering non Gaussian functional data. We propose a novel non Gaussian functional mixed-effects model without the prior information and clustering number. We use transformation functions to accommodate non Gaussian functional data. Smoothing spline ANOVA and cubic B-spline approximate unknown fixed effects and random effects, respectively. A penalized likelihood is used to estimate unknown parameters, and the consistency and asymptotic normality is provided after that. We take simulations for different measurement error distribution assumptions and adopt the air quality of Italian city data. Both simulation and actual data analysis show that the proposed method performs well and has a better clustering effect.

Compositional functional regression andisotemporal substitution analysis: Methods and application in time-use epidemiology.

The distribution of time that people spend in physical activity of various intensities has important health implications. Physical activity (commonly categorised by the intensity into light, moderate and vigorous physical activity), sedentary behaviour and sleep, should not be analysed separately, because they are parts of a time-use composition with a natural constraint of h/day. To find out how are relative reallocations of time between physical activity of various intensities associated with health, herewith we describe compositional scalar-on-function regression and a newly developed compositional functional isotemporal substitution analysis. Physical activity intensity data can be considered as probability density functions, which better reflects the continuous character of their measurement using accelerometers. These probability density functions are characterised by specific properties, such as scale invariance and relative scale, and they are geometrically represented using Bayes spaces with the Hilbert space structure. This makes possible to process them using standard methods of functional data analysis in the space, via centred logratio (clr) transformation. The scalar-on-function regression with clr transformation of the explanatory probability density functions and compositional functional isotemporal substitution analysis were applied to a dataset from a cross-sectional study on adiposity conducted among school-aged children in the Czech Republic. Theoretical reallocations of time to physical activity of higher intensities were found to be associated with larger and more progressive expected decreases in adiposity. We obtained a detailed insight into the dose-response relationship between physical activity intensity and adiposity, which was enabled by using the compositional functional approach.

A data-driven soft-sensing approach using probabilistic latent variable model with functional data framework

Functional principal component analysis (FPCA) and functional partial least squares (FPLS) are two mainstream functional data analysis (FDA) methods, which have been commonly used to extract deep information hidden in the original data space. However, the process data always contain random noise, which affects the performance of FDA models. To overcome this issue, two functional probabilistic latent variable models (FPLVMs), including functional probabilistic principal component analysis (FPPCA) and functional probabilistic partial least squares (FPPLS) are proposed in this work. First, the process data are converted into functional data using the FDA. Subsequently, a log-likelihood function considering the noise factor and functional latent variables is designed. Finally, the regression model parameters are estimated using an expectation–maximisation algorithm. In contrast to FPPCA, FPPLS decomposes the process data and the key variable with constrained latent variables, which is similar to the partial least squares (PLS) and the principal component analysis (PCA). Moreover, the degeneration mechanism from FPLVMs into probabilistic latent variable models and latent variable models is discussed. An adaptive strategy with functional covariance is used to satisfy the online predictive capabilities of the model. Finally, the proposed approach is validated using a numerical case, the Tennessee Eastman process and an industrial o-xylene distillation column for evaluation.

MAGIC (maternal glucose in pregnancy) understanding the glycemic profile of pregnancy, intensive CGM glucose profiling and its relationship to fetal growth: an observational study protocol

BackgroundContinuous glucose monitoring (CGM) provides the most objective method of assessing glucose in daily life. Although there have been small, short-term physiologic studies of glucose metabolism in ‘healthy’ pregnant women a comprehensive, longitudinal description of changes in glucose over the course of pregnancy and how glucose dysregulation earlier in pregnancy relates to traditional third trimester screening for gestational diabetes, fetal growth and pregnancy outcomes is lacking. This study aims to characterise longitudinal changes in glycemia across gestation using CGM, in order to understand the evolution of dysglycemia and its relationship to fetal growth.Method/designA multi-centre, prospective, observational, cohort study of 500 healthy pregnant women, recruited in the first trimester of pregnancy. Masked CGM will be performed for a 14-day period on five occasions across pregnancy at ~ 10–12, 18–20, 26–28, 34–36 weeks gestation and postnatally. Routinely collected anthropometric and sociodemographic information will be recorded at each visit including: weight, height, blood pressure, current medication. Age, parity, ethnicity, smoking will be recorded. Blood samples will be taken at each visit for HbA1c and a sample stored. Details on fetal growth from ultrasound scans and the OGTT results will be recorded. Maternal and neonatal outcomes will be collected. CGM glucose profiling is the exposure of interest, and will be performed using standard summary statistics, functional data analysis and glucotyping. The primary maternal outcome is clinical diagnosis of GDM. The primary neonatal outcome is large for gestational age (LGA) (> 90th centile defined by customised birthweight centile). The relationship of glucose to key secondary maternal and neonatal outcomes will be explored.DiscussionThis study will ascertain the relationship of maternal dysglycemia to fetal growth and outcomes. It will explore whether CGM glucose profiling can detect GDM before the OGTT; or indeed whether CGM glucose profiling may be more useful than the OGTT at detecting LGA and other perinatal outcomes.Trial registrationISRCTN 15,706,303 https://www.isrctn.com/ISRCTN15706303 Registration date: 13th March 2023.

Integrated omics approach for the identification of HDL structure-function relationships in PCSK9-related familial hypercholesterolemia

The role of proprotein convertase subtilisin/kexin type 9 (PCSK9) in dyslipidemia may go beyond its immediate effects on low-density lipoprotein receptor (LDL-R) activity. This study aimed to assess PCSK9-derived alterations of high-density lipoprotein (HDL) physiology, which bear a potential to contribute to cardiovascular risk profile. HDL was isolated from 33 patients with familial autosomal dominant hypercholesterolemia (FH), including those carrying PCSK9 gain-of-function (GOF) genetic variants (FH-PCSK9, n=11), together with two groups of dyslipidemic patients employed as controls and carrying genetic variants in the LDL-R not treated (ntFH-LDLR, n=11) and treated (tFH-LDLR, n=11) with statins, and 11 normolipidemic controls. Biological evaluations paralleled by proteomic, lipidomic and glycomic analyses were applied to characterize functional and compositional properties of HDL. Multiple deficiencies in the HDL function were identified in the FH-PCSK9 group relative to dyslipidemic FH-LDLR patients and normolipidemic controls, which involved reduced antioxidative, antiapoptotic, anti-thrombotic and anti-inflammatory activities. By contrast, cellular cholesterol efflux capacity of HDL was unchanged. In addition, multiple alterations of the proteomic, lipidomic and glycomic composition of HDL were found in the FH-PCSK9 group. Remarkably, HDLs from FH-PCSK9 patients were systematically enriched in several lysophospholipids as well as in A2G2S2 (GP13) glycan and apolipoprotein A-IV. Based on network analysis of functional and compositional data, a novel mosaic structure-function model of HDL biology involving FH was developed. Several metrics of anti-atherogenic HDL functionality are altered in FH-PCSK9 patients paralleled by distinct compositional alterations. These data provide a first-ever overview of the impact of GOF PCSK9 genetic variants on structure-function relationships in HDL.

Optimisation of LSTM neural networks with NSGA-II and FDA for PV installations characterisation

Solar PV generation is renewable energy source that in the last years has contributed to reduce the use of fossil fuels. Controlling the efficiency of photovoltaic (PV) installations is essential in order for its use to spread. Deep learning (DL) models have demonstrated their efficiency in this context. The purpose of this work is to show a methodology to optimised deep learning models and show a specific application of the optimised model to characterise PV installations. The estimations yielded by this model are obtained without the need for real PV data to the training process; synthetic data are used. The built multivariate neural network is optimised through the use of a multiobjective genetic algorithm and the use of a feature engineering tool based on functional data analysis (FDA) clustering. This method was applied on synthetic data and on a PV installation located in north-western Spain, where the number of parallel modules, the azimuth and the slope in the installation are estimated. The results show that the model optimised using the non-dominant sorting genetic algorithm (NSGA-II) and the customised dataset with FDA, achieve lower errors or in the same range by reducing the complexity of the model or the complexity of the dataset used. Specifically, the bests models generates estimations with an average error between 6% and 18% on synthetic data and between 6% and 12% on real data.

Kolmogorov Entropy for Convergence Rate in Incomplete Functional Time Series: Application to Percentile and Cumulative Estimation in High Dimensional Data.

The convergence rate for free-distribution functional data analyses is challenging. It requires some advanced pure mathematics functional analysis tools. This paper aims to bring several contributions to the existing functional data analysis literature. First, we prove in this work that Kolmogorov entropy is a fundamental tool in characterizing the convergence rate of the local linear estimation. Precisely, we use this tool to derive the uniform convergence rate of the local linear estimation of the conditional cumulative distribution function and the local linear estimation conditional quantile function. Second, a central limit theorem for the proposed estimators is established. These results are proved under general assumptions, allowing for the incomplete functional time series case to be covered. Specifically, we model the correlation using the ergodic assumption and assume that the response variable is collected with missing at random. Finally, we conduct Monte Carlo simulations to assess the finite sample performance of the proposed estimators.

Rethinking Theta/Beta Ratio in ADHD through Functional Data Analysis.

ADHD is a neurodevelopmental disorder largely diffused among children and adolescents. The current method of diagnosis is based on agreed clinical literature such as DSM-5, by identifying and evaluating signs of hyperactivity and inattention. Multiple reviews have assessed that EEG is not sufficiently reliable for the diagnosis of ADHD. Theta-Beta Ratio is now the sole EEG parameter considered for analysis, although it is not robust enough to be utilized as a confirmatory technique for diagnosis. In this setting, new objective approaches for reliably classifying neurotypical and ADHD subjects are required. As a result, we suggest a new methodology based on Functional Data Analysis, a statistical class of methods for dealing with curves and functions. The initial stage in our method is to separate frequency bands from the EEG signal using a wavelet decomposition. We next compute the Power Spectral Densities of each of these bands and represent them as mathematical functions via spline interpolation. Finally, the relevance of the collected features is assessed using the Permutation ANOVA test. Using this method, we can detect different patterns in the PSDs of the groups and identify statistically significant features, confirming prior findings in the literature. We validate the features using classification techniques such as Bagging trees, Random Forest, and AdaBoost. The latter reaches the highest accuracy score of 76.65%, confirming the relevance of the extracted features.

Bayesian functional data analysis over dependent regions and its application for identification of differentially methylated regions.

We consider a Bayesian functional data analysis for observations measured as extremely long sequences. Splitting the sequence into several small windows with manageable lengths, the windows may not be independent especially when they are neighboring each other. We propose to utilize Bayesian smoothing splines to estimate individual functional patterns within each window and to establish transition models for parameters involved in each window to address the dependence structure between windows. The functional difference of groups of individuals at each window can be evaluated by the Bayes factor based on Markov Chain Monte Carlo samples in the analysis. In this paper, we examine the proposed method through simulation studies and apply it to identify differentially methylated genetic regions in TCGA lung adenocarcinomadata.

Predicting rainfall and irrigation requirements of corn in Ecuador

This work is a case study whose objective is prediction of irrigation needs of corn crops in different regions of Ecuador; being this a fundamental basic food for the country's economy, as in the remaining countries of the Andean area. The proposed methodology seeks to help improving the quality of corn crop. Specifically, we propose the application of regression models, within the framework of Functional Data Analysis (FDA), to predict the amount of rainfall (scalar response variable) in the places with the highest production of corn in Ecuador, as a function of functional covariates such as temperature and wind speed. From the estimation of the amount of rainfall, effective precipitation is calculated. This is the fraction of water used by the crops, from which the value of real evapotranspiration or ETc is obtained and, more importantly, the irrigation requirements at each stage of the corn crop, for its adequate physiological development. Application of regression models based on functional basis, Functional Principal Components (FPC) or Functional Partial Least Squares (FPLS) for scalar response variable, allows us to use the information of variables such as wind speed and temperature (of functional nature) in a better way than using multivariate models, for predicting the amount of rainfall, obtaining, as a result, very explicative models, defined by a high goodness of fit (R2=0.97, with 6 significant parameters and an error of 0.14) and practical utility. The model has been also applied to North Peru regions, obtaining rainfall prediction errors between 9% and 22%. Thus, the geographical limitations of the model could be the Andean regions with similar climate. In addition, this study proposes the application of FDA exploratory analysis and FDA outlier detection techniques as a common and useful practice in the specific domain of rainfall prediction studies, prior to applying the regression models.

Detection of outlying patterns from sparse and irregularly sampled electronic health records data

Within the intensive care unit (ICU), vital signs such as arterial blood pressure (ABP) collected from electronic health records (EHRs) are typically recorded at different and uneven sampling frequencies and are often infrequently measured due to the nature of the medical treatment. Furthermore, from a temporal trajectory perspective, EHR data are likely to be corrupted by outlying patterns that deviate from normal samples in terms of the curves’ magnitude and shape. In this work, we propose a two-stage outlier detection approach for sparse and irregularly sampled (SiS) temporal data using functional data analysis (FDA) tools. In the first stage, an outlier identification measure is defined by a max–min statistic and a clean subset that contains nonoutliers. In the second stage, a multiple hypothesis testing problem is formulated based on the asymptotic distribution of the proposed measure. The simulation-based framework shows that the proposed method is robust to different types of shape and magnitude outliers. The detection results are more accurate than the widely used functional depth methods, especially in extremely sparse settings where the proportion of the observed data points over the entire time series is approximately 10%. Extensive experiments are also conducted on the real-world MIMIC-II dataset, which demonstrate that the method effectively detects clinically meaningful outlying patterns.

Conformal prediction bands for two-dimensional functional time series

Time evolving surfaces can be modeled as two-dimensional Functional time series, exploiting the tools of Functional data analysis. Leveraging this approach, a forecasting framework for such complex data is developed. The main focus revolves around Conformal Prediction, a versatile nonparametric paradigm used to quantify uncertainty in prediction problems. Building upon recent variations of Conformal Prediction for Functional time series, a probabilistic forecasting scheme for two-dimensional functional time series is presented, while providing an extension of Functional Autoregressive Processes of order one to this setting. Estimation techniques for the latter process are introduced, and their performance are compared in terms of the resulting prediction regions. Finally, the proposed forecasting procedure and the uncertainty quantification technique are applied to a real dataset, collecting daily observations of Sea Level Anomalies of the Black Sea.

The scalar-on-function modal regression for functional time series data

This paper develops a new nonparametric estimator of the scalar-on function modal regression that is used to analyse the co-variability between a functional regressor and a scalar output variable. The new estimator inherits the smoothness of the kernel method and the robustness of the quantile regression. We assume that the functional observations are structured as a strong mixing functional time series data and we establish the almost complete consistency (with rate) of the constructed estimator. A discussion highlighting the impact of this new estimator in nonparametric functional data analysis is also given. The usefulness of this new estimator is shown using an artificial data example.

Identifying multivariate disease trajectories and potential phenotypes of early knee osteoarthritis in the CHECK cohort.

To gain better understanding of osteoarthritis (OA) heterogeneity and its predictors for distinguishing OA phenotypes. This could provide the opportunity to tailor prevention and treatment strategies and thus improve care. Ten year follow-up data from CHECK (1002 early-OA subjects with first general practitioner visit for complaints ≤6 months before inclusion) was used. Data were collected on WOMAC (pain, function, stiffness), quantitative radiographic tibiofemoral (TF) OA characteristics, and semi-quantitative radiographic patellofemoral (PF) OA characteristics. Using functional data analysis, distinctive sets of trajectories were identified for WOMAC, TF and PF characteristics, based on model fit and clinical interpretation. The probabilities of knee membership to each trajectory were used in hierarchical cluster analyses to derive knee OA phenotypes. The number and composition of potential phenotypes was selected again based on model fit (silhouette score) and clinical interpretation. Five trajectories representing different constant levels or changing WOMAC scores were identified. For TF and PF OA, eight and six trajectories respectively were identified based on (changes in) joint space narrowing, osteophytes and sclerosis. Combining the probabilities of knees belonging to these different trajectories resulted in six clusters ('phenotypes') of knees with different degrees of functional (WOMAC) and radiographic (PF) parameters; TF parameters were found not to significantly contribute to clustering. Including baseline characteristics as well resulted in eight clusters of knees, dominated by sex, menopausal status and WOMAC scores, with only limited contribution of PF features. Several stable and progressive trajectories of OA symptoms and radiographic features were identified, resulting in phenotypes with relatively independent symptomatic and radiographic features. Sex and menopausal status may be especially important when phenotyping knee OA patients, while radiographic features contributed less. Possible phenotypes were identified that, after validation, could aid personalized treatments and patients selection.