Local Linear Approximation Research Articles

Modified Local Linear Estimators in Partially Linear Additive Models with Right-Censored Data Based on Different Censorship Solution Techniques.

This paper introduces a modified local linear estimator (LLR) for partially linear additive models (PLAM) when the response variable is subject to random right-censoring. In the case of modeling right-censored data, PLAM offers a more flexible and realistic approach to the estimation procedure by involving multiple parametric and nonparametric components. This differs from the widely used partially linear models that feature a univariate nonparametric function. The LLR method is employed to estimate unknown smooth functions using a modified backfitting algorithm, delivering a non-iterative solution for the right-censored PLAM. To address the censorship issue, three approaches are employed: synthetic data transformation (ST), Kaplan-Meier weights (KMW), and the kNN imputation technique (kNNI). Asymptotic properties of the modified backfitting estimators are detailed for both ST and KMW solutions. The advantages and disadvantages of these methods are discussed both theoretically and practically. Comprehensive simulation studies and real-world data examples are conducted to assess the performance of the introduced estimators. The results indicate that LLR performs well with both KMW and kNNI in the majority of scenarios, along with a real data example.

A robust method for state of charge estimation of lithium-ion batteries using adaptive nonlinear neural observer

To ensure the safety and functionality, it is crucial to evaluate the state of battery system in electric vehicles. Describing battery dynamic characteristic with a combined state space model, this paper presents a novel adaptive nonlinear neural observer based approach for state of charge estimation, which is composed of linear discriminant function, nonlinear neural proportional-integral observer and extended Kalman filter. It incorporates the local linear approximation capability of extended Kalman filter with the nonlinear mapping, self-learning and self-adjusting capabilities of neural proportional-integral observer, which is used to compensate the deviation resulted from the underestimated initial state, process noise and measurement noise. Taking the samples collected from lithium-ion battery test system for example, simulation is carried out to verify the proposed method. Results show that it is capable of evaluating the state of charge of cell with a rapid convergence and an error <2 % while remaining unaffected by the unknown initial cell states and the underestimation of the process noise and the measurement noise.

Kolmogorov Entropy for Convergence Rate in Incomplete Functional Time Series: Application to Percentile and Cumulative Estimation in High Dimensional Data.

The convergence rate for free-distribution functional data analyses is challenging. It requires some advanced pure mathematics functional analysis tools. This paper aims to bring several contributions to the existing functional data analysis literature. First, we prove in this work that Kolmogorov entropy is a fundamental tool in characterizing the convergence rate of the local linear estimation. Precisely, we use this tool to derive the uniform convergence rate of the local linear estimation of the conditional cumulative distribution function and the local linear estimation conditional quantile function. Second, a central limit theorem for the proposed estimators is established. These results are proved under general assumptions, allowing for the incomplete functional time series case to be covered. Specifically, we model the correlation using the ergodic assumption and assume that the response variable is collected with missing at random. Finally, we conduct Monte Carlo simulations to assess the finite sample performance of the proposed estimators.

Nonparametric Conditional Risk Mapping Under Heteroscedasticity

A nonparametric procedure to estimate the conditional probability that a nonstationary geostatistical process exceeds a certain threshold value is proposed. The method consists of a bootstrap algorithm that combines conditional simulation techniques with nonparametric estimations of the trend and the variability. The nonparametric local linear estimator, considering a bandwidth matrix selected by a method that takes the spatial dependence into account, is used to estimate the trend. The variability is modeled estimating the conditional variance and the variogram from corrected residuals to avoid the biasses. The proposed method allows to obtain estimates of the conditional exceedance risk in non-observed spatial locations. The performance of the approach is analyzed by simulation and illustrated with the application to a real data set of precipitations in the USA.Supplementary materials accompanying this paper appear on-line.

Is timing everything? Assessing the evidence on whether energy/electricity demand elasticities are time-varying

In this review we assess both the reasons/hypotheses for why GDP and price elasticities of energy/electricity demand might change over time, and the evidence from papers that have addressed that issue. We consider papers that have used parametric methods (e.g., endogenous or exogenous breaks, rolling window regressions) and semi- or non-parametric approaches (e.g., structural time series models, local linear estimators). The most convincing reasons for GDP and price elasticities varying over time were: (i) results from reactions to the energy price shocks of the 1970s and early-1980s (that would have affected OECD countries directly and non-OECD countries, possibly, via leapfrogging); (ii) saturation of the demand for energy services (typically at high income levels); and (iii) country-specific events such as policy changes and/or economic crises/recoveries/shocks/transitions. Despite those reasons, the weight of evidence points toward income/GDP and price elasticities of energy/electricity demand that are fairly stable over time. In addition to changes caused by reactions to the energy crises and (other) country specific events, the main exception to effectively constant elasticities is the GDP elasticity of economy-wide electricity demand in OECD countries. This elasticity may have declined substantially over time, perhaps, because the share of energy services provided by electricity has saturated. Lastly, it is recommended that modelers consider a combination of (parametric and nonparametric) approaches and search for consensus among the results.

Variable selection for joint models of multivariate skew-normal longitudinal and survival data.

Many joint models of multivariate skew-normal longitudinal and survival data have been presented to accommodate for the non-normality of longitudinal outcomes in recent years. But existing work did not consider variable selection. This article investigates simultaneous parameter estimation and variable selection in joint modeling of longitudinal and survival data. The penalized splines technique is used to estimate unknown log baseline hazard function, the rectangle integral method is adopted to approximate conditional survival function. Monte Carlo expectation-maximization algorithm is developed to estimate model parameters. Based on local linear approximations to conditional expectation of likelihood function and penalty function, a one-step sparse estimation procedure is proposed to circumvent the computationally challenge in optimizing the penalized conditional expectation of likelihood function, which is utilized to select significant covariates and trajectory functions, and identify the departure from normality of longitudinal data. The conditional expectation of likelihood function-based Bayesian information criterion is developed to select the optimal tuning parameter. Simulation studies and a real example from the clinical trial are used to illustrate the proposed methodologies.

Retire: Robust expectile regression in high dimensions

High-dimensional data can often display heterogeneity due to heteroscedastic variance or inhomogeneous covariate effects. Penalized quantile and expectile regression methods offer useful tools to detect heteroscedasticity in high-dimensional data. The former is computationally challenging due to the non-smooth nature of the check loss, and the latter is sensitive to heavy-tailed error distributions. In this paper, we propose and study (penalized) robust expectile regression (retire), with a focus on iteratively reweighted ℓ1-penalization which reduces the estimation bias from ℓ1-penalization and leads to oracle properties. Theoretically, we establish the statistical properties of the retire estimator under two regimes: (i) low-dimensional regime in which d≪n; (ii) high-dimensional regime in which s≪n≪d with s denoting the number of significant predictors. In the high-dimensional setting, we thoroughly analyze the statistical properties of the solution path of iteratively reweighted ℓ1-penalized retire estimation, adapted from the local linear approximation algorithm for folded-concave regularization. Under a mild minimum signal strength condition, we demonstrate that with as few as log(logd) iterations, the final iterate of our proposed approach achieves the oracle convergence rate. At each iteration, we solve the weighted ℓ1-penalized convex program using a semismooth Newton coordinate descent algorithm. Numerical studies demonstrate the promising performance of the proposed procedure in comparison to both non-robust and quantile regression based alternatives.

A method for detecting outliers in linear-circular non-parametric regression.

This study proposes a robust outlier detection method based on the circular median for non-parametric linear-circular regression in case the response variable includes outlier(s) and the residuals are Wrapped-Cauchy distributed. Nadaraya-Watson and local linear regression methods were employed to obtain non-parametric regression fits. The proposed method's performance was investigated by using a real dataset and a comprehensive simulation study with different sample sizes, contamination, and heterogeneity degrees. The method performs quite well in medium and higher contamination degrees, and its performance increases as the sample size and the homogeneity of data increase. In addition, when the response variable of linear-circular regression contains outliers, the Local Linear Estimation method fits the data set better than the Nadaraya Watson method.

Uniform convergence results for the local linear regression estimation of the conditional distribution

This paper examines the local linear regression (LLR) estimate of the conditional distribution function F(y|x). We derive three uniform convergence results: the uniform bias expansion, the uniform convergence rate, and the uniform asymptotic linear representation. The uniformity in the above results is with respect to both x and y and therefore has not previously been addressed in the literature on local polynomial regression. Such uniform convergence results are especially useful when the conditional distribution estimator is the first stage of a semiparametric estimator. We demonstrate the usefulness of these uniform results with two examples: the stochastic equicontinuity condition in y, and the estimation of the integrated conditional distribution function.

Extrapolation estimation for nonparametric regression with measurement error

AbstractFor the nonparametric regression models with covariates contaminated with the normal measurement errors, this paper proposes an extrapolation algorithm to estimate the regression functions. By applying the conditional expectation directly to the kernel‐weighted least squares of the deviations between the local linear approximation and the observed responses, the proposed algorithm successfully bypasses the simulation step in the classical simulation extrapolation, thus significantly reducing the computational time. It is noted that the proposed method also provides an exact form of the extrapolation function, although the extrapolation estimate generally cannot be obtained by simply setting the extrapolation variable to negative one in the fitted extrapolation function, if the bandwidth is less than the SD of the measurement error. Large sample properties of the proposed estimation procedure are discussed, as well as simulation studies and a real data example being conducted to illustrate its applications.

A mixed-effects stochastic model reveals clonal dominance in gene therapy safety studies

BackgroundMathematical models of haematopoiesis can provide insights on abnormal cell expansions (clonal dominance), and in turn can guide safety monitoring in gene therapy clinical applications. Clonal tracking is a recent high-throughput technology that can be used to quantify cells arising from a single haematopoietic stem cell ancestor after a gene therapy treatment. Thus, clonal tracking data can be used to calibrate the stochastic differential equations describing clonal population dynamics and hierarchical relationships in vivo.ResultsIn this work we propose a random-effects stochastic framework that allows to investigate the presence of events of clonal dominance from high-dimensional clonal tracking data. Our framework is based on the combination between stochastic reaction networks and mixed-effects generalized linear models. Starting from the Kramers–Moyal approximated Master equation, the dynamics of cells duplication, death and differentiation at clonal level, can be described by a local linear approximation. The parameters of this formulation, which are inferred using a maximum likelihood approach, are assumed to be shared across the clones and are not sufficient to describe situation in which clones exhibit heterogeneity in their fitness that can lead to clonal dominance. In order to overcome this limitation, we extend the base model by introducing random-effects for the clonal parameters. This extended formulation is calibrated to the clonal data using a tailor-made expectation-maximization algorithm. We also provide the companion package RestoreNet, publicly available for download at https://cran.r-project.org/package=RestoreNet.ConclusionsSimulation studies show that our proposed method outperforms the state-of-the-art. The application of our method in two in-vivo studies unveils the dynamics of clonal dominance. Our tool can provide statistical support to biologists in gene therapy safety analyses.

Method of Local Linear Approximation in the Theory of Nonlinear Impulsive Systems

Method of Local Linear Approximation in the Theory of Nonlinear Impulsive Systems

Exploring the dynamic and multifaceted effects of globalization on renewable energy sources through advanced nonparametric modeling techniques.

There is a growing global demand for clean energy, driven by concerns over global warming and the need to decrease greenhouse gas emissions. This research uses a nonparametric approach to analyze the correlation between industrialization and the use of clean energy sources in 16 countries between 1995 and 2020. We utilize a technique called the local linear dummy variable estimate to monitor the effects of globalization on the development of sustainable power over time. An unfavorable and economically important link was found between industrialization and the use of sustainable energy supplies from 2003 to 2012, as determined by an examination of nonparametric econometric techniques. Still, it started moving in the opposite direction, becoming significant and positive after 2014. In addition, we discover that globalization has various effects on the various indices of the use of renewable energy sources. The research also reveals that the effects of globalization on RES vary across different regions, with some regions experiencing more significant benefits than others. Overall, the study provides valuable insights into the complex relationship between globalization and RES and highlights the need for further research to inform policy decisions and promote sustainable development.

Bias reduction estimation for drift coefficient in diffusion models with jumps

In this paper, we reconstruct the local linear threshold estimator for the drift coefficient of a semimartingale with jumps. Under mild conditions, we provide the asymptotic normality of our estimator in the presence of finite activity jumps whether the underlying process is Harris recurrent or positive recurrent. Simulation studies for different models show that our estimator performs better than previous research in finite samples, which can correct the boundary bias automatically. Finally, the estimator is illustrated empirically through the stock index from Shanghai Stock Exchange in China under 15-minute high sampling frequency.

Online Smooth Backfitting for Generalized Additive Models

We propose an online smoothing backfitting method for generalized additive models coupled with local linear estimation. The idea can be extended to general nonlinear optimization problems. The strategy is to use an appropriate-order expansion to approximate the nonlinear equations and store the coefficients as sufficient statistics which can be updated in an online manner by the dynamic candidate bandwidth method. We investigate the statistical and algorithmic convergences of the proposed method. By defining the relative statistical efficiency and computational cost, we further establish a framework to characterize the tradeoff between estimation performance and computation performance. Simulations and real data examples are provided to illustrate the proposed method and algorithm. Supplementary materials for this article are available online.

UNIFORM CONVERGENCE OF NONPARAMETRIC CONDITIONAL HAZARD FUNCTION IN THE SINGLE FUNCTIONAL INDEX MODELING FOR DEPENDENT DATA

We study the nonparametric local linear estimation of the conditional hazard function of a scalar response variable given a functional explanatory variable, when the functional data are α-mixing dependency and we give the uniform almost complete convergence with rates of this function.

Characterization of variational quantum algorithms using free fermions

We study variational quantum algorithms from the perspective of free fermions. By deriving the explicit structure of the associated Lie algebras, we show that the Quantum Approximate Optimization Algorithm (QAOA) on a one-dimensional lattice – with and without decoupled angles – is able to prepare all fermionic Gaussian states respecting the symmetries of the circuit. Leveraging these results, we numerically study the interplay between these symmetries and the locality of the target state, and find that an absence of symmetries makes nonlocal states easier to prepare. An efficient classical simulation of Gaussian states, with system sizes up to80and deep circuits, is employed to study the behavior of the circuit when it is overparameterized. In this regime of optimization, we find that the number of iterations to converge to the solution scales linearly with system size. Moreover, we observe that the number of iterations to converge to the solution decreases exponentially with the depth of the circuit, until it saturates at a depth which is quadratic in system size. Finally, we conclude that the improvement in the optimization can be explained in terms of better local linear approximations provided by the gradients.

Local linear approximation with Laplacian smoothing penalty and application in biology.

Highly correlated structures appear in various fields, such as biology, biochemistry, and finance, with challenges of dimensionality and sparse estimation. To solve this problem, we propose an algorithm called local linear approximation with the Laplacian smoothing penalty (LLA-LSP). This method produces an accurate and smooth estimate that incorporates the correlation structure among predictors. We compare and discuss the difference between the Laplacian smoothing penalty and the total variance penalty. We prove that this algorithm converges to the oracle solution in a few iterations with a large probability. Numerical results show that the LLA-LSP has good performance in both variable selection and estimation. We apply the proposed algorithm to two biological datasets, a gene expression dataset and a chemical protein dataset, and provide meaningful insights.

Time-varying forecast combination for high-dimensional data

In this paper, we propose a new nonparametric estimator of time-varying forecast combination weights. When the number of individual forecasts is small, we study the asymptotic properties of the local linear estimator. When the number of candidate forecasts exceeds or diverges with the sample size, we consider penalized local linear estimation with the group SCAD penalty. We show that the estimator exhibits the oracle property and correctly selects relevant forecasts with probability approaching one. Simulations indicate that the proposed estimators outperform existing combination schemes when structural changes exist. An empirical application on inflation and unemployment forecasting highlights the merits of our approach relative to other popular methods in the literature.

High-Dimensional Censored Regression via the Penalized Tobit Likelihood

High-dimensional regression and regression with a left-censored response are each well-studied topics. In spite of this, few methods have been proposed which deal with both of these complications simultaneously. The Tobit model—long the standard method for censored regression in economics—has not been adapted for high-dimensional regression at all. To fill this gap and bring up-to-date techniques from high-dimensional statistics to the field of high-dimensional left-censored regression, we propose several penalized Tobit models. We develop a fast algorithm which combines quadratic majorization with coordinate descent to compute the penalized Tobit solution path. Theoretically, we analyze the Tobit lasso and Tobit with a folded concave penalty, bounding the estimation loss for the former and proving that a local linear approximation estimator for the latter possesses the strong oracle property. Through an extensive simulation study, we find that our penalized Tobit models provide more accurate predictions and parameter estimates than other methods on high-dimensional left-censored data. We use a penalized Tobit model to analyze high-dimensional left-censored HIV viral load data from the AIDS Clinical Trials Group and identify potential drug resistance mutations in the HIV genome. A supplementary file contains intermediate theoretical results and technical proofs.