Backfitting Algorithm Research Articles

Detection and estimation of weak pulse signal in chaotic background noise

As is well known, people has been suffering noise interference for a long time, and more and more researches show that a lot of weak signals such as pulse signal are embedded in the strong chaotic noise. The purpose of weak signal detection and recovery is to retrieve useful signal from strong noise. It is very difficult to detect and estimate the weak pulse signal which is mixed in the chaotic background interference. Therefore, the detection and recovery of weak signal are significant and have application value in signal processing area, especially for the weak pulse signal detection and recovery. By studying various methods of detecting and estimating the weak pulse signal in strong chaotic background noise, in this paper, we propose an efficient hybrid processing technique. First, based on the short-term predictability and sensitivity to the tiny disturbance, a new method is proposed, which can be used for detecting and estimating the weak pulse signals in chaotic background that the nonlinear mapping is unknown. We reconstruct a phase space according to Takens delay embedding theorem; then we establish the local linear autoregressive model to predict the short-term chaotic signal and obtain the fitting error, and judge whether there are weak pulse signals. Second, we establish a single-jump model for pulse signals, and combine the local linear autoregressive model with it to build a double local linear (DLL) model for estimating the weak pulse signal. DLL model contains two parameters, and the two parameters affect each other. We use the back-fitting algorithm to estimate model parameters and ultimately recover the weak pulse signals. Detecting and estimating the pulse signals in chaotic background turns into estimating the parameters of DLL model. The minimum fitting error criterion is used as the objective function to estimate the parameters of the DLL model. To make the estimation more exact, we can use the formula of mean square error. The new algorithm presented here in this paper does not need to know the prior knowledge of the chaotic background nor weak pulse signal, and this algorithm is also simple and effective. Finally, the simulation results show that the method is effective for detecting and estimating the weak pulse signals based on the chaotic background noise. Specifically, the weak pulse signal can be extracted well with low SNR and the minimum mean square error or the minimum normalized mean squared error is very low.

Geographically weighted regression with parameter-specific distance metrics

ABSTRACTGeographically weighted regression (GWR) is an important local technique to model spatially varying relationships. A single distance metric (Euclidean or non-Euclidean) is generally used to calibrate a standard GWR model. However, variations in spatial relationships within a GWR model might also vary in intensity with respect to location and direction. This assertion has led to extensions of the standard GWR model to mixed (or semiparametric) GWR and to flexible bandwidth GWR models. In this article, we present a strongly related extension in fitting a GWR model with parameter-specific distance metrics (PSDM GWR). As with mixed and flexible bandwidth GWR models, a back-fitting algorithm is used for the calibration of the PSDM GWR model. The value of this new GWR model is demonstrated using a London house price data set as a case study. The results indicate that the PSDM GWR model can clearly improve the model calibration in terms of both goodness of fit and prediction accuracy, in contrast to the model fits when only one metric is singly used. Moreover, the PSDM GWR model provides added value in understanding how a regression model’s relationships may vary at different spatial scales, according to the bandwidths and distance metrics selected. PSDM GWR deals with spatial heterogeneities in data relationships in a general way, although questions remain on its model diagnostics, distance metric specification, and computational efficiency, providing options for further research.

Semiparametric principal component poisson regression on clustered data

ABSTRACTIn modeling count data with multivariate predictors, we often encounter problems with clustering of observations and interdependency of predictors. We propose to use principal components of predictors to mitigate the multicollinearity problem and to abate information losses due to dimension reduction, a semiparametric link between the count dependent variable and the principal components is postulated. Clustering of observations is accounted into the model as a random component and the model is estimated via the backfitting algorithm. Simulation study illustrates the advantages of the proposed model over standard poisson regression in a wide range of scenarios.

Partially linear models and their applications to change point detection of chemical process data

ABSTRACTIn many chemical data sets, the amount of radiation absorbed (absorbance) is related to the concentration of the element in the sample by Lambert–Beer's law. However, this relation changes abruptly when the variable concentration reaches an unknown threshold level, the so-called change point. In the context of analytical chemistry, there are many methods that describe the relationship between absorbance and concentration, but none of them provide inferential procedures to detect change points. In this paper, we propose partially linear models with a change point separating the parametric and nonparametric components. The Schwarz information criterion is used to locate a change point. A back-fitting algorithm is presented to obtain parameter estimates and the penalized Fisher information matrix is obtained to calculate the standard errors of the parameter estimates. To examine the proposed method, we present a simulation study. Finally, we apply the method to data sets from the chemistry area. The partially linear models with a change point developed in this paper are useful supplements to other methods of absorbance–concentration analysis in chemical studies, for example, and in many other practical applications.

Extracting harmonic signal from a chaotic background with local linear model

In this paper, the problems of blind detection and estimation of harmonic signal in strong chaotic background are analyzed, and new methods by using local linear (LL) model are put forward. The LL model has been exhaustively researched and successfully applied for fitting and forecasting chaotic signal in many chaotic fields. We enlarge the modeling capacity substantially. Firstly, we can predict the short-term chaotic signal and obtain the fitting error based on the LL model. Then we detect the frequencies from the fitting error by periodogram, a property on the fitting error is proposed which has not been addressed before, and this property ensures that the detected frequencies are similar to that of harmonic signal. Secondly, we establish a two-layer LL model to estimate the determinate harmonic signal in strong chaotic background. To estimate this simply and effectively, we develop an efficient backfitting algorithm to select and optimize the parameters that are hard to be exhaustively searched for. In the method, based on sensitivity to initial value of chaos motion, the minimum fitting error criterion is used as the objective function to get the estimation of the parameters of the two-layer LL model. Simulation shows that the two-layer LL model and its estimation technique have appreciable flexibility to model the determinate harmonic signal in different chaotic backgrounds (Lorenz, Henon and Mackey–Glass (M–G) equations). Specifically, the harmonic signal can be extracted well with low SNR and the developed background algorithm satisfies the condition of convergence in repeated 3–5 times.

Forecasting SO2 pollution incidents by means of quantile curves based on additive models

More than 90% of the sulfur dioxide in the air comes from human sources. Because of the adverse health effects of high levels of sulfur dioxide, specific regulations have been adopted to manage and reduce the amount of sulfur dioxide produced. However, some SO2 emission incidents (i.e. emission exceeding the limits established by law) still occur. The aim of this paper is to predict time series of SO2 concentrations in order to estimate in advance high emission episodes and analyse the influence of previous series in the prediction. Previous studies aimed to forecast SO2 pollution incidents are based on estimating mean values. Instead, we propose the use of quantile curves obtained from additive models as they provide not only the mean but also the whole distribution of the pollution levels. A backfitting algorithm with local polynomial kernel smoothers was used to estimate the model, and critical values of the hypothesis test were obtained by means of bootstrapping. The performance of the method was evaluated using simulated data as well as real data drawn from an SO2 time series of a coal‐fired power station located in northern Spain. Copyright © 2016 John Wiley & Sons, Ltd.

An extension of log-symmetric regression models: R codes and applications

In the context of regression models, the data for which the response variable is continuous, strictly positive, and asymmetric are commonly employed in various fields of practice. The recent proposal of log-symmetric regression models can be used to describe this kind of data, because it allows that the median and the skewness (or the relative dispersion) of the response variable distribution are explicitly modelled. The purpose of this paper is to flexibilize the log-symmetric regression models by allowing that the median and the skewness (or the relative dispersion) may be described using an arbitrary number of non-parametric additive components, where the latter are approximated by P-splines and/or natural cubic splines with an arbitrary number of knots. An iterative process of parameter estimation based on Fisher scoring, expectation–maximization (EM) and backfitting algorithms is described. A computational implementation of the proposed methodology in the statistical computing environment is also presented. A real data set is analized to illustrate the flexibility of the addressed statistical and computational tools.

Application of the Vertex Exchange Method to estimate a semi-parametric mixture model for the MIC density of Escherichia coli isolates tested for susceptibility against ampicillin.

In the last decades, considerable attention has been paid to the collection of antimicrobial resistance data, with the aim of monitoring non-wild-type isolates. This monitoring is performed based on minimum inhibition concentration (MIC) values, which are collected through dilution experiments. We present a semi-parametric mixture model to estimate the entire MIC density on the continuous scale. The parametric first component is extended with a non-parametric second component and a new back-fitting algorithm, based on the Vertex Exchange Method, is proposed. Our data example shows how to estimate the MIC density for Escherichia coli tested for ampicillin and how to use this estimate for model-based classification. A simulation study was performed, showing the promising behavior of the new method, both in terms of density estimation as well as classification.

Partially linear models with first-order autoregressive symmetric errors

In this paper we discuss estimation and diagnostic procedures for partially linear models with first-order autoregressive [AR(1)] symmetric errors. The symmetric class includes all symmetric continuous distributions, particularly distributions with heavier and lighter tails than the normal ones, such as Student-t, power exponential and logistic, among others. Estimation is performed by maximum penalized likelihood and by using natural cubic splines. We derive the penalized score functions and the penalized Fisher information matrices for the parameters in the model. A reweighted iterative process based on the back-fitting algorithm is derived for the parameter estimation and the inference is based on the penalized Fisher information matrix. We discuss the effective degrees of freedom estimation and procedures for selecting the smoothing parameter. A small simulation study is performed for assessing the empirical distribution of the parameter estimates obtained from partially linear models with AR(1) errors. Residual analysis and derivation of conformal normal curvatures of local influence for some perturbation schemes are also given. Finally, a real data set is analyzed under partially linear models with AR(1) symmetric errors.

Automatic structure recovery for additive models.

We propose an automatic structure recovery method for additive models, based on a backfitting algorithm coupled with local polynomial smoothing, in conjunction with a new kernel-based variable selection strategy. Our method produces estimates of the set of noise predictors, the sets of predictors that contribute polynomially at different degrees up to a specified degree M, and the set of predictors that contribute beyond polynomially of degree M. We prove consistency of the proposed method, and describe an extension to partially linear models. Finite-sample performance of the method is illustrated via Monte Carlo studies and a real-data example.

Asymptotics for nonparametric and semiparametric fixed effects panel models

In this paper, we investigate the problem of estimating nonparametric and semiparametric panel data models with fixed effects. We focus on establishing the asymptotic results for estimators using smooth backfitting methods. We consider two estimators for the smooth unknown function in nonparametric panel regressions. One is a local linear estimator constructed similar as the one in Mammen et al. (2009) which was proposed for the additive nonparametric panel model. The other is the local profile likelihood based estimator proposed by Henderson et al. (2008) (HCL hereafter). We build the link and compare the difference between these two estimators which are constructed under different sets of conditions. We put both of these estimators in the smooth backfitting algorithm framework discussed in Mammen et al. (1999). Following the recently developed theories on backfitting kernel estimates in Mammen et al. (2009), we establish the asymptotic normality of these estimators, and hence verify the conjectures made by HCL and complement their paper. Further, we consider a partially linear fixed effects panel data model with the nonparametric component estimated using the methods discussed in the first part of the paper. We give the asymptotic result for the estimators of finite dimensional parameters, which shows that the first-step plug-in estimators will not affect the asymptotic variance in the second-step estimation.

Iterative isotonic regression

This article introduces a new nonparametric method for estimating a univariate regression function of bounded variation. The method exploits the Jordan decomposition which states that a function of bounded variation can be decomposed as the sum of a non-decreasing function and a non-increasing function. This suggests combining the backfitting algorithm for estimating additive functions with isotonic regression for estimating monotone functions. The resulting iterative algorithm is called Iterative Isotonic Regression (I.I.R.). The main technical result in this paper is the consistency of the proposed estimator when the number of iterations $k_n$ grows appropriately with the sample size $n$. The proof requires two auxiliary results that are of interest in and by themselves: firstly, we generalize the well-known consistency property of isotonic regression to the framework of a non-monotone regression function, and secondly, we relate the backfitting algorithm to Von Neumann's algorithm in convex analysis.

Differencing techniques in semi-parametric panel data varying coefficient models with fixed effects: a Monte Carlo study

Recently, some new techniques have been proposed for the estimation of semi-parametric fixed effects varying coefficient panel data models. These new techniques fall within the class of the so-called differencing estimators. In particular, we consider first-differences and within local linear regression estimators. Analyzing their asymptotic properties it turns out that, keeping the same order of magnitude for the bias term, these estimators exhibit different asymptotic bounds for the variance. In both cases, the consequences are suboptimal non-parametric rates of convergence. In order to solve this problem, by exploiting the additive structure of this model, a one-step backfitting algorithm is proposed. Under fairly general conditions, it turns out that the resulting estimators show optimal rates of convergence and exhibit the oracle efficiency property. Since both estimators are asymptotically equivalent, it is of interest to analyze their behavior in small sample sizes. In a fully parametric context, it is well-known that, under strict exogeneity assumptions the performance of both first-differences and within estimators is going to depend on the stochastic structure of the idiosyncratic random errors. However, in the non-parametric setting, apart from the previous issues other factors such as dimensionality or sample size are of great interest. In particular, we would be interested in learning about their relative average mean square error under different scenarios. The simulation results basically confirm the theoretical findings for both local linear regression and one-step backfitting estimators. However, we have found out that within estimators are rather sensitive to the size of number of time observations.

Sparse Semiparametric Nonlinear Model With Application to Chromatographic Fingerprints

Traditional Chinese herbal medications (TCHMs) are composed of a multitude of compounds and the identification of their active composition is an important area of research. Chromatography provides a visual representation of a TCHM sample’s composition by outputting a curve characterized by spikes corresponding to compounds in the sample. Across different experimental conditions, the location of the spikes can be shifted, preventing direct comparison of curves and forcing compound identification to be possible only within each experiment. In this article, we propose a sparse semiparametric nonlinear modeling framework for the establishment of a standardized chromatographic fingerprint. Data-driven basis expansion is used to model the common shape of the curves, while a parametric time warping function registers across individual curves. Penalized weighted least-squares with the adaptive lasso penalty provides a unified criterion for registration, model selection, and estimation. Furthermore, the adaptive lasso estimators possess attractive sampling properties. A back-fitting algorithm is proposed for estimation. Performance is assessed through simulation and we apply the model to chromatographic data of rhubarb collected from different experimental conditions and establish a standardized fingerprint as a first step in TCHM research.

Nonparametric estimation of fixed effects panel data varying coefficient models

In this paper, we consider the nonparametric estimation of a varying coefficient fixed effect panel data model. The estimator is based in a within (un-smoothed) transformation of the regression model and then a local linear regression is applied to estimate the unknown varying coefficient functions. It turns out that the standard use of this technique produces a non-negligible asymptotic bias. In order to avoid it, a high dimensional kernel weight is introduced in the estimation procedure. As a consequence, the asymptotic bias is removed but the variance is enlarged, and therefore the estimator shows a very slow rate of convergence. In order to achieve the optimal rate, we propose a one-step backfitting algorithm. The resulting two-step estimator is shown to be asymptotically normal and its rate of convergence is optimal within its class of smoothness functions. It is also oracle efficient. Further, this estimator is compared both theoretically and by Monte-Carlo simulation against other estimators that are based in a within (smoothed) transformation of the regression model. More precisely the profile least-squares estimator proposed in this context in Sun et al. (2009). It turns out that the smoothness in the transformation enlarges the bias and it makes the estimator more difficult to analyze from the statistical point of view. However, the first step estimator, as expected, shows a bad performance when compared against both the two step backfitting algorithm and the profile least-squares estimator.

Efficient Estimation of the Partly Linear Additive Hazards Model with Current Status Data

AbstractThis paper focuses on efficient estimation, optimal rates of convergence and effective algorithms in the partly linear additive hazards regression model with current status data. We use polynomial splines to estimate both cumulative baseline hazard function with monotonicity constraint and nonparametric regression functions with no such constraint. We propose a simultaneous sieve maximum likelihood estimation for regression parameters and nuisance parameters and show that the resultant estimator of regression parameter vector is asymptotically normal and achieves the semiparametric information bound. In addition, we show that rates of convergence for the estimators of nonparametric functions are optimal. We implement the proposed estimation through a backfitting algorithm on generalized linear models. We conduct simulation studies to examine the finite‐sample performance of the proposed estimation method and present an analysis of renal function recovery data for illustration.

Estimation Procedure for a Multiple Time Series Model

Given a multiple time series sharing common autoregressive patterns, we estimate an additive model. The autoregressive component and the individual random effects are estimated by integrating maximum likelihood estimation and best linear unbiased predictions in a backfitting algorithm. The simulation study illustrated that the estimation procedure provides an alternative to the Arellano–Bond generalized method of moments (GMM) estimator of the panel model when T > N and the Arellano–Bond generally diverges. The estimator has high predictive ability. In cases where T ≤ N, the backfitting estimator is at least comparable to Arellano–Bond estimator.

On concurvity in nonlinear and nonparametric regression models

When data are affected by multicollinearity in the linear regression framework, then concurvity will be present in fitting a generalized additive model (GAM). The term concurvity describes nonlinear dependencies among the predictor variables. As collinearity results in inflated variance of the estimated regression coefficients in the linear regression model, the result of the presence of concurvity leads to instability of the estimated coefficients in GAMs. Even if the backfitting algorithm will always converge to a solution, in case of concurvity the final solution of the backfitting procedure in fitting a GAM is influenced by the starting functions. While exact concurvity is highly unlikely, approximate concurvity, the analogue of multicollinearity, is of practical concern as it can lead to upwardly biased estimates of the parameters and to underestimation of their standard errors, increasing the risk of committing type I error. We compare the existing approaches to detect concurvity, pointing out their advantages and drawbacks, using simulated and real data sets. As a result, this paper will provide a general criterion to detect concurvity in nonlinear and non parametric regression models.

Direct semi-parametric estimation of fixed effects panel data varying coefficient models

Summary In this paper, we present a new technique to estimate varying coefficient models of unknown form in a panel data framework where individual effects are arbitrarily correlated with the explanatory variables in an unknown way. The estimator is based on first differences and then a local linear regression is applied to estimate the unknown coefficients. To avoid a non-negligible asymptotic bias, we need to introduce a higher-dimensional kernel weight. This enables us to remove the bias at the price of enlarging the variance term and, hence, achieving a slower rate of convergence. To overcome this problem, we propose a one-step backfitting algorithm that enables the resulting estimator to achieve optimal rates of convergence for this type of problem. It also exhibits the so-called oracle efficiency property. We also obtain the asymptotic distribution. Because the estimation procedure depends on the choice of a bandwidth matrix, we also provide a method to compute this matrix empirically. The Monte Carlo results indicate the good performance of the estimator in finite samples.

Time-Varying Additive Models for Longitudinal Data

The additive model is an effective dimension-reduction approach that also provides flexibility in modeling the relation between a response variable and key covariates. The literature is largely developed to scalar response and vector covariates. In this article, more complex data are of interest, where both the response and the covariates are functions. We propose a functional additive model together with a new backfitting algorithm to estimate the unknown regression functions, whose components are time-dependent additive functions of the covariates. Such functional data may not be completely observed since measurements may only be collected intermittently at discrete time points. We develop a unified platform and an efficient approach that can cover both dense and sparse functional data and the needed theory for statistical inference. We also establish the oracle properties of the proposed estimators of the component functions.