Local Linear Method Research Articles

Partially adapting multi-step local linear prediction

An adaptive multi-step local linear prediction method is derived for nonlinear time series by partially updating the predictor with an implicit vector inner product involved in every prediction. The predictor mainly consists of two parts, the basic vector which is calculated at one step based on the nearest neighbors of the current state point, and the updating vector which ensures the validity of the multi-step prediction. Analyses on the precision and computational complexity are made with close comparisons with other prediction methods based on the local linear model. Straightforward performance comparisons made with the classical Lorenz series show that while preserving an admirable precision, the proposed method reduces the computational complexity, thus rendering it more applicable of real time signal processing than the others based on the local linear model. Experiments on voiced sounds are also made and the results demonstrate that, as compared with the traditional linear prediction (LP) method, a remarkable performance gain in precision is achieved with the proposed method, while the computations it costs are similar to that consumed by the LP method.

Statistical inference in the partial linear models with the double smoothing local linear regression method

Statistical inference about a partial linear model includes two parts, one linear and the other nonparametric. The double smoothing method proposed by He and Huang (2009) is a progressive local smoothing method for nonparametric curve estimation. In this paper, we discuss its extension to partial linear models, accompanied with difference-based estimation method for the linear part. Asymptotic theory of the proposed method is developed. The results of simulation studies and real data examples demonstrate that our approach is effective even for data with moderate sample sizes.

Local rank estimation and related test for varying-coefficient partially linear models

This paper develops a robust estimation procedure for the varying-coefficient partially linear model via local rank technique. The new procedure provides a highly efficient and robust alternative to the local linear least-squares method. In other words, the proposed method is highly efficient across a wide class of non-normal error distributions and it only loses a small amount of efficiency for normal error. Moreover, a test for the hypothesis of constancy for the nonparametric component is proposed. The test statistic is simple and thus the test procedure can be easily implemented. We conduct Monte Carlo simulation to examine the finite sample performance of the proposed procedures and apply them to analyse the environment data set. Both the theoretical and the numerical results demonstrate that the performance of our approach is at least comparable to those existing competitors.

On the practical use of the spectral homotopy analysis method and local linearisation method for unsteady boundary-layer flows caused by an impulsively stretching plate

This paper expands the ideas of the spectral homotopy analysis method to apply them, for the first time, on non-linear partial differential equations. The spectral homotopy analysis method (SHAM) is a numerical version of the homotopy analysis method (HAM) which has only been previously used to solve non-linear ordinary differential equations. In this work, the modified version of the SHAM is used to solve a partial differential equation (PDE) that models the problem of unsteady boundary layer flow caused by an impulsively stretching plate. The robustness of the SHAM approach is demonstrated by its flexibility to allow linear operators that are partial derivatives with variable coefficients. This is seen to significantly improve the convergence and accuracy of the method. To validate accuracy of the the present SHAM results, the governing PDEs are also solved using a novel local linearisation technique coupled with an implicit finite difference approach. The two approaches are compared in terms of accuracy, speed of convergence and computational efficiency.

Nonparametric Estimation and Parametric Calibration of Time-Varying Coefficient Realized Volatility Models

This paper introduces a new speci…cation for the heterogeneous autoregressive (HAR) model for the realized volatility of S&P500 index returns. In this new model, the coe¢ cients of the HAR are allowed to be time-varying with unknown functional forms. We propose a local linear method for estimating this TVC-HAR model as well as a bootstrap method for constructing con…dence intervals for the time varying coef…cient functions. In addition, the estimated nonparametric TVC-HAR was calibrated by …tting parametric polynomial functions by minimising the L2-type criterion. The calibrated TVC-HAR and the simple HAR models were tested separately against the nonparametric TVC-HAR model. The test statistics constructed based on the generalised likelihood ratio method augmented with bootstrap method provide evidence in favour of calibrated TVC-HAR model. More importantly, the results of conditional predictive ability test developed by Giacomini and White (2006) indicate that the nonparametric TVC-HAR model consistently outperforms its calibrated counterpart as well as the simple HAR and the HAR-GARCH models in out-of-sample forecasting.

Robust variable selection for the varying coefficient model based on composite L 1–L 2 regression

The varying coefficient model (VCM) is an important generalization of the linear regression model and many existing estimation procedures for VCM were built on L 2 loss, which is popular for its mathematical beauty but is not robust to non-normal errors and outliers. In this paper, we address the problem of both robustness and efficiency of estimation and variable selection procedure based on the convex combined loss of L 1 and L 2 instead of only quadratic loss for VCM. By using local linear modeling method, the asymptotic normality of estimation is driven and a useful selection method is proposed for the weight of composite L 1 and L 2. Then the variable selection procedure is given by combining local kernel smoothing with adaptive group LASSO. With appropriate selection of tuning parameters by Bayesian information criterion (BIC) the theoretical properties of the new procedure, including consistency in variable selection and the oracle property in estimation, are established. The finite sample performance of the new method is investigated through simulation studies and the analysis of body fat data. Numerical studies show that the new method is better than or at least as well as the least square-based method in terms of both robustness and efficiency for variable selection.

Optical property dimensionality reduction techniques for accelerated radiative transfer performance: Application to remote sensing total ozone retrievals

In this paper, we introduce several dimensionality reduction techniques for optical parameters. We consider the principal component analysis, the local linear embedding methods (locality pursuit embedding, locality preserving projection, locally embedded analysis), and discrete orthogonal transforms (cosine, Legendre, wavelet). The principle component analysis has already been shown to be an effective and accurate method of enhancing radiative transfer performance for simulations in an absorbing and a scattering atmosphere. By linearizing the corresponding radiative transfer model, we analyze the applicability of the proposed methods to a practical problem of total ozone column retrieval from UV-backscatter measurements.

Object recognition and pose estimation using appearance manifolds

Conventionally, image object recognition and pose estimation are two independent components in machine vision. This paper presented a simple but effective method KNN-SNG, which tightly couples these two components within a single algorithm framework. The basic idea of this method came from the bionic pattern recognition and the manifold ways of perception. Firstly, the shortest neighborhood graphs (SNG) are established for each registered object. SNG can be regarded as a covering and triangulation for a hypersurface on which the training data are distributed. Then for recognition task, the determined test image lies on which SNG by employing the parameter “k”, which could be calculated adaptively. Finally, the local linear approximation method was adopted to build a local map between high-dimensional image space and low-dimensional manifold for pose estimation. The projective coordinates on manifold can depict the pose of object. Experiment results manifested the effectiveness of the method.

A TWO-STAGE PEAK ALIGNMENT ALGORITHM FOR TWO-DIMENSIONAL GAS CHROMATOGRAPHY TIME-OF-FLIGHT MASS SPECTROMETRY-BASED METABOLOMICS

Comprehensive two-dimensional gas chromatography coupled with time-of-flight mass spectrometry (GC×GC/TOF-MS) has been applied to metabolomics analyses recently. However, retention time shifts in the two-dimensional gas chromatography will introduce difficulty to compare compound profiles obtained from multiple samples. In this work, a novel two-stage peak alignment algorithm has been developed for data analysis of GC×GC/TOF-MS. In the first stage, our algorithm detects and merges multiple peak entries of the same metabolite into one peak entry. After a z-score transformation of metabolite retention times, landmark peaks will be selected from all samples based on both two-dimensional retention times and mass spectrum similarity of fragment ions measured by Pearson's correlation coefficient. In the second stage, the original two-dimensional retention time shift will be corrected using a local linear fitting method. A progressive retention time map searching method is used to align peaks in all samples together based on the parameters optimized in the first stage. Our algorithm can avoid defining a threshold of retention time window and spectrum similarity, which is very difficult for the users since the experimental condition is always changed in different experimental runs, even for the repeat experiments. The experimental results show that our algorithm can work well in peak alignment from real biological samples, which is very important for the further analysis.

Reduced bases for nonlinear structural dynamic systems: A comparative study

The presented work provides an overview of some commonly used approaches for generating reduced bases for discrete nonlinear dynamic systems. It investigates the performance and the robustness of these bases if they are applied in a reduction-by-projection procedure on different test cases. The bases are created from the Linear Normal Modes, the Ritz-vectors, the Proper and the Smooth Orthogonal Decomposition method, the A Priori Reduction, the Centroidal Voronoi Tessellation and the Local Equivalent Linear Stiffness Method. Second-Order Terms and an Enhanced Proper Orthogonal Decomposition formulation are included as variants. The test cases are small dimensional, locally or entirely nonlinear system subjected to a harmonic or an impulse force excitation. The double objective of this numerical study is, first, to determine which bases are most adequate for a given combination of nonlinearity and excitation and, second, to which extend the bases exhibit an inherent robustness if the parameterisation of the excitation is changed. A specific multicriteria decision analysis score is developed to assess the bases' performance. As a major result, a strong dependence of the performance of the bases on the type of excitation is established and thus some bases become more adequate for a certain situation than others. Also a lack of robustness for all considered bases can be observed. This situation improves in most cases if the basis is generated with the most critical values of the parameter.

Local Linear GMM Estimation of Functional Coefficient IV Models With an Application to Estimating the Rate of Return to Schooling

We consider the local linear generalized method of moment (GMM) estimation of functional coefficient models with a mix of discrete and continuous data and in the presence of endogenous regressors. We establish the asymptotic normality of the estimator and derive the optimal instrumental variable that minimizes the asymptotic variance-covariance matrix among the class of all local linear GMM estimators. Data-dependent bandwidth sequences are also allowed for. We propose a nonparametric test for the constancy of the functional coefficients, study its asymptotic properties under the null hypothesis as well as a sequence of local alternatives and global alternatives, and propose a bootstrap version for it. Simulations are conducted to evaluate both the estimator and test. Applications to the 1985 Australian Longitudinal Survey data indicate a clear rejection of the null hypothesis of the constant rate of return to education, and that the returns to education obtained in earlier studies tend to be overestimated for all the work experience.

Weighted local linear composite quantile estimation for the case of general error distributions

It is known that for nonparametric regression, local linear composite quantile regression (local linear CQR) is a more competitive technique than classical local linear regression since it can significantly improve estimation efficiency under a class of non-normal and symmetric error distributions. However, this method only applies to symmetric errors because, without symmetric condition, the estimation bias is non-negligible and therefore the resulting estimator is inconsistent. In this paper, we propose a weighted local linear CQR method for general error conditions. This method applies to both symmetric and asymmetric random errors. Because of the use of weights, the estimation bias is eliminated asymptotically and the asymptotic normality is established. Furthermore, by minimizing asymptotic variance, the optimal weights are computed and consequently the optimal estimate (the most efficient estimate) is obtained. By comparing relative efficiency theoretically or numerically, we can ensure that the new estimation outperforms the local linear CQR estimation. Finite sample behaviors conducted by simulation studies further illustrate the theoretical findings.

Estimating Smooth Structural Change in Cointegration Models

This paper studies nonlinear cointegration models in which the structural coefficients may evolve smoothly over time. These time-varying coefficient functions are well-suited to many practical applications and can be estimated conveniently by nonparametric kernel methods. It is shown that the usual asymptotic methods of kernel estimation completely break down in this setting when the functional coefficients are multivariate. The reason for this breakdown is a kernel-induced degeneracy in the weighted signal matrix associated with the nonstationary regressors, a new phenomenon in the kernel regression literature. Some new techniques are developed to address the degeneracy and resolve the asymptotics, using a path-dependent local coordinate transformation to re-orient coordinates and accommodate the degeneracy. The resulting asymptotic theory is fundamentally different from the existing kernel literature, giving two different limit distributions with different convergence rates in the different directions (or combinations) of the (functional) parameter space. Both rates are faster than the usual rate for nonlinear models with smoothly changing coefficients and local stationarity. Hence two types of super-consistency apply in nonparametric kernel estimation of time-varying coefficient cointegration models. The higher rate of convergence lies in the direction of the nonstationary regressor vector at the local coordinate point. The lower rate lies in the degenerate directions but is still super-consistent for nonparametric estimators. In addition, local linear methods are used to reduce asymptotic bias and a fully modified kernel regression method is proposed to deal with the general endogenous nonstationary regressor case. Simulations are conducted to explore the finite sample properties of the methods and a practical application is given to examine time varying empirical relationships involving consumption, disposable income, investment and real interest rates.

Estimating Smooth Structural Change in Cointegration Models

This paper studies nonlinear cointegration models in which the structural coefficients may evolve smoothly over time. These time-varying coefficient functions are well-suited to many practical applications and can be estimated conveniently by nonparametric kernel methods. It is shown that the usual asymptotic methods of kernel estimation completely break down in this setting when the functional coefficients are multivariate. The reason for this breakdown is a kernel-induced degeneracy in the weighted signal matrix associated with the nonstationary regressors, a new phenomenon in the kernel regression literature. Some new techniques are developed to address the degeneracy and resolve the asymptotics, using a path-dependent local coordinate transformation to re-orient coordinates and accommodate the degeneracy. The resulting asymptotic theory is fundamentally different from the existing kernel literature, giving two different limit distributions with different convergence rates in the different directions (or combinations) of the (functional) parameter space. Both rates are faster than the usual (?nh) rate for nonlinear models with smoothly changing coefficients and local stationarity. Hence two types of super-consistency apply in nonparametric kernel estimation of time-varying coefficient cointegration models. The higher rate of convergence (n?h) lies in the direction of the nonstationary regressor vector at the local coordinate point. The lower rate (nh) lies in the degenerate directions but is still super-consistent for nonparametric estimators. In addition, local linear methods are used to reduce asymptotic bias and a fully modified kernel regression method is proposed to deal with the general endogenous nonstationary regressor case. Simulations are conducted to explore the finite sample properties of the methods and a practical application is given to examine time varying empirical relationships involving consumption, disposable income, investment and real interest rates.

A Nonlinear System Identification Method Based on Local Linear PLS Method

A Nonlinear System Identification Method Based on Local Linear PLS Method

Semisupervised Local Discriminant Analysis for Feature Extraction in Hyperspectral Images

We propose a novel semisupervised local discriminant analysis method for feature extraction in hyperspectral remote sensing imagery, with improved performance in both ill-posed and poor-posed conditions. The proposed method combines unsupervised methods (local linear feature extraction methods and supervised method (linear discriminant analysis) in a novel framework without any free parameters. The underlying idea is to design an optimal projection matrix, which preserves the local neighborhood information inferred from unlabeled samples, while simultaneously maximizing the class discrimination of the data inferred from the labeled samples. Experimental results on four real hyperspectral images demonstrate that the proposed method compares favorably with conventional feature extraction methods.

Nonlinear prediction of small scale network traffic based on local relevance vector machine regression model

Based on the nonlinear time series local prediction method and the relevance vector machine regression model, the local relevance vector machine prediction method is proposed and applied to predict the small scale traffic measurement data, and the BIC-based neighbor point selection method is used to choose the number of nearest-neighbor points for the local relevance vector machine regression model. We also compare the performance of the local relevance vector machine regression model with the feed-forward neural network optimized by particle swarm optimization for the same problem. Experimental results show that the local relevance vector machine prediction method whose neighboring points have been optimized can effectively predict the small scale traffic measurement data, can reproduce the statistical features of real small scale traffic measurements, and the prediction accuracy of the local relevance vector machine regression model is superior to that of the feedforward neural network optimized by PSO and the local linear prediction method.

Spectral Local Linearisation Approach for Natural Convection Boundary Layer Flow

The present work introduces a spectral local linearisation method (SLLM) to solve a natural convection boundary layer flow problem with domain transformation. It is customary to find solutions of semi-infinite interval problems by first truncating the interval and subsequently applying a suitable numerical method. However, this gives rise to increased error terms in the numerical solution. Carrying out a transformation of the semi-infinite interval problems into singular problems posed on a finite interval can avoid the domain truncation error and enables the efficient application of collocation methods. The SLLM is based on linearising and decoupling nonlinear systems of equations into a sequence or subsystems of differential equations which are then solved using spectral collocation methods. A comparative study between the SLLM and existing results in the literature was carried out to validate the results. The method has shown to be a promising efficient tool for nonlinear boundary value problems as it gives converging results after very few iterations.

A New Spectral Local Linearization Method for Nonlinear Boundary Layer Flow Problems

We propose a simple and efficient method for solving highly nonlinear systems of boundary layer flow problems with exponentially decaying profiles. The algorithm of the proposed method is based on an innovative idea of linearizing and decoupling the governing systems of equations and reducing them into a sequence of subsystems of differential equations which are solved using spectral collocation methods. The applicability of the proposed method, hereinafter referred to as the spectral local linearization method (SLLM), is tested on some well-known boundary layer flow equations. The numerical results presented in this investigation indicate that the proposed method, despite being easy to develop and numerically implement, is very robust in that it converges rapidly to yield accurate results and is more efficient in solving very large systems of nonlinear boundary value problems of the similarity variable boundary layer type. The accuracy and numerical stability of the SLLM can further be improved by using successive overrelaxation techniques.

Empirical likelihood inference in mixture of semiparametric varying-coefficient models for longitudinal data with non-ignorable dropout

In this paper, empirical likelihood inference in mixture of semiparametric varying-coefficient models for longitudinal data with non-ignorable dropout is investigated. We estimate the non-parametric function based on the estimating equations and the local linear profile-kernel method. An empirical log-likelihood ratio statistic for parametric components is proposed to construct confidence regions and is shown to be an asymptotically chi-squared distribution. The non-parametric version of Wilk's theorem is also derived. A simulation study is undertaken to illustrate the finite sample performance of the proposed method.