Local Linear Approximation Research Articles

Semantic types of legal norms in German laws: classification and analysis using local linear explanations

This paper describes the automated classification of legal norms in German statutes with regard to their semantic type. We propose a semantic type taxonomy for norms in the German civil law domain consisting of nine different types focusing on functional aspects, such as Duties, Prohibitions, Permissions, etc. We performed four iterations in classifying legal norms with a rule-based approach using a manually labeled dataset, i.e., tenancy law, of the German Civil Code ($$\hbox {n} = 601$$). During this experiment the $$F_1$$ score continuously improved from 0.52 to 0.78. In contrast, a machine learning based approach for the classification was implemented. A performance of $$F_1 = 0.83$$ was reached. Traditionally, machine learning classifiers lack of transparency with regard to their decisions. We extended our approach using so-called local linear approximations, which is a novel technique to analyze and inspect a trained classifier’s behavior. We can show that there are significant similarities of manually crafted knowledge, i.e., rules and pattern definitions, and the trained decision structures of machine learning approaches.

Adaptive extended Kalman filter based state of charge determination for lithium-ion batteries

The accurate online estimation of cell state of charge in electric vehicles is challenging due to extensive computational requirement, measurement noise and convergence issues. This paper proposes an adaptive extended Kalman filter based state of charge determination method, which employs improved simulated annealing method to access the uncertain states and adaptive switch mechanism to adjust the estimation algorithm. It combines the advantages of the local linear approximation and noise reduction capability from extended Kalman filter with the global optimization from improved simulated annealing. The method is verified by the samples collected from battery test system. Results illustrate that the proposed method estimates the cell remained capacity with great performance despite a range of common errors from initialization, current disturbance and measurement noise.

Averaging estimators for kernel regressions

This paper considers model averaging for kernel regressions. We construct a weighted average of the local constant and local linear estimators at each point of estimation. We propose a two-step cross-validation method for bandwidths and weights selection, and derive the rate of convergence of the cross-validation weights to their optimal benchmark values.

Asymptotic normality of some conditional nonparametric functional parameters in high-dimensional statistics

This paper deals with the convergence in distribution of estimators of some conditional parameters in the Functional Data Analysis framework. In fact, we consider models where the input is of functional kind and the output is a scalar. Then, we establish the asymptotic normality of the nonparametric local linear estimators of (1) the conditional distribution function and (2) the successive derivatives of the conditional density. Moreover, as by-product, we deduce the asymptotic normality of the local linear estimator of the conditional mode. Finally, to show interests of our results, on the practical point of view, we have conducted a computational study, first on a simulated data and, then on some real data concerning the forage quality.

Structure identification and model selection in geographically weighted quantile regression models

Geographical weighted quantile regression (GWQR) is an important tool for exploring the spatial non-stationarity of the regression relationship and providing the entire description of the response distribution, which is useful in practice because of its robustness against outliers and flexibility in dealing with non-normal distributions. This paper proposes the GWQlasso method for structure identification and variable selection in GWQR models. The proposed method combines the local-linear estimation of the GWQR model and the adaptive group lasso, which can simultaneously identify spatially varying coefficients, non-zero constant coefficients and zero coefficients. The selection consistency and the oracle property of the proposal are studied. Moreover, the derived algorithm for the GWQlasso method and the selection of the tuning parameter by the BIC criterion are established. Simulations and real examples are used to illustrate the method.

Visual Servoing on the Generalized Voronoi Diagram Using an Omnidirectional Camera

The Generalized Voronoi Diagram (GVD) is a powerful environment representation, since, among other reasons, it defines a set of paths at maximal distance from the obstacles. Many works implicitly use this property to define safe navigation strategies for a mobile robot, but, in practice, only a few explicitly extract in real time the GVD from its perception to induce motion. This article addresses this challenge for a mobile robot solely equipped with an omnidirectional camera, and proposes an autonomous navigation strategy in unknown indoor and/or outdoor environments. The problem is formulated as a visual servoing task, and uses an anisotropic skeletonization algorithm to identify the projection in the image of the local GVD, while suppressing unsafe navigable paths thanks to a context-defined pruning parameter. Unlike rangefinder based system, the use of an omnidirectional camera allows to efficiently deal with outdoor scenes, since the navigable space is extracted using photometric cues. The servoing is then performed using a local linear approximation of the GVD, defined by a conic extracted in real time in the omnidirectional image. To assert the relevancy and efficiency of the proposed approach, extended experimental results are presented, both on indoor and outdoor scenarios.

I-LAMM FOR SPARSE LEARNING: SIMULTANEOUS CONTROL OF ALGORITHMIC COMPLEXITY AND STATISTICAL ERROR.

We propose a computational framework named iterative local adaptive majorize-minimization (I-LAMM) to simultaneously control algorithmic complexity and statistical error when fitting high dimensional models. I-LAMM is a two-stage algorithmic implementation of the local linear approximation to a family of folded concave penalized quasi-likelihood. The first stage solves a convex program with a crude precision tolerance to obtain a coarse initial estimator, which is further refined in the second stage by iteratively solving a sequence of convex programs with smaller precision tolerances. Theoretically, we establish a phase transition: the first stage has a sublinear iteration complexity, while the second stage achieves an improved linear rate of convergence. Though this framework is completely algorithmic, it provides solutions with optimal statistical performances and controlled algorithmic complexity for a large family of nonconvex optimization problems. The iteration effects on statistical errors are clearly demonstrated via a contraction property. Our theory relies on a localized version of the sparse/restricted eigenvalue condition, which allows us to analyze a large family of loss and penalty functions and provide optimality guarantees under very weak assumptions (For example, I-LAMM requires much weaker minimal signal strength than other procedures). Thorough numerical results are provided to support the obtained theory.

On the local linear estimate for functional regression: Uniform in bandwidth consistency

ABSTRACTWe consider the problem of local linear estimation of the regression function when the regressor is functional. The main result of this paper is to prove the strong convergence (with rates), uniformly in bandwidth parameters (UIB), of the considered estimator. The main interest of this result is the possibility to derive the asymptotic properties of our estimate even if the bandwidth parameter is a random variable.

Local linear estimation for spatial random processes with stochastic trend and stationary noise.

We consider the problem of estimating the trend for a spatial random process model expressed as Z(x) = μ(x) + ε(x) + δ(x), where the trend μ is a smooth random function, ε(x) is a mean zero, stationary random process, and {δ(x)} are assumed to be i.i.d. noise with zero mean. We propose a new model for stochastic trend in by generalizing the notion of a structural model for trend in time series. We estimate the stochastic trend nonparametrically using a local linear regression method and derive the asymptotic mean squared error of the trend estimate under the proposed model for trend. Our results show that the asymptotic mean squared error for the stochastic trend is of the same order of magnitude as that of a deterministic trend of comparable complexity. This result suggests from the point of view of estimation under stationary noise, it is immaterial whether the trend is treated as deterministic or stochastic. Moreover, we show that the rate of convergence of the estimator is determined by the degree of decay of the correlation function of the stationary process ε(x) and this rate can be different from the usual rate of convergence found in the literature on nonparametric function estimation. We also propose a data dependent selection procedure for the bandwidth parameter which is based on a generalization of Mallow's C p criterion. We illustrate the methodology by simulation studies and by analyzing a data on surface temperature anomalies.

Simultaneous selection of optimal bandwidths for the sharp regression discontinuity estimator

A new bandwidth selection rule that uses different bandwidths for the local linear regression estimators on the left and the right of the cut-off point is proposed for the sharp regression discontinuity estimator of the mean program impact at the cut-off point. The asymptotic mean squared error of the estimator using the proposed bandwidth selection rule is shown to be smaller than other bandwidth selection rules proposed in the literature. An extensive simulation study shows that the proposed method's performances for the sample sizes 500, 2000, and 5000 closely match the theoretical predictions.

Variable bandwidth local maximum likelihood type estimation for diffusion processes

The method of robust approach is applied to estimate drift function and diffusion function of diffusion processes with discrete-time observations. The proposed method combines the ideas of local linear regression technique and maximum likelihood type estimation technique, so the advantages of local linear estimators persist and overcome the disadvantages of least-squares estimator. Moreover, a variable bandwidth instead of a constant bandwidth is considered in the local maximum likelihood type estimators. The consistency and asymptotic normality of the local maximum likelihood type estimators for drift and diffusion functions are developed under some given conditions. We perform a simulation study to evaluate the robust performances of the proposed estimators.

A chaos genetic algorithm based extended Kalman filter for the available capacity evaluation of lithium-ion batteries

Lithium-ion batteries are developed rapidly in electric vehicles, whose safety and functional capabilities are influenced greatly by the evaluation of available capacity. Combined with the evolution trend description of state of charge from extended Kalman filter and an adaptive switch mechanism, this paper advances an adaptive chaos genetic algorithm based extended Kalman filter for the state of charge determination of lithium-ion batteries, where a combined state space model is used for simulating their dynamics. It combines the advantage of local linear approximation capability from extended Kalman filter with the global optimal search mechanism from chaos genetic algorithm. The method is applied for the state of charge determination of lithium-ion batteries, and results of lab tests on physical cells, compared with model prediction, are presented. Furthermore, the innovation magnitude bound test and innovation whiteness test are employed for verifying the performance of the proposed method. Results confirm that the advanced method may quickly evaluate state of charge with high accuracy and has great robustness without being affected by the uncertain initial value.

On Local Linear Regression Estimation of Finite Population Totals in Model Based Surveys

In this paper, nonparametric regression is employed which provides an estimation of unknown finite population totals. A robust estimator of finite population totals in model based inference is constructed using the procedure of local linear regression. In particular, robustness properties of the proposed estimator are derived and a brief comparison between the performances of the derived estimator and some existing estimators is made in terms of bias, MSE and relative efficiency. Results indicate that the local linear regression estimator is more efficient and performing better than the Horvitz-Thompson and Dorfman estimators, regardless of whether the model is specified or mispecified. The local linear regression estimator also outperforms the linear regression estimator in all the populations except when the population is linear. The confidence intervals generated by the model based local linear regression method are much tighter than those generated by the design based Horvitz-Thompson method. Generally the model based approach outperforms the design based approach regardless of whether the underlying model is correctly specified or not but that effect decreases as the model variance increases.

Simultaneous Confidence Bands in Nonlinear Regression Models with Nonstationarity

We consider nonparametric estimation of the regression function g(·) in a nonlinear regression model Yt = g(Xt) + σ(Xt)et, where the regressor (Xt) is a nonstationary unit root process and the error (et) is a sequence of independent and identically distributed (i.i.d.) random variables. With proper centering and scaling, the maximum deviation of the local linear estimator of the regression function g is shown to be asymptotically Gumbel. Based on the latter result, we construct simultaneous confidence bands for g, which can be used to test patterns of the regression function. Our results substantially extend existing ones which typically require independent or stationary weakly dependent regressors. Furthermore, we examine the finite sample behavior of the proposed approach via the simulated and real data examples.

Non‐convex penalized multitask regression using data depth‐based penalties

We propose a new class of non‐convex penalties based on data depth functions for multitask sparse penalized regression. These penalties quantify the relative position of rows of the coefficient matrix from a fixed distribution centred at the origin. We derive the theoretical properties of an approximate one‐step sparse estimator of the coefficient matrix using local linear approximation of the penalty function and provide an algorithm for its computation. For the orthogonal design and independent responses, the resulting thresholding rule enjoys near‐minimax optimal risk performance, similar to the adaptive lasso (Zou, H (2006), ‘The adaptive lasso and its oracle properties’, Journal of the American Statistical Association, 101, 1418–1429). A simulation study and real data analysis demonstrate its effectiveness compared with some of the present methods that provide sparse solutions in multitask regression. Copyright © 2018 John Wiley & Sons, Ltd.

Scalable aggregation predictive analytics

We introduce a predictive modeling solution that provides high quality predictive analytics over aggregation queries in Big Data environments. Our predictive methodology is generally applicable in environments in which large-scale data owners may or may not restrict access to their data and allow only aggregation operators like COUNT to be executed over their data. In this context, our methodology is based on historical queries and their answers to accurately predict ad-hoc queries’ answers. We focus on the widely used set-cardinality, i.e., COUNT, aggregation query, as COUNT is a fundamental operator for both internal data system optimizations and for aggregation-oriented data exploration and predictive analytics. We contribute a novel, query-driven Machine Learning (ML) model whose goals are to: (i) learn the query-answer space from past issued queries, (ii) associate the query space with local linear regression & associative function estimators, (iii) define query similarity, and (iv) predict the cardinality of the answer set of unseen incoming queries, referred to the Set Cardinality Prediction (SCP) problem. Our ML model incorporates incremental ML algorithms for ensuring high quality prediction results. The significance of contribution lies in that it (i) is the only query-driven solution applicable over general Big Data environments, which include restricted-access data, (ii) offers incremental learning adjusted for arriving ad-hoc queries, which is well suited for query-driven data exploration, and (iii) offers a performance (in terms of scalability, SCP accuracy, processing time, and memory requirements) that is superior to data-centric approaches. We provide a comprehensive performance evaluation of our model evaluating its sensitivity, scalability and efficiency for quality predictive analytics. In addition, we report on the development and incorporation of our ML model in Spark showing its superior performance compared to the Spark’s COUNT method.

Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization

Hyperspectral image (HSI) possesses three intrinsic characteristics: the global correlation across spectral domain, the nonlocal self-similarity across spatial domain, and the local smooth structure across both spatial and spectral domains. This paper proposes a novel tensor based approach to handle the problem of HSI spatial super-resolution by modeling such three underlying characteristics. Specifically, a noncovex tensor penalty is used to exploit the former two intrinsic characteristics hidden in several 4D tensors formed by nonlocal similar patches within the 3D HSI. In addition, the local smoothness in both spatial and spectral modes of the HSI cube is characterized by a 3D total variation (TV) term. Then, we develop an effective algorithm for solving the resulting optimization by using the local linear approximation (LLA) strategy and the alternative direction method of multipliers (ADMM). A series of experiments are carried out to illustrate the superiority of the proposed approach over some state-of-the-art approaches.

Local linear estimation of a generalized regression function with functional dependent data

ABSTRACTThis work deals with a local linear non parametric estimation of the generalized regression function in the case of a scalar response variable given a random variable taking values in a semimetric space. The rates of pointwise and uniform almost complete convergence are established for the studied estimator when the sample is an α-mixing sequence. Two real datasets are used to illustrate the performance of a studied estimator with respect to the kernel method.

Functional coefficient time series models with trending regressors

ABSTRACTThis paper studies a functional coefficient time series model with trending regressors, where the coefficients are unknown functions of time and random variables. We propose a local linear estimation method to estimate the unknown coefficient functions, and establish the corresponding asymptotic theory under mild conditions. We also develop a test procedure to see if the functional coefficients take particular parametric forms. For practical use, we further propose a Bayesian approach to select the bandwidths, and conduct several numerical experiments to examine the finite sample performance of our proposed local linear estimator and the test procedure. The results show that the local linear estimator works well and the proposed test has satisfactory size and power. In addition, our simulation studies show that the Bayesian bandwidth selection method performs better than the cross-validation method. Furthermore, we use the functional coefficient model to study the relationship between consumption per capita and income per capita in United States, and it was shown that the functional coefficient model with our proposed local linear estimator and Bayesian bandwidth selection method performs well in both in-sample fitting and out-of-sample forecasting.

Consistent estimator of nonparametric structural spurious regression model for high frequency data

We propose a new nonparametric estimator for continuous-time regression models with nonstationary error terms. While other conventional nonparametric estimators such as the Nadaraya–Watson and local linear estimators are not consistent, our estimator achieves consistency and asymptotic normality.