Single Smoothing Parameter Research Articles

Accommodating heterogeneity in brand loyalty estimation: application to the U.S. beer retail market

The assessment of consumer behavior in brand-choice models may be greatly influenced by accurately modeling and evaluating a brand-loyalty parameter. The most well-known approaches for estimating brand loyalty employ a household’s past purchase data and account for habit formation with a single smoothing parameter that indicates the weight assigned to households’ current decisions versus their distant shopping history. In this study, we present a method for estimating time-varying smoothing parameters for heterogeneous households. We estimate smoothing parameters for American households from 2014 to 2017 using Nielsen Homescan data for the US beer retail market. Using this more flexible method, we discover that the smoothing parameter varies significantly among households and time frames (for the same households). We next include a brand-loyalty index based on this approach into a brand-choice model of the American beer retail market, demonstrating that this new method improves estimation results.

Smoothing Methods for Numerical Differentiation to Identify Electrochemical Reactions from Open-Circuit-Potential Data

We develop and compare methods to determine when electrochemical reactions take place within intercalation electrodes. We employ open-circuit data from the Chevrolet Bolt EV negative and positive electrodes, lithiated graphite and NMC (Ni0.6Mn0.2Co0.2O2), respectively. Electrochemical reactions correspond to peaks in plots of dx/dU vs. U, where U is the electrode potential and x is the fractional state of charge (0 ⩽ x ⩽ 1). We employ three nonparametric methods in this work to smooth and subsequently differentiate the data: a windowed cubic polynomial of fixed and variable window length and a cubic smoothing spline. The standard deviation in the measurements can be used to set appropriately the window length and the analogous smoothing parameter for the spline. The cubic polynomial with a fixed window length and the smoothing spline each have a single smoothing parameter; both methods are shown to work well and yield comparable results. The best results were achieved with the cubic polynomial incorporating a varying, adaptive window length, and the increased fidelity enables a completely automated procedure to identify reactions. The analysis clarifies the multi-reaction nature of lithiated graphite, which must be addressed in cell modeling to simulate fast-charge behavior.

Identifying local smoothness for spatially inhomogeneous functions

We consider a problem of estimating local smoothness of a spatially inhomogeneous function from noisy data under the framework of smoothing splines. Most existing studies related to this problem deal with estimation induced by a single smoothing parameter or partially local smoothing parameters, which may not be efficient to characterize various degrees of smoothness of the underlying function when it is spatially varying. In this paper, we propose a new nonparametric method to estimate local smoothness of the function based on a moving local risk minimization coupled with spatially adaptive smoothing splines. The proposed method provides full information of the local smoothness at every location on the entire data domain, so that it is able to understand the degrees of spatial inhomogeneity of the function. A successful estimate of the local smoothness is useful for identifying abrupt changes of smoothness of the data, performing functional clustering and improving the uniformity of coverage of the confidence intervals of smoothing splines. We further consider a nontrivial extension of the local smoothness of inhomogeneous two-dimensional functions or spatial fields. Empirical performance of the proposed method is evaluated through numerical examples, which demonstrates promising results of the proposed method.

Application of Reinforcement Learning Algorithms for the Adaptive Computation of the Smoothing Parameter for Probabilistic Neural Network.

In this paper, we propose new methods for the choice and adaptation of the smoothing parameter of the probabilistic neural network (PNN). These methods are based on three reinforcement learning algorithms: Q(0)-learning, Q(λ)-learning, and stateless Q-learning. We regard three types of PNN classifiers: the model that uses single smoothing parameter for the whole network, the model that utilizes single smoothing parameter for each data attribute, and the model that possesses the matrix of smoothing parameters different for each data variable and data class. Reinforcement learning is applied as the method of finding such a value of the smoothing parameter, which ensures the maximization of the prediction ability. PNN models with smoothing parameters computed according to the proposed algorithms are tested on eight databases by calculating the test error with the use of the cross validation procedure. The results are compared with state-of-the-art methods for PNN training published in the literature up to date and, additionally, with PNN whose sigma is determined by means of the conjugate gradient approach. The results demonstrate that the proposed approaches can be used as alternative PNN training procedures.

Multivariate Exponential Smoothing and Dynamic Factor Model Applied to Hourly Electricity Price Analysis

Thanks to its very simple recursive computing scheme, exponential smoothing has become a popular technique to forecast time series. In this work, we show the advantages of its multivariate version and present some properties of the model, which allows us to perform a dynamic factor analysis. This analysis leads to a simple methodology to reduce the number of parameters (useful when the dimension of observations is large) via a linear transformation that decomposes the multivariate process into independent univariate exponential smoothing processes, characterized by a single smoothing parameter that goes from zero (white-noise process) to one (random walk process). A computer implementation of the expectation-maximization (EM) algorithm has been built for the maximum likelihood estimation of the models. The practicality of the method is demonstrated by its application to hourly electricity price predictions in some day-ahead markets, such as Omel, Powernext, and Nord Pool markets, whose forecasts are given as examples. This article has supplementary material online.

A non-iterative optimization method for smoothness in penalized spline regression

Typically, an optimal smoothing parameter in a penalized spline regression is determined by minimizing an information criterion, such as one of the C p , CV and GCV criteria. Since an explicit solution to the minimization problem for an information criterion cannot be obtained, it is necessary to carry out an iterative procedure to search for the optimal smoothing parameter. In order to avoid such extra calculation, a non-iterative optimization method for smoothness in penalized spline regression is proposed using the formulation of generalized ridge regression. By conducting numerical simulations, we verify that our method has better performance than other methods which optimize the number of basis functions and the single smoothing parameter by means of the CV or GCV criteria.

Bayesian multiscale smoothing in supervised and semi-supervised kernel discriminant analysis

In kernel discriminant analysis, it is common practice to select the smoothing parameter (bandwidth) based on the training data and use it for classifying all unlabeled observations. But this method of selecting a single scale of smoothing ignores the major issue of model uncertainty. Moreover, in addition to depending on the training sample, a good choice of bandwidth may also depend on the observation to be classified, and a fixed level of smoothing may not work well in all parts of the measurement space. So, instead of using a single smoothing parameter, it may be more useful in practice to study classification results for multiple scales of smoothing and judiciously aggregate them to arrive at the final decision. This paper adopts a Bayesian approach to carry out one such multiscale analysis using a probabilistic framework. This framework also helps us to extend our multiscale method for semi-supervised classification, where, in addition to the training sample, one uses unlabeled test set observations to form the decision rule. Some well-known benchmark data sets are analyzed to show the utility of these proposed methods.

Enhancement of spatially adaptive smoothing splines via parameterization of smoothing parameters

This paper considers the problem of estimating curve and surface functions when the structures of an unknown function vary spatially. Classical approaches such as using smoothing splines, which are controlled by a single smoothing parameter, are inefficient in estimating the underlying function that consists of different spatial structures. In this paper, we propose a blockwise method of fitting smoothing splines wherein the smoothing parameter λ varies spatially, in order to accommodate possible spatial nonhomogeneity of the regression function. A key feature of the proposed blockwise method is the parameterization of a smoothing parameter function λ ( x ) that produces a continuous spatially adaptive fit over the entire range of design points. The proposed parameterization requires two important ingredients: (1) a blocking scheme that divides the data into several blocks according to the degree of spatial variation of the data; and (2) a method for choosing smoothing parameters of blocks. We propose a block selection approach that is based on the adaptive thinning algorithm and a choice of smoothing parameters that minimize a newly defined blockwise risk. The results obtained from numerical experiments validate the effectiveness of the proposed method.

Modeling Individual Damped Linear Oscillator Processes with Differential Equations: Using Surrogate Data Analysis to Estimate the Smoothing Parameter

Among the many methods available for modeling intraindividual time series, differential equation modeling has several advantages that make it promising for applications to psychological data. One interesting differential equation model is that of the damped linear oscillator (DLO), which can be used to model variables that have a tendency to fluctuate around some typical, or equilibrium, value. Methods available for fitting the damped linear oscillator model using differential equation modeling can yield biased parameter estimates when applied to univariate time series. The degree of this bias depends on a smoothing–like parameter, which balances the need for increasing smoothing to minimize error variance but not smoothing so much as to obscure change of interest. This article explores a technique that uses surrogate data analysis to select such a parameter, thereby producing approximately unbiased parameter estimates. Furthermore the smoothing parameter, which is usually researcher-selected, is produced in an automated manner so as to reduce the experience required by researchers to apply these methods. Focus is placed on the damped linear model; however, similar issues are expected with other differential equation models and other techniques in which parameter estimates depend on a smoothing parameter. An example using affect data from the Notre Dame Longitudinal Study of Aging (2004) is presented, which contrasts the use of a single smoothing parameter for all individuals versus use of a smoothing parameter for each individual.

A Fully Bayesian Inversion for Spatial Distribution of Fault Slip with Objective Smoothing

Abstract In inversion of geodetic data for spatial distribution of fault slip under smoothing and positivity constraints, subjective or incorrect techniques are often employed to select a smoothing parameter that determines the relative weight placed on fitting the data versus smoothing the slip distribution. A popular objective method based on Akaike’s Bayesian information criterion (ABIC) is incorrect if positivity constraints are employed. We introduce a fully probabilistic inversion method to simultaneously estimate the slip distribution and smoothing parameter objectively and correctly in a Bayesian framework. The complete solution to the inverse problem, the joint posterior probability density function of slip and smoothing parameter, is formulated using Bayes’ theorem and sampled with a Markov chain Monte Carlo method. We apply the new method to coseismic displacement data from the 1999 Chi-Chi, Taiwan, earthquake and compare the results of our method with conventional methods. The slip distribution estimated with our method is significantly smoother than that estimated by the ABIC-based method, demonstrating that the ABIC-based method may in general generate significantly undersmoothed or oversmoothed slip distributions under positivity constraints. On the other hand, the estimated slip distribution is similar to that estimated using another conventional method based on cross validation. Multiple trade-off curve plots between slip roughness and data misfit show that trade-off curves provide no basis for selecting a single smoothing parameter. The new method is easily generalized to slip inversions of multiple data sets with unknown relative weighting and to slip inversions with unknown fault geometry parameters.

A practical sub-space adaptive filter

A Sub-Space Adaptive Filter (SSAF) model is developed using, as a basis, the Modified Probabilistic Neural Network (MPNN) and its extension the Tuneable Approximate Piecewise Linear Regression (TAPLR) model. The TAPLR model can be adjusted by a single smoothing parameter continuously from the best piecewise linear model in each sub-space to the best approximately piecewise linear model over the whole data space. A suitable value in between ensures that all neighbouring piecewise linear models merge together smoothly at their boundaries. This model was developed by altering the form of the MPNN, a network used for general nonlinear regression. The MPNNs special structure allows it to be easily used to model a process by appropriately weighting piecewise linear models associated with each of the network's radial basis functions. The model has now been further extended to allow each piecewise linear model section to be adapted separately as new data flows through it. By doing this, the proposed SSAF model represents a learning/filtering method for nonlinear processes that provides one solution to the stability/plasticity dilemma associated with standard adaptive filters.

Smoothing the canonical piecewise-linear model: an efficient and derivable large-signal model for MESFET/HEMT transistors

In this paper we present the smoothed piecewise-linear (SPWL) model as a useful tool in the device modeling field. The SPWL model is an extension of the well-known canonical piecewise-linear model proposed by Chua, which substitutes the abrupt absolute value function for a smoothing function (the logarithm of hyperbolic cosine), thus providing the model with several interesting properties. In particular, this function makes the model derivable, which is important to predict the intermodulation distortion behavior. Moreover, it allows one to control the smoothness of the global model by means of a single smoothing parameter. The parameters of the model are adapted to fit the nonlinear function, while the smoothing parameter is selected according to derivative constraints. The applied learning algorithm is a second-order gradient method. The proposed SPWL model is successfully applied to model a microwave HEMT transistor under optical illumination using real measurements. The model receives as input the bias voltages of the transistor, the instantaneous voltages, and the optical power and provides the drain to source current. The performance and computational burden of the SPWL model is compared with an empirical model and with some neural networks-based alternatives.

Confidence Intervals for Nonparametric Curve Estimates

Numerous nonparametric regression methods exist that yield consistent estimators of function curves. Often, one is also interested in constructing confidence intervals for the unknown function. When a function estimate is based on a single global smoothing parameter the resulting confidence intervals may hold their desired confidence level 1 – α on average but because bias in nonparametric estimation is not uniform, they do not hold the desired level uniformly at all design points. Most research in this area has focused on mean squared error properties of the estimator, for example MISE, itself a global measure. In addition, measures like MISE are one step removed from the practical issue of coverage probability. Recent work that focuses on coverage probability has considered only coverage in an average sense, ignoring the important issue of uniformity of coverage across the design space. To deal with this problem, a new estimator is developed which uses a local cross-validation criterion (LCV) to determine a separate smoothing parameter for each design point. The local smoothing parameters are then used to compute the point estimators of the regression curve and the corresponding pointwise confidence intervals. Incorporation of local information through the new method is shown, via Monte Carlo simulation, to yield more uniformly valid pointwise confidence intervals for nonparametric regression curves. Diagnostic plots are developed (Breakout Plots) to visually inspect the degree of uniformity of coverage of the confidence intervals. The approach, here applied to cubic smoothing splines, easily generalizes to many other nonparametric regression estimators. The improved curve estimation is not a solely theoretical improvement such as providing an estimator that has a faster EASE convergence rate but shows its worth empirically by yielding improved coverage probabilities through reliable pointwise confidence intervals.

Smoothing methods for complementarity problems in process engineering

AbstractComplementarity is central to all constrained optimization problems. However, direct enforcement of complementarity conditions is difficult because of the inherent nondifferentiability associated with them. Here, a class of smoothing methods for solving the complementarity problem by using a continuation algorithm to solve nonlinear equations is studied. The applicability of smoothing methods to approximate complicated nested derivative discontinuities is investigated using simple functions with a single smoothing parameter. In addition, an equation‐based formulation for solving the phase equilibrium problem with complementarity conditions is formulated. This approach can model the appearance and disappearance of phases directly in phase equilibrium problems. Moreover, it is shown how smoothing methods can be used to solve limiting distillation cases, such as dry and vaporless trays, modeled within an equation‐based formulation.

Smoothed Nonparametric Estimation of the Luminosity Function for Flux-limited Samples

A smoothed nonparametric method for estimating the intrinsic distribution of truncated sample data is described. This method determines both the cumulative and differential forms of the distribution, and it does not suffer from the numerical fluctuations associated with histogram distributions commonly used to describe such data. Variation of a single smoothing parameter provides a convenient means of error estimation and allows for differentiation between random fluctuations and subtle features which may exist in the parent distribution. The smoothing method is useful for obtaining distribution functions for flux or magnitude-limited astronomical data, and as an example, we calculate the smoothed luminosity function of a flux-limited sample of ∼500 quasars.

How easy is a given density to estimate?

In data analytic applications of density estimation one is usually interested in estimating the density over its support. However, common estimators such as the basic kernel estimator use a single smoothing parameter over the whole of the support. While this will be adequate for some densities there will be other densities that will be very difficult to estimate using this approach. The purpose of this article is to quantify how easy a particular density is to estimate using a global smoothing parameter. By considering the asymptotic expected L 1 error we obtain a scale invariant functional that is useful for measuring degree of estimation difficulty. Implications for the transformation kernel density estimators, which attempt to overcome the inadequacy of the basic kernel estimator, are also discussed.

Comparison of Smoothing Parameterizations in Bivariate Kernel Density Estimation

Abstract The basic kernel density estimator in one dimension has a single smoothing parameter, usually referred to as the bandwidth. For higher dimensions, however, there are several options for smoothing parameterization of the kernel estimator. For the bivariate case, there can be between one and three independent smoothing parameters in the estimator, which leads to a flexibility versus complexity trade-off when using this estimator in practice. In this article the performances of the different possible smoothing parameterizations are compared, using both the asymptotic and exact mean integrated squared error. Our results show that it is important to have independent smoothing parameters for each of the coordinate directions. Although this is enough for many situations, for densities with high amounts of curvature in directions different to those of the coordinate axes, substantial gains can be made by allowing the kernel mass to have arbitrary orientations. The “sphering” approaches to choosing this orientation are shown to be detrimental in general, however.

Error analysis for general multtvariate kernel estimators

Kernel estimators for d dimensional data are usually parametrized by either a single smoothing parameter, or d smoothing parameters corresponding to each of the coordinate directions. A generalization of each of these parameterizations is to use a d× d matrix which allows smoothing in arbitrary directions. We demonstrate that, at this level of generality, the usual error approximations and their numerical minimization can be done quite simply using matrix algebra. The minimization formulas have the practical importance that they can be applied to data-driven selection of the smoothing parameters using a ”plug-in approach. Particular attention is paid to the special case of kernel estimation of multivariate normal mixture densities where it is shown that the numerical evaluation and minimization of both asymptotic and exact mean integrated squared error can be set up in a matrix algebraic formulation which requires no numerical integration. This provides a flexible family of multivariate smoothing problems...

Smoothing noisy data using dynamic programming and generalized cross-validation.

Smoothing and differentiation of noisy data using spline functions requires the selection of an unknown smoothing parameter. The method of generalized cross-validation provides an excellent estimate of the smoothing parameter from the data itself even when the amount of noise associated with the data is unknown. In the present model only a single smoothing parameter must be obtained, but in a more general context the number may be larger. In an earlier work, smoothing of the data was accomplished by solving a minimization problem using the technique of dynamic programming. This paper shows how the computations required by generalized cross-validation can be performed as a simple extension of the dynamic programming formulas. The results of numerical experiments are also included.