Algorithm For Nonparametric Estimation Research Articles

A PRticle filter algorithm for nonparametric estimation of multivariate mixing distributions

Predictive recursion (PR) is a fast, recursive algorithm that gives a smooth estimate of the mixing distribution under the general mixture model. However, the PR algorithm requires evaluation of a normalizing constant at each iteration. When the support of the mixing distribution is of relatively low dimension, this is not a problem since quadrature methods can be used and are very efficient. But when the support is of higher dimension, quadrature methods are inefficient and there is no obvious Monte Carlo-based alternative. In this paper, we propose a new strategy, which we refer to as PRticle filter, wherein we augment the basic PR algorithm with a filtering mechanism that adaptively reweights an initial set of particles along the updating sequence which are used to obtain Monte Carlo approximations of the normalizing constants. Convergence properties of the PRticle filter approximation are established and its empirical accuracy is demonstrated with simulation studies and a marked spatial point process data analysis.

Revisiting consistency of a recursive estimator of mixing distributions

Estimation of the mixing distribution under a general mixture model is a very difficult problem, especially when the mixing distribution is assumed to have a density. Predictive recursion (PR) is a fast, recursive algorithm for nonparametric estimation of a mixing distribution/density in general mixture models. However, the existing PR consistency results make rather strong assumptions, some of which fail for a class of mixture models relevant for monotone density estimation, namely, scale mixtures of uniform kernels. In this paper, we develop new consistency results for PR under weaker conditions. Armed with this new theory, we prove that PR is consistent for the scale mixture of uniforms problem, and we show that the corresponding PR mixture density estimator has very good practical performance compared to several existing methods for monotone density estimation.

An Algorithm for Nonparametric Estimation of a Multivariate Mixing Distribution with Applications to Population Pharmacokinetics.

Population pharmacokinetic (PK) modeling has become a cornerstone of drug development and optimal patient dosing. This approach offers great benefits for datasets with sparse sampling, such as in pediatric patients, and can describe between-patient variability. While most current algorithms assume normal or log-normal distributions for PK parameters, we present a mathematically consistent nonparametric maximum likelihood (NPML) method for estimating multivariate mixing distributions without any assumption about the shape of the distribution. This approach can handle distributions with any shape for all PK parameters. It is shown in convexity theory that the NPML estimator is discrete, meaning that it has finite number of points with nonzero probability. In fact, there are at most N points where N is the number of observed subjects. The original infinite NPML problem then becomes the finite dimensional problem of finding the location and probability of the support points. In the simplest case, each point essentially represents the set of PK parameters for one patient. The probability of the points is found by a primal-dual interior-point method; the location of the support points is found by an adaptive grid method. Our method is able to handle high-dimensional and complex multivariate mixture models. An important application is discussed for the problem of population pharmacokinetics and a nontrivial example is treated. Our algorithm has been successfully applied in hundreds of published pharmacometric studies. In addition to population pharmacokinetics, this research also applies to empirical Bayes estimation and many other areas of applied mathematics. Thereby, this approach presents an important addition to the pharmacometric toolbox for drug development and optimal patient dosing.

An algorithm for non-parametric estimation in state–space models

State–space models are ubiquitous in the statistical literature since they provide a flexible and interpretable framework for analyzing many time series. In most practical applications, the state–space model is specified through a parametric model. However, the specification of such a parametric model may require an important modeling effort or may lead to models which are not flexible enough to reproduce all the complexity of the phenomenon of interest. In such situations, an appealing alternative consists in inferring the state–space model directly from the data using a non-parametric framework. The recent developments of powerful simulation techniques have permitted to improve the statistical inference for parametric state–space models. It is proposed to combine two of these techniques, namely the Stochastic Expectation–Maximization (SEM) algorithm and Sequential Monte Carlo (SMC) approaches, for non-parametric estimation in state–space models. The performance of the proposed algorithm is assessed though simulations on toy models and an application to environmental data is discussed.

The discrete Chebyshev algorithm for nonparametric estimation of autocorrelation function of electrochemical random time series

The discrete Chebyshev algorithm for nonparametric estimation of autocorrelation function of electrochemical random time series is presented. The algorithm is resistant to a trend of electrochemical noise. The discrete Chebyshev algorithm is tested using the model electrochemical noise corresponding to the equilibrium two-element Voigt circuit. The algorithm is used for nonparametric estimation of autocorrelation function of corrosion noise and electronic noise of measuring instrument.

A PV power interval forecasting based on seasonal model and nonparametric estimation algorithm

With the continuous increase of grid-connected photovoltaic (PV), high-precision PV power prediction is increasingly important. Extant deterministic forecasting methods do not facilitate fully effective dispatching decisions or power grid risk analysis. Furthermore, the single model also has insufficient generalization ability and unstable forecasting performance in PV power forecasting. This paper proposes an alternative multi-model PV power interval forecasting method which takes into account the seasonal distribution of power fluctuation characteristics. PV output power, absolute power deviation, and relative variation rate are first analyzed for the seasonal distribution characteristics of PV output as they fluctuate over time. Seasonal multi-models for deterministic forecasting of PV power are then built based on an extreme learning machine (ELM). Deterministic forecasting error is fitted by kernel density estimation to complete the PV power interval forecast. The effectiveness of the proposed method is validated by comparison against other methods.

An extended instrument variable approach for nonparametric LPV model identification

Abstract Linear parameter varying models (LPV) have proven to be effective to describe non–linearities and time–varying behaviors. In this work, a new non-parametric estimation algorithm for state-space LPV models based on support vector machines is presented. This technique allows the functional dependence between the model coefficients and the scheduling signal to be “learned” from the input and output data. The proposed algorithm is formulated in the context of instrumental (IV) estimators, in order to obtain consistent estimates for general noise conditions. The method is based on a canonical state–space representation and admits a predictor form that has shown to be suitable for system identification, as it leads to a convenient regression form. In addition, this predictor has an inherent filtering feature. In the context of vector support machines, such filtering mechanism leads to two–dimensional data processing, which can be used to decrease the variance of estimates due to noisy data. The performance of the proposed approach is evaluated from simulated data subject to different noise scenarios. The technique was able to reduce the error due to the variance of the estimator in most of the analyzed scenarios.

An EM algorithm for nonparametric estimation of the cumulative incidence function from repeated imperfect test results.

In screening and surveillance studies, event times are interval censored. Besides, screening tests are imperfect so that the interval at which an event takes place may be uncertain. We describe an expectation-maximization algorithm to find the nonparametric maximum likelihood estimator of the cumulative incidence function of an event based on screening test data. Our algorithm has a closed-form solution for the combined expectation and maximization step and is computationally undemanding. A simulation study indicated that the bias of the estimator tends to zero for large sample size, and its mean squared error is in general lower than the mean squared error of the estimator that assumes the screening test is perfect. We apply the algorithm to follow-up data from women treated for cervical precancer. Copyright © 2017 John Wiley & Sons, Ltd.

Online Bayesian Inference in Some Time-Frequency Representations of Non-Stationary Processes

The use of Bayesian inference in the inference of time-frequency representations has, thus far, been limited to offline analysis of signals, using a smoothing spline based model of the time-frequency plane. In this paper we introduce a new framework that allows the routine use of Bayesian inference for online estimation of the time-varying spectral density of a locally stationary Gaussian process. The core of our approach is the use of a likelihood inspired by a local Whittle approximation. This choice, along with the use of a recursive algorithm for non-parametric estimation of the local spectral density, permits the use of a particle filter for estimating the time-varying spectral density online. We provide demonstrations of the algorithm through tracking chirps and the analysis of musical data.

Regularized spectrum estimation using stable spline kernels

This paper presents a new regularized kernel-based approach for the estimation of the second order moments of stationary stochastic processes. The proposed estimator is defined by a Tikhonov-type variational problem. It contains few unknown parameters which can be estimated by cross validation solving a sequence of problems whose computational complexity scales linearly with the number of noisy moments (derived from the samples of the process). The correlation functions are assumed to be summable and the hypothesis space is a reproducing kernel Hilbert space induced by the recently introduced stable spline kernel. In this way, information on the decay to zero of the functions to be reconstructed is incorporated in the estimation process. An application to the identification of transfer functions in the case of white noise as input is also presented. Numerical simulations show that the proposed method compares favorably with respect to standard nonparametric estimation algorithms that exploit an oracle-type tuning of the parameters.

A comprehensive approach to optimal software rejuvenation

Software aging is caused by resource exhaustion and can lead to progressive performance degradation or result in a crash. We develop experiments that simulate an on-line bookstore application, using the standard configuration of TPC-W benchmark. We study application failures due to memory leaks, using the accelerated life testing (ALT). ALT significantly reduces the time needed to estimate the time to failure at normal level. We then select the Weibull time to failure distribution at normal level, to be used in a semi-Markov model so as to optimize the software rejuvenation trigger interval. Then we derive the optimal rejuvenation schedule interval by fixed point iteration and by an alternative non-parametric estimation algorithm. Finally, we develop a simulation model using importance sampling (IS) to cross validate the ALT experimental results and the semi-Markov model, and also we apply the non-parametric method to cross validate the optimized trigger intervals by comparing the availabilities obtained from the semi-Markov model and those from IS simulation using the non-parametric method.

A pool-adjacent-violators type algorithm for non-parametric estimation of current status data with dependent censoring

A likelihood based approach to obtaining non-parametric estimates of the failure time distribution is developed for the copula based model of Wang et al. (Lifetime Data Anal 18:434-445, 2012) for current status data under dependent observation. Maximization of the likelihood involves a generalized pool-adjacent violators algorithm. The estimator coincides with the standard non-parametric maximum likelihood estimate under an independence model. Confidence intervals for the estimator are constructed based on a smoothed bootstrap. It is also shown that the non-parametric failure distribution is only identifiable if the copula linking the observation and failure time distributions is fully-specified. The method is illustrated on a previously analyzed tumorigenicity dataset.

Convergence rate for predictive recursion estimation of finite mixtures

Predictive recursion (PR) is a fast stochastic algorithm for nonparametric estimation of mixing distributions in mixture models. It is known that the PR estimates of both the mixing and mixture densities are consistent under fairly mild conditions, but currently very little is known about the rate of convergence. Here I first investigate asymptotic convergence properties of the PR estimate under model misspecification in the special case of finite mixtures with known support. Tools from stochastic approximation theory are used to prove that the PR estimates converge, to the best Kullback–Leibler approximation, at a nearly root- n rate. When the support is unknown, PR can be used to construct an objective function which, when optimized, yields an estimate of the support. I apply the known-support results to derive a rate of convergence for this modified PR estimate in the unknown support case, which compares favorably to known optimal rates.

Kernel-smoothed conditional quantiles of correlated bivariate discrete data

Socio-economic variables are often measured on a discrete scale or rounded to protect confidentiality. Nevertheless, when exploring the effect of a relevant covariate on the outcome distribution of a discrete response variable, virtually all common quantile regression methods require the distribution of the covariate to be continuous. This paper departs from this basic requirement by presenting an algorithm for nonparametric estimation of conditional quantiles when both the response variable and the covariate are discrete. Moreover, we allow the variables of interest to be pairwise correlated. For computational efficiency, we aggregate the data into smaller subsets by a binning operation, and make inference on the resulting prebinned data. Specifically, we propose two kernel-based binned conditional quantile estimators, one for untransformed discrete response data and one for rank-transformed response data. We establish asymptotic properties of both estimators. A practical procedure for jointly selecting band- and binwidth parameters is also presented. Simulation results show excellent estimation accuracy in terms of bias, mean squared error, and confidence interval coverage. Typically prebinning the data leads to considerable computational savings when large datasets are under study, as compared to direct (un)conditional quantile kernel estimation of multivariate data. With this in mind, we illustrate the proposed methodology with an application to a large dataset concerning US hospital patients with congestive heart failure.

Semiparametric inference in mixture models with predictive recursion marginal likelihood

Predictive recursion is an accurate and computationally efficient algorithm for nonparametric estimation of mixing densities in mixture models. In semiparametric mixture models, however, the algorithm fails to account for any uncertainty in the additional unknown structural parameter. As an alternative to existing profile likelihood methods, we treat predictive recursion as a filter approximation to fitting a fully Bayes model, whereby an approximate marginal likelihood of the structural parameter emerges and can be used for inference. We call this the predictive recursion marginal likelihood. Convergence properties of predictive recursion under model mis-specification also lead to an attractive construction of this new procedure. We show pointwise convergence of a normalized version of this marginal likelihood function. Simulations compare the performance of this new marginal likelihood approach that of existing profile likelihood methods as well as Dirichlet process mixtures in density estimation. Mixed-effects models and an empirical Bayes multiple testing application in time series analysis are also considered.

선형-원형배열 안테나에 따른 MIMO의 DOA 추정과 적응 빔성형 분석

Abstract In this paper, DOA(direction of arrival) of multiple incident signals received from linear array antenna and circular array antenna, which is based on nonparametric estimation algorithm, and adaptive beam-forming algorithm are studied and analyzed. In nonparametric estimation algorithm, we minimize a regularized objective function for finding a estimate of the signal energy as a function of angle, using nonquadratic norm which leads to supper resolution and noise suppression. And then, DOA is estimated by the signal and noise spatial steering vector, and adaptive beam-forming pattern is improved by weight vectors obtained from the spatial vector. Especially, the discrimination ability of DOA and the adaptive beam-forming ability according to antenna array methods and the number of array elements are compared and considered. Key Words : linear array, Circular array, DOA, Nonparametric Estimation algorithm, Adaptive Beam-forming pattern, Weight vector, Super resolution 본 논문은 2011년도 대진대학교 학술연구지원비로 수행되었음.

Adaptive AM–FM Signal Decomposition With Application to Speech Analysis

In this paper, we present an iterative method for the accurate estimation of amplitude and frequency modulations (AM-FM) in time-varying multi-component quasi-periodic signals such as voiced speech. Based on a deterministic plus noise representation of speech initially suggested by Laroche (“HNM: A simple, efficient harmonic plus noise model for speech,” Proc. WASPAA, Oct., 1993, pp. 169-172), and focusing on the deterministic representation, we reveal the properties of the model showing that such a representation is equivalent to a time-varying quasi-harmonic representation of voiced speech. Next, we show how this representation can be used for the estimation of amplitude and frequency modulations and provide the conditions under which such an estimation is valid. Finally, we suggest an adaptive algorithm for nonparametric estimation of AM-FM components in voiced speech. Based on the estimated amplitude and frequency components, a high-resolution time-frequency representation is obtained. The suggested approach was evaluated on synthetic AM-FM signals, while using the estimated AM-FM information, speech signal reconstruction was performed, resulting in a high signal-to-reconstruction error ratio (around 30 dB).

Kernel-Smoothed Conditional Quantiles of Correlated Bivariate Discrete Data

Often socio-economic variables are measured on a discrete scale or rounded to protect confidentiality. Nevertheless, when exploring the effect of a relevant covariate on the whole outcome distribution of a discrete response variable, virtually all common quantile regression methods require the distribution of the covariate to be continuous. This paper departs from this basic requirement by presenting an algorithm for nonparametric estimation of conditional quantiles when both the response variable and the covariate are discretely distributed. Moreover, we allow the variables of interest to be pairwise correlated. For computational efficiency, we aggregate the data into smaller subsets by a binning operation, and make inference on the resulting prebinned data. Specifically, we propose two kernel-based binned conditional quantile estimators, one for untransformed discrete response data and one for rank-transformed response data. We establish asymptotic properties of both estimators. A practical procedure for jointly selecting band- and binwidth parameters is also presented. Simulation results show excellent estimation accuracy in terms of bias, mean squared error, and confidence interval coverage. Typically prebinning the data leads to considerable computational savings when large datasets are under study, as compared to direct (un)conditional quantile kernel estimation of multivariate data. With this in mind, we illustrate the proposed methodology with an application to a large real dataset concerning US hospital patients with congestive heart failure.

이동통신 환경에서 다중신호의 DOA 추정과 적응 빔성형

The DOA(direction of arrival), which is based on parametric and nonparametric estimation algorithm, and adaptive beamforming algorithm for mobile communication environments are researched and analyzed. In parametric estimation algorithm, eigenvalues of the signal component and the noise component are obtained from correlation matrix of received signal by array antenna and power spectrum of the received signal is discriminated from them. Otherwise, in nonparametric estimation algorithm, we minimize a regularized objective function for finding a estimate of the signal energy as a function of angle, using nonquadratic norm which leads to supper resolution and noise suppression. And then, DOA is estimated by the signal and noise spatial steering vector, and adaptive beam-forming pattern is improved by weight vectors obtained from the spatial vector. Therefore, the improved directional estimation algorithm with regularizing sparsity constraints offers super-resolution and noise suppression compared to other algorithms.

Nonparametric inference and uniqueness for periodically observed progressive disease models

In many studies examining the progression of HIV and other chronic diseases, subjects are periodically monitored to assess their progression through disease states. This gives rise to a specific type of panel data which have been termed "chain-of-events data"; e.g. data that result from periodic observation of a progressive disease process whose states occur in a prescribed order and where state transitions are not observable. Using a discrete time semi-Markov model, we develop an algorithm for nonparametric estimation of the distribution functions of sojourn times in a J state progressive disease model. Issues of uniqueness for chain-of-events data are not well-understood. Thus, a main goal of this paper is to determine the uniqueness of the nonparametric estimators of the distribution functions of sojourn times within states. We develop sufficient conditions for uniqueness of the nonparametric maximum likelihood estimator, including situations where some but not all of its components are unique. We illustrate the methods with three examples.