Local Maximum Likelihood Estimation Research Articles

Hypothesis Testing for Class-Conditional Noise Using Local Maximum Likelihood

In supervised learning, automatically assessing the quality of the labels before any learning takes place remains an open research question. In certain particular cases, hypothesis testing procedures have been proposed to assess whether a given instance-label dataset is contaminated with class-conditional label noise, as opposed to uniform label noise. The existing theory builds on the asymptotic properties of the Maximum Likelihood Estimate for parametric logistic regression. However, the parametric assumptions on top of which these approaches are constructed are often too strong and unrealistic in practice. To alleviate this problem, in this paper we propose an alternative path by showing how similar procedures can be followed when the underlying model is a product of Local Maximum Likelihood Estimation that leads to more flexible nonparametric logistic regression models, which in turn are less susceptible to model misspecification. This different view allows for wider applicability of the tests by offering users access to a richer model class. Similarly to existing works, we assume we have access to anchor points which are provided by the users. We introduce the necessary ingredients for the adaptation of the hypothesis tests to the case of nonparametric logistic regression and empirically compare against the parametric approach presenting both synthetic and real-world case studies and discussing the advantages and limitations of the proposed approach.

Temporally Local Maximum Likelihood with Application to SIS Model.

The parametric estimators applied by rolling are commonly used for the analysis of time series with nonlinear patterns, including time varying parameters and local trends. This paper examines the properties of rolling estimators in the class of temporally local maximum likelihood (TLML) estimators. We consider the TLML estimators of (a) constant parameters, (b) stochastic, stationary parameters and (c) parameters with the ultra-long run (ULR) dynamics bridging the gap between the constant and stochastic parameters. We show that the weights used in the TLML estimators have a strong impact on the inference. For illustration, we provide a simulation study of the epidemiological susceptible-infected-susceptible (SIS) model, which explores the finite sample performance of TLML estimators of a time varying contagion parameter.

Generalized varying-coefficient additive model for locally stationary time series

ABSTRACT Recently, the varying-coefficient additive model (VCAM) has received much attention due to its flexibility and interpretability. In this paper, we consider its generalized form, called a generalized varying-coefficient additive model (GVCAM), which can model discrete and continuous responses for locally stationary covariates. Specifically, the conditional mean function is specified via a link function g and a nonparametric part given by a VCAM. We develop a pilot-estimation-based local linear maximum likelihood estimator (PEB-LLE) to approximate the univariate component function. We also show the proposed estimators are oracle efficient and present the pointwise asymptotic distribution as well as simultaneous confidence bands. Simulation studies investigating the finite-sample performance of the proposed methodology are considered, and a real data application illustrating our method is also given.

Global optimization for accurate and efficient parameter estimation in nanofiltration

One of the most well-established frameworks for modeling multicomponent transport in nanofiltration (NF) is the Donnan-Steric Pore Model with Dielectric Exclusion (DSPM-DE). Conventional DSPM-DE characterizes transport across NF membranes through four governing membrane parameters: (1) pore radius; (2) effective membrane thickness; (3) membrane charge density; and (4) the dielectric constant inside the membrane pores. The process for quantifying these parameters is typically sequential. First, neutral solute experiments are performed to determine pore radius and effective membrane thickness. Next, charged species experiments are conducted, and the data is used to regress out the remaining parameters. The resulting regressions are often performed using local search algorithms that can struggle to provide low residuals with robust fits. In addition, this two-step approach tends to: (1) require a substantial number of charged and uncharged solute experiments; and (2) introduce assumed relationships between pore size and water flux, such as the Hagen-Poiseuille equation, which may not be representative of transport through complex pore networks. To address these issues, we propose the use of metaheuristic global optimization techniques supplemented with gradient-free local search and maximum likelihood estimation to simultaneously regress all four membrane parameters directly from charged species experiments. We validate our approach against eight independent datasets across diverse input salinities, compositions, and membranes.

Regression Discontinuity for Binary Response and Local Maximum Likelihood Estimator to Extrapolate Treatment.

Regression discontinuity is popular in finding treatment/policy effects when the treatment is determined by a continuous variable crossing a cutoff. Typically, a local linear regression (LLR) estimator is used to find the effects. For binary response, however, LLR is not suitable in extrapolating the treatment, as in doubling/tripling the treatment dose/intensity. The reason is that doubling/tripling the LLR estimate can give a number out of the bound , despite that the effect should be a change in probability. We propose local maximum likelihood estimators which overcome these shortcomings, while giving almost the same estimates as the LLR estimator does for the original treatment. A simulation study and an empirical analysis for effects of an income subsidy program on religion demonstrate these points.

Modeling nonstationary extremes of storm severity: Comparing parametric and semiparametric inference

AbstractThis article compares the modeling of nonstationary extreme events using parametric models with local parametric and semiparametric approaches also motivated by extreme value theory. Specifically, three estimators are compared based on (a) (local) semiparametric moment estimation, (b) (local) maximum likelihood estimation, and (c) spline‐based maximum likelihood estimation. Inference is performed in a sequential manner, highlighting the synergies between the different approaches to estimating extreme quantiles, including the T‐year level and right endpoint when finite. We present a novel heuristic to estimate nonstationary extreme value threshold with exceedances varying on a circular domain, and hypothesis‐testing procedures for identifying max‐domain of attraction in the nonstationary setting. Bootstrapping is used to estimate nonstationary confidence bounds throughout. We provide step‐by‐step guides for estimation, and explore the different inference strategies in application to directional modeling of hindcast storm peak significant wave heights recorded in the North Sea.

Estimation of nonparametric ordinal logistic regression model using local maximum likelihood estimation

Ordinal logistic regression is a statistical method used to analyze the ordinal response variable with three or more categories and predictor variables that are categorical or continuous. The parametric models for ordinal response variable assume that the predictor is given by a linear form of covariates. In this study, the parametric models are extended to include smooth components based on nonparametric approach. The covariates are modeled as unspecified but smooth functions. Estimation is based on local maximum likelihood estimation (LMLE).

A local maximum likelihood model of crop yield distributions

AbstractIn this note, we propose a local maximum likelihood estimator for spatially‐dependent distributions. Our estimator adopts the Poisson regression approach for density ratio models and incorporates spatial smoothing via local regression. We also present a method of smoothing parameter selection. We illustrate this easy‐to‐implement estimator with an application to the estimation of corn yield distributions of Iowa counties. The usefulness of the approach is further demonstrated via an application to the estimation of crop insurance premium.

Guided trilateral filter and its application to ultrasound image despeckling

Speckle reduction has been a hot research topic in ultrasound imaging community. However, designing a method with better despeckling performance and lower algorithm complexity is still an active pursuit. In this paper, we propose a novel local spatial filtering framework named as the guided trilateral filter (GTF) to implement restoration of a noisy image with a well-fitting statistical distribution model. The GTF is derived as the local maximum likelihood estimation from the probabilities of the residuals between the filtered/guided image and the noisy image. The resulting iteration algorithm is essentially a local weighted filtering whose weights come from the guided image, including spatial distance, range discrepancy and statistical distribution. Choosing specific functions for these trilateral weights can direct to some classical local spatial filters, such as classical bilateral filter, robust bilateral filter, joint/cross bilateral filter, speckle reduction bilateral filter, etc. As a despeckling application of the GTF, we embed the Fisher-Tippett distribution model of ultrasound image into the GTF and thereby present the guided trilateral despeckling filter (GTDF). Experimental results on synthetic and real ultrasound images have demonstrated that the GTDF not only can achieve superior performance against state of the art methods, but also has fast and robust convergence and parameter setting insensitivity.

Non-separable models with high-dimensional data

This paper studies non-separable models with a continuous treatment when the dimension of the control variables is high and potentially larger than the effective sample size. We propose a three-step estimation procedure to estimate the average, quantile, and marginal treatment effects. In the first stage we estimate the conditional mean, distribution, and density objects by penalized local least squares, penalized local maximum likelihood estimation, and numerical differentiation, respectively, where control variables are selected via a localized method of L1-penalization at each value of the continuous treatment. In the second stage we estimate the average and marginal distribution of the potential outcome via the plug-in principle. In the third stage, we estimate the quantile and marginal treatment effects by inverting the estimated distribution function and using the local linear regression, respectively. We study the asymptotic properties of these estimators and propose a weighted-bootstrap method for inference. Using simulated and real datasets, we demonstrate that the proposed estimators perform well in finite samples.

Generalized random forests

We propose generalized random forests, a method for nonparametric statistical estimation based on random forests (Breiman [Mach. Learn. 45 (2001) 5–32]) that can be used to fit any quantity of interest identified as the solution to a set of local moment equations. Following the literature on local maximum likelihood estimation, our method considers a weighted set of nearby training examples; however, instead of using classical kernel weighting functions that are prone to a strong curse of dimensionality, we use an adaptive weighting function derived from a forest designed to express heterogeneity in the specified quantity of interest. We propose a flexible, computationally efficient algorithm for growing generalized random forests, develop a large sample theory for our method showing that our estimates are consistent and asymptotically Gaussian and provide an estimator for their asymptotic variance that enables valid confidence intervals. We use our approach to develop new methods for three statistical tasks: nonparametric quantile regression, conditional average partial effect estimation and heterogeneous treatment effect estimation via instrumental variables. A software implementation, grf for R and C++, is available from CRAN.

A Maximum Likelihood Approach for Depth Field Estimation Based on Epipolar Plane Images.

In this paper, a multi-resolution method for depth estimation from dense image arrays is presented. Recent progress in consumer electronics has enabled the development of low cost hand-held plenoptic cameras. In these systems, multiple views of a scene are captured in a single shot by means of a micro-lens array placed on the focal point of the first camera lens, in front of the imaging sensor. These views can be processed jointly to obtain accurate depth maps. In this contribution, to reduce the computational complexity associated to global optimization schemes based on match cost functions, we make a local estimate based on the maximization of the total log-likelihood spatial density aggregated along the epipolar lines corresponding to each view pair. This method includes the local maximum likelihood estimation of the depth field based on epipolar plane images. To face the potential accuracy losses associated to the ambiguity problem that arises in flat surface regions while preserving bandwidth in correspondence of the edges, we adopt a multi-resolution scheme. In practice, the depth map resolution is reduced in regions where maximizing the higher resolution functional is ill-conditioned. The main benefits of the proposed system are in a reduced computational complexity and a high accuracy of the estimated depth. Experimental results show that the proposed scheme represents a good tradeoff among accuracy, robustness, and discontinuities handling.

Statistical inference for the doubly stochastic self-exciting process

We introduce and show the existence of a Hawkes self-exciting point process with exponentially-decreasing kernel and where parameters are time-varying. The quantity of interest is defined as the integrated parameter $T^{-1}\int_{0}^{T}\theta_{t}^{*}\,dt$, where $\theta_{t}^{*}$ is the time-varying parameter, and we consider the high-frequency asymptotics. To estimate it naively, we chop the data into several blocks, compute the maximum likelihood estimator (MLE) on each block, and take the average of the local estimates. The asymptotic bias explodes asymptotically, thus we provide a non-naive estimator which is constructed as the naive one when applying a first-order bias reduction to the local MLE. We show the associated central limit theorem. Monte Carlo simulations show the importance of the bias correction and that the method performs well in finite sample, whereas the empirical study discusses the implementation in practice and documents the stochastic behavior of the parameters.

A Fast, Decentralized Covariance Selection-Based Approach to Detect Cyber Attacks in Smart Grids

Recent studies have shown that an attacker can compromise some of the power grid measurements to mislead the conventional state estimators (SEs), since the manipulated measurements can pass the SE residue tests. Statistical structure learning-based approaches have been recently introduced as a powerful tool to detect some of the most complicated cyber attacks. However, the expensive computational complexity of the learning process limits the applicability of these approaches for real time cyber attack detection. This paper proposes a fast and decentralized approach for cyber attack detection based on a maximum likelihood (ML) estimation which exploits the near chordal sparsity of power grids to establish an efficient framework to solve the associated ML estimation problem. The proposed detection method is then decomposed to several local ML estimation problems; this would ensure privacy and reduce the complexity of the underlying problem. The simulation studies validate the efficiency of the proposed method in detecting truly complicated stealthy false data injection attacks.

On estimation of insurance risk parameters by combining local regression and distribution fitting ideas

The problem of premium estimation is an essential part of the insurance mathematics. Often the problem is divided into two parts: estimation of claim number (or frequency) and the estimation of individual claim amounts (severities). In this paper, we will focus on the former. More precisely, we are looking for certain semiparametric dynamic regression type model to avoid the "price shock" issue of static classication. We apply locally the regression method, use local maximum likelihood estimation for the parameters of the model and cross-validation techniques to determine the optimal size of a neighborhood. A case study with real vehicle casco insurance dataset is included, the results obtained by proposed method are compared by the ones obtained by global regression and the classification and regression trees (C&RT) approach.

Estimation of Nuclear Counting by a Nonlinear Filter Based on a Hypothesis Test and a Double Exponential Smoothing

Online nuclear counting represents a challenge due to the stochastic nature of radioactivity. The counting data have to be filtered in order to provide a precise and accurate estimation of the count rate, while ensuring a response time compatible with the application in view. An innovative filter is presented in this paper to address this issue. The filter is nonlinear and based on a Centered Significance Test (CST) providing a local maximum likelihood estimation of the signal. This nonlinear approach allows enables to smooth the counting signal while maintaining a fast response when brutal change in activity occurs. The filter is then improved by the implementation of a Brown’s double Exponential Smoothing (BES). The filter has been validated and compared to other state-of-the-art smoothing filters. The CST* filter shows a significant improvement compared to all tested smoothing filters.

Nonparametric incidence estimation and bootstrap bandwidth selection in mixture cure models

A completely nonparametric method for the estimation of mixture cure models is proposed. A nonparametric estimator of the incidence is extensively studied and a nonparametric estimator of the latency is presented. These estimators, which are based on the Beran estimator of the conditional survival function, are proved to be the local maximum likelihood estimators. An i.i.d. representation is obtained for the nonparametric incidence estimator. As a consequence, an asymptotically optimal bandwidth is found. Moreover, a bootstrap bandwidth selection method for the nonparametric incidence estimator is proposed. The introduced nonparametric estimators are compared with existing semiparametric approaches in a simulation study, in which the performance of the bootstrap bandwidth selector is also assessed. Finally, the method is applied to a database of colorectal cancer from the University Hospital of A Coruña (CHUAC).

Time-varying copula models for financial time series

AbstractWe perform an analysis of the potential time inhomogeneity in the dependence between multiple financial time series. To this end, we use the framework of copula theory and tackle the question of whether dependencies in such a case can be assumed constant throughout time or rather have to be modeled in a time-inhomogeneous way. We focus on parametric copula models and suitable inference techniques in the context of a special copula-based multivariate time series model. A recent result due to Chanet al.(2009) is used to derive the joint limiting distribution of local maximum-likelihood estimators on overlapping samples. By restricting the overlap to be fixed, we establish the limiting law of the maximum of the estimator series. Based on the limiting distributions, we develop statistical homogeneity tests, and investigate their local power properties. A Monte Carlo simulation study demonstrates that bootstrapped variance estimates are needed in finite samples. Empirical analyses on real-world financial data finally confirm that time-varying parameters are an exception rather than the rule.

Implementation of a nonlinear filter for online nuclear counting

Nuclear counting is a challenging task for nuclear instrumentation because of the stochastic nature of radioactivity. Event counting has to be processed and filtered to determine a stable count rate value and perform variation monitoring of the measured event. An innovative approach for nuclear counting is presented in this study, improving response time and maintaining count rate stability. Some nonlinear filters providing a local maximum likelihood estimation of the signal have been recently developed, which have been tested and compared with conventional linear filters. A nonlinear filter thus developed shows significant performance in terms of response time and measurement precision. The filter also presents the specificity of easy embedment into digital signal processor (DSP) electronics based on field-programmable gate arrays (FPGA) or microcontrollers, compatible with real-time requirements. © 2001 Elsevier Science. All rights reserved.

Model selection and structure specification in ultra-high dimensional generalised semi-varying coefficient models

In this paper, we study the model selection and structure specification for the generalised semi-varying coefficient models (GSVCMs), where the number of potential covariates is allowed to be larger than the sample size. We first propose a penalised likelihood method with the LASSO penalty function to obtain the preliminary estimates of the functional coefficients. Then, using the quadratic approximation for the local log-likelihood function and the adaptive group LASSO penalty (or the local linear approximation of the group SCAD penalty) with the help of the preliminary estimation of the functional coefficients, we introduce a novel penalised weighted least squares procedure to select the significant covariates and identify the constant coefficients among the coefficients of the selected covariates, which could thus specify the semiparametric modelling structure. The developed model selection and structure specification approach not only inherits many nice statistical properties from the local maximum likelihood estimation and nonconcave penalised likelihood method, but also computationally attractive thanks to the computational algorithm that is proposed to implement our method. Under some mild conditions, we establish the asymptotic properties for the proposed model selection and estimation procedure such as the sparsity and oracle property. We also conduct simulation studies to examine the finite sample performance of the proposed method, and finally apply the method to analyse a real data set, which leads to some interesting findings.