Minimax Efficiency Research Articles

Efficient parametric estimation for a signal-plus-noise Gaussian model from discrete time observations

This paper deals with the parametric inference for integrated continuous time signals embedded in an additive Gaussian noise and observed at deterministic discrete instants which are not necessarily equidistant. The unknown parameter ismultidimensional and compounded of a signal-of-interest parameter and a variance parameter of the noise.We state the consistency and the minimax efficiency of the maximum likelihood estimator and of the Bayesian estimator when the time of observation tends to infinity and the delays between two consecutive observations tend to 0 or are only bounded. The class of signals in consideration contains among others, almost periodic signals and also non-continuous periodic signals. However the problem of frequency estimation is not considered here. Furthermore, in this paper the signal-plus-noise discretely observed in time model is considered as a particular case of a more general model of independent Gaussian observations forming a triangular array.

Research on Different Heuristics for Minimax Algorithm Insight from Connect-4 Game

Minimax algorithm and machine learning technologies have been studied for decades to reach an ideal optimization in game areas such as chess and backgammon. In these fields, several generations try to optimize the code for pruning and effectiveness of evaluation function. Thus, there are well-armed algorithms to deal with various sophisticated situations in gaming occasion. However, as a traditional zero-sum game, Connect-4 receives less attention compared with the other members of its zero-sum family using traditional minimax algorithm. In recent years, new generation of heuristics is created to address this problem based on research conclusions, expertise and gaming experiences. However, this paper mainly introduced a self-developed heuristics supported by well-demonstrated result from researches and our own experiences which fighting against the available version of Connect-4 system online. While most previous works focused on winning algorithms and knowledge based approaches, we complement these works with analysis of heuristics. We have conducted three experiments on the relationship among functionality, depth of searching and number of features and doing contrastive test with sample online. Different from the sample based on summarized experience and generalized features, our heuristics have a basic concentration on detailed connection between pieces on board. By analysing the winning percentages when our version fights against the online sample with different searching depths, we find that our heuristics with minimax algorithm is perfect on the early stages of the zero-sum game playing. Because some nodes in the game tree have no influence on the final decision of minimax algorithm, we use alpha-beta pruning to decrease the number of meaningless node which greatly increases the minimax efficiency. During the contrastive experiment with the online sample, this paper also verifies basic characters of the minimax algorithm including depths and quantity of features. According to the experiment, these two characters can both effect the decision for each step and none of them can be absolutely in charge. Besides, we also explore some potential future issues in Connect-4 game optimization such as precise adjustment on heuristic values and inefficiency pruning on the search tree.

Local linear smoothers using inverse Gaussian regression

Local linear fitting has been widely used for estimating the univariate regression function, it has numerous fantastic properties like minimax efficiency and boundary correction. The asymmetric kernel functions match the support of the explanatory variables, and we discover the inverse Gaussian (IG) kernel function is identical to the normal kernel with variable bandwidth. Based on these, this paper proposes a new local linear estimation procedure with the IG kernel when the covariates are supported on \((0,\infty )\). Asymptotic theories of the proposed estimator are systematically studied, including the conditional mean squared error, the asymptotic normality and the uniform almost sure convergence. A simulation study and a data example indicate that the proposed estimator works efficiently.

Linear and Conic Programming Estimators in High Dimensional Errors-in-variables Models

Summary We consider the linear regression model with observation error in the design. In this setting, we allow the number of covariates to be much larger than the sample size. Several new estimation methods have been recently introduced for this model. Indeed, the standard lasso estimator or Dantzig selector turns out to become unreliable when only noisy regressors are available, which is quite common in practice. In this work, we propose and analyse a new estimator for the errors-in-variables model. Under suitable sparsity assumptions, we show that this estimator attains the minimax efficiency bound. Importantly, this estimator can be written as a second-order cone programming minimization problem which can be solved numerically in polynomial time. Finally, we show that the procedure introduced by Rosenbaum and Tsybakov, which is almost optimal in a minimax sense, can be efficiently computed by a single linear programming problem despite non-convexities.

Efficient shrinkage in parametric models

This paper introduces shrinkage for general parametric models. We show how to shrink maximum likelihood estimators towards parameter subspaces defined by general nonlinear restrictions. We derive the asymptotic distribution and risk of our shrinkage estimator using a local asymptotic framework. We show that if the shrinkage dimension exceeds two, the asymptotic risk of the shrinkage estimator is strictly less than that of the maximum likelihood estimator (MLE). This reduction holds globally in the parameter space. We show that the reduction in asymptotic risk is substantial, even for moderately large values of the parameters.We also provide a new high-dimensional large sample local minimax efficiency bound. The bound is the lowest possible asymptotic risk, uniformly in a local region of the parameter space. Local minimax bounds are a stronger efficiency characterization than global minimax bounds. We show that our shrinkage estimator asymptotically achieves this local asymptotic minimax bound, while the MLE does not. Thus the shrinkage estimator, unlike the MLE, is locally minimax efficient.This theory is a combination and extension of standard asymptotic efficiency theory (Hájek, 1972) and local minimax efficiency theory for Gaussian models (Pinsker, 1980).

Local polynomial quasi-likelihood regression with truncated and dependent data

The local polynomial quasi-likelihood estimation has several good statistical properties such as high minimax efficiency and adaptation of edge effects. In this paper, we construct a local quasi-likelihood regression estimator for a left truncated model, and establish the asymptotic normality of the proposed estimator when the observations form a stationary and α-mixing sequence, such that the corresponding result of Fan et al. [Local polynomial kernel regression for generalized linear models and quasilikelihood functions, J. Amer. Statist. Assoc. 90 (1995), pp. 141–150] is extended from the independent and complete data to the dependent and truncated one. Finite sample behaviour of the estimator is investigated via simulations too.

LOCAL POLYNOMIAL QUASI-LIKELIHOOD REGRESSION ON RANDOM FIELDS

Summary In this paper, local quasi-likelihood regression is considered for stationary random fields of dependent variables. In the case of independent data, local polynomial quasi-likelihood regression is known to have several appealing features such as minimax efficiency, design adaptivity and good boundary behaviour. These properties are shown to carry over to the case of random fields. The asymptotic normality of the regression estimator is established and explicit formulae for its asymptotic bias and variance are derived for strongly mixing stationary random fields. The extension to multi-dimensional covariates is also provided in full generality. Moreover, evaluation of the finite sample performance is made through a simulation study.

Nonparametric estimation of mixing densities for discrete distributions

By a mixture density is meant a density of the form πμ(⋅)=∫πθ(⋅)×μ(dθ), where (πθ)θ∈Θ is a family of probability densities and μ is a probability measure on Θ. We consider the problem of identifying the unknown part of this model, the mixing distribution μ, from a finite sample of independent observations from πμ. Assuming that the mixing distribution has a density function, we wish to estimate this density within appropriate function classes. A general approach is proposed and its scope of application is investigated in the case of discrete distributions. Mixtures of power series distributions are more specifically studied. Standard methods for density estimation, such as kernel estimators, are available in this context, and it has been shown that these methods are rate optimal or almost rate optimal in balls of various smoothness spaces. For instance, these results apply to mixtures of the Poisson distribution parameterized by its mean. Estimators based on orthogonal polynomial sequences have also been proposed and shown to achieve similar rates. The general approach of this paper extends and simplifies such results. For instance, it allows us to prove asymptotic minimax efficiency over certain smoothness classes of the above-mentioned polynomial estimator in the Poisson case. We also study discrete location mixtures, or discrete deconvolution, and mixtures of discrete uniform distributions.

On Fractal Distribution Function Estimation and Applications

In this paper we review some recent results concerning the approximations of distribution functions and measures on [0,1] based on iterated function systems. The two different approaches available in the literature are considered and their relation are investigated in the statistical perspective. In the second part of the paper we propose a new class of estimators for the distribution function and the related characteristic and density functions. Glivenko-Cantelli, LIL properties and local asymptotic minimax efficiency are established for some of the proposed estimators. Via Monte Carlo analysis we show that, for small sample sizes, the proposed estimator can be as efficient or even better than the empirical distribution function and the kernel density estimator respectively. This paper is to be considered as a first attempt in the construction of new class of estimators based on fractal objects. Pontential applications to survival analysis with random censoring are proposed at the end of the paper.

Linear minimax efficiency of local polynomial regression smoothers

This paper proves that local polynomial regression smoothers achieve linear minimax efficiency over a class of functions, generalizing a result of Fan (1993) for local linear smoothers and proving that a conjecture of Fan and Gijbels (1996) is true. Consequences are also illustrated.

Weighted Nadaraya–Watson regression estimation

In this article, we study nonparametric estimation of regression function by using the weighted Nadaraya–Watson approach. We establish the asymptotic normality and weak consistency of the resulting estimator for α-mixing time series at both boundary and interior points, and we show that the weighted Nadaraya–Watson estimator not only preserves the bias, variance, and more importantly, automatic good boundary behavior properties of local linear estimator, but also makes computation fast. Furthermore, the asymptotic minimax efficiency is discussed. Finally, comparisons between weighted Nadaraya–Watson approach and local linear fitting are given.

Local asymptotic normality and asymptotical minimax efficiency of the MLE under random censorship

Here we study the problems of local asymptotic normality of the parametric family of distributions and asymptotic minimax efficient estimators when the observations are subject to right censoring. Local asymptotic normality will be established under some mild regularity conditions. A lower bound for local asymptotic minimax risk is given with respect to a bowl-shaped loss function, and furthermore a necessary and sufficient condition is given in order to achieve this lower bound. Finally, we show that this lower bound can be attained by the maximum likelihood estimator in the censored case and hence it is local asymptotic minimax efficient.

Local Polynomial Regression: Optimal Kernels and Asymptotic Minimax Efficiency

We consider local polynomial fitting for estimating a regression function and its derivatives nonparametrically. This method possesses many nice features, among which automatic adaptation to the boundary and adaptation to various designs. A first contribution of this paper is the derivation of an optimal kernel for local polynomial regression, revealing that there is a universal optimal weighting scheme. Fan (1993, Ann. Statist., 21, 196-216) showed that the univariate local linear regression estimator is the best linear smoother, meaning that it attains the asymptotic linear minimax risk. Moreover, this smoother has high minimax risk. We show that this property also holds for the multivariate local linear regression estimator. In the univariate case we investigate minimax efficiency of local polynomial regression estimators, and find that the asymptotic minimax efficiency for commonly-used orders of fit is 100% among the class of all linear smoothers. Further, we quantify the loss in efficiency when going beyond this class.

Minimax detection of a signal in ?p metrics

The problem of the minimax detection of a signal is considered in a Gaussian white noise of intensity e → 0. The set of the signals represents an ellipsoid in the lp-metric with lengths of semiaxes ak−1 → 0 as k → ∞, from which a ball of radius ρe → 0 as e → 0 in the lp-metric has been removed. Asymptotically minimax tests have been constructed for the cases 1 < p ≤ ∞ and asymptotically sharp estimates of the minimax efficiency have been obtained.

Locally minimax efficiency of nonparametric estimates of square-integrable densities

Locally minimax efficiency of nonparametric estimates of square-integrable densities

ASYMPTOTIC ABSOLUTE EFFICIENCY OF PROJECTION TESTS WITH AN APPLICATION TO N‐RANKING TESTS

AbstractA class of χ2‐type goodness‐of‐fit tests for the hypotheses of equal cell probabilities is introduced and the quality of the tests under arbitrary model restrictions is compared, where a suitable efficiency concept is used. As a special case, several nonparametric tests on treatment effects in the randomized block design can be considered. Some well known cases, such as the Anderson‐, Friedman‐ and the Page‐test, turn out to possess a certain minimax efficiency property under reasonable model assumptions. Finally a direct comparison between the Anderson‐ and Friedman‐tests is given.