Maximum Likelihood Type Estimators Research Articles

Tests of Statistical Methods for Estimating Galaxy Luminosity Function and Applications to the Hubble Deep Field

We study statistical methods for the estimation of the luminosity function (LF) of galaxies. We focus on four nonparametric estimators: the 1/Vmax estimator, the stepwise maximum-likelihood estimator, Chołoniewski's estimator, and the improved Lynden-Bell estimator. The performance of the 1/Vmax estimator has recently been questioned, especially for the faint-end estimation of the LF. We improve these estimators for studies of the distant universe, and examine their performances for various classes of functional forms by Monte Carlo simulations. We also apply these estimation methods to the mock Two-Degree Field (2dF) redshift survey catalog prepared by Cole et al. We find that the 1/Vmax estimator yields a completely unbiased result if there is no inhomogeneity, but is not robust against clusters or voids. This is consistent with the well-known results, and we do not confirm the bias trend of 1/Vmax claimed by Willmer in the case of a homogeneous sample. We also find that the other three maximum-likelihood type estimators are quite robust and give consistent results with each other. In practice, we recommend Chołoniewski's estimator for two reasons: (1) it simultaneously provides the shape and normalization of the LF; and (2) it is the fastest among these four estimators, because of its algorithmic simplicity. We then analyze the photometric redshift data of the Hubble Deep Field prepared by Fernández-Soto et al. using the above four methods. We also derive the luminosity density, ρL, at the B and I bands. Our B-band estimation is roughly consistent with that of Sawicki, Lin, & Yee, but it is a few times lower at 2.0 < z < 3.0. The evolution of ρL(I) is found to be less prominent.

Cramer-Rao bounds and maximum likelihood estimation for random amplitude phase-modulated signals

The problem of estimating the phase parameters of a phase-modulated signal in the presence of colored multiplicative noise (random amplitude modulation) and additive white noise (both Gaussian) is addressed. Closed-form expressions for the exact and large-sample Cramer-Rao Bounds (CRBs) are derived. It is shown that the CRB is significantly affected by the color of the modulating process when the signal-to-noise ratio (SNR) or the intrinsic SNR is small. Maximum likelihood type estimators that ignore the noise color and optimize a criterion with respect to only the phase parameters are proposed. These estimators are shown to be equivalent to the nonlinear least squares estimators, which consist of matching the squared observations with a constant amplitude phase-modulated signal when the mean of the multiplicative noise is forced to zero. Closed-form expressions are derived for the efficiency of these estimators and are verified via simulations.

Robustness of the student t based M-estimator

This paper considers the maximum likelihood type (M) estimator based on Student's t distribution for the location/scale model. The Student t M-estimator is generally thought to be robust to outliers. This paper shows that this is only true if the degrees of freedom parameter is kept fixed. By contrast, if the degrees of freedom parameter is also estimated from the data, the influence functions for the scale and degrees of freedom parameter become unbounded. Moreover, the influence function of the location parameter remains bounded, but its change-of-variance function is unboi~nded. The intuitioil behind these results is explained in the paper. The rates at which both the influence functions and the change-of-variance function diverge to infinity, are very slow. Tliis implies that outliers have to be extremely large in order to become detrimental to the performance of the Student t based M-estimator with estimated degrees of freedom. The theoretical results are illustrated in a a simulation experiment using several related competing estimators and several distributions for the error process.

A Central Limit Theorem for M‐estimators by the von Mises Method

Asymptotic normality of M‐ or maximum likelihood type estimators was established in a classic paper by Huber (1967). Reeds (1976) argued that this could have been obtained simply as an application of the delta‐method, using the tool of compactly differentiating von Mises functionals with respect to the empirical distribution function Fn. His proof however contains some errors and has been largely ignored. A corrected version of the proof is given.

Maximum Likelihood Type Estimation for Nearly Nonstationary Autoregressive Time Series

The nearly nonstationary first-order autoregression is a sequence of autoregressive processes $y_n(k + 1) = \phi_ny_n(k) + \varepsilon(k + 1), 0 \leq k \leq n$, where the $\varepsilon(k)$'s are iid mean zero shocks and the autoregressive coefficient $\phi_n = 1 - \beta/n$ for some $\beta > 0$, so that $\phi_n \rightarrow 1$ as $n \rightarrow \infty$. We consider a class of maximum likelihood type estimators called $M$ estimators, which are not necessarily robust. The estimates are obtained as the solution $\hat{\phi}_n$ of an equation of the form $\sum^{n - 1}_{k = 0}y_n(k)\psi(y_n(k + 1) - \phi y_n(k)) = 0,$ where $\psi$ is a given "score" function. Assuming the shocks have $2 + \delta$ moments and that $\psi$ satisfies some regularity conditions, it is shown that the limiting distribution of $n(\hat{\phi}_n - \phi_n)$ is given by the ratio of two stochastic integrals. For a given shock density $f$ satisfying regularity conditions, it is shown that the optimal $\psi$ function for minimizing asymptotic mean squared error is not the maximum likelihood score in general, but a linear combination of the maximum likelihood score and least squares score. However, numerical calculations under the constraint $y_n(0) = 0$ show that the maximum likelihood score has asymptotic efficiency no lower than 40${\tt\%}$.

Robust bayesian estimation

Classical least squares Bayesian estimation consists of minimizing the sum of the squared residuals of observations and the corrections to prior estimates of parameters.Many authors have produced more robust versions of this estimation by replacing the square by something else, such as the absolute value. In this article, three robust (M-LS, LS-M and M-M) estimators for three corresponding error models are described based on the principle of maximum likelihood type estimates (M-estimates). The influence functions of the three robust Bayesian estimators are given. The algorithm implementation problems are discussed and the expressions for the posterior variance-covariance are derived.

Robust Parameter Estimation Procedure for ɛ-Contaminated Probability Distribution Models with Uncertain Gross-Error

Recently the concept of robustness has attracted considerable attentions of control engineers, and robust procedures have been developed for several control problems. One of the most important ideas on this direction is Huber’s M-estimator (maximum likelihood type estimator). He first identified neighborhoods of stochastic models by in terms of the class of ε-contaminated probability distributions. Then, he derived an estimator that minimizes the maximum degradation of performance possible for an ε-deviation from the assumption. The idea, however, is not applicable without the exact knowledge of the gross-error ε. So, an M-estimator is derived in this paper, which is applicable only with a priori knowledge of the upper bound of the un-ceratin gross-error. A numerical example illustrates the usefulness of the proposed robust estimation procedure.

Convergence results for maximum likelihood type estimators in multivariable ARMA models II

The consistency proof for the (Gaussian quasi) maximum likelihood estimator in multivariable ARMA models as given in Dunsmuir and Hannan (1976, Adv, in Appl. Probab. 8, 339–364) rests on a certain property of the underlying parameter space, called B6 in their paper. It is not known whether the usual parameter spaces like the manifold M( n) or the parameter spaces corresponding to echelon forms satisfy condition B6, since the argument given by Dunsmuir and Hannan to establish this fact is inconclusive. In Pötscher (1987, J. Multivariate Anal. 21 29–52) it was shown how consistency can be proved without relying on B6 if the data generating process is Gaussian. In this note we show that the Gaussianity assumption can be replaced by ergodicity thus restoring Dunsmuir and Hannan's consistency proof to its full generality and extending it to parameter spaces which do not satisfy condition B6.

Convergence results for maximum likelihood type estimators in multivariable ARMA models

General convergence results for maximum likelihood type estimators in multivariable ARMA-models under very weak assumptions are given. This extends results by Dunsmuir and Hannan (1976, Advan. Appl. Probab. 8 339–364) and Deistler, Dunsmuir, and Hannan (1978, Advan. Appl. Probab. 10 360–372). In particular it is shown that consistency can be achieved without imposing a certain assumption used in Dunsmuir and Hannan which is related to the zeroes of the spectral density if one is willing to make stronger assumptions concerning the probabilistic structure of the process.

Ordinary and proper location M-estimates for autoregressive-moving average models

SUMMARY Proper location M-estimates for a model with non-Gaussian autoregressive-moving average type errors are genuine maximum likelihood type estimates, whereas ordinary location M-estimates are those introduced by P. Huber for independent and identically distributed errors. The relative behaviour of ordinary location M-estimates and proper location M-estimates is studied for situations with dependent errors of purely autoregressive and purely moving average type. It is shown through asymptotic calculations and finite-sample size Monte Carlo studies that although ordinary location M-estimates are adequate for weak dependency structure, they can be quite inefficient compared with proper M-estimates of location when the non-Gaussian errors have a moderate to strong dependency structure.

Small-sample asymptotic distributions of M-estimators of location

Asymptotic formulae for the distribution of M-estimators, i.e. maximum likelihood type estimators, of location, including the arithmetic mean, are derived which numerical studies show to give relative errors for densities and tail areas of the order of magnitude of 1% down to sample sizes 3 and 4 even in the extreme tails. The paper is the continuation of earlier work by the second author and is also closely related to Daniels's work on the saddlepoint approximation. The method consists in expanding the derivative of the logarithm of the unstandardized density of the estimator in powers of 1/n at each point, using recentring by means of conjugate distributions. This method yields a unified point of view for the comparison of other asymptotic methods, namely saddlepoint method, Edgeworth expansion and large deviations approach, which are also compared numerically.

Second order asymptotic relations of M-estimators and R-estimators in two-sample location model

Let T M be an M-estimator (maximum likelihood type estimator) and T R be an R-estimator (Hodges-Lehmann's estimator) of the shift parameter Δ in the two-sample location model. The asymptotic representation of √ N ( T M - T R ) up to a term of the order O p ( N - 1 4 ) is derived which is valid if the functions Ψ and ϕ generating T M and T R , respectively, decompose into an absolutely continuous and a step-function components; the order O p ( N - 1 4 ) cannot be improved unless the discontinuous components vanish. As a consequence, the conditions under which √ N ( T M - T R )= O p ( N - 1 4 ) are obtained. The main tool for obtaining the results is the second order asymptotic linearity of the pertaining linear rank statistics which is proved here under the assumption that the score-generating function ϕ has some jump-discontinuities.

Robust Analysis of Variance Based Upon a Likelihood Ratio Criterion

Robust tests of general linear hypotheses in linear models are developed. These are likelihood ratio type tests in the same sense that M -estimates are maximum likelihood type estimates. Construction of the tests suggests a decomposition of the data into terms analogous to classical sums of squares, providing a robust analysis of variance. Asymptotic efficiency and robustness properties of the tests are the same as those of the M -estimates upon which they are based.

Robust analysis of variance based upon a likelihood ratio criterion

SUMMARY Robust tests of general linear hypotheses in linear models are developed. These are likeli- hood ratio type tests in the same sense that M-estimates are maximum likelihood type estimates. Construction of the tests suggests a decomposition of the data into terms analogous to classical sums of squares, providing a robust analysis of variance. Asymptotic efficiency and robustness properties of the tests are the same as those of the M-estimates upon which they are based. Parameter estimation is usually only a first step in the analysis of data arising fromn a linear model. A classical least squares analysis often focuses upon the analysis of variance, which tests simultaneous hypotheses on large subsets of the parameters. Since the terms in a classical analysis of variance are quadratic forms in least squares estimates, one would expect that the sensitivity of the estimates to departures from normality should be inherited by the tests. In fact, for moderate to heavy tailed error distributions or in the presence of outliers, it appears that the classical F test does lose power. Calculations of relative efficiency for proce- dures proposed in this paper substantiate on theoretical grounds the possible inefficiency and lack of power of classical F tests. In this paper, Huber's M-estimates are adapted to hypothesis tests which can be termed likelihood ratio type tests. These procedures naturally generalize and bear a striking re- semblance to classical F tests. Robustness and efficiency properties of M-estimates apply directly to the proposed tests. Hence the case to be made for using likelihood ratio type tests rather than classical F tests is the same as that for using M-estimates in favour of least squares estimates: possible poor performance of the classical methods may be overcome with methods which perform well both when classical assumptions are met and when they are not. The proposed methods are natural, intuitive and as easily computed as M-estimates.

Asymptotic Relations of $M$-Estimates and $R$-Estimates in Linear Regression Model

Let $\hat{\mathbf{\Delta}}_M$ be an $M$-estimator (maximum-likelihood type estimator) and $\hat{\mathbf{\Delta}}_R$ be an $R$-estimator (rank estimator) of the parameter $\mathbf{\Delta} = (\Delta_1,\cdots, \Delta_p)$ in the linear regression model $X_{Ni} = \sum^p_{j=1} \Delta_jc_{ji} + e_i, i = 1,\cdots, N$. The asymptotic distribution of $\hat\mathbf{\Delta}_M - \hat\mathbf{\Delta}_R$ is derived for $p$ fixed and $N \rightarrow \infty,$ under some assumptions on the design matrix, on the error distribution $F$ and on the functions generating the respective estimators. The result has several consequences which have an interest of their own; among others, it is shown that to any $M$-estimator corresponds an $R$-estimator such that the estimators asymptotically equivalent, and conversely. A special case when $\hat\mathbf{\Delta}_M$ is the maximum likelihood estimator and $\hat\mathbf{\Delta}_R$ the $R$-estimator, both asymptotically efficient for some distribution $G$, is also considered.

Robust Estimates of Location: Symmetry and Asymmetric Contamination

The problem of finding location estimators which are robust against deviations from normality has received increasing attention in the last several years. See, for example, Tukey (1960), Huber (1968), and papers cited therein. In the theoretical work done on the estimation of a location parameter, the underlying distribution is usually assumed to be symmetric, and the estimand is taken to be the center of symmetry, a natural quantity to estimate in this situation. Since the finite sample size properties of many proposed estimators are difficult to study analytically, most research has focussed on their more easily ascertainable asymptotic properties, which, it is hoped, will provide useful approximations to the finite sample size case. Most of the estimators commonly studied are, under suitable regularity conditions, asymptotically normal about the center of symmetry, with asymptotic variance depending on the underlying distribution. We thus have a simple criterion, the asymptotic variance, for comparing the performance of different estimators for a given underlying distribution, and of a given estimator for different underlying distributions. Huber (1964) has formulated and solved some minimax problems, in which the estimators are judged by their asymptotic variance. In Section 2 we define and state the asymptotic variances which have been found for the three most commonly studied types of location estimators. In Section 3 we demonstrate some relationships among the three types of estimators, and in Section 4 we show that Huber's minimax result applies to all three types. Then, in Section 5 we consider an aspect of the more general estimation problem in which the distributions are not assumed symmetric. A model of asymmetric contamination of a symmetric distribution is formulated, in which the amount of asymmetry tends to zero as the sample size increases. The estimators here are thought of as estimating the center of the symmetric component of the distribution. The maximum likelihood type estimators are shown to be asymptotically normal under this model, but with a bias that tends to zero as the sample size increases. The estimators may be judged by their asymptotic mean squared error, a concept which is made meaningful by the model. We conclude in Section 6 with a minimax result analogous to Huber's, for which we allow both symmetric and asymmetric contamination of a given distribution and judge the estimators by their asymptotic mean squared error.

Small Sample Properties of Selected Distributed Lag Estimators

VARIOUS TECHNIQUES have been proposed for estimating the parameters in distributed lag schemes. Of these, ordinary least squares is known to be inconsistent, and the technique proposed by Koyck [8] and elaborated by Klein [7] has been shown by Amemiya and Fuller [1] to be less efficient than maximum likelihood or Aitken type estimates.2 Liviatan [9] has proposed an instrumental variables approach which is consistent but inefficient. Hannan [5] has proposed a method of estimating regression coefficients by spectral techniques and in [6] has applied the method to the estimation of distributed lag schemes. Amemiya and Fuller [1] have shown that Hannan's estimates have the same asymptotic distribution as the maximum likelihood estimates in the distributed lag model