Maximum Probability Estimation Research Articles

On the asymptotic behaviour of general maximum likelihood estimates for the nonregular case under nonstandard conditions

SUMMARY In this paper we generalize Huber's (1967) results to include the nonregular case. Resistant estimators which turn out to be asymptotically unbiased in a neighbourhood of the model are given for some univariate models. Their order of consistency achieves the rate of the maximum likelihood estimates. Moreover, some of the estimates are asymptotically efficient under the central model, in the sense that they have the same The consistency of the maximum likelihood estimators has been studied by several authors, such as Doob (1934) and Cramer (1946). Later Wald (1949) gave, for the regular case, a simpler proof with weaker hypotheses that do not involve second and higher derivatives of the likelihood function. Weiss & Wolfowitz (1966, 1974) considered some nonregular cases via maximum probability estimators. The classical regularity conditions for the asymptotic distribution of maximum likelihood estimates are not satisfied by nonregular families. In some cases the estimates have the same asymptotic properties as in the regular case, while in others they are not asymptotically normally distributed. The case of a translation parameter of a truncated distribution p(x, 0) = p(x - 0), where p(t) vanishes on (-oo, 0) and satisfies p(t)-cytyas t -- 0, with y a 1 and c> 0, has been consider by Woodroofe (1972, 1974), Weiss & Wolfowitz (1973), Akahira (1975a, b) and Smith (1985). Woodroofe (1972) showed that, for y > 2, the maximum likelihood estimators have the same asymptotic properties as in the regular case and that, for y = 2, they have an asymptotic normal distribution but the order of consistency is O{(n log n)2}.

Estimation in the General Linear Model when the Accuracy is Specified Before Data Collection

An estimator $\hat{\beta}$ of $\beta$ is accurate with accuracy $A$ and confidence $\gamma, 0 < \gamma < 1,$ if $P(\hat{\beta} - \beta \in A) \geq \gamma$ for all $\beta.$ Given a sequence $Y_1, Y_2, \cdots$ of independent vector-valued homoscedastic normally-distributed random variables generated via the general linear model $Y_i = X_i\beta + \varepsilon,$ the $k$-dimensional parameter $\beta$ is accurately estimated using a sequential version of the maximum probability estimator developed by L. Weiss and J. Wolfowitz. The procedure given also generalizes C. Stein's fixed-width confidence sets to several dimensions.

On estimator efficiency in stochastic processes

It is shown, under mild regularity conditions on the random information matrix, that the maximum likelihood estimator is efficient in the sense of having asymptotically maximum probability of concentration about the true parameter value. In the case of a single parameter, the conditions are improvements of those used by Heyde (1978). The proof is based on the idea of maximum probability estimators introduced by Weiss and Wolfowitz (1967).

Small-sample properties of maximum probability estimators

The theory of maximum probability estimation is predominantly asymptotic. In this paper it is shown that in many cases maximum probability estimators based on small samples are admissible for all practical purposes, in the sense that their expected gain function is arbitrarily close to the expected gain function of an admissible estimator.

Estimation of the Larger Mean

In the present paper estimation of the larger of two normal means is studied. Two new estimators are added to the class of possible estimators of the larger normal mean, namely, the maximum probability estimator and the iterated bias elimination estimators. If the magnitude of the difference between the two population means is not close to zero (as evidenced by its strongly consistent estimator, the magnitude of the difference between the two sample means) a suitably chosen maximum probability estimator is seen to be best as regards both bias and mean-squared error.

On the detection of jumps in a Markovian piecewise-constant video-signal

A Markovian piecewise-constant signal which changes by abrupt jumps may serve as a model of many types of signals. In the video-signal a jump is equivalent to the contour in the image line. The maximum posterior probability estimation of jump distribution is used. Optimal method of estimating is based on dynamic programming with reducing the number of variants from 2 n−1 (full number of variants) to 1 6 n(n − 1)(n − 2) + n . The suboptimal iterative technique of jumps detection is suggested also, as well as the parallel scheme for its implementation. The lower boundaries of probabilities of edge missing and of false detection were obtained.

Estimation of location of discontinuity in a density

Here we propose a few estimators of θ, in addition to those studied in Goria (1978), the point of discontinuity of the probability density $$f(x,\theta ) = \frac{1}{{2\Gamma (\alpha )}}e^{ - |x - \theta |} |x - \theta |^{\alpha - 1} ,$$ for $$0< \alpha< 1, - \infty< x< \infty , - \infty< \theta< \infty .$$ We establish the consistency and the optimality of the Bayes and the maximum probability estimators. Despite their nice properties, these estimators are not easy to compute in this case and their effective computation depends on the knowledge of the exponent α. Hence, we propose another class of estimators, dependent upon the spacings of the observations, computable without actual knowledge of the value of α as long as it is known that α < α0 < 1: we show that these estimators converge at the best possible rate. We further demonstrate, using a modified version of the maximum probability estimator's technique, that the tails of the density do not substantially effect their efficiency. Finally a bivariate family of densities, having a ridge dependent on the parameter θ, is considered and it is shown that this family exhibits features similar to the univariate case, and thus, the necessary modifications of the arguments of the univariate case are utilized for the estimation of θ in this bivariate example.

Maximum Probability Estimation of Location Parameters: Minimaxity and Admissibility

SYNOPTIC ABSTRACTIt is shown that, under a mild condition, an invariant maximum probability estimator of a location parameter is minimax. An example shows that such an estimator may be inadmissible.

Form and meaning in an Australian language.

The distribution of phonemes and phoneme sequences is shown to be the result of the interaction of semantic features. Each semantic feature has a phonological canonical form, and the probability of a word realizing a given semantic feature complex is inversely dependent on its distance from the canonical forms concerned. Each word is the maximum probability estimator of its semantic features. It is suggested that this relation may hold in all languages, and that it requires considerable revision of traditional views of the lexicon.

Evidence for a general template in central optimal processing for pitch of complex tones.

Optimum processor theory successfully accounts for earlier pitch data by including the constraint that component tones in a complex stimulus are estimated as successive harmonics. This constraint gives the paradoxical prediction that a periodic complex tone comprising nonsuccessive harmonics cannot evoke periodicity pitch corresponding to its period. Most published data from pitch-shift experiments imply the necessity for this constraint. New periodicity pitch experiments on pitch shift and musical interval recognition were performed which prove that the theoretical constraint is not generally true. New and old data are reconciled by replacing the maximum likelihood estimation of the theory with maximum posterior probability estimation and removing the successive harmonic constraint. Periodicity pitch is estimated by optimizing the match between the aurally measured frequencies of stimulus components and a general harmonic template over some a priori expected pitch range. The new, more general, formulation reduces in many experimental situations to the successive harmonic constraint as a special case.

A note on the maximum probability property of MLE's in multiparameter exponential families

For multiparameter exponential models a short and direct proof is given that the maximum likelihood estimator is a maximum probability estimator with respect to a certain sequence of convex and bounded sets inR(k) that are symmetric about the origin; asymptotically these sets are allowed to be unbounded.

Asymptotic properties of the maximum probability estimates in Markov processes

Asymptotic properties of the maximum probability estimates in Markov processes

Adaptive line enhancement and spectrum analysis

The notion that adaptive filters may estimate and track the frequency of a phase-modulated sinusoid in noise better than a spectrum analyzer is examined. It is argued, using both theoretical and experimental results, that spectrum analysis performs better than adaptive filtering. In addition, the method of spectrum analysis can be improved for such applications by the use of frequency-slope processing, or by the generation of a partially coherent reference waveform by maximum posterior probability estimation of the phase function.

Maximum Probability Estimators in the Classical Case and in the “Almost Smooth” Case

Previous article Next article Maximum Probability Estimators in the Classical Case and in the “Almost Smooth” CaseJ. WolfowitzJ. Wolfowitzhttps://doi.org/10.1137/1120039PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] I. A. Ibragimov and , R. Z. Khas'minskii, Asymptotic behavior of statistical estimators in the smooth case, Theory Prob. Applications, 17 (1972), 445–462 10.1137/1117054 0273.62019 LinkGoogle Scholar[2] L. Weiss and , J. Wolfowitz, Maximum probability estimators, Ann. Inst. Statist. Math., 19 (1967), 193–206 MR0232488 0183.21203 CrossrefGoogle Scholar[3] Lionel Weiss and , Jacob Wolfowitz, Maximum probability estimators and related topics, Springer-Verlag, Berlin, 1974v+106, Lecture Notes in Mathematics, Vol. 424 MR0359152 0297.62015 Google Scholar[4] L. Weiss and , J. Wolfowitz, Maximum probability estimators and asymptotic sufficiency, Ann. Inst. Statist. Math., 22 (1970), 225–244 MR0278345 0218.62035 CrossrefGoogle Scholar[5] Lionel Weiss and , J. Wolfowitz, Maximum probability estimators with a general loss functionProbability and Information Theory (Proc. Internat. Sympos., McMaster Univ., Hamilton, Ont., 1968), Springer, Berlin, 1969, 232–256, Lecture Notes in Mathematics, Vol. 89, Heidelberg–New York MR0269014 0188.49901 CrossrefGoogle Scholar[6] Harald Cramér, Mathematical Methods of Statistics, Princeton Mathematical Series, vol. 9, Princeton University Press, Princeton, N. J., 1946xvi+575 MR0016588 0063.01014 Google Scholar[7] Michel Loève, Probability theory, Third edition, D. Van Nostrand Co., Inc., Princeton, N.J.-Toronto, Ont.-London, 1963xvi+685 MR0203748 0108.14202 Google Scholar[8] I. A. Ibragimov and , R. Z. Khas'minskii, Asymptotic analysis of statistical estimates for the “almost smooth” case, Theory Prob. Applications, 18 (1973), 241–252 10.1137/1118027 0295.62027 LinkGoogle Scholar[9] Michael Woodroofe, Maximum likelihood estimation of a translation parameter of a truncated distribution, Ann. Math. Statist., 43 (1972), 113–122 MR0298817 0251.62018 CrossrefGoogle Scholar[10] L. Weiss and , J. Wolfowitz, Maximum likelihood estimation of a translation parameter of a truncated distribution, Ann. Statist., 1 (1973), 944–947 MR0341727 0271.62043 CrossrefGoogle Scholar Previous article Next article FiguresRelatedReferencesCited byDetails Volume 20, Issue 2| 1976Theory of Probability & Its Applications History Submitted:30 May 1974Published online:17 July 2006 InformationCopyright © 1976 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1120039Article page range:pp. 363-371ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics

Maximum probability estimators in the case of exponential distribution

In 1966–1969L. Weiss andJ. Wolfowitz developed the theory of „maximum probability” estimators (m.p.e.'s). M.p.e.'s have the property of minimizing the limiting value of the risk (see (2.10).) In the present paper, therfore, after a short description of the new method, a fundamental loss function is introduced, for which—in the so-called regular case—the optimality property of the maximum probability estimators yields the classical result ofR.A. Fisher on the asymptotic efficiency of the maximum likelihood estimator. Thereby it turns out that the m.p.e.'s possess still another important optimality property for this loss function. For the latter the parameters of the exponential distribution—in the one-and the two-dimensional case—are estimated by the new method; for the estimation of the location parameter aWeibull distribution — in a more general sense — is taken as a basis. This shows that the maximum likelihood estimators involve a greater risk than the m.p.e.'s.

Maximum probability estimators for ranked means

Suppose tha t observations f rom populations ,~1," "' , ~ (k>_2) are normally distributed with unknown means #~, . . . , #~ (respectively) and a common known variance o ~. Let ~ c 1 ~ _ . . . ~ denote the ranked means. Several ranking and selection procedures take n independent observations from each population, denote the sample mean of the n observations from ~ by .~ ( i = 1 , . . . , k), and utilize the ranked sample means Xcl j_~. . .~X~l . (See Dudewicz [3] for details.) We assume throughout tha t both the numerical values of ~1,-.-, pk and the pairings of the Pro,'" ", pr~ with the populations z~,.. , ~ are completely unknown and consider problems of estimation of p~ ( l ~ i _ k ) based on the statistics provided by the single-stage rule stated above, and utilizing recent work of Weiss and Wolfowitz. Generalized maximum likelihood estimators (GMLE's), introduced by Weiss and Wolfowitz [5], provide (where available) asymptotically efficient estimators, whereas this is not always t rue for MLE's even if the lat ter can be found. Most classical MLE theory assumes i.i.d, observations and is therefore not applicable in our case; the theory of Weiss and Wolfowitz [5] allows for more general situations (although most of their applications are to i.i.d. "non-regular" cases). (Corrections to Weiss and Wolfowitz [5] are contained in Weiss and Wolfowitz [6], in Weiss and Wolfowitz [8], and in Dudewicz [2]. An additional example is given in Weiss and Wolfowitz [7].) Maximum probability estimators (MPE's) were introduced by Weiss and Wolfowitz [6] for much the same reason as GMLE's were introduced by Weiss and Wolfowitz [5]. Weiss and Wolfowitz [6], pp. 202-203, proved that , for the case of m = l parameter , every GMLE is an MPE; thus MPE's extend the notion of GMLE's (and by finding a GMLE we find a f o r t i o r i an MPE). Below we study the extension of this result to m ~ l parameters, first summarizing Weiss and Wolfowitz's results.

Convergence of Estimates Under Dimensionality Restrictions

Consider independent identically distributed observations whose distribution depends on a parameter $\theta$. Measure the distance between two parameter points $\theta_1, \theta_2$ by the Hellinger distance $h(\theta_1, \theta_2)$. Suppose that for $n$ observations there is a good but not perfect test of $\theta_0$ against $\theta_n$. Then $n^{\frac{1}{2}}h(\theta_0, \theta_n)$ stays away from zero and infinity. The usual parametric examples, regular or irregular, also have the property that there are estimates $\hat{\theta}_n$ such that $n^{\frac{1}{2}}h(\hat{\theta}_n, \theta_0)$ stays bounded in probability, so that rates of separation for tests and estimates are essentially the same. The present paper shows that need not be true in general but is correct under certain metric dimensionality assumptions on the parameter set. It is then shown that these assumptions imply convergence at the required rate of the Bayes estimates or maximum probability estimates.

ASYMPTOTIC BEHAVIOR OF STATISTICAL ESTIMATES FOR SAMPLES WITH A DISCONTINUOUS DENSITY

In this paper the authors study the asymptotic behavior of the joint distribution and moments of generalized Bayesian estimates, maximum likelihood estimates and maximum probability estimates with respect to intervals constructed from independent observations with a density having a discontinuity of the first kind. Bibliography: 9 entries.

Maximum probability estimators and asymptotic sufficiency

Maximum probability estimators and asymptotic sufficiency