Maximum Likelihood Prediction Research Articles

ART neural networks for remote sensing: vegetation classification from Landsat TM and terrain data

A new methodology for automatic mapping from Landsat thematic mapper (TM) and terrain data, based on the fuzzy ARTMAP neural network, is developed. System capabilities are tested on a challenging remote sensing classification problem, using spectral and terrain features for vegetation classification in the Cleveland National Forest. After training at the pixel level, system performance is tested at the stand level, using sites not seen during training. Results are compared to those of maximum likelihood classifiers, as well as back propagation neural networks and K nearest neighbor algorithms. ARTMAP dynamics are fast, stable, and scalable, overcoming common limitations of back propagation. Best results are obtained using a hybrid system based on a convex combination of fuzzy ARTMAP and maximum likelihood predictions. A prototype remote sensing example introduces each aspect of data processing and fuzzy ARTMAP classification. The example shows how the network automatically constructs a minimal number of recognition categories to meet accuracy criteria. A voting strategy improves prediction and assigns confidence estimates by training the system several times on different orderings of an input set.

Gaussian mixture density modeling of non-Gaussian source for autoregressive process

A new approach is taken to model non-Gaussian sources of AR processes using Gaussian mixture densities that are known to be effective for approximating wide varieties of probability distributions. A maximum likelihood estimation algorithm is derived for estimating the AR parameters by solving a generalized normal equation, and a clustering algorithm is used for estimating the parameters of Gaussian mixture density of the source signals. The correlation matrix of the generalized normal equation is not Toeplitz but is symmetric and in general positive definite. Higher order statistics of skewness and kurtosis are used for identifying the source distribution as being Gaussian or non-Gaussian and, consequently, determining the parameter estimation technique between the conventional method and the proposed method. Experiments on non-Gaussian source AR processes demonstrate that under high SNR conditions (SNR/spl ges/20 dB), the proposed algorithm outperforms the conventional AR estimation algorithm and the cumulant-based algorithm by an order-of-magnitude reduction of average estimation errors. The proposed algorithm also has very low estimation errors with short data records. Finally, a maximum likelihood prediction method is formulated for non-Gaussian source AR processes that has shown potential in achieving higher efficiency signal coding than linear predictive coding. >

Corrections: Comparisons of Prediction Methods When Only a Few Components are Relevant

Proposition 3 in this article is said to give an asymptotic formula for the prediction error of the (modified) maximum likelihood predictor under the hypothesis of one relevant component for prediction. It is more correct to say that this formula gives the Cramer-Rao lower bound and the asymptotical prediction error for an efficient version of the maximum likelihood predictor. The point is that the likelihood function has several local maxima. By Lehmann (1983), theorem 6.2.3, any consistent sequence of roots of the likelihood equation is efficient, and because the PLS estimator is consistent in this situation, it is better to choose the root of the likelihood equation that is closest to PLS (as measured by the closeness of the regression vectors) than to search for the absolute maximum. Doing this in the simulations systematically leads to improvement of the MML solution, except for the largest irrelevant eigenvalues in Table 1, correspondinig to a rather extreme situation. The values for MML in Table 2 turn out to be about the same as those for PLS. When n is large or the signalto-noise ratio is small, MML performs slightly better than PLS. For instance, in a factorially designed simulation experiment with m = 1 relevant component and two values for n, for p, for the squared multiple correlation coefficient Y? 2, and for each of three parameters characterizing the irrelevant eigenvalues, the mean values of the mean squared prediction errors were:

Modified Maximum Likelihood Predictors of Future Order Statistics from Rayleigh Samples

Modified Maximum Likelihood Predictors of Future Order Statistics from Rayleigh Samples

Prediction Theory for Finite Populations.

A large number of papers have appeared in the past 20 years on estimating and predicting characteristics of finite populations. This monograph is designed to present this modern theory in a systematic and consistent manner. The author's approach is that of superpopulation models in which values of the population elements are considered as random variables having joint distributions. Throughout, the emphasis is on the analysis of data rather than on the design of samples. Topics covered include: optimal predictors for various superpopulation models, Bayes, minimax, and maximum likelihood predictors, classical and Bayesian prediction internals, model robustness, and models with measurement errors. Each chapter contains numerous examples, and exercises which extend and illustrate the themes in the text. As a result, this book will be ideal for all those research workers seeking an up-to-date and well-referenced introduction to the subject.

Maximum likelihood estimation and prediction mean square error in the spatial linear model

This paper is concerned with prediction in the spatial linear model using the maximum likelihood estimation of parameters in this model. In particular, we give some properties of predictors obtained on substituting the maximum likelihood estimators of model parameters into the form of the best-in the sense of minimum mean square prediction error-predictor. Such predictors are not optimal but we show them to be asymptotically equivalent to the optimum. We discuss practical aspects of this work and conclude by considering the connection with other areas.

Maximum likelihood prediction of lake acidity based on sedimented diatoms

Abstract. As an example of ecological gradient analysis, Gaussian response functions, with Poisson or quasi‐Poisson error distribution, were fitted for diatom taxa on a pH gradient. It is possible to predict or infer the pH of lake water from the fitted curves using the method of maximum likelihood, which is easily implemented in standard non‐linear regressionprograms. Due to overdis‐persion with respect to the Poisson distribution, moment estimates forthe negative binomial distribution were also applied, both in estimating the species response curves and in prediction. Simulations indicated that the theoretical maximum precision (measuredby standard deviation of prediction errors) in our data set was 0.17 pH units. The observed errors were much greater (SD 0.35 to 0.43). It seems that roughly equal proportions of the excess error were caused (1) by systematic differences between the training (estimation) data and the validation (prediction) data, and (2) from a misspecified model. It is suggested that the error due to model misspecification consists of inadequacy of the presumed error distribution and of inadequacy of the simple Gaussian response function.

Some shrunken predictors in finite populations with a multinormal superpopulation model

In this paper we derive some alternative estimators of the superparameter and predictors of the population total for a multinormal superpopulation model. The estimators and predictors obtained are better than the maximum likelihood predictor near the “natural origin” though possibly worse farther away. The technique employed is to shrink the maximum likelihood predictor towards a “natural origin”. With a numerical example, it is shown that the shrunken predictor of the population total works better than the maximum likelihood predictor for small sample sizes and near the “natural origin”. A combined predictor which is not as good as the shrunken predictor near the origin but not disasterously far away from the origin is introduced.

Prediction of IBNR claim counts by modelling the distribution of report lags

Property/liability insurance companies write insurance to cover many possible events for many different kinds of insured groups. Examples are: (1) workers insured against work related injuries; (2) doctors (and other professionals) insured against malpractice; (3) automobile owners insured against fire, theft and liability for injury; (4) homeowners insured against a multitude of hazards, etcetera. The company must, among other things, have a good idea of how many claims have yet to be reported on accidents which have already happened, and this must be done for each different kind of insured group. A claim which has been incurred (i.e., the accident has already happened) but not yet reported to the insurance company is called an IBNR claim. In this paper, we discuss some methods of predicting the number of IBNR claims. These methods center on the principle of maximum likelihood prediction, and are based upon suitable assumptions about the nature of accident frequency and the delay (lag) between the accident and the report. A later paper will include simulations, examples with real data, and asymptotic results. A special feature of the present paper is that grouping of the data as it arises in insurance data collection is fully taken into account. An example illustrating the basic ideas is presented at the end of the paper.

A note on the missing value principle and the EM-Algorithm for estimation and prediction in sampling from finite populations with a multinormal superpopulation model

It is shown that both the missing value principle (MVP) of Orchard and Woodbury (1972) and the EM-Algorithm of Dempster, Laird and Rubin (1972) yield a unique predictor of the population total, under a superpopulation multinormal model. The predictor obtained is the maximum likelihood predictor introduced by Royal (1976).

Maximum likelihood prediction

The principle of maximum likelihood is applied to the joint prediction and estimation of a future random variable and an unknown parameter. We assume dependence between present and future, and the approach is non-Bayesian. Our principal application is to the prediction of higher order statistics from lower ones in Type II censored random samples. Some simple criteria for existence and uniqueness of the predictor are given for this situation and the methods are illustrated with several examples.

A bayesian analysis of some threshold switching models

This paper presents a Bayesian analysis of various threshold switching regression models, including simple time series models, where the change of regime is governed by a known function of exogenous variables. Some special features arising from the choice of a two-dimensional linear dichotomy function are then discussed and the concepts of concurrency and duality introduced. Within this framework, we compare the maximum likelihood and Bayesian methodologies for inference and prediction. In particular, we show that the Bayesian approach solves the non-uniqueness problem which affects maximum likelihood prediction in certain situations. These results are illustrated with two numerical examples.

Predictive source coding techniques using maximum likelihood prediction for compression of digitized images

A predictive compression technique is examined, using maximum likelihood prediction of the image pixel based on the Markov mesh model, that encodes the differences via Gordon block-bit-plane (GBBP) encoding. The procedure is very efficient in that it requires a bit rate near the entropy of the source. For images with many quantization levels, maximum likelihood prediction can be cumbersome to implement. Thus, a suboptimal procedure called differential bit-plane coding (DBPC) is investigated. This is easily implemented, even for a large number of quantization levels, and is reasonably efficient.

A maximum likelihood prediction function for the linear model with consistency results

An alternative technique to current methods for constructing a prediction function for the normal linear regression model is proposed based on the concept of maximum likelihood. The form of this prediction function is evaluated and normalized to produce a multivariate Student's t-density. Consistency properties are established under regularity conditions, and an empirical comparison, based on the Kullback-Leibler information divergence, is made with some other prediction functions.

Bit Rate Reduction of Digitized Speech Using Entropy Techniques

We present the results of a study to reduce the bit rate of speech that has been digitized with a continuously variable slope delta modulator (CVSD) operating at 16, 24, and 32 kbits/s. The theoretical reduction is found from the bit stream entropy. The actual reduction, via Huffman coding, is within 1-2 Percent of the theoretical value. The conditional entropy indicates that additional bit rate reduction can be achieved if we use a set of Huffman codes, conditioned on the past CVSD bits. A third technique, tandem coding, using a maximum likelihood predictor in tandem with run length and Huffman coding, is also investigated. Using these entropy techniques, bit rate reductions of 11-25 percent are achieved for the CVSD rates considered. The paper concludes with a study of the buffer requirements needed to support these entropy coders.