Use of Maximum Likelihood for Estimating Error Variance from a Collection of Analysis of Variance Mean Squares

Analysis of Variance Through Examples

SUMMARY By constructing a sequence of multinomial approximations and related maximum likelihood estimators, we derive a Cramér-Rao lower bound for nonparametric estimators of the mixture proportions and thereby characterize asymptotically optimal estimators. For the case of the sampling model M2 of Hosmer (1973) it is shown that the sequence of maximum likelihood estimators, which can be obtained explicitly, is asymptotically optimal in this sense. The results hold true even when the multinomial approximations involve cells chosen adaptively, from the data, in a well-specified way.

The most commonly used estimator for a log-normal mean is the sample mean. In this paper, we show that this estimator can have a large mean square error, even for large samples. Then, we study three main alternative estimators: (i) a uniformly minimum variance unbiased (UMVU) estimator; (ii) a maximum likelihood (ML) estimator; (iii) a conditionally minimal mean square error (MSE) estimator. We find that the conditionally minimal MSE estimator has the smallest mean square error among the four estimators considered here, regardless of the sample size and the skewness of the log-normal population. However, for large samples (n > or = 200), the UMVU estimator, the ML estimator, and the conditionally minimal MSE estimators have very similar mean square errors. Since the ML estimator is the easiest to compute among these three estimators, for large samples we recommend the use of the ML estimator. For small to moderate samples, we recommend the use of the conditionally minimal MSE estimator.

Using mean square error as the criterion, we compare two least squares estimates of the Weibull parameters based on non‐parametric estimates of the unreliability with the maximum likelihood estimates (MLEs). The two non‐parametric estimators are that of Herd–Johnson and one recently proposed by Zimmer. Data was generated using computer simulation with three small sample sizes (5, 10 and 15) with three multiply‐censored patterns for each sample size. Our results indicate that the MLE is a better estimator of the Weibull characteristic value, θ, than the least squares estimators considered. No firm conclusions may be made regarding the best estimate of the Weibull shape parameter, although the use of maximum likelihood is not recommended for small sample sizes. Whenever least squares estimation of both Weibull parameters is appropriate, we recommend the use of the Zimmer estimator of reliability. Copyright © 2001 John Wiley & Sons, Ltd.

The problem considered requires estimation of parameters characterizing a serially correlated hydrologic time series {yt}, where data are also available from a longer time series {xt}, itself serially correlated and cross correlated with {yt}. On the assumption that both series are lag one autoregressions (when {yt} is characterized by three parameters μy, β, and σϵ2), large‐sample variances for the autoregressive parameter β are derived from the short series {yt} alone or from both series {xt} and {yt}, and the relative information is investigated numerically. It is concluded that, when the two serial correlations are approximately equal and the length of {xt} is twice that of {yt}, the gain in precision of the estimate β is about 4% when the cross correlation is 0.2, about 15% when the cross correlation is 0.4, about 31% when the cross correlation is 0.6, and about 49% when the cross correlation is 0.8. The equations giving maximum likelihood (ML) estimates are examined, and a relatively simple numerical technique is developed for one particular case of some practical importance. Small‐sample properties of both ML estimates and an estimate derived from work by Matalas are investigated, and tentative conclusions are that (1) the modified Matalas estimates have smaller mean square error than ML estimates, and (2) ML estimates of μy and β derived from both series have smaller mean square error than ML estimates of μy and β derived from the single series only, the converse being true for σϵ2.

This study focuses on optimizing the process of biofuel production from citrus peel using the Design of Experiments (DOE) technique. This study aims to determine the optimal values for the variables that have a significant impact on the production of biofuel. The variance within and between data groups was determined using the analysis of variance (ANOVA) table. The ANOVA table shows how much of the response variable's variation (biofuel production) can be explained by the independent variables (A, B, C, D, E, AB, AC, AD, AE, and BJ) and how much is caused by random error. The ANOVA table comprises of three primary parts: the F-statistic, the p-value, the df, the mean square (MS), the source of variation, and the sum of squares (SS). The wellspring of variety alludes to the beginning of the information variety, which can be either the lingering or the model. The amount of squares estimates the information's changeability, with the absolute amount of squares addressing the amount of the squared deviations of the genuine qualities from the mean worth. The residual is the sum of the squared deviations from the predicted values of the actual values, while the model's sum of squares is the sum of the squared deviations from the mean of the predicted values. The model has 10 degrees of freedom (the number of independent variables) and the residual has 4 degrees of freedom (the number of observations minus the number of independent variables). These degrees of freedom represent the number of independent pieces of information used to estimate a parameter. The mean square, which indicates the typical amount of variation for each variation source, is calculated by dividing the sum of squares by the degrees of freedom. The degree to which the model explains the variation in the data is indicated by the F-statistic, which is the ratio of the model's mean square to the residual's mean square. The probability of obtaining an F-statistic that is as large as the one observed if the null hypothesis is true is represented by the p-value. The independent variables' insignificant impact on biofuel production is the null hypothesis in this instance. The model's p-esteem in this study is under 0.05, demonstrating that the free factors essentially affect biofuel creation and that the model is genuinely huge. In addition, the model is significant because the F-statistic is relatively large in comparison to the F-distribution for the 10 and 4 degrees of freedom, respectively. The estimated coefficients for the linear regression model used to investigate the production of biofuel from citrus peel can be found in the ANOVA coefficients table. The table provides a list of the intercept and independent variables' coefficients, standard errors, t-values, and p-values. When all of the independent variables are zero, the intercept has a coefficient of 0.0672, indicating the estimated value of the response variable. The fact that the intercept does not differ significantly from zero is supported by the fact that its p-value is not significant. The fact that the coefficients of the independent variables A, E, AC, AD, AE, and BJ are not statistically significant indicates that these variables have little impact on the response variable. On the other hand, the positive coefficients and significant p-values of the independent variables B and C suggest that an increase in their values could result in an increase in the production of biofuel from citrus peel. In conclusion, the key variables that influence the production of biofuel from citrus peel have been identified thanks to the use of the Design of Experiments (DOE) method. According to the findings of this study, an increase in the production of biofuel from citrus peel may result from an increase in the values of the independent variables B and C. The development of environmentally friendly energy sources and the optimization of biofuel production processes will benefit greatly from these findings

Effects of data imbalance on bias, sampling variance and mean square error of heritability estimated with variance components were examined using a random two-way nested classification. Four designs, ranging from zero imbalance (balanced data) to "low", "medium" and "high" imbalance, were considered for each of four combinations of heritability (h(2)=0.2 and 0.4) and sample size (N=120 and 600). Observations were simulated for each design by drawing independent pseudo-random deviates from normal distributions with zero means, and variances determined by heritability. There were 100 replicates of each simulation; the same design matrix was used in all replications. Variance components were estimated by analysis of variance (Henderson's Method 1) and by maximum likelihood (ML). For the design and model used in this study, bias in heritability based on Method 1 and ML estimates of variance components was negligible. Effect of imbalance on variance of heritability was smaller for ML than for Method 1 estimation, and was smaller for heritability based on estimates of sire-plus-dam variance components than for heritability based on estimates of sire or dam variance components. Mean square error for heritability based on estimates of sire-plus-dam variance components appears to be less sensitive to data imbalance than heritability based on estimates of sire or dam variance components, especially when using Method 1 estimation. Estimation of heritability from sire-plus-dam components was insensitive to differences in data imbalance, especially for the larger sample size.

The scintillation detectors commonly used in SPECT and PET imaging and in Compton cameras require estimation of the position and energy of each gamma ray interaction. Ideally, this process would yield images with no spatial distortion and the best possible spatial resolution. In addition, especially for Compton cameras, the computation must yield the best possible estimate of the energy of each interacting gamma ray. These goals can be achieved by use of maximum-likelihood (ML) estimation of the event parameters, but in the past the search for an ML estimate has not been computationally feasible. Now, however, graphics processing units (GPUs) make it possible to produce optimal, real-time estimates of position and energy, even from scintillation cameras with a large number of photodetectors. In addition, the mathematical properties of ML estimates make them very attractive for use as list entries in list-mode ML image reconstruction. This two-step ML process-using ML estimation once to get the list data and again to reconstruct the object-allows accurate modeling of the detector blur and, potentially, considerable improvement in reconstructed spatial resolution.

THE analysis of variance may be thought of as a method of ranking a set of independent mean sums of squares (mean squares), or more specifically, of separating the significantly larger mean squares from a null set of homogeneous ones. When only a poor estimate of a2, the expected mean square under the null hypothesis, is available, or when there is no estimate of U2 at all, there is no standard way of doing this analysis. In such cases, the method to be presented below may be useful as a supplementary tool in the analysis of variance. As in other ranking problems, the first step will be to bring the mean squares onto a common scale which is well understood statistically. This is not quite straightforward in the general case when U2 is unknown and the individual mean squares have different degrees of freedom. It is instructive first to consider the problem in the simpler situation when the degrees of freedom are all equal to vo, say. Then one can plot the ordered sums of squares against the corresponding quantities of the chi-squared distribution with vo degrees of freedom. The plot may then be analysed in the same way as the half-normal plot of Daniel (1959) which represents the case for vo = 1. Another simple case is when a2 is known. Let the independent sums of squares be SS ,..., SSp

In this paper, a new generalized distribution called the gamma log-logistic Weibull (GLLoGW) distribution is proposed and studied. The GLLoGW distribution include the gamma log-logistic, gamma log-logistic Rayleigh, gamma log logistic exponential, log-logistic Weibull, log-logistic Rayleigh, log-logistic exponential, log-logistic as well as other special cases as sub-models. Some mathematical properties of the new distribution including moments, conditional moments, mean and median deviations, Bonferroni and Lorenz curves, distribution of the order statistics and R\'enyi entropy are derived. Maximum likelihood estimation technique is used to estimate the model parameters. A Monte Carlo simulation study to examine the bias and mean square error of the maximum likelihood estimators is presented and an application to real dataset to illustrate the usefulness of the model is given.

This paper presents a performance analysis of the maximum likelihood (ML) estimator for finding the directions of arrival (DOAs) with a sensor array. The asymptotic properties of this estimator are well known. In this paper, the performance under conditions of low signal-to-noise ratio (SNR) and a small number of array snapshots is investigated. It is well known that the ML estimator exhibits a threshold effect, i.e., a rapid deterioration of estimation accuracy below a certain SNR or number of snapshots. This effect is caused by outliers and is not captured by standard techniques such as the Crame/spl acute/r-Rao bound and asymptotic analysis. In this paper, approximations to the mean square estimation error and probability of outlier are derived that can be used to predict the threshold region performance of the ML estimator with high accuracy. Both the deterministic ML and stochastic ML estimators are treated for the single-source and multisource estimation problems. These approximations alleviate the need for time-consuming computer simulations when evaluating the threshold region performance. For the special case of a single stochastic source signal and a single snapshot, it is shown that the ML estimator is not statistically efficient as SNR/spl rarr//spl infin/ due to the effect of outliers.

This paper presents a performance analysis of the maximum likelihood (ML) estimator for finding the directions of arrival (DOAs) with a sensor array. The asymptotic properties of this estimator are well known. In this paper, the performance under conditions of low signal-to-noise ratio (SNR) and a small number of array snapshots is investigated. It is well known that the ML estimator exhibits a threshold effect, i.e., a rapid deterioration of estimation accuracy below a certain SNR or number of snapshots. This effect is caused by outliers and is not captured by standard techniques such as the Cram??r-Rao bound and asymptotic analysis. In this paper, approximations to the mean square estimation error and probability of outlier are derived that can be used to predict the threshold region performance of the ML estimator with high accuracy. Both the deterministic ML and stochastic ML estimators are treated for the single-source and multisource estimation problems. These approximations alleviate the need for time-consuming computer simulations when evaluating the threshold region performance. For the special case of a single stochastic source signal and a single snapshot, it is shown that the ML estimator is not statistically efficient as SNR ?> ?> due to the effect of outliers.

This paper examines the detection problem of the preamble sequence index in the WiMAX system. The mobile station receiver knows all the possible preamble sequences and should estimate which preamble sequence has been transmitted from the base station. Since the preamble in the orthogonal frequency division multiplexing (OFDM) transmission is usually the first received symbol, the channel is unknown to the receiver, which makes the problem of preamble sequence estimation complicated. In this paper, this problem is addressed by developing the joint maximum likelihood (ML) estimator of the preamble sequence and the channel. A simple decoupled estimator and a minimum mean square error (MMSE) estimator are also presented as benchmarks for the joint ML estimator. Then it is shown how the joint ML estimator can be used for the segment detection. Since the joint ML estimator can be computationally complex in its general form, low-complexity algorithms are developed depending on the type of pilot subcarrier locations for general OFDM systems including WiMAX. The simulation results show that the joint ML estimator detects the preamble sequence index very well in the absence of the channel knowledge.

In this paper, the data aided (DA) and non‐data aided (NDA) maximum likelihood (ML) symbol timing estimators and their corresponding conditional Cramer–Rao bound (CCRB) and modified Cramer–Rao bound (MCRB) in multiple‐input‐multiple‐output (MIMO) correlated flat‐fading channels are derived. It is shown that the approximated ML algorithm in References [4,13] is just a special case of the DA ML estimator; while the extended squaring algorithm in Reference [14] is just a special case of the NDA ML estimator. For the DA case, the optimal orthogonal training sequences are also derived. It is found that the optimal orthogonal sequences resemble the Walsh sequences, but present different envelopes. Simulation results under different operating conditions (e.g. number of antennas and correlation between antennas) are given to assess and compare the performances of the DA and NDA ML estimators with respect to their corresponding CCRBs and MCRBs. It is found that (i) the mean square error (MSE) of the DA ML estimator is close to the CCRB and MCRB, (ii) the MSE of the NDA ML estimator is close to the CCRB but not to the MCRB, (iii) the MSEs of both DA and NDA ML estimators are approximately independent of the number of transmit antennas and are inversely proportional to the number of receive antennas, (iv) correlation between antennas has little effect on the MSEs of DA and NDA ML estimators and (v) DA ML estimator performs better than NDA ML estimator at the cost of lower transmission efficiency and higher implementation complexity. Copyright © 2004 John Wiley & Sons, Ltd.

The reverberation power spectral density (PSD) is often required for dereverberation and noise reduction algorithms. In this work, we compare two maximum likelihood (ML) estimators of the reverberation PSD in a noisy environment. In the first estimator, the direct path is first blocked. Then, the ML criterion for estimating the reverberation PSD is stated according to the probability density function of the blocking matrix (BM) outputs. In the second estimator, the speech component is not blocked. Instead, the ML criterion for estimating the speech and reverberation PSD is stated according to the probability density function of the microphone signals. To compare the expected mean square error (MSE) between the two ML estimators of the reverberation PSD, the Cramer–Rao Bounds (CRBs) for the two ML estimators are derived. We show that the CRB for the joint reverberation and speech PSD estimator is lower than the CRB for estimating the reverberation PSD from the BM outputs. Experimental results show that the MSE of the two estimators indeed obeys the CRB curves. Experimental results of multimicrophone dereverberation and noise reduction algorithm show the benefits of using the ML estimators in comparison with another baseline estimators.