Maximum likelihood and the maximum product of spacings from the viewpoint of the method of weighted residuals

Abstract

In parameter estimation, the maximum-likelihood method (ML) does not work for some distributions. In such cases, the maximum product of spacings method (MPS) is used as an alternative. However, the advantages and disadvantages of the MPS, its variants, and the ML are still unclear. These methods are based on the Kullback–Leibler divergence (KLD), and we consider applying the method of weighted residuals (MWR) to it. We prove that, after transforming the KLD to the integral over [0, 1], the application of the collocation method yields the ML, and that of the Galerkin method yields the MPS and Jiang’s modified MPS (JMMPS); and the application of zero boundary conditions yields the ML and JMMPS, and that of non-zero boundary conditions yields the MPS. Additionally, we establish formulas for the approximate difference among the ML, MPS, and JMMPS estimators. Our simulation for seven distributions demonstrates that, for zero boundary condition parameters, for the bias convergence rate, ML and JMMPS are better than the MPS; however, regarding the MSE for small samples, the relative performance of the methods differs according to the distributions and parameters. For non-zero boundary condition parameters, the MPS outperforms the other methods: the MPS yields an unbiased estimator and the smallest MSE among the methods. We demonstrate that from the viewpoint of the MWR, the counterpart of the ML is JMMPS not the MPS. Using this KLD-MWR approach, we introduce a unified view for comparing estimators, and provide a new tool for analyzing and selecting estimators.