A Novel Nonparametric Maximum Likelihood Estimator for Probability Density Functions

Examining the robustness properties of maximum likelihood (ML) estimators of parameters in exponential power and generalized t distributions has been considered together. The well-known asymptotic properties of ML estimators of location, scale and added skewness parameters in these distributions are studied. The ML estimators for location, scale and scale variant (skewness) parameters are represented as an iterative reweighting algorithm (IRA) to compute the estimates of these parameters simultaneously. The artificial data are generated to examine performance of IRA for ML estimators of parameters simultaneously. We make a comparison between these two distributions to test the fitting performance on real data sets. The goodness of fit test and information criteria approve that robustness and fitting performance should be considered together as a key for modeling issue to have the best information from real data sets.

Maximum likelihood (ML) estimation of parameters of the generalized gamma (GG) distribution has been considered in several papers, and some of them stated that the ML estimation has some computational difficulties. Therefore, different approaches including numerical methods have been proposed for the ML estimation of parameters of the GG distribution. However, it is known that using numerical methods may have some drawbacks, e.g., non-convergence of iterations, multiple roots, and convergence to the wrong root. In this study, we rehabilitate the ML procedure via the modified ML (MML) methodology and obtain the likelihood equations in which two of them have explicit solutions, and the remaining one should be solved numerically. Since the MML methodology explicitly solves two of three likelihood equations, the mentioned drawbacks are alleviated. We also propose a simple algorithm to obtain the estimates of the parameters of the GG distribution. Then, the GG distribution is used for modeling the real data sets, and the performance of the proposed algorithm is compared with the Broyden–Fletcher–Goldfarby–Shanno (BFGS) and Nelder–Mead (NM) algorithms. The results show that the proposed algorithm is preferable to the BFGS and NM algorithms in terms of computational sense when considering the GG distribution.

This paper describes an EM algorithm for nonparametric maximum likelihood (ML) estimation in generalized linear models with variance component structure. The algorithm provides an alternative analysis to approximate MQL and PQL analyses (McGilchrist and Aisbett, 1991, Biometrical Journal 33, 131-141; Breslow and Clayton, 1993; Journal of the American Statistical Association 88, 9-25; McGilchrist, 1994, Journal of the Royal Statistical Society, Series B 56, 61-69; Goldstein, 1995, Multilevel Statistical Models) and to GEE analyses (Liang and Zeger, 1986, Biometrika 73, 13-22). The algorithm, first given by Hinde and Wood (1987, in Longitudinal Data Analysis, 110-126), is a generalization of that for random effect models for overdispersion in generalized linear models, described in Aitkin (1996, Statistics and Computing 6, 251-262). The algorithm is initially derived as a form of Gaussian quadrature assuming a normal mixing distribution, but with only slight variation it can be used for a completely unknown mixing distribution, giving a straightforward method for the fully nonparametric ML estimation of this distribution. This is of value because the ML estimates of the GLM parameters can be sensitive to the specification of a parametric form for the mixing distribution. The nonparametric analysis can be extended straightforwardly to general random parameter models, with full NPML estimation of the joint distribution of the random parameters. This can produce substantial computational saving compared with full numerical integration over a specified parametric distribution for the random parameters. A simple method is described for obtaining correct standard errors for parameter estimates when using the EM algorithm. Several examples are discussed involving simple variance component and longitudinal models, and small-area estimation.

Manufacturers need to evaluate the reliability of their products in order to increase the customer satisfaction. Proper analysis of reliability also requires an effective study of the failure process of a product, especially its failure time. So, the Failure Process Modeling (FPM) plays a key role in the reliability analysis of the system that has been less focused on. This paper introduces a framework defining an approach for the failure process modeling with censored data in Constant Stress Accelerated Life Tests (CSALTs). For the first time, various types of censoring schemes are considered in this study. Usually, in data analysis, it is impossible to get closed form of estimates of the unknown parameter due to complex and nonlinear likelihood equations. As a new approach, a mathematical programming problem is formed and the Maximum Likelihood Estimation (MLE) of parameters is obtained to maximize the likelihood function. A case study in red Light- Emitting Diode (LED) lamps is also presented. The MLE of parameters is obtained using genetic algorithm (GA). Furthermore, the Fisher information matrix is obtained for constructing the asymptotic variances and the approximate confidence intervals of estimates of the parameters.

The multinomial logistic response model has been used in the analysis of data from longitudinal studies of RERF's mortality cohort population. The model was restricted to linear and quadratic doseresponses for practical as well as biological reasons. The advantages and disadvantages of the multinomial logistic model are pointed out. Numerical comparison is made of the maximum likelihood (ML) estimates of parameters obtained by binomial and multinomial logistic procedures. The dose-response difference between two independent “same age” groups is evaluated from the ML estimates of parameters under a linear logistic response model. A significant dose-response difference between two independent “same age” groups in the years 1950–1959 and 1960–1969 is noted only for the 15–24 age group for all cancers other than leukemia.

The importance of providing structural validity evidence for test score(s) derived from psychometric test instruments is highlighted by several institutions; for example, the American Psychological Association (2014) demands that evidence for the validity of an instruments' internal structure and its underlying measurement model must be provided before it is applied in psychological assessment. The knowledge about the latent structure of data obtained with tests addressing the major question "What is/are the construct[s] being measured" by psychological tests under investigation (Ziegler, 2014 (Ziegler, , 2020)) . The study of structural validity is typically addressed with factor analyses when the test scores reflect continuous latent traits. As most submissions to Psychological Test Adaptation and Development (PTAD) deal with the adaptation and further development of existing measures, authors typically test a measurement model that is based on theoretical considerations and prior findings on original versions (or adaptations) of the test under investigation. Our literature review of PTAD's publications showed that more than 90% of the articles contain at least one confirmatory factor analysis (CFA). As editor and reviewers of PTAD, we appreciate that authors are rigorous in providing evidence on the structural validity of their tests' data. However, since PTAD's inception in 2019, we experience that one comment is frequently communicated to authors during the review process, namely, the request to adjust the analytic approach in CFA from maximum likelihood (ML) estimation toward using the mean-and variance-adjusted weighted least squares (WLSMV; Muthén et al., 1997) estimator to account for the ordinal nature of the data that psychological instruments typically generate on the item level. In this editorial, we discuss the rationale behind choosing the WLSMV estimator when analyzing test adaptations and developments that are based on ordinal categorical data and concisely illustrate the problems associated with using the ML estimator (potentially in combination with robust tests of model fit) for such data.

The events of interest in any survival analysis study are regularly subject to censoring. There are various censoring schemes, including right or left censoring, and interval censoring. The most frequent censoring scheme is the right censoring, where subjects might drop out of the study or simply because not all events of interest occur before the end of the study. Moreover, for each subject, additional information referred to as covariates is registered at the beginning or throughout the study, such as age, sex, undergoing treatment, etc. The classical model to study the distribution of the events of interest, while accounting for additional information, is the Cox model. The Cox model expresses the hazard function of a subject given a set of covariates in terms of a baseline hazard, for which all covariates are zero, and an exponential function of the covariates and corresponding regression parameters. The baseline hazard can be left completely unspecified while estimating the regression parameters. Nonetheless, in practice, there are numerous studies in which the baseline hazard appears to be monotone. Time to death or to the onset of a disease are observed to have a nondecreasing baseline hazard, while the survival or recovery time after a successful medical treatment usually exhibit a nonincreasing baseline hazard. The aim of this thesis is to study the behavior of nonparametric baseline hazard and baseline density estimators in the Cox model under monotonicity constraints. The event times are assumed to be right censored and the censoring mechanism is assumed to be independent of the event of interest and non-informative. The covariates are assumed to be time-independent, usually recorded at the beginning of the study. In addition to point estimates, interval estimates of a monotone baseline hazard will be provided, based on a likelihood ratio method, along with testing at a fixed point. Furthermore, kernel smoothed estimates of a monotone baseline hazard will be defined and their behavior will be investigated. In Chapter 2, we propose several nonparametric monotone estimators of a baseline hazard or a baseline density within the Cox model. We derive the nonparametric maximum likelihood estimator of a nondecreasing baseline hazard and we consider a Grenander-type estimator, defined as the left-hand slope of the greatest convex minorant of the Breslow estimator. The two estimators are then shown to be strongly consistent and asymptotically equivalent. Moreover, we derive their common limit distribution at a fixed point. The two equivalent estimators of a nonincreasing baseline hazard and their asymptotic properties are acquired similarly. Furthermore, we introduce a Grenander-type estimator of a nonincreasing baseline density, defined as the left-hand slope of the least concave majorant of an estimator of the baseline cumulative distribution function derived from the Breslow estimator. This estimator is proven to be strongly consistent and its asymptotic distribution at a fixed point is derived. Chapter 3 provides an asymptotic linear representation of the Breslow estimator of the baseline cumulative hazard function in the Cox model. This representation can be used to derive the asymptotic distribution of the Grenander type estimator of a monotone baseline hazard estimator. The representation consists of an average of independent random variables and a term involving the difference between the maximum partial likelihood estimator and the underlying regression parameter. The order of the remainder term is arbitrarily close to n^-1. Chapter 4 focuses on interval estimation and on testing whether a monotone baseline hazard function in the Cox model has a particular value at a fixed point, via a likelihood ratio method. Nonparametric maximum likelihood estimators under the null hypothesis are defined for both nondecreasing and nonincreasing baseline hazard functions. These characterizations, along with those of the monotone nonparametric maximum likelihood estimators provide the asymptotic distribution of the likelihood ratio test. This asymptotic distribution enables, via inversion, the construction of pointwise confidence intervals. This method of constructing confidence intervals avoids the issue of estimating the nuisance parameters, as in the case of confidence intervals based on the asymptotic distribution of the estimators. Simulations indicate that the two methods yield confidence intervals with comparable coverage probabilities. Nonetheless, the confidence intervals based on the likelihood ratio are smaller, on average. Finally, in chapter 5 we consider smooth baseline hazard estimators. The estimators are obtained by kernel smoothing the maximum likelihood and Grenander-type estimators of a monotone baseline hazard function. Three different estimators are proposed for a nondecreasing baseline hazard, which are provided by the interchange of the smoothing and isotonization step. With this respect, we define a smoothed maximum likelihood estimator (SMLE), as well as a smoothed Grenander type (SG) estimator and a Grenander type smoothed (GS) estimator. All estimators are shown to be strongly pointwise or uniformly consistent.

The maximum likelihood estimation of shape parameters of Lomax distribution under bilateral Type-I Censored Sample is studied, the explicit expression of shape parameter estimation cannot be obtained, but it is proved that the maximum likelihood estimation of this parameter is unique. Then the EM estimation of shape parameters is obtained by using EM algorithm, and the asymptotic variance and approximate confidence interval of the EM estimation of shape parameters are also given; Finally, the effects of maximum likelihood estimation and EM estimation of shape parameters of Lomax distribution are numerically simulated. By comparing the relative deviation, it shows that the estimation effect of using EM algorithm to calculate shape parameters is relatively good, and the asymptotic variance and asymptotic confidence interval of EM estimation are given.

Misspecification of noncausal order in autoregressive processes

Recent research in the area of numerical optimization has led to development of the efficient algorithms based on Update Methods and Model Trust Region Techniques. The update methods are a class of iterative schemes that avoids expensive evaluations of (approximate) Hessians, yet retains the rapid convergence properties of Newton-like methods that require second-derivative (Hessian) information. Model Trust Region techniques have recently been proposed as a way of avoiding costly step-length calculations that are required by standard iterative methods based on approximate Newton methods. The purpose of this paper is to describe briefly the most sucessful of the update methods and to show how it, or more conventional methods such as scoring, can be combined with a model trust region technique to produce numerical algorithms that are ideally suited to maximum likelihood (ML) parameter estimation. Specific properties of these algorithms that are important for ML parameter estimation include a fast (superlinear) rate of convergence together with the ability to handle parameter constraints easily and efficiently.

Compound-Gaussian model with the texture of inverse gamma distribution is regarded as one of the best models to characterize the high-resolution sea clutter at low grazing angles. The model parameters are usually estimated through moments or maximum likelihood function of sample. The former one is of low precision since the using of high order moments. The latter is difficult to implement due to its high computing complexity. In this paper, a zFIogz estimator is proposed to decrease the order of moments, and an iterative maximum likelihood (ML) estimator is constructed to simplify the computation of ML estimator. The experiments based on simulated and real sea clutter data show that the proposed methods perform better than the method of moments while computing more efficiently than the ML estimator.

A new approach using the genetic algorithm for parameter estimation in multiple linear regression with long-tailed symmetric distributed error terms: An application to the Covid-19 data

When modelling the survival distribution of a disease for which the symptomatic progression of the associated condition is insidious, it is not always clear how to measure the failure/censoring times from some true date of disease onset. In a prevalent cohort study with follow-up, one approach for removing any potential influence from the uncertainty in the measurement of the true onset dates is through the utilization of only the residual lifetimes. As the residual lifetimes are measured from a well-defined screening date (prevalence day) to failure/censoring, these observed time durations are essentially error free. Using residual lifetime data, the nonparametric maximum likelihood estimator (NPMLE) may be used to estimate the underlying survival function. However, the resulting estimator can yield exceptionally wide confidence intervals. Alternatively, while parametric maximum likelihood estimation can yield narrower confidence intervals, it may not be robust to model misspecification. Using only right-censored residual lifetime data, we propose a stacking procedure to overcome the non-robustness of model misspecification; our proposed estimator comprises a linear combination of individual nonparametric/parametric survival function estimators, with optimal stacking weights obtained by minimizing a Brier Score loss function.

This paper considers the problem of maximum likelihood (ML) estimation for reduced-rank linear regression equations with noise of arbitrary covariance. The rank-reduced matrix of regression coefficients is parameterized as the product of two full-rank factor matrices. This parameterization is essentially constraint free, but it is not unique, which renders the associated ML estimation problem rather nonstandard. Nevertheless, the problem turns out to be tractable, and the following results are obtained. An explicit expression is derived for the ML estimate of the regression matrix in terms of the data covariances and their eigenelements. Furthermore, a detailed analysis of the statistical properties of the ML parameter estimate is performed. Additionally, a generalized likelihood ratio test (GLRT) is proposed for estimating the rank of the regression matrix. The paper also presents the results of some simulation exercises, which lend empirical support to the theoretical findings.

The discrete Coefficient of Determination (CoD) has become a key component of inference methods for stochastic Boolean models. We develop a parametric maximum-likelihood (ML) method for the inference of the discrete CoD for static Boolean systems and for dynamical Boolean systems in the steady state. Using analytical and numerical approaches, we compare the performance of the parametric ML approach against that of common nonparametric alternatives for CoD estimation, which show that the parametric approach has the least bias, variance, and root mean-square (RMS) error, provided that the system noise level is not too high. Next we consider the application of the proposed estimation approach to the problem of system identification, where only partial knowledge about the system is available. Inference procedures are proposed for both the static and dynamical cases, and their performance in logic gate and wiring identification is assessed through numerical experiments. The results indicate that identification rates converge to 100% as sample size increases, and that the convergence rate is much faster as more prior knowledge is available. For wiring identification, the parametric ML approach is compared to the nonparametric approaches, and it produced superior identification rates, though as the amount of prior knowledge is reduced, its performance approaches that of the nonparametric ML estimator, which was generally the best nonparametric approach in our experiments.