Standard Maximum Likelihood Estimation Research Articles

Statistical Reliability Estimation of Deep Space Satellites and Launch Vehicles: 1958–2022

On-orbit flight data for deep space satellites and space launch vehicles (SLVs) servicing deep space missions operated between 1958 and 2022 is analyzed due to a lack of reliability research combining both satellites and SLVs for this class of mission. Satellite reliability is first estimated by the Kaplan–Meier estimator, and then parameterized through the Weibull distribution. This process is applied to the general deep space satellite data set as well as the delimitation of the data according to decade. SLV reliability is computed using standard maximum likelihood estimation and an evolving first-level Bayesian estimation scheme. Results from the satellite analysis demonstrated the differences between the parameterization methods used. The bulk satellite data set indicated that deep space satellites suffer from infant mortality, while certain subsets were found to have a primary wear-out failure mode. In general, U.S. deep space missions demonstrated the highest reliability. Deep space satellite reliability was also found to have increased as a function of time. Results from the SLV data set revealed that deep space launches are primarily performed by four main launch vehicle families. Additionally, most deep space SLV failures occurred in the space age (1959–1975), with the USA and USSR/Russia being the only two countries to experience deep space launch failures. With both deep space SLV and satellite reliability increasing over time, reliability is not expected to be a major hindrance to future deep space missions.

Parameter Estimation and Hypothesis Testing on Bivariate Log-Normal Regression Models

This study aims to introduce a bivariate Log-Normal regression model and to develop a technique for parameter estimation and hypothesis testing. We term the model Bivariate Log-Normal Regression (BLNR). The estimation procedure is conducted by the standard Maximum Likelihood Estimation (MLE) employing the Newton-Raphson method. To perform hypothesis testing, we adapt the Maximum Likelihood Ratio Test (MLRT) for simultaneous testing with test statistics which, for large n, follows Chi-Square distribution with degrees of freedom p. In addition, the partial testing is derived from a central limit theorem which results in a Z-test statistic.
 Keywords: parameter estimation, hypothesis testing, bivariate log, normal regression

MWDINet: A multilevel wavelet decomposition interaction network for stock price prediction

Stock price prediction is a classical interdisciplinary issue drawn from finance, computer science, econometrics, and mathematics. Most stock price data are nonlinear, nonstationary, and highly complex, making stock price prediction challenging. Recently, deep neural networks (DNNs) have demonstrated powerful learning capabilities and have yielded notable results in stock price prediction tasks. Most existing deep learning solutions, however, only consider time-domain information or lack effective modeling of frequency-domain information, thus failing to effectively utilize both time-domain and frequency-domain information of the data. Meanwhile, existing methods ignore autocorrelated errors in the stock price forecasting task due to missing valid information data, i.e., they do not consider the correlation between the error at the current time step and the error at the previous time step, which undermines the standard maximum likelihood estimation (MLE) assumption, thereby weakening the model’s performance. We propose a multilevel wavelet decomposition interaction network (MWDINet), an end-to-end framework for stock price prediction. MWDINet employs the multiscale wavelet decomposition interaction module (MWDI-Block) and the Hull Moving Average module (HMA-Block) to extract the data’s frequency-domain and time-domain information, respectively. In MWDI-Block, the traditional signal processing method of Maximum Overlapping Discrete Wavelet Transform (MODWT) is seamlessly embedded into a deep learning framework (named DMODWT). The DMODWT algorithm not only automatically extracts the frequency-domain information from the data, but also fine-tunes the wavelet filters. With HMA-Block, we improve the Hull Moving Average (HMA), commonly used in the industry, into a deep learning module, which learns how changes in different markets over time. Inspired by the research on correcting autocorrelated errors in linear models in econometrics, we further design a deep difference module (DIF-Block) to correct autocorrelated errors and thus improve the prediction performance of the model. Moreover, all components are integrated seamlessly in a unified end-to-end framework. Extensive experiments on real-world datasets demonstrate that MWDINet outperforms the state-of-the-art models and has remarkable potential in stock price prediction.

Nonparametric Bayesian modeling on infinite mixture Student t copulas

This article introduces an infinite mixture Student t copula model using a nonparametric Bayesian approach. We establish a corresponding Markov chain Monte Carlo sampler for this model. In contrast to the normal mixture model, our proposed model is more suitable for data exhibiting tail dependence, which is frequently encountered in financial risk management. We evaluated the proposed algorithm through theoretical simulations and real data analysis. Parameter estimation results from the simulations demonstrate that our approach is competitive when compared to the standard maximum likelihood estimation method. The analysis of real financial data supports the validity of our approach and highlights the importance of applying a t copula in the presence of heavy tails.

Multi-pass Bayesian estimation: a robust Bayesian method

The prior plays a central role in Bayesian inference but specifying a prior is often difficult and a prior considered appropriate by a modeler may be significantly biased. We propose multi-pass Bayesian estimation (MBE), a robust Bayesian method capable of adjusting the prior’s influence on the inference result based on the prior’s quality. MBE adjusts the relative importance of the prior and the data by iteratively performing approximate Bayesian updates on the given data, with the number of updates determined using a cross-validation method. The repeated use of the data resembles the data cloning method, but data cloning performs maximum likelihood estimation (MLE), while MBE interpolates between standard Bayesian inference and MLE; there are also algorithmic differences in how MBE and data cloning make repeated use of the data. Alternatively, MBE can be considered a method for constructing a new prior from the given initial prior and the data. We additionally provide a new non-asymptotic bound on the convergence of data cloning, and provide an MBE-like iterative heuristic approach which achieves faster convergence speed by boosting posterior variance. In numerical simulations on several simulated and real-world datasets, MBE provides robust inference results as compared to standard Bayesian inference and MLE.

Population, Institutions, and Violent Conflicts – How Important is Population Pressure in Violent Resource-Based Conflicts?

Abstract This paper examines the moderating role of institutions in explaining the effects of population density, income, and high-valued natural resources (oil) on violent conflict events. Panel-Corrected Standard Errors and Poisson Pseudo Maximum Likelihood estimators were applied. Results show that population density beyond 2293 persons per square kilometre increases violent conflict events. Further, institutional quality has a moderating effect on violent conflict events – stronger corruption control reduces the positive effect of significant oil production on violent conflict events and weakens the negative effect of per capita income on such events. The results suggest that reducing violent conflict events requires at least three things; (1) keeping population density below 2293 persons per square kilometre, (2) investing in institutional quality improvements, and (3) raising incomes.

Unsupervised mixture estimation via approximate maximum likelihood based on the Cramér - von Mises distance

Mixture distributions with dynamic weights are an efficient way of modeling loss data characterized by heavy tails. However, maximum likelihood estimation of this family of models is difficult, mostly because of the need to evaluate numerically an intractable normalizing constant. In such a setup, simulation-based estimation methods are an appealing alternative. The approximate maximum likelihood estimation (AMLE) approach is employed. It is a general method that can be applied to mixtures with any component densities, as long as simulation is feasible. The focus is on the dynamic lognormal-generalized Pareto distribution, and the Cramér - von Mises distance is used to measure the discrepancy between observed and simulated samples. After deriving the theoretical properties of the estimators, a hybrid procedure is developed, where standard maximum likelihood is first employed to determine the bounds of the uniform priors required as input for AMLE. Simulation experiments and two real-data applications suggest that this approach yields a major improvement with respect to standard maximum likelihood estimation.

On the Robust Estimation of Spatial Autoregressive Models

The spatial autoregressive (SAR) model is commonly used for cross-sectional data with spatial dependence in the response variable. A robust M-estimator for the SAR model with independent normal random errors is proposed, which is resistant to extreme contamination or outliers in the data. The proposed estimator is defined by a set of robust estimating functions derived by modifying the estimating functions for the standard maximum likelihood (ML) estimator using the Huber function. An iterative algorithm based on iteratively reweighted least squares and a fixed point method is provided to compute the robust M-estimator. The proposed estimator is shown to be Fisher-consistent under the core model, has a bounded influence function, and is asymptotically normally distributed. A closed-form expression for the asymptotic covariance matrix estimator under the core model is derived, and used to study the asymptotic relative efficiency of the proposed estimator. Simulation studies demonstrate that the proposed robust M-estimator greatly outperforms the ML estimator as well as several other non-robust estimators of the SAR model, in terms of both estimation and inference under different contamination scenarios. Additionally, the proposed estimator offers comparable performance to some robust estimators of the SAR model under contamination. Applying the robust M-estimator for the SAR model to the U.S. federal grants and U.S. Nielsen Local Television View datasets reveals strong positive spatial dependence in the grant value among counties, insignificant positive effect of news watching on grant distribution, and evidence of outliers in the data. This results in different conclusions from those obtained using the ML estimator.

New exponentiated generalized censored regression models: Monte Carlo simulation and application

The standard maximum likelihood estimator of censored regression models is constructed under the assumption of the normal distribution and it yields inefficient results in case of non-normal asymmetric data or/and presence of outlier(s). In the analysis of censored real-life data encountered in many disciplines such as finance, medical sciences, and engineering, there is often a violation of the assumption of normality. In this paper, we propose a generalization of the censored normal regression based on the extended normal distribution (EGT). The generalized model, with two additional shape parameters as a and b, includes classical Tobit model for a = 0 and b = 0 and Alpha-Power Tobit model for a = 1. It provides very flexible estimation in cases symmetric-asymmetric data and especially in case non-normal asymmetric data or/and presence of outlier(s). In addition, Lehmann type II-G Tobit model (LTII-GT) is also proposed for b = 1 as a special case of generalization. The performance of proposed two new generalizations of Tobit, namely EGT and LTII-GT has been compared with Tobit, Alpha-Power Tobit, and classical estimators by means of a comprehensive Monte Carlo simulation study. The performance of proposed models' ML estimators is illustrated on a real data set. The superiority of the proposed estimators is shown.

A Double-Penalized Estimator to Combat Separation and Multicollinearity in Logistic Regression

When developing prediction models for small or sparse binary data with many highly correlated covariates, logistic regression often encounters separation or multicollinearity problems, resulting serious bias and even the nonexistence of standard maximum likelihood estimates. The combination of separation and multicollinearity makes the task of logistic regression more difficult, and a few studies addressed separation and multicollinearity simultaneously. In this paper, we propose a double-penalized method called lFRE to combat separation and multicollinearity in logistic regression. lFRE combines the logF-type penalty with the ridge penalty. The results indicate that compared with other penalty methods, lFRE can not only effectively remove bias from predicted probabilities but also provide the minimum mean squared prediction error. Aside from that, a real dataset is also employed to test the performance of the lFRE algorithm compared with several existing methods. The result shows that lFRE has strong competitiveness compared with them and can be used as an alternative algorithm in logistic regression to solve separation and multicollinearity problems.

Robust prediction interval estimation for Gaussian processes by cross-validation method

Probabilistic regression models typically use the Maximum Likelihood Estimation or Cross-Validation to fit parameters. These methods can give an advantage to the solutions that fit observations on average, but they do not pay attention to the coverage and the width of Prediction Intervals. A robust two-step approach is used to address the problem of adjusting and calibrating Prediction Intervals for Gaussian Processes Regression. First, the covariance hyperparameters are determined by a standard Cross-Validation or Maximum Likelihood Estimation method. A Leave-One-Out Coverage Probability is introduced as a metric to adjust the covariance hyperparameters and assess the optimal type II Coverage Probability to a nominal level. Then a relaxation method is applied to choose the hyperparameters that minimize the Wasserstein distance between the Gaussian distribution with the initial hyperparameters (obtained by Cross-Validation or Maximum Likelihood Estimation) and the proposed Gaussian distribution with the hyperparameters that achieve the desired Coverage Probability. The method gives Prediction Intervals with appropriate coverage probabilities and small widths.

Integration of model-based recursive partitioning with bias reduction estimation: a case study assessing the impact of Oliver’s four factors on the probability of winning a basketball game

In this contribution, we investigate the importance of Oliver’s Four Factors, proposed in the literature to identify a basketball team’s strengths and weaknesses in terms of shooting, turnovers, rebounding and free throws, as success drivers of a basketball game. In order to investigate the role of each factor in the success of a team in a match, we applied the MOdel-Based recursive partitioning (MOB) algorithm to real data concerning 19,138 matches of 16 National Basketball Association (NBA) regular seasons (from 2004–2005 to 2019–2020). MOB, instead of fitting one global Generalized Linear Model (GLM) to all observations, partitions the observations according to selected partitioning variables and estimates several ad hoc local GLMs for subgroups of observations. The manuscript’s aim is twofold: (1) in order to deal with (quasi) separation problems leading to convergence problems in the numerical solution of Maximum Likelihood (ML) estimation in MOB, we propose a methodological extension of GLM-based recursive partitioning from standard ML estimation to bias-reduced (BR) estimation; and (2) we apply the BR-based GLM trees to basketball analytics. The results show models very easy to interpret that can provide useful support to coaching staff’s decisions.

Generative adversarial network enables rapid and robust fluorescence lifetime image analysis in live cells

Fluorescence lifetime imaging microscopy (FLIM) is a powerful tool to quantify molecular compositions and study molecular states in complex cellular environment as the lifetime readings are not biased by fluorophore concentration or excitation power. However, the current methods to generate FLIM images are either computationally intensive or unreliable when the number of photons acquired at each pixel is low. Here we introduce a new deep learning-based method termed flimGANE (fluorescence lifetime imaging based on Generative Adversarial Network Estimation) that can rapidly generate accurate and high-quality FLIM images even in the photon-starved conditions. We demonstrated our model is up to 2,800 times faster than the gold standard time-domain maximum likelihood estimation (TD_MLE) and that flimGANE provides a more accurate analysis of low-photon-count histograms in barcode identification, cellular structure visualization, Förster resonance energy transfer characterization, and metabolic state analysis in live cells. With its advantages in speed and reliability, flimGANE is particularly useful in fundamental biological research and clinical applications, where high-speed analysis is critical.

Learning Markov Models Via Low-Rank Optimization

Taming high-dimensional Markov models In “Learning Markov models via low-rank optimization”, Z. Zhu, X. Li, M. Wang, and A. Zhang focus on learning a high-dimensional Markov model with low-dimensional latent structure from a single trajectory of states. To overcome the curse of high dimensions, the authors propose to equip the standard MLE (maximum-likelihood estimation) with either nuclear norm regularization or rank constraint. They show that both approaches can estimate the full transition matrix accurately using a trajectory of length that is merely proportional to the number of states. To solve the rank-constrained MLE, which is a nonconvex problem, the authors develop a new DC (difference) programming algorithm. Finally, they apply the proposed methods to analyze taxi trips on the Manhattan island and partition the island based on the destination preference of customers; this partition can help balance supply and demand of taxi service and optimize the allocation of traffic resources.

Estimation of Evapotranspiration Rates and Root Water Uptake Profiles From Soil Moisture Sensor Array Data

AbstractEvapotranspiration is arguably the least quantified component of the hydrologic cycle. We propose two complementary strategies for estimation of evapotranspiration rates and root water uptake profiles from soil‐moisture sensor‐array data. One is our implementation of ensemble Kalman filter (EnKF); it treats the evapotranspiration sink term in the Richards equation, rather than soil moisture, as the observable to update. The other is a maximum likelihood estimator (MLE) applied to the same observable; it is supplemented with the Fisher information matrix to quantify uncertainty in its predictions. We use numerical experiments to demonstrate the accuracy and computational efficiency of these techniques. We found our EnKF implementation to be two orders of magnitude faster than either the standard EnKF or MLE, and our MLE procedure to require an order of magnitude fewer iterations to converge than its counterpart applied to soil moisture. These findings render our methodologies a viable and practical tool for estimation of the root water uptake profiles and evaporation rates, with the MLE technique to be used when the prior knowledge about evapotranspiration at the site is elusive.

Minimum Message Length Inference of the Exponential Distribution with Type I Censoring.

Data with censoring is common in many areas of science and the associated statistical models are generally estimated with the method of maximum likelihood combined with a model selection criterion such as Akaike’s information criterion. This manuscript demonstrates how the information theoretic minimum message length principle can be used to estimate statistical models in the presence of type I random and fixed censoring data. The exponential distribution with fixed and random censoring is used as an example to demonstrate the process where we observe that the minimum message length estimate of mean survival time has some advantages over the standard maximum likelihood estimate.

On the consistency of Bayes estimates for the infinite continuous mixture of Dirichlet distributions

In this paper, we introduced the infinite continuous mixture of Dirichlet distributions as a generalization of the infinite mixture of Dirichlet ones, in order to avoid the limitation of choosing the a priori sample size for the expectation \textit{a posteriori} estimator. Monte-Carlo sampling was used in order to obtain the \textit{posterior} distributions mixture, since this mixture is difficult to get analytically. A new parametrization of this proposed distribution was achieved. Then, we suggested a mixture expectation \textit{a posteriori} estimator of the unknown parameters. The proposed estimator solves the problem of how to construct a Bayesian estimation of proportions without specifying particular parameters and sample size of the prior knowledge. Some asymptotic properties of this estimator were derived, specifically, its bias and variance. The consistency and asymptotic normality of the estimator were also established when the sample size tends to infinity and its credible interval was determined. The performance of the proposed estimator was illustrated theoretically and by means of a simulation study. Ultimately, a comparative simulation study between the learned estimates, the proposed mixture expectation \textit{a posteriori}, standard Bayesian estimator, maximum likelihood and Jeffreys estimator, was established. According to this simulation, we were able to conclude that the prior infinite mixture of Dirichlet distributions offers higher accuracy and flexibility for modeling and learning data.

Inference for a two-stage enrichment design

Two-stage enrichment designs can be used to target the benefiting population in clinical trials based on patients’ biomarkers. In the case of continuous biomarkers, we show that using a bivariate model that treats biomarkers as random variables more accurately identifies a treatment-benefiting enriched population than assuming biomarkers are fixed. Additionally, we show that under the bivariate model, the maximum likelihood estimators (MLEs) follow a randomly scaled mixture of normal distributions. Using random normings, we obtain asymptotically standard normal MLEs and construct hypothesis tests. Finally, in a simulation study, we demonstrate that our proposed design is more powerful than a single stage design when outcomes and biomarkers are correlated; the model-based estimators have smaller bias and mean square error (MSE) than weighted average estimators.

Statistical analysis of flash flood events for designing water harvesting systems in an extremely arid environment

AbstractA water harvesting system for research purposes has been established in the Lisan Peninsula of the Dead Sea in the middle of the Jordan Rift Valley, where no authorized guideline is available for designing water harvesting systems. Rainfall and runoff, which occurred as flash floods, were observed at the downstream end of a gorge with a 1.12 km2 barren catchment area from October 2014 through July 2019. Due to the extremely arid environment, runoff from the catchment is ephemeral, and the flash flood events can be clearly distinguishable from each other. Thirteen flash flood events with a total runoff volume of more than 100 m3 were successfully recorded during the five rainy seasons. Pearson and Spearman correlations between duration, total rainfall depths at two points, total runoff volume, maximum runoff discharge, bulk runoff coefficient, total variation in runoff discharge and maximum variation in runoff discharge of each flash flood event were examined, revealing no straightforward relationship between rainfall and runoff. The performance of the conventional SCS runoff curve number method was also deficient in reproducing any rainfall–runoff relationship. Therefore, probability distribution fitting was performed for each random variable, focusing on the lognormal distribution with three parameters and the generalized extreme value distribution. The maximum goodness‐of‐fit estimation turns out to be a more rational and efficient method in obtaining the parameter values of those probability distributions rather than the standard maximum likelihood estimation, which has known disadvantages. Results support the design of the water harvesting system and provide quantitative information for designing and operating similar systems in the future.

Bayesian Estimation of The Ex-Gaussian Distribution

Fitting of the exponential modified Gaussian distribution to model reaction times and drawing conclusions from its estimated parameter values is one of the most popular method used in psychology. This paper aims to develop a Bayesian approach to estimate the parameters of the ex-Gaussian distribution. Since the chosen priors yield to posterior densities that are not of known form and that they are not always log-concave, we suggest to use the adaptive rejection Metropolis sampling method. Applications on simulated data and on real data are provided to compare this method to the standard maximum likelihood estimation method as well as the quantile maximum likelihood estimation. Results shows the effectiveness of the proposed Bayesian method by computing the root mean square error of the estimated parameters using the three methods.