Traditional Maximum Likelihood Estimation Research Articles

Optimal Subsampling for Functional Quasi-Mode Regression with Big Data

We propose investigating optimal subsampling for functional regression with massive datasets based on the mode value, which is referred to as functional quasi-mode regression, to reduce data volume and alleviate computational burden. Using data-adaptive weights derived from regression residuals, the suggested regression offers enhanced robustness against nonnormal errors compared to traditional least squares or maximum likelihood estimation methods. To estimate the model, we employ B-spline basis functions to approximate the functional coefficient and include a penalty term in the objective function for enforcing smoothness in the resulting estimator. We adopt a computationally efficient mode-expectation-maximization algorithm, augmented by a Gaussian kernel, for numerical estimation. Under mild regularity conditions, we derive the asymptotic distributions of both full data and subsample quasi-mode estimators. The optimal subsampling probabilities by minimizing the asymptotic variance-covariance matrix under A- and L-optimality criteria are identified. These optimal probabilities rely on the full data estimate, prompting the development of a two-step algorithm to approximate the optimal subsampling procedure. The resultant algorithm is processing-efficient and can significantly reduce computational time compared to the full data approach. We also establish the asymptotic normality of the quasi-mode estimator obtained through this two-step algorithm. To assess finite sample performance, we conduct Monte Carlo simulations and analyze air quality data, showcasing the effectiveness of the developed estimator. Supplemental materials for this article are available online.

Deep regression learning with optimal loss function

In this paper, we develop a novel efficient and robust nonparametric regression estimator under a framework of a feedforward neural network (FNN). There are several interesting characteristics for the proposed estimator. First, the loss function is built upon an estimated maximum likelihood function, which integrates the information from observed data as well as the information from the data distribution. Consequently, the resulting estimator has desirable optimal properties, such as efficiency. Second, different from the traditional maximum likelihood estimation (MLE), the proposed method avoids the specification of the distribution, making it adaptable to various distributions such as heavy tails and multimodal or heterogeneous distributions. Third, the proposed loss function relies on probabilities rather than direct observations as in least square loss, contributing to the robustness of the proposed estimator. Finally, the proposed loss function involves a nonparametric regression function only. This enables the direct application of the existing packages, simplifying the computational and programming requirements. We establish the large sample property of the proposed estimator in terms of its excess risk and minimax near-optimal rate. The theoretical results demonstrate that the proposed estimator is equivalent to the true MLE where the density function is known in terms of excess risk. Our simulation studies show that the proposed estimator outperforms the existing methods based on prediction accuracy, efficiency and robustness. Particularly, it is comparable to the MLE with the known density and even gets slightly better as the sample size increases. This implies that the adaptive and data-driven loss function from the estimated density may offer an additional avenue for capturing valuable information. We further apply the proposed method to four real data examples, resulting in significantly reduced out-of-sample prediction errors compared to existing methods.

HBMIRT: A SAS macro for estimating uni- and multidimensional 1- and 2-parameter item response models in small (and large!) samples

Item response theory (IRT) has evolved as a standard psychometric approach in recent years, in particular for test construction based on dichotomous (i.e., true/false) items. Unfortunately, large samples are typically needed for item refinement in unidimensional models and even more so in the multidimensional case. However, Bayesian IRT approaches with hierarchical priors have recently been shown to be promising for estimating even complex models in small samples. Still, it may be challenging for applied researchers to set up such IRT models in general purpose or specialized statistical computer programs. Therefore, we developed a user-friendly tool – a SAS macro called HBMIRT – that allows to estimate uni- and multidimensional IRT models with dichotomous items. We explain the capabilities and features of the macro and demonstrate the particular advantages of the implemented hierarchical priors in rather small samples over weakly informative priors and traditional maximum likelihood estimation with the help of a simulation study. The macro can also be used with the online version of SAS OnDemand for Academics that is freely accessible for academic researchers.

Joint estimation model for FSO channel parameters and performance evaluation based on CNNs.

Free space optical (FSO) communication systems experience turbulence-induced fading. As a possible solution, adaptive transmission, which adjusts transmitter parameters based on instantaneous channel state information (CSI), can be used. Most of the existing channel estimation methods ignore the impact of detection noise at the receiver, which will lead to additional estimation errors. In this paper, a joint estimation model based on convolutional neural networks (CNNs) is proposed to estimate detection noise and turbulence fading parameters. We obtained turbulence channel simulation data sets considering the background of detection noise based on the edge probability distribution function of the receive signal. The training of the CNN estimator is carried out through maximum pooling, adaptive learning rate, and regularization, ultimately accurately estimating channel characteristics based on the optimal output results of the network. The simulation results show that the proposed CNN joint estimator performs better in high-detection-noise environments compared with traditional maximum likelihood estimators, and it has better generalization ability in different real atmospheric environments.

Heuristic techniques for maximum likelihood localization of radioactive sources via a sensor network

Maximum likelihood estimation (MLE) is an effective method for localizing radioactive sources in a given area. However, it requires an exhaustive search for parameter estimation, which is time-consuming. In this study, heuristic techniques were employed to search for radiation source parameters that provide the maximum likelihood by using a network of sensors. Hence, the time consumption of MLE would be effectively reduced. First, the radiation source was detected using the k-sigma method. Subsequently, the MLE was applied for parameter estimation using the readings and positions of the detectors that have detected the radiation source. A comparative study was performed in which the estimation accuracy and time consumption of the MLE were evaluated for traditional methods and heuristic techniques. The traditional MLE was performed via a grid search method using fixed and multiple resolutions. Additionally, four commonly used heuristic algorithms were applied: the firefly algorithm (FFA), particle swarm optimization (PSO), ant colony optimization (ACO), and artificial bee colony (ABC). The experiment was conducted using real data collected by the Low Scatter Irradiator facility at the Savannah River National Laboratory as part of the Intelligent Radiation Sensing System program. The comparative study showed that the estimation time was 3.27 s using fixed resolution MLE and 0.59 s using multi-resolution MLE. The time consumption for the heuristic-based MLE was 0.75, 0.03, 0.02, and 0.059 s for FFA, PSO, ACO, and ABC, respectively. The location estimation error was approximately 0.4 m using either the grid search-based MLE or the heuristic-based MLE. Hence, heuristic-based MLE can provide comparable estimation accuracy through a less time-consuming process than traditional MLE.

A simulation study on the insurance claims distribution using Weibull distribution

<p style="text-align:justify"><span style="font-size:10.5pt"><span style="font-family:等线"><span dir="ltr" lang="EN-US" style="font-family:"Cambria",serif"><span style="color:black">The Weibull distribution is extensively useful in the field of finance, insurance and natural disasters. Recently, It has been considered as one of the most frequently used statistical distributions in modelling and analyzing stock pricing movement and uncertain prediction in financial and investment data sets, such as insurance claims distribution. It is well known that the Bayes estimators of the two-parameter Weibull distribution do not have a compact form and the closed-form expression of the Bayes estimators cannot be obtained. In this paper and the Bayesian setting, it is assumed that the scale parameter of the Weibull model has a gamma prior under the assumption that its shape parameter is known. A simulation study is performed using random claims amount to compare the performance of the Bayesian approach with traditional maximum likelihood estimators in terms of Root Mean Square Errors (RMSE) and Mean Absolute Error (MAE) for different sample sizes, with specific values of the scale parameter and shape parameters. The results have been compared with the estimated result via the maximum likelihood method. The result revealed that the Bayesian approach behaves similarly to the maximum likelihood method when the sample size is small. Nevertheless, in all cases for both methods, the RMSE and MAE decrease as the sample size increases. Finally, applications of the proposed model to the insurance claim data set have been presented.</span></span></span></span></p>

Recurrent neural network based parameter estimation of Hawkes model on high-frequency financial data

This study examines the use of a recurrent neural network for estimating the parameters of a Hawkes model based on high-frequency financial data, and subsequently, for computing volatility. Neural networks have shown promising results in various fields, and interest in finance is also growing. Our approach demonstrates significantly faster computational performance compared to traditional maximum likelihood estimation methods while yielding comparable accuracy in both simulation and empirical studies. Furthermore, we demonstrate the application of this method for real-time volatility measurement, enabling the continuous estimation of financial volatility as new price data keeps coming from the market.

On Fitting the Lomax Distribution: A Comparison between Minimum Distance Estimators and Other Estimation Techniques

In this paper, we investigate the performance of a variety of frequentist estimation techniques for the scale and shape parameters of the Lomax distribution. These methods include traditional methods such as the maximum likelihood estimator and the method of moments estimator. A version of the maximum likelihood estimator adjusted for bias is included as well. Furthermore, an alternative moment-based estimation technique, the L-moment estimator, is included, along with three different minimum distance estimators. The finite sample performances of each of these estimators are compared in an extensive Monte Carlo study. We find that no single estimator outperforms its competitors uniformly. We recommend one of the minimum distance estimators for use with smaller samples, while a bias-reduced version of maximum likelihood estimation is recommended for use with larger samples. In addition, the desirable asymptotic properties of traditional maximum likelihood estimators make them appealing for larger samples. We include a practical application demonstrating the use of the described techniques on observed data.

New Cure Rate Survival Models Generated by Poisson Distribution and Different Regression Structures with Applications to Cancer Data Set

A new cure rate model is presented by assuming that, conditional on η , the number of competing causes of the event of interest follows the Poisson distribution, where η is assumed a random variable with gamma and generalized exponential distributions. For the time-to-event of the concurrent causes, we assumed the recently introduced truncated Nadarajah–Haghighi model. The model is parameterized directly in terms of the cure term, and then different symmetric and asymmetric link functions are used to assess the effects of covariates, such as logit, probit, log-log, Cauchit, Aranda-Ordaz, skewed probit, and skewed logit. Parameters estimation for the model is approached based on the traditional maximum likelihood estimation method. We achieve a simulation study in order to investigate the performance of these estimators under different scenarios. Finally, the model is illustrated in data sets related to two kinds of cancers (melanoma and colon cancer).

Risk preferences, gender effects and Bayesian econometrics

Gender differences in decision making is a topic that has attracted much attention in the literature and the debate seems to be inconclusive. A method that is often used in the economics literature to account for gender effects is by estimating econometric structural models and testing the significance of the estimated parameters. In this paper we focus on estimations of preference models and we show how omitting to account for behavioural heterogeneity can lead to failures in identifying potential differences. Using data from risky choice experiments, we compare the traditional representative agent Maximum Likelihood Estimation approach against two more flexible inference methods that allow for heterogeneity at the individual level, the Maximum Simulated Likelihood Estimation, and the Hierarchical Bayesian modelling. We show how ignoring heterogeneity may lead to failures capturing gender differences and we suggest the use of Bayesian modelling to effectively estimate the underlying parameters.

Bootstrap-based inferential improvements to the simplex nonlinear regression model.

In this paper we evaluate the performance of point and interval estimators based on the maximum likelihood(ML) method for the nonlinear simplex regression model. Inferences based on traditional maximum likelihood estimation have good asymptotic properties, but their performance in small samples may not be satisfactory. At out set we consider the maximum likelihood estimation for the parameters of the nonlinear simplex regression model, and so we introduced a bootstrap-based correction for such estimators of this model. We also develop the percentile and bootstrapt confidence intervals for those parameters as competitors to the traditional approximate confidence interval based on the asymptotic normality of the maximum likelihood estimators (MLEs). We then numerically evaluate the performance of these different methods for estimating the simplex regression model. The numerical evidence favors inference based on the bootstrap method, in special the bootstrapt interval, which was decisive in an application to real data.

Two‐parameter estimation for Tobit model: An application to national health and nutrition examination survey dataset

AbstractFixed censorship is usually encountered in medical fields, as well as biological, econometric, and engineering researches. It is quite possible to come across censored data with multicollinearity in real‐life studies. The multicollinearity affects traditional Tobit maximum likelihood estimation, which is often used in presence of censored data. Biased estimators such as Tobit Liu estimator with one biasing parameter can be used for solving the multicollinearity. In this study, Tobit two‐parameter estimator with two biasing parameters is proposed as an alternative to the classical Tobit maximum likelihood and Tobit Liu estimators for the censored data. This new estimator provides advantages from two different aspects thanks to having two biasing parameters. With a simulation study consisting of several scenarios where various levels of factors like the number of independent variables, sample size, correlation degree, variance, and censorship are examined, an in‐depth experimental analysis is conducted. In these scenarios, two different biasing parameters' selection methods where one of the biasing parameters is fixed to determine the other one are used and their corresponding outputs are obtained. In addition, a real‐life application is carried out with the National Health and Nutrition Examination Survey public‐use dataset which includes some demographic and health‐related laboratory measurements. The simulation study and data analysis results show that the proposed estimator is more advantageous to its competitors.

An Improved Mixture Density Network Via Wasserstein Distance Based Adversarial Learning for Probabilistic Wind Speed Predictions

This paper develops a novel improved mixture density network via Wasserstein distance-based adversarial learning (WA-IMDN) for achieving more accurate probabilistic wind speed predictions (PWSP). The proposed method utilizes the historical supervisory control and data acquisition (SCADA) system data collected from multiple wind turbines (WTs) in different wind farms to predict the wind speed probability density function (PDF) of a targeted WT at the next timestamp. To better capture the fluctuation pattern of historical wind speed sequences and estimate parameters of the probability mixture model for approximating the wind speed PDF, an improved mixture density network (IMDN) is proposed. To address drawbacks of the traditional maximum likelihood estimation (MLE) on training the mixture density network, a Wasserstein distance (WD)-based adversarial learning is developed and the reparameterization trick is employed for the gradient delivery. The effectiveness of the proposed WA-IMDN is validated based on SCADA data (One dataset is publicly accessible) by benchmarking against a set of the commonly considered and recently reported PWSP methods, such as the mixture density network (MDN), maximum likelihood estimation (MLE) based mixture density attention network (MLE-IMDN), recent DMDNN and Improved Deep Mixture Density Network (IDMDN). Results demonstrate the superior performance of the proposed WA-IMDN on the PWSP. To demonstrate the repeatability of the presented research, we release our code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/IkeYang/WA-IMDN-</uri> .

Residual life estimation of lithium-ion batteries based on nonlinear Wiener process with measurement error

Residual life (RL) estimation is a key issue in the prognostics and health management (PHM). This paper proposes a heuristic algorithm for RL estimation based on the nonlinear Wiener process with measurement error (ME) and also proposes an unbiased parameters estimation method. First, we use the nonlinear Wiener process with ME to model the degradation process. Then, an analytical expression of parameters estimation results with restriction of the nonlinear coefficient and variance of ME is obtained and an unbiased parameters estimation method is also derived by analyzing the natures of parameters estimation. Moreover, an empirical unbiased parameters estimation method for the degradation data with different measurement times is also proposed. After that, we extend the heuristic algorithm to the nonlinear Wiener process with ME and some relevant conclusions are proved. Finally, some simulation examples and a case study of lithium-ion batteries are used for experimental verification. The results show that the unbiased parameters estimation method is superior to the traditional maximum likelihood estimation (MLE) method and the heuristic RL estimation method can overcome the influence of imperfect prior information for lithium-ion batteries based on the nonlinear Wiener process with ME.

Automatic and Accurate Vision-Based Measurement of Camber and Toe-In Alignment of Vehicle Wheel

Camber and toe-in are important alignment parameters of vehicle wheels, which determine the operation stability and safety during vehicle driving. An automatic and accurate measurement method is proposed for camber and toe-in alignment without vision system calibration. Different from the previous target-based vision measurement (TVM) based on the maximum likelihood estimation (MLE), the RVM-DBSC model is proposed to achieve the automatic measurement of camber and toe-in alignment. Furthermore, the camber and toe-in measurement model is established by the maximum a posteriori (MAP) and the alternating direction method of multipliers (ADMM), rather than the traditional MLE. The contrast experiments are performed by the verification instrument, which indicates that the proposed method achieves effectiveness and high accuracy of camber and toe-in measurement.

On the mean-parameterized Bell–Touchard regression model for count data

The two-parameter Bell–Touchard distribution corresponds to a quite flexible yet tractable parametric family of discrete distributions, which arises prominently as an empirical model for count data. In this paper, we introduce a novel parametric regression model for count response variables on the basis of the mean-parameterized Bell–Touchard distribution, which allows interpretation of the regression coefficients in terms of the expectation of the count response variable, like the generalized linear model framework. The unknown parameters of the mean-parameterized Bell–Touchard regression model are estimated using the traditional maximum likelihood estimation procedure. We also consider the deviance residuals to assess departures from model assumptions. Empirical applications are provided to illustrate the mean-parameterized Bell–Touchard regression model in practice, and comparisons with some popular existing count regression models are made.

A restricted gamma ridge regression estimator combining the gamma ridge regression and the restricted maximum likelihood methods of estimation

In this article, we propose a restricted gamma ridge regression estimator (RGRRE) by combining the gamma ridge regression (GRR) and restricted maximum likelihood estimator (RMLE) to combat multicollinearity problem for estimating the parameter in the gamma regression model. The properties of the new estimator are discussed, and its superiority over the GRR, RMLE and traditional maximum likelihood estimator is theoretically analysed under different conditions. We also suggest some estimating methods to find the optimal value of the shrinkage parameter. A Monte Carlo simulation study is conducted to judge the performance of the proposed estimator. Finally, an empirical application is analysed to show the benefit of RGRRE over the existing estimators.

Multivariate zero-inflated Bell distribution and its inference and applications

In this paper we introduce a multivariate family of distributions for multivariate count data with excess zeros, which is a multivariate extension of the univariate zero-inflated Bell distribution. We derive various general properties of this multivariate distribution. In particular, the marginal distributions are univariate zero-inflated Bell distributions. The model parameters are estimated using the traditional maximum likelihood estimation method. In addition, we develop a simple EM algorithm to compute the maximum likelihood estimates of the parameters of the new multivariate distribution with closed-form expressions for the maximum likelihood estimators. Empirical applications that employ real multivariate count data are considered to illustrate the usefulness of the new class of multivariate distributions, and comparisons with the multivariate zero-inflated Poisson distribution, multivariate zero-adjusted Poisson distributions, and multivariate zero-inflated generalized Poisson distribution are made.

Generalized two-parameter estimators in the multinomial logit regression model: methods, simulation and application

In this article, we propose generalized two-parameter (GTP) estimators and an algorithm for the estimation of shrinkage parameters to combat multicollinearity in the multinomial logit regression model. In addition, the mean squared error properties of the estimators are derived. A simulation study is conducted to investigate the performance of proposed estimators for different sample sizes, degrees of multicollinearity, and the number of explanatory variables. Swedish football league dataset is analyzed to show the benefits of the GTP estimators over the traditional maximum likelihood estimator (MLE). The empirical results of this article revealed that GTP estimators have a smaller mean squared error than the MLE and can be recommended for practitioners.

Are Cramér-Rao lower bounds an accurate estimate for standard deviations in in vivo magnetic resonance spectroscopy?

Due to inherent time constraints for in vivo experiments, it is infeasible to repeat multiple MRS scans to estimate standard deviations on the desired measured parameters. As such, the Cramér-Rao lower bounds (CRLBs) have become the routine method to approximate standard deviations for in vivo experiments. Cramér-Rao lower bounds, however, as the name suggests, are theoretically a lower bound on the standard deviation and it is not clear if and under what circumstances this approximation is valid. Realistic synthetic 3 T spectra were used to investigate the relationship between estimated CRLBs, true CRLBs and standard deviations. Here we demonstrate that, although the CRLBs are theoretically truly a lower bound on the standard deviation (not an equality) for the problem typically encountered in quantification, they are still an adequate approximation to standard deviation as long as the model perfectly characterizes the data. In the case when the macromolecule basis deviates from the measured macromolecules it was shown that the CRLBs can deviate from standard deviations by approximately 50% for N-acetylaspartic acid, creatine and glutamate and of the order of 100% or more for myo-inositol and γ-aminobutyric acid. In the case when the model perfectly reflects the data the CRLBs are within approximately 10% of standard deviations for all metabolites. The result of the CRLB being within 10% of standard deviations means that, for an accurate model, novel quantification methods such as machine learning or deep learning will not be able to obtain substantially more precise estimates for the desired parameters than traditional maximum-likelihood estimation.