Exponential Family Of Distributions Research Articles

Prior effective sample size for exponential family distributions with multiple parameters

AbstractThe setting of priors is an important issue in Bayesian analysis. In particular, when external information is applied, a prior with too much information can dominate the posterior inferences. To prevent this effect, the effective sample size (ESS) can be used. Various ESSs have been proposed recently; however, all have the problem of limiting the applicable prior distributions. For example, one ESS can only be used with a prior that can be approximated by a normal distribution, and another ESS cannot be applied when the parameters are multidimensional. We propose an ESS to be applied to more prior distributions when the sampling model belongs to an exponential family (including the normal model and logistic regression models). This ESS has the predictive consistency and can be used with multidimensional parameters. It is confirmed from normally distributed data with the Student's‐t priors that this ESS behaves as well as an existing predictively consistent ESS for one‐parameter exponential families. As examples of multivariate parameters, ESSs for linear and logistic regression models are also discussed.

A dynamic count process

The current paper aims to complement the recent development of the observation-driven models of dynamic counts with a parametric-driven one for a general case, particularly discrete two parameters exponential family distributions. The current paper proposes a finite semiparametric exponential mixture of SETAR processes of the conditional mean of counts to capture the nonlinearity and complexity. Because of the intrinsic latency of the conditional mean, the general additive state-space representation of dynamic counts is firstly proposed then stationarity and geometric ergodicity are established under a mild set of conditions. We also propose to estimate the unknown parameters by using quasi maximum likelihood estimation and establishes the asymptotic properties of the quasi maximum likelihood estimators (QMLEs), particularly T-consistency and normality under the relatively mild set of conditions. Furthermore, the finite sample properties of the QMLEs are investigated via simulation exercises and an illustration of the proposed process is presented by applying the proposed method to the intraday transaction counts per minute of AstraZeneca stock.

ERPCA: Robust Principal Component Analysis for Exponential Family Distributions

AbstractRobust principal component analysis (RPCA) is a widely used method for recovering low‐rank structure from data matrices corrupted by significant and sparse outliers. These corruptions may arise from occlusions, malicious tampering, or other causes for anomalies, and the joint identification of such corruptions with low‐rank background is critical for process monitoring and diagnosis. However, existing RPCA methods and their extensions largely do not account for the underlying probabilistic distribution for the data matrices, which in many applications are known and can be highly non‐Gaussian. We thus propose a new method called RPCA for exponential family distributions (), which can perform the desired decomposition into low‐rank and sparse matrices when such a distribution falls within the exponential family. We present a novel alternating direction method of multiplier optimization algorithm for efficient decomposition, under either its natural or canonical parametrization. The effectiveness of is then demonstrated in two applications: the first for steel sheet defect detection and the second for crime activity monitoring in the Atlanta metropolitan area.

Cross-entropy based importance sampling for composite systems reliability evaluation with consideration of multivariate dependence

Multivariate kernel density estimation (KDE) is powerful for dependence modeling, but producing a non-sparse probability density estimation model. To address this issue, a sparse density estimation technique termed as generalized cross-entropy (GCE) density estimator, is proposed to estimate the joint probability density function (PDF) of correlated random variables (RVs). Moreover, to effectively accelerate the composite systems reliability evaluation, a unified cross-entropy based importance sampling (CEIS) method accommodating both GCE and KDE density estimators is presented, and a creative CE optimization approach is also developed based on the composition sampling algorithm, through which the analytical updating rules of IS-PDF parameters can be derived even though GCE and KDE estimators do not belong to the exponential distribution family (EF). Compared to the KDE estimator, the GCE estimator encounters much less dimensionality difficulty in CE optimization due to the model sparsity merit, and consequently achieves more accurate IS-PDF parameters and better CEIS efficiency. The superiority of GCE estimator over KDE estimator in correlation modeling and CEIS is verified by several numerical tests.

On the Efficiency of the newly Proposed Convex Olanrewaju-Olanrewaju Lo-oλγ(|θ|) Penalized Regression-Type Estimator via GLMs Technique.

In this article, we proposed a novel convex penalized regression-type estimator, termed Olanrewaju-Olanrewaju penalized regression-type estimator, denoted by Lo-oλγ(|θ|) for ultra and high-dimensional data consideration whenever the number of covariates (p) is strictly greater than the sample size(n⁡). The newly proposed penalized estimator was formulated via the unique combination of arctangent and hyperbolic functions. It was noted that the newly proposed estimator could efficiently work under the constraint p<n formulated for, and as well as for the conventional regression methods of n>p. The proposed convex penalized regression-type estimator was formulated via three symmetric distributional noises of Gaussian, Laplace, and Normal Inverse Gaussian (NIG), considered being the three certified symmetric distributions that belong to the exponential family of distributions. The regularity axioms and oracle properties of the newly proposed penalized estimator were worked-out, as well as the parameter estimation for the penalized regression-type estimator for the three symmetric noises via canonical expressible form of Generalized Linear Models (GLMs) and first-order Karush-Kuhn-Tucker (KKT) condition. The Lo-oλγ(|θ|) regression-type estimator was subjected to simulation studies of well-generated symmetric observations. The US Longley macroeconomic dataset with the constraint p<n was fitted to Lo-oλγ(|θ|) regression-type estimator under the umbrella of three symmetric distributional noises. In conclusion, Gaussian noise for Lo-oλγ(|θ|) penalized regression-type estimator under the umbrella of p<n for the simulation and real life analyzes was considered effectively efficient because of its most reduced Mean Square Error (mse) measured error-type in comparison to Gaussian, Laplace, and NIG noises for LASSO and Atan penalties; and as well for Laplace, and NIG noises for Lo-oλγ(|θ|).

Interval estimation of the overlapping coefficients in an exponential family of distributions based on upper record values

This paper investigates interval estimation for measures of overlap, namely Matusita's measure, Weitzman's measure and based on Kullback–Leibler. Two types of sampling procedures, namely, Simple Random Sample and Upper Record Values from two exponential populations. Bootstrap method series approximation is used to construct confidence intervals for the overlap measures. To illustrate the performance of the likelihood confidence intervals obtained for this overlapping coefficient, under our proposal, we carry out some simulation studies that yield adequate coverage frequencies, this study conducts for comparing the performances of various competing estimators. A real data set is analysed to exemplify our proposal.

Thompson Sampling-Based Partially Observable Online Change Detection for Exponential Families

This paper proposes a holistic sequential change detection framework for partially observable high-dimensional data streams with exponential-family distributions. The framework first proposes a general composite decomposition for exponential-family distributed data by projecting its natural parameter onto normal bases and abnormal bases, which enables efficient inference for sparse changes. Then, the inference results are used for detection scheme construction, and different types of test statistics can be compacted in our framework. Last, by further designing the test statistic as the reward function in the combinatorial multi-armed bandit problem, a Thompson sampling-based sensor allocation strategy is constructed to select the most anomalous variables. Theoretical properties of the detection framework are discussed. Finally, examples of Gaussian, Poisson, and binomial distributed data streams are given in numerical studies and case studies to evaluate the performance of our proposed method. History: Bianca Maria Colosimo served as the senior editor. Funding: C. Zhang is partially supported by the NSFC [Grants 71932006 and 72271138], the BNSF [Grant 9222014], and the ASFC [Grant 2020Z063058001]. H. Yan is partially supported by NIH [R21 AI157618] and NSF [CMMI 2316654]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijds.2022.00011 . Data Ethics & Reproducibility Note: No data ethics considerations are foreseen related to this paper. The code capsule is available on Code Ocean at https://codeocean.com/capsule/8794940/tree/v1 and in the e-Companion to this article (available at https://doi.org/10.1287/ijds.2022.00011 ).

A General Weighted Exponentiated Family of Distributions with Application to Carbon Fiber and Petroleum Rock Data

A General Weighted Exponentiated Family of Distributions with Application to Carbon Fiber and Petroleum Rock Data

Classification of household poverty in West Java using the generalized mixed-effects trees model

Dealing with fixed effects and random effects can be accomplished by combining statistical modeling and machine learning techniques. This paper discusses the modeling of fixed effects and random effects using a statistical machine-learning approach. We used the generalized mixed-effects trees (GMET), a tree-based mixed-effect model for dealing with response variables that belong to the exponential family of distributions. In this study, both simulation and actual/empirical data utilized the GMET method to discover data conditions that were appropriate for employing this approach. The simulation data was generated using different response variable generations, as well as different values of the variance of random effect and fixed effect coefficients. The findings indicated that the GMET performs similarly for different response variable generation scenarios. However, it performed better when the fixed effect value and the variance of random effects were large. When applied to the empirical data, the GMET method describes fixed effects and random effects and classifies household poverty status quite well based on the area under curve (AUC) value. It has also revealed that important variables for poverty classification are the number of household members, owning land, the type of main fuel used for cooking, and the main source of water used for drinking. In order to address the socioeconomic disparity that leads to poverty, the government may become concerned about these factors. In addition to that information, the use of regional typology as a random effect in the model has also contributed to the variation of household poverty status. Based on research, the fixed effects in mixed models do not need to be linear and GMET may be employed in grouped data structures, giving the GMET technique the ability to compete with other approaches/methods.

Asymptotic loss of the MLE of a truncation parameter in the presence of a nuisance parameter for a one-sided truncated family of distributions

For a truncated family of distributions with a truncation parameter γ and a parameter θ as a nuisance parameter, we derive the stochastic expansions of bias-adjusted maximum likelihood estimators and of γ based on a sample of size n when θ is known and when θ is unknown, respectively. The asymptotic loss of relative to is obtained up to the second order, that is the order n −1. The results are a generalization of those for a one-sided truncated exponential family of distributions. Its application to truncated t-distributions is also given.

Supervised Dimensionality Reduction of Proportional Data Using Exponential Family Distributions

Most well-known supervised dimensionality reduction algorithms suffer from the curse of dimensionality while handling high-dimensional sparse data due to ill-conditioned second-order statistics matrices. They also do not deal with multi-modal data properly since they construct neighborhood graphs that do not discriminate between multi-modal classes of data and single-modal ones. In this paper, a novel method that mitigates the above problems is proposed. In this method, assuming the data is from two classes, they are projected into the low-dimensional space in the first step which removes sparsity from the data and reduces the time complexity of any operation drastically afterwards. These projected data are modeled using a mixture of exponential family distributions for each class, allowing the modeling of multi-modal data. A measure for the similarity between the two projected classes is used as an objective function for constructing an optimization problem, which is then solved using a heuristic search algorithm to find the best separating projection. The conducted experiments show that the proposed method outperforms the rest of the compared algorithms and provides a robust effective solution to the problem of dimensionality reduction even in the presence of multi-modal and sparse data.

The Pezeta regression model: an alternative to unit Lindley regression model

A new probability distribution is proposed in this paper. This new distribution has support on the interval and was obtained after transforming the random variable with exponential distribution. The mode, quantile function, median, ordinary moments and density function belongs to exponential family of distributions are demonstrated. The maximum likelihood method is used to obtain the parameter estimate. A regression model for the median of the distribution is also proposed. Closed-form expressions for the score vector and Fisher’s information matrix are demonstrated. A simulation study and an application to real data showed the good performance of the proposed regression model.

A Copula Based Investigation of Reliability for the Multivariate Exponential Family of Distributions

This paper explores the possibility for using the copulas in the context of evaluating the reliability for the exponential family of distributions.

Adaptive Perturbation-Based Gradient Estimation for Discrete Latent Variable Models

The integration of discrete algorithmic components in deep learning architectures has numerous applications. Recently, Implicit Maximum Likelihood Estimation, a class of gradient estimators for discrete exponential family distributions, was proposed by combining implicit differentiation through perturbation with the path-wise gradient estimator. However, due to the finite difference approximation of the gradients, it is especially sensitive to the choice of the finite difference step size, which needs to be specified by the user. In this work, we present Adaptive IMLE (AIMLE), the first adaptive gradient estimator for complex discrete distributions: it adaptively identifies the target distribution for IMLE by trading off the density of gradient information with the degree of bias in the gradient estimates. We empirically evaluate our estimator on synthetic examples, as well as on Learning to Explain, Discrete Variational Auto-Encoders, and Neural Relational Inference tasks. In our experiments, we show that our adaptive gradient estimator can produce faithful estimates while requiring orders of magnitude fewer samples than other gradient estimators.

Low-rank latent matrix-factor prediction modeling for generalized high-dimensional matrix-variate regression.

Motivated by diagnosing the COVID-19 disease using two-dimensional (2D) image biomarkers from computed tomography (CT) scans, we propose a novel latent matrix-factor regression model to predict responses that may come from an exponential distribution family, where covariates include high-dimensional matrix-variate biomarkers. A latent generalized matrix regression (LaGMaR) is formulated, where the latent predictor is a low-dimensional matrix factor score extracted from the low-rank signal of the matrix variate through a cutting-edge matrix factor model. Unlike the general spirit of penalizing vectorization plus the necessity of tuning parameters in the literature, instead, our prediction modeling in LaGMaR conducts dimension reduction that respects the geometric characteristic of intrinsic 2D structure of the matrix covariate and thus avoids iteration. This greatly relieves the computation burden, and meanwhile maintains structural information so that the latent matrix factor feature can perfectly replace the intractable matrix-variate owing to high-dimensionality. The estimation procedure of LaGMaR is subtly derived by transforming the bilinear form matrix factor model onto a high-dimensional vector factor model, so that the method of principle components can be applied. We establish bilinear-form consistency of the estimated matrix coefficient of the latent predictor and consistency of prediction. The proposed approach can be implemented conveniently. Through simulation experiments, the prediction capability of LaGMaR is shown to outperform some existing penalized methods under diverse scenarios of generalized matrix regressions. Through the application to a real COVID-19 dataset, the proposed approach is shown to predict efficiently the COVID-19.

Frequency Analysis of Extreme Events Using the Univariate Beta Family Probability Distributions

This manuscript presents three families of distributions, namely the Beta, Beta Prime and Beta Exponential families of distributions. From all the distributions of these families, 14 statistical distributions of three, four and five parameters are presented that have applicability in the analysis of extreme phenomena in hydrology. These families of distributions were analyzed regarding the improvement of the existing legislation for the determination of extreme events, specifically the elaboration of a norm regarding frequency analysis in hydrology. To estimate the parameters of the analyzed distributions, the method of ordinary moments and the method of linear moments were used; the latter conforms to the current trend for estimating the parameters of statistical distributions. The main purpose of the manuscript was to identify other distributions from these three families with applicability in flood frequency analysis compared to the distributions already used in the literature from these families, such as the Log–logistic distribution, the Dagum distribution and the Kumaraswamy distribution. The manuscript does not exclude the applicability of other distributions from other families in the frequency analysis of extreme values, especially since these families were also analyzed within the research carried out in the Faculty of Hydrotechnics and presented in other materials. All the necessary elements for their use are presented, including the probability density functions, the complementary cumulative distribution functions, the quantile functions and the exact and approximate relations for estimating parameters. A flood frequency analysis case study was carried out for the Prigor RiverRiver, to numerically present the proposed distributions. The performance of this distributions were evaluated using the relative mean error, the relative absolute error and the L-skewness–L-kurtosis diagram. The best fit distributions are the Kumaraswamy, the Generalized Beta Exponential and the Generalized Beta distributions, which presented a stability related to both the length of the data and the presence of outliers.

Numerical solution of Kiefer-Weiss problems when sampling from continuous exponential families

In this article, we deal with problems of testing hypotheses in the framework of sequential statistical analysis. The main concern is the optimal design and performance evaluation of sampling plans in Kiefer-Weiss problems. The main goal of the Kiefer-Weiss problem is designing hypothesis tests that minimize the maximum average sample number, over all parameter values, as opposed to both the sequential probability tests (SPRTs) minimizing the average sample number only at two hypothesis points and the classical fixed-sample-size test. For observations that follow a distribution from an exponential family of the continuous type, we provide algorithms for optimal design in the modified Kiefer-Weiss problem and obtain formulas for evaluating the performance of sequential tests by calculating the operating characteristic function, the average sample number, and some related characteristics. These formulas cover, as a particular case, the SPRTs and their truncated versions, as well as optimal finite-horizon sequential tests. In the setting of the original Kiefer-Weiss problem we apply the method of our recent work (Sequential Analysis 2022, 41(2), 198–219) for numerical construction of the optimal tests. For the particular case of sampling from a normal distribution with a known variance, we make numerical comparisons of the Kiefer-Weiss solution with the SPRT and the fixed-sample-size test provided that the three tests have the same levels of the error probabilities. All of the algorithms are implemented in the form of computer code written in the R programming language and are available at the GitHub public repository (https://github.com/tosinabase/Kiefer-Weiss). Guidelines on the adaptation of the program code to other exponential family distributions are provided.

On Bayesian credibility mean for finite mixture distributions

AbstractConsider the problem of determining the Bayesian credibility mean $E(X_{n+1}|X_1,\cdots, X_n),$ whenever the random claims $X_1,\cdots, X_n,$ given parameter vector $\boldsymbol{\Psi},$ are sampled from the K-component mixture family of distributions, whose members are the union of different families of distributions. This article begins by deriving a recursive formula for such a Bayesian credibility mean. Moreover, under the assumption that using additional information $Z_{i,1},\cdots,Z_{i,m},$ one may probabilistically determine a random claim $X_i$ belongs to a given population (or a distribution), the above recursive formula simplifies to an exact Bayesian credibility mean whenever all components of the mixture distribution belong to the exponential families of distributions. For a situation where a 2-component mixture family of distributions is an appropriate choice for data modelling, using the logistic regression model, it shows that: how one may employ such additional information to derive the Bayesian credibility model, say Logistic Regression Credibility model, for a finite mixture of distributions. A comparison between the Logistic Regression Credibility (LRC) model and its competitor, the Regression Tree Credibility (RTC) model, has been given. More precisely, it shows that under the squared error loss function, it shows the LRC’s risk function dominates the RTC’s risk function at least in an interval which about $0.5.$ Several examples have been given to illustrate the practical application of our findings.

Modeling Partitions of Individuals

Despite the central role of self-assembled groups in animal and human societies, statistical tools to explain their composition are limited. The authors introduce a statistical framework for cross-sectional observations of groups with exclusive membership to illuminate the social and organizational mechanisms that bring people together. Drawing from stochastic models for networks and partitions, the proposed framework introduces an exponential family of distributions for partitions. The authors derive its main mathematical properties and suggest strategies to specify and estimate such models. A case study on hackathon events applies the developed framework to the study of mechanisms underlying the formation of self-assembled project teams.

Assisting the decision making-A generalization of choice models to handle the binary choices

<abstract> <p>This research fundamentally aims at providing a generalized framework to assist the launch of paired comparison models while dealing with discrete binary choices. The purpose is served by exploiting the fundaments of the exponential family of distributions. The proposed generalization is proved to cater to seven paired comparison models as members of this newly developed mechanism. The legitimacy of the devised scheme is demonstrated through rigorous simulation-based investigation as well as keenly persuaded empirical evaluations. A detailed analysis, covering a wide range of parametric settings, through the launch of Gibbs Sampler—a notable extension of Markov Chain Monte Carlo methods, is conducted under the Bayesian paradigm. The outcomes of this research substantiate the legitimacy of the devised general structure by not only successfully retaining the preference ordering but also by staying consistent with the established theoretical framework of comparative models.</p> </abstract>